Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle local paths without file:/// #142

Merged
merged 1 commit into from
Oct 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ documentation = "https://appliedcomputing.io/simkube/"
license-file = "LICENSE"
readme = "README.md"
edition = "2021"
rust-version = "1.79"

[profile.dev.package."*"]
debug = false
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ APP_VERSION=$(shell $(APP_VERSION_CMD))
include build/base.mk
include build/k8s.mk

RUST_BUILD_IMAGE ?= rust:buster
RUST_BUILD_IMAGE ?= rust:1.79-bullseye

main:
docker run $(DOCKER_ARGS) -u `id -u`:`id -g` -w /build -v `pwd`:/build:rw -v $(BUILD_DIR):/build/.build:rw $(RUST_BUILD_IMAGE) make build-docker
Expand Down
2 changes: 1 addition & 1 deletion docs/intro/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This guide will walk you through installing the various SimKube components

The following prereqs are required for all components:

- [Rust (including Cargo)](https://www.rust-lang.org/learn/get-started) >= 1.71
- [Rust (including Cargo)](https://www.rust-lang.org/learn/get-started) >= 1.79
- [Docker](https://docs.docker.com/get-started/)
- [kubectl](https://kubernetes.io/docs/tasks/tools/) >= 1.27
- a Kubernetes cluster running at least v1.27
Expand Down
2 changes: 1 addition & 1 deletion images/Dockerfile.sk-ctrl
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM debian:buster-slim
FROM debian:bullseye-slim

ARG K8S_VERSION=v1.29.5

Expand Down
2 changes: 1 addition & 1 deletion images/Dockerfile.sk-driver
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM debian:buster-slim
FROM debian:bullseye-slim

ARG K8S_VERSION=v1.29.5

Expand Down
2 changes: 1 addition & 1 deletion images/Dockerfile.sk-tracer
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM debian:buster-slim
FROM debian:bullseye-slim

RUN apt-get update && apt-get install -y \
ca-certificates \
Expand Down
1 change: 1 addition & 0 deletions sk-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ thiserror = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
url = { workspace = true }

# testutils dependencies
http = { workspace = true, optional = true }
Expand Down
30 changes: 28 additions & 2 deletions sk-core/src/external_storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@
/// Set the `GOOGLE_SERVICE_ACCOUNT` environment variable to the path for your service account JSON
/// file (if you're running inside a container, you'll need that file injected as well). Pass in a
/// URL like `gs://bucket/path/to/resource`.
use std::path::{
absolute,
PathBuf,
};

use anyhow::anyhow;
use async_trait::async_trait;
use bytes::Bytes;
#[cfg(feature = "testutils")]
Expand Down Expand Up @@ -53,8 +59,7 @@ pub struct SkObjectStore {

impl SkObjectStore {
pub fn new(path_str: &str) -> anyhow::Result<SkObjectStore> {
let url = Url::parse(path_str)?;
let (scheme, path) = ObjectStoreScheme::parse(&url)?;
let (scheme, path) = parse_path(path_str)?;
let store: Box<DynObjectStore> = match scheme {
ObjectStoreScheme::Local => Box::new(object_store::local::LocalFileSystem::new()),
ObjectStoreScheme::Memory => Box::new(object_store::memory::InMemory::new()),
Expand Down Expand Up @@ -96,6 +101,18 @@ impl ObjectStoreWrapper for SkObjectStore {
}
}

fn parse_path(path_str: &str) -> anyhow::Result<(ObjectStoreScheme, Path)> {
let url = match Url::parse(path_str) {
Err(url::ParseError::RelativeUrlWithoutBase) => {
let path = absolute(PathBuf::from(path_str))?;
Url::from_file_path(path).map_err(|e| anyhow!("could not create URL from file path: {e:?}"))?
},
res => res?,
};

Ok(ObjectStoreScheme::parse(&url)?)
}

#[cfg(test)]
mod test {
use rstest::*;
Expand All @@ -112,4 +129,13 @@ mod test {
let store = SkObjectStore::new("s3://foo/bar").unwrap();
assert_eq!(store.scheme(), ObjectStoreScheme::AmazonS3);
}

#[rstest]
#[case::with_base("file:///tmp/foo")]
#[case::without_base("/tmp/foo")]
#[case::relative("foo")]
fn test_new_sk_object_store_local_path(#[case] path: &str) {
let store = SkObjectStore::new(path).unwrap();
assert_eq!(store.scheme(), ObjectStoreScheme::Local);
}
}
Loading