Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: skip unsupported codecs/storage transformers with "must_understand": false (ZEP0009) #150

Merged
merged 7 commits into from
Mar 9, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion zarrs/src/array/codec/array_to_bytes/codec_chain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,17 @@
let mut array_to_bytes: Option<NamedArrayToBytesCodec> = None;
let mut bytes_to_bytes: Vec<NamedBytesToBytesCodec> = vec![];
for metadata in metadatas {
let codec = Codec::from_metadata(metadata)?;
let codec = match Codec::from_metadata(metadata) {
Ok(codec) => Ok(codec),
Err(err) => {
if metadata.must_understand() {
Err(err)

Check warning on line 141 in zarrs/src/array/codec/array_to_bytes/codec_chain.rs

View check run for this annotation

Codecov / codecov/patch

zarrs/src/array/codec/array_to_bytes/codec_chain.rs#L139-L141

Added lines #L139 - L141 were not covered by tests
} else {
continue;

Check warning on line 143 in zarrs/src/array/codec/array_to_bytes/codec_chain.rs

View check run for this annotation

Codecov / codecov/patch

zarrs/src/array/codec/array_to_bytes/codec_chain.rs#L143

Added line #L143 was not covered by tests
}
}
}?;

Check warning on line 146 in zarrs/src/array/codec/array_to_bytes/codec_chain.rs

View check run for this annotation

Codecov / codecov/patch

zarrs/src/array/codec/array_to_bytes/codec_chain.rs#L146

Added line #L146 was not covered by tests

match codec {
Codec::ArrayToArray(codec) => {
array_to_array.push(NamedArrayToArrayCodec::new(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,16 @@
) -> Result<Self, PluginCreateError> {
let mut storage_transformers = Vec::with_capacity(metadatas.len());
for metadata in metadatas {
let storage_transformer = try_create_storage_transformer(metadata, path)?;
let storage_transformer = match try_create_storage_transformer(metadata, path) {
Ok(storage_transformer) => Ok(storage_transformer),
Err(err) => {
if metadata.must_understand() {
Err(err)

Check warning on line 44 in zarrs/src/array/storage_transformer/storage_transformer_chain.rs

View check run for this annotation

Codecov / codecov/patch

zarrs/src/array/storage_transformer/storage_transformer_chain.rs#L40-L44

Added lines #L40 - L44 were not covered by tests
} else {
continue;

Check warning on line 46 in zarrs/src/array/storage_transformer/storage_transformer_chain.rs

View check run for this annotation

Codecov / codecov/patch

zarrs/src/array/storage_transformer/storage_transformer_chain.rs#L46

Added line #L46 was not covered by tests
}
}
}?;

Check warning on line 49 in zarrs/src/array/storage_transformer/storage_transformer_chain.rs

View check run for this annotation

Codecov / codecov/patch

zarrs/src/array/storage_transformer/storage_transformer_chain.rs#L49

Added line #L49 was not covered by tests
storage_transformers.push(storage_transformer);
}
Ok(Self(storage_transformers))
Expand Down
1 change: 1 addition & 0 deletions zarrs_metadata/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Add support for a `must_understand` field to `MetadataV3` (ZEP0009)
- Extensions can now be parsed in more than just the additional fields of array/group metadata (e.g. codecs)
- Automatically skip unsupported codecs/storage transformers with `"must_understand": false`
- Add `CodecMap` and `CodecName` for codec `nam` overriding and aliasing
- Implement `From<T> for MetadataConfiguration` for all codec configuration enums
- Implement `Copy` for `ZstdCompressionLevel`
Expand Down