-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Balinus/patch 1 savedoc #438
Balinus/patch 1 savedoc #438
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #438 +/- ##
=======================================
Coverage 68.39% 68.40%
=======================================
Files 12 12
Lines 1778 1807 +29
=======================================
+ Hits 1216 1236 +20
- Misses 562 571 +9 ☔ View full report in Codecov by Sentry. |
Pull Request Test Coverage Report for Build 10853841802Details
💛 - Coveralls |
Save a dataset to Zarr format with compression: | ||
|
||
````@example write | ||
using Blosc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you need to add Blosc
to the Project toml, explicitly. Or, loaded it from whatever other package is also available.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding Blosc to YAXArrays's Project.toml? or perhaps you mean YAXArrays/docs/Project.toml ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to docs 😄 .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can access the compressor within Zarr.jl directly. I will adapt the code
savedataset(ds, path="ds.zarr", driver=:zarr, compressor=compression) | ||
nothing # hide | ||
```` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should have more than 1 compression level, and compare them both at the end, namely, what their disk sizes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean, compare on my computer and report sizes in the documentation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here, directly. Let's say a ds_level_1.zarr
and ds_level4.zarr
. And then query their sizes and print them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had never use this argument, so..., also I don't know what are the defaults, and hence how different things actually are.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh, I see I didn't even thought that the code were actually run! We'll look into it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Humm, strangely, the compression does not seems to work as expected. There is little effect on Zarr folders and there is some on netcdf files...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
76M test1.nc
103M test_default.nc
75M test9.nc
84M test1.zarr
83M test_default.zarr
82M test9.zarr
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks a lot! some minor comments.
Co-authored-by: Lazaro Alonso <[email protected]>
Documentation about compression #215