dwarfs-0.7.0-RC1

[mhx]

Bugfixes

• Fixed extract_block.py, which was incorrectly using printf instead of print.

• Support LZ4 compression levels above 9.

Features

• Added --filter option to support simple (rsync-like) filter rules. This was driven by a discussion on github Enhance mkdwarfs to support specifying a list of files to include (similar to cpio) #6.

• Added --input-list option to support reading a list of input files from a file or stdin. At least partially fixes github Enhance mkdwarfs to support specifying a list of files to include (similar to cpio) #6.

• The compression code has been made more modular. This should make it much easier to add support for more compression algorithms in the future.

• Added support for Brotli compression. This is generally much slower at compression than ZSTD or LZMA, but faster than LZMA, while offering a compression ratio better than ZSTD. Fixes github new compression algo #76.

• Added support for choosing the file hashing algorithm using the --file-hash option. This allows you to pick a secure hash instead of the default XXH3. Also fixes github Choice of hash for duplicate detection #92.

• Improved de-duplication algorithm to only hash files with the same size. File hashing is delayed until at least one more file with the same size is discovered. This happens automatically and should improve scanning speed, especially on slow file systems.

• Added --max-similarity-size option to prevent similarity hashing of huge files. This saves scanning time, especially on slow file systems, while it shouldn't affect compression ratio too much.

• Honour user locale when formatting numbers.

• Added --num-scanner-workers option.

• Added support for extracting corrupted file systems with dwarfsextract. This is enabled using the --continue-on-error and, if really needed, --disable-integrity-check options. Fixes github dwarfsextract aborts instead of skipping when corrupt files are encountered #51.

Other

• Added unit tests for progress class.

• Lots of internal cleanups.

---

This discussion was created from the release dwarfs-0.7.0-RC1.