Add Blocked GZip Format (BGZF) #19
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
removed print statements put put_trailer back in original spot
added more tests for edge cases and gzip compatibility
|
👍 I would find this feature quite useful. I also use BGZF. |
|
I just noticed that |
|
👍 for compatibility with bgzip! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch adds the ability for pigz to compress and uncompress the Blocked GNU Zip Format (BGZF) variant of gzip whose specification can be found here in the SAM specification. http://samtools.github.io/hts-specs/SAMv1.pdf
This format uses an extra field in the gzip header to encode each independent block's length, thereby allowing parallel decompression and even fast random access of the compressed archive (with indexing or other tricks)
I hope that people interested in parallel compression will be willing to sacrifice a small amount of compression-space efficiency for the ability to greatly accelerate the speed and parallelism of decompression.