Faster startup using lz4?

[jb55]

I think this would require a custom archiver that can bootstrap lz4 decompression

[probonopd]

@jb55 have you run benchmarks between squashfs and lz4? We'd be interested in those for AppImageKit, too.

[jb55]

@probonopd I'm going off generic compression benchmarks where lz4 wins in speed by a large margin. I haven't tested lz4 in squashfs, although it looks like it's supported?

[probonopd]

You are right...

[domenkozar]

We did some measurements using gzip (instead of bzip) and the difference is not too big (a few percent quicker). Would be interesting to give lz4 also a try (but arx doesn't implement it).

[matthewbauer]

Yeah I think biggest savings will come with "mounting" instead of "extracting". FUSE should give us that and I believe it's supported on most systems.

[cleverca22]

https://github.com/input-output-hk/daedalus/blob/10c9d349bcb964c0c5693a9bd484759d372b21f6/nix-bundle.nix#L63

over here, the bundle pre-unpacks the entire thing to a known directory, and generates things like a .desktop file
then when the user launches the application, its already unpacked, and the helper script just runs nix-user-chroot against the pre-unpacked dir

[jb55]

@cleverca22 clever

[cleverca22]

but ive also ran into an issue, i'm treating the bundle purely as an installer/updater, but i cant run it from within the namespace for self-updating

[probonopd]

the bundle pre-unpacks the entire thing to a known directory

That will be much slower than loop-mounting using FUSE (as AppImage does), and will need additional space. Think about bundles that are GBs large.

[cleverca22]

yeah, depends on your use-case and how big things are

i also recently discovered pigz and pixz, which just perform gzip or xz across multiple cores, but the user would have to have them installed before arx could make use of it

[DavHau]

According to multiple benchmarks online (like for example: https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/), bzip2 seems to be significantly slower to unpack than gzip and lzma.

gzip would be the fastest but doesn't have good compression ratio.
In #69 I swapped out bzip2 with xz -1 which should improve upon compression/decompression speed and also file size.
Also it now uses all available cores for compression.