Which compression algorithm to choose in mkinitcpio.conf?

General system
22

I wanted to add some data to this discussion. We are talking about the boot process. So I benchmarked the boot time (systemd-analayze time) and the size of the initramfs files for the 5 compression algorithms gzip, xz, lz4, bzip2 and zstd. I used my endeavourOS virtualbox for that. 5 boot process per compression algorithm and I always discarded the slowest result in a series.

Here it is:

        absolut values      ratios
        time    size        time    size 
        [s]     [MB]        [%]     [%]
gzip    7,72    36,4        104     142
xz      7,78    25,6        104     100
lz4     7,44    53,0        100     207
bzip2   8,48    34,2        114     134
zstd    7,66    29,2        103     114

Conclusion:

  • bzip2 should not be used. I am taking it out of consideration.
  • best boot time is with lz4
  • best compression is with xz
  • taking out bzip2 the boot time variation is 3-4 %. That is not much.
  • taking out bzip2 the compression variation is up to 207 %. That is a lot.

For me personally xz is the winner. It boots in 7,78 s. This is just 0,33 s slower than the fastest algorithm lz4. But xz creates the smallest files which are half the size of what lz4 is doing.

We also need to keep in mind that these are really fast boot times because my virtualbox is lean with no extra services. In real life boot times would be longer because of all the service start activities. These extra seconds are more than compensating the performance gain of lz4.

5 Likes
23

Thanks for the figures and especially for pointing it out it is a VBox.

What hardware did you run this on?
Because obviously if you have a fast Ryzen the differences in time are small.
Try it on a Core 2 Duo and it will look differently - especially with the memory intensive XZ.

One additional thing though: you were measuring the default parameters which are as follows (I leave out bzip2):

gzip -6
xz -6 [= dictionary of 8 MiB, preset 6]
lz4 -1 [fastest, lowest compression ratio]
zstd -3 [also very fast]

EDIT: are you sure that systemd-analyze is the right tool for this? When initram is uncompressed, systemd isnā€™t loaded yet, AFAIK it only measures time spent in the already uncompressed initram.

2 Likes
24

Yes, my PC is fast: Ryzen 7 3700X.

But anyways, I tend to believe that the ratios are largely hardware independent. If it is 4 % difference on a fast PC it is most likely 4 % difference on a slow PC as well.

And if you have a slow PC and you start more services in real life, these 4 % difference diminish even further.

And yes, I believe that systemd-analyze time is the right command for that:

systemd-analyze time
       This command prints the time spent in the kernel before userspace has been reached, the time spent in the initial RAM disk
       (initrd) before normal system userspace has been reached, and the time normal system userspace took to initialize. Note that
       these measurements simply measure the time passed up to the point where all system services have been spawned, but not
       necessarily until they fully finished initialization or the disk is idle.

But I am open to any suggestion to improve the benchmark. Just let me know how I should measure instead and I will run another test.

1 Like
25

I will investigate and report back if I find something.

I donā€™t believe systemd-analyze is the right command, from your citation:

the time spent in the initial RAM disk (initrd) before 
normal system userspace has been reached

So it is already in the initram. But my interpretation can be wrong of course.

While on the decompression side, you might be right, donā€™t forget compression! XZ is soooo slooooow on old hardware.

And even on this Ryzen 9 I get this:
XZ: mkinitcpio -P with three kernels and four initrams: 75 seconds
ZSTD: 17 seconds.
Thatā€™s almost an entire minute gained!

3 Likes
26

That is indeed an issue. xz is really slow when compressing. But because compressing is only a one time task during installation of the kernel I did not pay much attention to it. May be I should.

27

I havenā€™t found any other method :frowning:

However, your numbers in systemd-analyze appear to be correct.
A manual unpacking of the initram/lz4 takes 0.04 seconds here, whereas xz takes 0.66 seconds.

28

In a scenario where compression ratio is not relevant, would you say that lz4 is faster than zstd when it comes to both compressing and decompressing? Will the boot time benefit from a faster decompressing?

29

See @mbodā€™s results above. The difference seems to be minimal on a fast computer.
Itā€™s the compression part that makes a big difference.

2 Likes
30

Yes it is but the difference is small.
The benefit of zstd is, that it is almost as fast as lz4 but with a much, much better compression rate.

If you dont care about file size (compression rate) you are better off with lz4.

1 Like
31

Thanks @mbod for the reply!

As these stuff is kind of new to me, at this stage I am experimenting with different options.
Since my previous post, I did actually changed to lz4. I ran a mkinitcpio -P. I didnā€™t take the time, but it was considerably faster than zstd. Watching the htop as well showed considerable less cpu usage.
Since the size of the compressed images is not of a concern, for now I think I stick with lz4.

32

Just switched to zstd. I think my system spends more time running mkinitcpio than it does decompressing the initramfs. Hence, on balance, this is a better trade-off for me.

4 Likes