Which compression algorithm to choose in mkinitcpio.conf?

As long as you mount the subvolume @, yes.

  1. Boot into the live environment
  2. In case your device is encrypted, decrypt and mount your luks partition (e. g. /dev/sda2); otherwise leave out the decryption part and use /dev/sda2 instead of /dev/mapper/crypt:
sudo cryptsetup luksOpen /dev/sda2 crypt &&
sudo mount -o compress=zstd,subvol=@ /dev/mapper/crypt /mnt
 
  1. Only on UEFI systems: Mount your EFI boot folder (e. g. /dev/sda1)
sudo mount /dev/sda1 /mnt/boot/efi
  1. chroot into your system
sudo arch-chroot /mnt
  1. Do some root stuff …
2 Likes

Thank you so much for your answer! I appreciate it!

Have a great day!
:sun_with_face:

1 Like

It doesnt matter. And I honestly do not know why there are so many choices. This is just over complicating things. I stick with “xz”.

2 Likes

Because it’s Linux, which is all about choice.

But you’re not wrong, bzip2 and lzma are superseded by faster algos with better compression ratio. Gzip, xz, lz4 and zstd are valid and useful alternatives.

That said, XZ in my opinion only makes sense if you want to squeeze out a few KB/MB of free space. It’s rather slow both in compression and in decompression.

3 Likes

In terms of speed and/or CPU usage how would you rate the algorithm that you have mentioned above if space is not a concern.

lz4 or zstd. Depends on the exact options and of course your hardware.
I use zstd -15 as a good compromise.
Somewhere I have a little script that logs total and cpu time, and file size for zstd and xz, might post it some day (I don’t have it on this computer).

(and there’s always “uncompressed” but I’m not sure if it’s really faster than e.g. lz4)

1 Like

Thanks for the reply and explanation!
This machine got a AMD Ryzen 7 4800U. So if I were to use zstd -15, where do I need to make the change?

I know that I need to read up on these stuff and perhaps at the end of the day the difference is in order of some hundreds of a second but it is just nice to know that you can speed up things a bit when possible.

That would be nice!

To use a specific compression algorithm simply uncomment it in mkinitcpio.conf (and comment all the others).

For options, see COMPRESSION_OPTIONS array.
Important: Do not use the above as I think it applies to all compression algos.
From manpage:

This option is generally not used. It can be potentially dangerous and may cause invalid images to be generated without any sign of an error.

If you’re adventurous, you can modify /usr/bin/mkinitcpio like I already described here. I don’t support or endorse this :slight_smile:

1 Like

Great!

This sounds very tempting now! I’ll have a look!

No problem! See my username? :sweat_smile:

1 Like

I wanted to add some data to this discussion. We are talking about the boot process. So I benchmarked the boot time (systemd-analayze time) and the size of the initramfs files for the 5 compression algorithms gzip, xz, lz4, bzip2 and zstd. I used my endeavourOS virtualbox for that. 5 boot process per compression algorithm and I always discarded the slowest result in a series.

Here it is:

        absolut values      ratios
        time    size        time    size 
        [s]     [MB]        [%]     [%]
gzip    7,72    36,4        104     142
xz      7,78    25,6        104     100
lz4     7,44    53,0        100     207
bzip2   8,48    34,2        114     134
zstd    7,66    29,2        103     114

Conclusion:

  • bzip2 should not be used. I am taking it out of consideration.
  • best boot time is with lz4
  • best compression is with xz
  • taking out bzip2 the boot time variation is 3-4 %. That is not much.
  • taking out bzip2 the compression variation is up to 207 %. That is a lot.

For me personally xz is the winner. It boots in 7,78 s. This is just 0,33 s slower than the fastest algorithm lz4. But xz creates the smallest files which are half the size of what lz4 is doing.

We also need to keep in mind that these are really fast boot times because my virtualbox is lean with no extra services. In real life boot times would be longer because of all the service start activities. These extra seconds are more than compensating the performance gain of lz4.

5 Likes

Thanks for the figures and especially for pointing it out it is a VBox.

What hardware did you run this on?
Because obviously if you have a fast Ryzen the differences in time are small.
Try it on a Core 2 Duo and it will look differently - especially with the memory intensive XZ.

One additional thing though: you were measuring the default parameters which are as follows (I leave out bzip2):

gzip -6
xz -6 [= dictionary of 8 MiB, preset 6]
lz4 -1 [fastest, lowest compression ratio]
zstd -3 [also very fast]

EDIT: are you sure that systemd-analyze is the right tool for this? When initram is uncompressed, systemd isn’t loaded yet, AFAIK it only measures time spent in the already uncompressed initram.

2 Likes

Yes, my PC is fast: Ryzen 7 3700X.

But anyways, I tend to believe that the ratios are largely hardware independent. If it is 4 % difference on a fast PC it is most likely 4 % difference on a slow PC as well.

And if you have a slow PC and you start more services in real life, these 4 % difference diminish even further.

And yes, I believe that systemd-analyze time is the right command for that:

systemd-analyze time
       This command prints the time spent in the kernel before userspace has been reached, the time spent in the initial RAM disk
       (initrd) before normal system userspace has been reached, and the time normal system userspace took to initialize. Note that
       these measurements simply measure the time passed up to the point where all system services have been spawned, but not
       necessarily until they fully finished initialization or the disk is idle.

But I am open to any suggestion to improve the benchmark. Just let me know how I should measure instead and I will run another test.

1 Like

I will investigate and report back if I find something.

I don’t believe systemd-analyze is the right command, from your citation:

the time spent in the initial RAM disk (initrd) before 
normal system userspace has been reached

So it is already in the initram. But my interpretation can be wrong of course.

While on the decompression side, you might be right, don’t forget compression! XZ is soooo slooooow on old hardware.

And even on this Ryzen 9 I get this:
XZ: mkinitcpio -P with three kernels and four initrams: 75 seconds
ZSTD: 17 seconds.
That’s almost an entire minute gained!

3 Likes

That is indeed an issue. xz is really slow when compressing. But because compressing is only a one time task during installation of the kernel I did not pay much attention to it. May be I should.

I haven’t found any other method :frowning:

However, your numbers in systemd-analyze appear to be correct.
A manual unpacking of the initram/lz4 takes 0.04 seconds here, whereas xz takes 0.66 seconds.

In a scenario where compression ratio is not relevant, would you say that lz4 is faster than zstd when it comes to both compressing and decompressing? Will the boot time benefit from a faster decompressing?

See @mbod’s results above. The difference seems to be minimal on a fast computer.
It’s the compression part that makes a big difference.

2 Likes

Yes it is but the difference is small.
The benefit of zstd is, that it is almost as fast as lz4 but with a much, much better compression rate.

If you dont care about file size (compression rate) you are better off with lz4.

1 Like

Thanks @mbod for the reply!

As these stuff is kind of new to me, at this stage I am experimenting with different options.
Since my previous post, I did actually changed to lz4. I ran a mkinitcpio -P. I didn’t take the time, but it was considerably faster than zstd. Watching the htop as well showed considerable less cpu usage.
Since the size of the compressed images is not of a concern, for now I think I stick with lz4.

Just switched to zstd. I think my system spends more time running mkinitcpio than it does decompressing the initramfs. Hence, on balance, this is a better trade-off for me.

4 Likes