PSA: New linux-firmware package requires kernel >= 5.3

The linux-firmware package 20220119.0c6a7b3-2 implements kernel firmware compression. Linux kernel from 5.3 on support loading from xz compressed firmware.
CONFIG_FW_LOADER_COMPRESS kernel option must be enabled. All official Arch Linux kernel support this for a long time. [1]

The linux-firmware package has been split into smaller packages to further reduce required disk space. Some big firmware files of rarely used hardware have been split into separate packages.
This affects firmware for Mellanox Spectrum switches, Marvell devices, Qualcomm SoCs, Cavium LiquidIO server adapters, QLogic devices, Broadcom NetXtreme II 10Gb ethernet adapters.

Make sure to install additional firmware packages if needed. [2]

[1] FS#72899
[2] FS#72559 + svn commit

If you are still using an older kernel, like 4.19, you might have to figure out what this means for you.

10 Likes

This an Arch specific linux-firmware packaging change purely to reduce package size?

So the uncompressed upstream linux-firmware could potentially become an AUR package, or a third party repo package, in order to support lts kernels <= 4.19?

that current package is split into smaller chunks like linux-firmware and linux-firmware-extra

This affects firmware for Mellanox Spectrum switches, Marvell devices, Qualcomm SoCs, Cavium LiquidIO server adapters, QLogic devices, Broadcom NetXtreme II 10Gb ethernet adapters.

Is there a more complete list of which hardware has been separated into extra packages?

@jonathon might look into this for your older kernels repo :upside_down_face:

2 Likes

Looking into it. 5.4 and 5.10 are fine, earlier kernels will have issues and there’s no clean patch backport available. 4.19 might be easiest as there’s less of a gap between it and 5.4, but the underlying firmware_loader this feature needs doesn’t exist in 4.4, 4.9, or 4.14.

This looks like the easiest option - just need to check it’s OK as it essentially duplicates something in the repos.

5 Likes

I just got a linux-firmware update.

[ricklinux@eos-kde ~]$ pacman -Qi linux-firmware
Name            : linux-firmware
Version         : 20220119.0c6a7b3-2
Description     : Firmware files for Linux
Architecture    : any
URL             : https://git.kernel.org/?p=linux/kernel/git/firmware/linux-firmware.git;a=summary
Licenses        : GPL2  GPL3  custom
Groups          : None
Provides        : None
Depends On      : linux-firmware-whence
Optional Deps   : None
Required By     : None
Optional For    : linux
Conflicts With  : None
Replaces        : None
Installed Size  : 149.48 MiB
Packager        : Andreas Radke <andyrtr@archlinux.org>
Build Date      : Wed 19 Jan 2022 05:00:49 PM
Install Date    : Fri 21 Jan 2022 02:31:41 PM
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : Signature

[ricklinux@eos-kde ~]$ 

linux-firmware-uncompressed

1 Like

I’m waiting for an official opinion before I upload something. It can go in kernel-lts anyway, but an AUR package would help too.

2 Likes

Without an uncompressed firmware AUR package the existing AUR lts kernel packages would become defunct anyway.

1 Like

I’ve got these two now when updating:

[2022-01-20T03:52:18+0100] [ALPM] installed linux-firmware-whence (20220119.0c6a7b3-2)
[2022-01-20T03:52:19+0100] [ALPM] upgraded linux-firmware (20220111.13dca28-1 → 20220119.0c6a7b3-2)

1 Like

https://aur.archlinux.org/pkgbase/linux-firmware-uncompressed/

It’s also in kernel-lts for people who need it.

5 Likes

yeah, not a big fan of that although I use linux-zen. It notably slows down boot time because it now has to decompress the firmware package with a rather slow decompression standard. I now have a delay between “loading linux-zen” and the “starting systemd” message.

From what I read, the only way of disabling the compression at the moment is by switching to an uncompressed firmware-package?

Edit:

Did you leave out the “whence” package by mistake or by purpose?

1 Like

Yup - the files are compressed before being packaged.

The repo package for that is fine - it’s a single text file, no need to repackage it.

4 Likes

So in order to eliminate this delay, one can just install the package linux-firmware-uncompressed provided by @jonathon replacing the one from repos ? And the whence one will be alright as is?

well … Seems like it was only imaginary. Took the time trying to prove my observation and tried several combinations, measuring time using systemd-analyze on my system (Ryzen 2700X on a X470 chipset, 32GB RAM@3466Mhz, booting from a Samsung SSD 960 EVO 500GB)
Booted two times per test:

uncompressed firmware package and cat mkinitcpio:
Startup finished in 10.699s (firmware) + 2.436s (loader) + 4.862s (kernel) + 1.835s (userspace) = 19.834s 
Startup finished in 10.695s (firmware) + 2.436s (loader) + 4.865s (kernel) + 1.609s (userspace) = 19.606s 

compressed firmware package and cat mkinitcpio:
Startup finished in 10.690s (firmware) + 2.436s (loader) + 4.857s (kernel) + 1.850s (userspace) = 19.835s 
Startup finished in 10.688s (firmware) + 2.433s (loader) + 4.858s (kernel) + 1.732s (userspace) = 19.713s 

compressed firmware package and zstd mkinitcpio:
Startup finished in 10.680s (firmware) + 2.408s (loader) + 4.865s (kernel) + 1.705s (userspace) = 19.659s 
Startup finished in 10.669s (firmware) + 2.410s (loader) + 4.873s (kernel) + 1.730s (userspace) = 19.684s 

uncompressed firmware package and zstd mkinitcpio:
Startup finished in 10.707s (firmware) + 2.412s (loader) + 4.867s (kernel) + 1.819s (userspace) = 19.805s 
Startup finished in 10.725s (firmware) + 2.412s (loader) + 4.867s (kernel) + 1.751s (userspace) = 19.756s

Interestingly enough, the only thing having a real influence on boot time are my changing wallpapers in KDE (seen in userspace times - i just had a boot outside of the test where userspace took over 2 seconds to load :sweat_smile: )
And zst compressed mkinitcpio actually is a bit faster then uncompressed / cat mkinitcpio.
Edit: And “loader” only takes over 2 seconds because grub has a 2 second timeout on my machine that I let run out for comparability.

4 Likes

:wink: :sweat_smile:

1 Like

That is great … I’m still :laughing:

2 Likes

I just upgraded and was surprised with the size difference of linux-firmware, so I searched and found this discussion.
I had a reported difference of about 120MB (the lowest I can be sure, it may have been higher… memory issues…).
The packages info show the difference is not that much. Has anyone noticed the reported size diff during update?

This is from my cache

ls -l /var/cache/pacman/pkg/linux-firmware-*.zst | tail
-rw-r--r-- 1 root root 180251651 Ιουλ 18  2021 /var/cache/pacman/pkg/linux-firmware-20210716.b7c134f-1-any.pkg.tar.zst
-rw-r--r-- 1 root root 181729606 Αυγ  18 14:25 /var/cache/pacman/pkg/linux-firmware-20210818.c46b8c3-1-any.pkg.tar.zst
-rw-r--r-- 1 root root 187344858 Σεπ  20 21:51 /var/cache/pacman/pkg/linux-firmware-20210919.d526e04-1-any.pkg.tar.zst
-rw-r--r-- 1 root root 192313501 Νοε   2 20:44 /var/cache/pacman/pkg/linux-firmware-20211027.1d00989-1-any.pkg.tar.zst
-rw-r--r-- 1 root root 208595172 Ιαν  10 21:38 /var/cache/pacman/pkg/linux-firmware-20211216.f682ecb-1-any.pkg.tar.zst
-rw-r--r-- 1 root root 140657181 Ιαν  20 00:12 /var/cache/pacman/pkg/linux-firmware-20220119.0c6a7b3-2-any.pkg.tar.zst
-rw-r--r-- 1 root root     29116 Ιαν  20 00:13 /var/cache/pacman/pkg/linux-firmware-whence-20220119.0c6a7b3-2-any.pkg.tar.zst

Installed size should be a little higher than package size. The difference is at least 30%.

pacman -Si linux-firmware | grep -i size
Download Size   : 134.14 MiB
Installed Size  : 149.48 MiB

It’s a considerable on-disk size difference (though anyone using filesystem compression won’t see such a large effective difference):

$ sudo pacman -S linux-firmware-uncompressed
[sudo] password for jonathon: 
resolving dependencies...
looking for conflicting packages...
:: linux-firmware-uncompressed and linux-firmware are in conflict. Remove linux-firmware? [y/N] y

Packages (2) linux-firmware-20220119.0c6a7b3-2 [removal]  linux-firmware-uncompressed-20220119.0c6a7b3-2

Total Download Size:   121.79 MiB
Total Installed Size:  431.07 MiB
Net Upgrade Size:      281.59 MiB

:: Proceed with installation? [Y/n]
2 Likes

Interesting.

This would have more effect, on times where normal spinning HDDs was popular.

On my System, its not noticeable.

In theory, if you compress your kernel and everything needed by kernel, it should boot faster, because the compressed files get first loaded into ram and then decompressed. The theory says because ram is faster, it should decompress and load faster.

But today? i guess 99% with a “modern” machine use at least a ssd and shouldnt be noticeable.