Crash during os update - issues repairing using arch-chroot

After a crash during a system update where I had to hard reboot, the system failed to boot and stuck at switch to root message.
So I followed instructions about repairing my system using live usb and arch-chroot https://discovery.endeavouros.com/system-rescue/arch-chroot/2022/12/

After mounting EFI and main partitions in /mnt, I logged with arch-chroot to repair broken system.

Yet, when I try to use pacman commands like pacman -Syu, I get errors like

pacman: error while loading shared libraries: /usr/lib/libarchive.so.13: file too short

I found some topics here talking about such issues like this one https://forum.endeavouros.com/t/kernel-panic-during-update-issues-with-pacman-sysroot/52530

But when I try to get pacman-static, I end up with similar error

[root@EndeavourOS /]# wget https://pkgbuild.com/~morganamilo/pacman-static/x86_64/bin/pacman-static
./pacman-static -Syu
wget: error while loading shared libraries: /usr/lib/libpcre2-8.so.0: file too short

Can you help me solve this issue ?

Download pacman-static with wget before chrooting into your system.

Then copy/move it to your mounted system partition, chroot and run ./pacman-static -Syu again

Thank you for your feedback @jake99.
After copying pacman-static executable to mounted partition, I get the following error

[root@EndeavourOS /]# ./pacman-static -Syu
:: Synchronizing package databases...
error: failed to synchronize all databases (unable to lock database)

I found some instructions here https://wiki.archlinux.org/title/Pacman#.22Failed_to_init_transaction_.28unable_to_lock_database.29.22_error so I deleted /var/lib/pacman/db.lck file.

After running pacman-static once again, I get the following errors

[root@EndeavourOS /]# ./pacman-static -Syu
:: Synchronizing package databases...
 endeavouros is up to date
 core                                                                                                             128.4 KiB   419 KiB/s 00:00 [---------------------------------------------------------------------------------------] 100%
 extra                                                                                                              8.1 MiB  13.3 MiB/s 00:01 [---------------------------------------------------------------------------------------] 100%
 community is up to date
 multilib is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Package (3)             Old Version  New Version  Net Change  Download Size

core/linux-lts          6.6.23-1     6.6.24-1      -0.03 MiB     128.78 MiB
core/linux-lts-headers  6.6.23-1     6.6.24-1       0.01 MiB      24.70 MiB
extra/nvidia-lts        1:550.67-2   1:550.67-3     0.00 MiB      40.46 MiB

Total Download Size:   193.94 MiB
Total Installed Size:  293.87 MiB
Net Upgrade Size:       -0.03 MiB

:: Proceed with installation? [Y/n] Y
:: Retrieving packages...
 linux-lts-headers-6.6.24-1-x86_64                                                                                 24.7 MiB  13.0 MiB/s 00:02 [---------------------------------------------------------------------------------------] 100%
 nvidia-lts-1:550.67-3-x86_64                                                                                      40.5 MiB  16.6 MiB/s 00:02 [---------------------------------------------------------------------------------------] 100%
 linux-lts-6.6.24-1-x86_64                                                                                        128.8 MiB  32.0 MiB/s 00:04 [---------------------------------------------------------------------------------------] 100%
 Total (3/3)                                                                                                      193.9 MiB  47.9 MiB/s 00:04 [---------------------------------------------------------------------------------------] 100%
(3/3) checking keys in keyring                                                                                                                [---------------------------------------------------------------------------------------] 100%
error: GPGME error: Invalid crypto engine
error: GPGME error: Invalid crypto engine
error: GPGME error: Invalid crypto engine
(3/3) checking package integrity                                                                                                              [---------------------------------------------------------------------------------------] 100%
error: GPGME error: Invalid crypto engine
error: GPGME error: Invalid crypto engine
error: GPGME error: Invalid crypto engine
error: linux-lts: missing required signature
:: File /var/cache/pacman/pkg/linux-lts-6.6.24-1-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] Y
error: linux-lts-headers: missing required signature
:: File /var/cache/pacman/pkg/linux-lts-headers-6.6.24-1-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] Y
error: nvidia-lts: missing required signature
:: File /var/cache/pacman/pkg/nvidia-lts-1:550.67-3-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] Y
error: failed to commit transaction (invalid or corrupted package (PGP signature))
Errors occurred, no packages were upgraded.

Do you have any pointers on how to go further ?

After some additional research, I tried to update signatures with ./pacman-static -S archlinux-keyring on the mounted system.
I got same kind of error

[root@EndeavourOS /]# ./pacman-static -S archlinux-keyring
warning: archlinux-keyring-20240313-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Package (1)             Old Version  New Version  Net Change

core/archlinux-keyring  20240313-1   20240313-1     0.00 MiB

Total Installed Size:  1.66 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] Y
(1/1) checking keys in keyring                                                                                                                [---------------------------------------------------------------------------------------] 100%
error: GPGME error: Invalid crypto engine
(1/1) checking package integrity                                                                                                              [---------------------------------------------------------------------------------------] 100%
error: GPGME error: Invalid crypto engine
error: archlinux-keyring: missing required signature
:: File /var/cache/pacman/pkg/archlinux-keyring-20240313-1-any.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] Y
error: failed to commit transaction (invalid or corrupted package (PGP signature))
Errors occurred, no packages were upgraded.

Also, I wonder what is the difference between using pacman-static on the mounted system, and using commands like pacman --sysroot /mnt -Syu as in https://wiki.archlinux.org/title/Pacman#Pacman_crashes_during_an_upgrade ?

Some more info on my issue following similar issue described here https://forum.endeavouros.com/t/kernel-panic-during-update-issues-with-pacman-sysroot/52530/11 :

[root@EndeavourOS /]# pacman-key --init
pacman-conf: error while loading shared libraries: /usr/lib/libarchive.so.13: file too short
mkdir: cannot create directory ‘’: No such file or directory
gpg: error while loading shared libraries: /usr/lib/libassuan.so.0: file too short
chmod: cannot access '/trustdb.gpg': No such file or directory
gpg: error while loading shared libraries: /usr/lib/libassuan.so.0: file too short
gpg: error while loading shared libraries: /usr/lib/libassuan.so.0: file too short
==> Generating pacman master key. This may take some time.
gpg: error while loading shared libraries: /usr/lib/libassuan.so.0: file too short
==> Updating trust database...
gpg: error while loading shared libraries: /usr/lib/libassuan.so.0: file too short
==> ERROR: Trust database could not be updated.

And when I try the following commands from the arch-chroot

wget https://arch.phinau.de/core/os/x86_64/gnupg-2.4.5-1-x86_64.pkg.tar.zst
wget https://arch.phinau.de/core/os/x86_64/gpgme-1.23.2-1-x86_64.pkg.tar.zst
./pacman-static -U ./gnupg-2.4.5-1-x86_64.pkg.tar.zst ./gpgme-1.23.2-1-x86_64.pkg.tar.zst
./pacman-static -Syu

The first pacman-static command outputs bunch of errors like below:

error: GPGME error: Invalid crypto engine
warning: Public keyring not found; have you run 'pacman-key --init'?
error: GPGME error: Invalid crypto engine
error: './var/cache/pacman/pkg/btrfs-progs-6.3.2-1-x86_64.pkg.tar.zst': invalid or corrupted package (PGP signature)
warning: gnupg-2.4.5-1 is up to date -- reinstalling
warning: gpgme-1.23.2-1 is up to date -- reinstalling
error: could not open file ./usr/lib/firmware/iwlwifi-so-a0-gf-a0-81.ucode.zst: Unrecognized archive format
error: './usr/lib/firmware/iwlwifi-so-a0-gf-a0-81.ucode.zst': cannot open package file
error: could not open file ./usr/lib/firmware/sxg/saharadbgdownloadB.sys.zst: Unrecognized archive format
error: './usr/lib/firmware/sxg/saharadbgdownloadB.sys.zst': cannot open package file
error: could not open file ./usr/lib/firmware/sxg/saharadownloadB.sys.zst: Unrecognized archive format
error: './usr/lib/firmware/sxg/saharadownloadB.sys.zst': cannot open package file
error: could not open file ./usr/lib/firmware/amd/amd_sev_fam19h_model1xh.sbin.zst: Unrecognized archive format
error: './usr/lib/firmware/amd/amd_sev_fam19h_model1xh.sbin.zst': cannot open package file
...

After a lot of unsuccessful tries with pacman-static, I found that I may be using liveISO pacman instead, so here is what I did :

sudo pacman --root /mnt --cachedir /mnt/var/cache/pacman/pkg -Syu

Packages seem to be downloaded but when the script starts to check conflicts, I get lots of errors like below :

(19/19) checking for file conflicts                                                                                                           [---------------------------------------------------------------------------------------] 100%
error: failed to commit transaction (conflicting files)
libxmlb: /mnt/usr/bin/xb-tool exists in filesystem
libxmlb: /mnt/usr/include/libxmlb-2/libxmlb/xb-builder-fixup.h exists in filesystem

And sudo pacman --root /mnt -Qkk outputs lots of

...
warning: libpipewire: /mnt/usr/lib/pkgconfig/libspa-0.2.pc (MD5 checksum mismatch)
warning: libpipewire: /mnt/usr/share/licenses/libpipewire/COPYING (MD5 checksum mismatch)
libpipewire: 237 total files, 207 altered files
...

I can provide the full output which is very long if needed.

At this point, Im quite lost on what I should do, so any help from the community would be of great help !

I finally succeeded to solve my issue and get back to a functional system.
Here is the commands I finally use from the live ISO to get out of that situation, based on the Arch Wiki https://wiki.archlinux.org/title/Pacman#Pacman_crashes_during_an_upgrade :

  1. First mounted system and efi, then /proc, /sys, /dev
  2. sudo pacman --sysroot /mnt -Syu did not succeed so I had to follow https://wiki.archlinux.org/title/Pacman/Tips_and_tricks#Reinstalling_all_packages with the following command
sudo pacman --root /mnt -Qqn | sudo pacman --root /mnt pacman --overwite=* -S -

I also reinstalled kernel
I still had some broken packages, plus additional kernels, but I could reboot to the main system, update GPG keys with pacman-key --init and pacman-key --populate archlinux endeavouros, and clean the old and unused kernels with kernel-install remove

I may have to check if some damaged packages still remain though…

Et voilà ! A bit painful though for a first time. This can be closed !

Also, for the record, asking on ArchLinux forum as endeavouros user is unfortunately seen as help vampirism, and not appreciated.

It’s not seen as Help Vampirism, it’s flagrantly breaking the rules of the forum:

2 Likes

Yes, indeed, but I also read something about help vampirism here though, I thought this was related.
But it was my entire fault, I didn’t pay attention to the rules when first posting, as a lot of relevant and useful info can be found on the arch wiki and the arch forum ! Naively thinking vanilla arch and derivatives were one community with specificities of course.

I was thinking it may be useful to remind newbie people like me on endeavouros that this is forbidden.

1 Like

Indeed it is, and I just wanted to point out that it’s not because of help vampirism, which is a separate thing that’s not tolerated there, but because forum rules specifically state that the forum is for Arch only - not “Arch-based” distros.

The main reason for this is that Arch-based distros are pre-configured, so the user usually does not know how to answer questions about what configurations/daemons/services are in use, whereas with an actual Arch installation the user would know because they set it up themselves. Arch is a DIY distro.

I would also like to point out for the people who say the Arch forum is rude and mean and nasty that you got a polite, useful answer to your question there in the same post that pointed out the rules to you :+1:.

2 Likes

Interesting, I get it, thank you for this clarification ! Despite this pre-configuration, is it correct that one could find the relevant info by digging into his pre-configured system, thus being able to tell what is actually in use ? Also, do features like pacman present any pre-configuration that would make endeavouros specific on this layer ?

Don’t know if you said that because of my message, but despite my foolishness, I did not feel like the Arch forum was rude or mean, I would rather truly say it is a masterpiece of useful info and skilled people :slight_smile: I will definitely be more watchful in the future (I admit I was in a hurry) !

(and please excuse me for being quite off-topic, but I find the topic interesting ! I’ve been a Linux user for a long time, but I consider myself quite ignorant on philosophical questions about distributions and related communities)

Yes, but that has no bearing on the fact that the Arch forum is for support of Arch only, not Arch-based distros. The rule is clear, and when creating an account an individual agrees to abide by forum rules.

These boards are for the support of Arch Linux, and Arch ONLY

If you have installed Archbang, Artix, Chakra, EndeavourOS, Evo/Lution, Manjaro, Whatever, you are NOT running Arch Linux. Similarly, if you followed some random video on YouTube or used an automated script you found on a blog, you are NOT running Arch Linux, so do not expect any support, sympathy or anything but your thread being closed and told to move along.

Don’t break a forum’s rules. You’re not using Arch, you’re using an Arch-based distro, so the place to ask for assistance is that distro’s forums. Uisng the Arch forum to get assistance for Arco, Endeavour, Manjaro, Garuda, etc is wasting the time and effort of the volunteers who are there to help Arch users.

Feel free to browse the Arch forum for pertinent information, and the Arch wiki is available for everyone (and, like the Gentoo wiki, the Arch wiki has information that is useful no matter what distro is being used). But don’t post on the Arch forums if you haven’t installed Arch itself.

No, I wasn’t directing that at you, but to:

2 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.