Waiting for disk UUID during boot that doesn't exist

dwagner · March 17, 2024, 5:15pm

Hello!

I’m having an issue that seems very similar to other posts here, but after combing through the forums I haven’t found a fix.

I had a hard system crash during a pacman update, which I’ve recovered from, but it introduced an issue where during boot, A start job is running for dev/disk/by-uuid/<non-extant-uuid> would begin with an unlimited timeout.

I went through many of the solutions: checked /etc/fstab and the UUID wasn’t there, checked my /efi/loader/entries files, and found that each entry had a resume=<non-extant-uuid> module. So, I removed that and ran dracut-rebuild.

Now, the same thing happens, but at least there is only a 2min timeout instead of unlimited, and the system will finally boot.

I’m really having trouble finding where this ghost UUID is coming from, though, so I can eliminate this start job. Where else should I look to fix this? Thanks for any insight.

pebcak · March 17, 2024, 5:31pm

If you are using systemd-boot and dracut, other places to look into would be:

/etc/kernel/cmdline

and

/etc/dracut.conf.d/resume

See if you have any “residue” in there. If there is, remove it and run:

sudo reinstall-kernels

dwagner · March 17, 2024, 7:57pm

Thank you. I did find the resume= hook again in /etc/kernel/cmdline, removed it, and ran sudo reinstall-kernels. Issue persisted after this, however.

I checked out /etc/dracut.conf.d/ and found resume.conf in there. It has a single line: add_dracutmodules+=" resume ". Could this be causing the issue?

I double checked the files in /efi/loader/entries again, since there was a kernel update and they were regenerated, and the resume= hooks are still gone from when I removed them.

Just so we’re clear, I can’t find the UUID reference in the start job anywhere else on the system. Definitely not corresponding to anything currently present

/etc/dracut.conf.d🔒 took 6s 
❯ lsblk -f
NAME FSTYPE FSVER LABEL  UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
nvme0n1
                                                                             
├─nvme0n1p1
│    vfat   FAT32 SYSTEM 20F5-6FF2                                           
├─nvme0n1p2
│                                                                            
├─nvme0n1p3
│    ntfs         OS     8AF0F71BF0F70BE7                                    
├─nvme0n1p4
│    vfat   FAT32        85D7-AA10                              18.6M    96% /efi
└─nvme0n1p5
     ext4   1.0          6266af6a-7791-4c57-811c-96f88ed9ffa2  341.3G    37% /

pebcak · March 17, 2024, 8:04pm

Did you use a swap partition previously?

dwagner · March 17, 2024, 8:08pm

Yes, a few months ago I upgraded the SSD, so I cloned my partitions but left out the swap partition. The system was running without this issue up until this week, though, when I had a hard freeze during upgrade and had to recover from arch-chroot.

pebcak · March 17, 2024, 8:10pm

Could you post the output of: systemctl --all | grep swap ?

dwagner · March 17, 2024, 8:13pm

Nothing listed

~ via 🌙 v5.4.6 took 2m57s 
❯ systemctl --all | grep swap 
  swap.target                                                                                                                         loaded    active   active    Swaps

pebcak · March 17, 2024, 8:22pm

I am not sure really if this would cause the issue. However, you won’t be needing it if you are no longer using hibernation-resume.

Also, this may come across as a dirty fix, but if you are up for testing you could mask the swap.target and reboot the system and see if you still will have the issue.

dwagner · March 17, 2024, 8:25pm

Hmm, I may try that if I can’t figure this out.

I was reading the Arch WIKI on systemd-boot, and eventually got to looking at my EFI variables. I decided to checkout HibernateLocation:

~/.config/hypr took 10s 
❯ efivar -n 8cf2644b-4b0b-428f-9387-6d876050dc67-HibernateLocation -p
GUID: 8cf2644b-4b0b-428f-9387-6d876050dc67
Name: "HibernateLocation"
Attributes:
	Non-Volatile
	Boot Service Access
	Runtime Service Access
Value:
00000000  7b 00 22 00 75 00 75 00  69 00 64 00 22 00 3a 00  |{.".u.u.i.d.".:.|
00000010  22 00 66 00 65 00 30 00  39 00 66 00 38 00 38 00  |".f.e.0.9.f.8.8.|
00000020  64 00 2d 00 64 00 37 00  63 00 35 00 2d 00 34 00  |d.-.d.7.c.5.-.4.|
00000030  64 00 66 00 61 00 2d 00  61 00 33 00 64 00 31 00  |d.f.a.-.a.3.d.1.|
00000040  2d 00 65 00 65 00 34 00  63 00 32 00 36 00 35 00  |-.e.e.4.c.2.6.5.|
00000050  66 00 30 00 64 00 36 00  39 00 22 00 2c 00 22 00  |f.0.d.6.9.".,.".|
00000060  6f 00 66 00 66 00 73 00  65 00 74 00 22 00 3a 00  |o.f.f.s.e.t.".:.|
00000070  30 00 2c 00 22 00 6b 00  65 00 72 00 6e 00 65 00  |0.,.".k.e.r.n.e.|
00000080  6c 00 56 00 65 00 72 00  73 00 69 00 6f 00 6e 00  |l.V.e.r.s.i.o.n.|
00000090  22 00 3a 00 22 00 36 00  2e 00 36 00 2e 00 36 00  |".:.".6...6...6.|
000000a0  2d 00 61 00 72 00 63 00  68 00 31 00 2d 00 31 00  |-.a.r.c.h.1.-.1.|
000000b0  2d 00 67 00 31 00 34 00  22 00 2c 00 22 00 6f 00  |-.g.1.4.".,.".o.|
000000c0  73 00 52 00 65 00 6c 00  65 00 61 00 73 00 65 00  |s.R.e.l.e.a.s.e.|
000000d0  49 00 64 00 22 00 3a 00  22 00 65 00 6e 00 64 00  |I.d.".:.".e.n.d.|
000000e0  65 00 61 00 76 00 6f 00  75 00 72 00 6f 00 73 00  |e.a.v.o.u.r.o.s.|
000000f0  22 00 7d 00 00 00                                 |".}...          |

The UUID in the ASCII column definitely corresponds to the ghost UUID on the startup job. Should I try modifying directly HibernateLocation or is something else setting this? Gonna keep reading in the mean time.

pebcak · March 17, 2024, 8:31pm

I am out of my depth on this one.

Btw, you mentioned that you had recovered your system from a crash. Out of curiosity, how did you do that? What method?

dwagner · March 17, 2024, 8:33pm

Lol no worries. Thanks for the tips otherwise.

For future reference, I guess, I found this SE post about how to remove an undesired ghost EFI variable: https://superuser.com/questions/1346163/how-to-safely-set-change-remove-values-of-efi-variables

Going to try it out and report back.

To recover, I booted from EOS live usb, mounted / and /efi partitions, arch-chrooted and ran some pacman commands to reinstall all packages:

pacman -Qqen > pkglist.txt
pacman --overwrite "*"  -S $(< pkglist.txt)

dwagner · March 17, 2024, 8:46pm

Alright, for posterity, I was able to remove this undesired EFI variable that was somehow overriding the options parameters in my systemd-boot loader entries and totally fix the issue by running

sudo chattr -i /sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67

followed by

sudo rm /sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67

pebcak · March 17, 2024, 8:51pm

Nice troubleshooting and solution! Thanks for sharing it!

And also, welcome to EnOS’ community!

system · March 20, 2024, 8:50pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.