Migration to systemd-boot with luks encrypted partition - screen turns black when external monitor attached

Hi everyone,

I just followed the tutorials and migrated 1st from mkinitcpio to dracut and right after that (reboot in between was successful) I 2nd migrated away from grub to system-boot including luks encr<pted disk

All works on two laptops i migrated one after the other.

Weird effect. The second laptop is usually attached to a DELL UW3821 monitor via USB-C. The monitor has a built in dock. When I boot up systemd-boot boots up fine up to the “enter password for luks partition stuff”. This line is being shown for a second and then the screen turns off and even a blindly typed password is not accepted. I can reboot the laptop with a bunch of ctrl-alt-del keystrikes but with the monitor attached “something” happens. A standalone boot works fine.

Any ideas where to start investigation?

Did you have any kernel options in your /etc/default/grub that did not get migrated to /etc/kernel/cmdline?

in /etc/default/grub I had

GRUB_CMDLINE_LINUX_DEFAULT="quiet cryptdevice=UUID=91386e25-1e7b-4df5-819d-96198a54eb82:luks-91386e25-1e7b-4df5-819d-96198a54eb82 root=/dev/mapper/luks-91386e25-1e7b-4df5-819d-96198a54eb82 loglevel=3 nowatchdog nvme_load=YES resume= resume_offset=119984128 resume=/dev/mapper/luks-91386e25-1e7b-4df5-819d-96198a54eb82 resume_offset=119984128 amd_pstate.shared_mem=1"

(I remember I added the resume part for hibernation. But I disabled hibernationit in /etc/systemd/sleep.conf)

A /etc/kernel/cmdline does not exist…

How did you convert to systemd-boot?

If of interest as I did that right before: my steps to convert to dracut in short…

sudo pacman -S dracut
sudo pacman -S eos-dracut
sudo pacman -Rc mkinitcpio
sudo micro /etc/dracut.conf.d/encryption.conf  # put the encryption line in there (install_items+=...)
curl https://raw.githubusercontent.com/endeavouros-team/EndeavourOS-ISO/main/airootfs/etc/dracut.conf.d/eos-defaults.conf | sudo tee /etc/dracut.conf.d/eos-defaults.conf
sudo dracut-rebuild

Then I converted to systemd-boot following your tutorial:

sudo pacman -Rc grub

sudo mkdir /efi
efidevice=$(findmnt /boot/efi -no SOURCE) # save the efi partition location
sudo umount /boot/efi
sudo mount ${efidevice} /efi

# cleanup a bit
sudo rm -r /boot/efi /boot/grub /boot/initramfs* /boot/vmlinuz*

sudo micro /etc/fstab # /boot/efi -> /efi -> save
sudo bootctl install

sudo micro /efi/loader/loader.conf # uncomment timeout
sudo pacman -S kernel-install-for-dracut # eos-dracut automatically offered to remove. Did that.
sudo ./migrate-systemd-boot-helper.sh # executed your script - saved it as a file
sudo reinstall-kernels
sudo reboot

# reboot to check if all works - not being asked for a password for luks as keyfile still there in /
sudo cryptsetup luksRemoveKey /dev/nvme0n1p2 /crypto_keyfile.bin
sudo reinstall-kernels

# reboot again for check - standalone laptop no problems - I am asked for luks password and can boot
# when docked laptop display turns black

What I discovered in between: when I configure Laptop to boot with internal display instead of external display in bios I can see that without my interaction “something” seems to send an enter twice so password entry fails and cryptsetup drops me to a press Ctrl-D or login as root shell - but none of the options work here.

The second laptop boots fine with or without external monitor. I can enter the passphrase and boot. Only difference here is that I never implemented hibernation… AND there a /etc/kernel/cmdline exists

EDIT - though I don’t have a /etc/kernel/cmdline on my laptop the resume parameters are there when looking here:

cat /proc/cmdline
initrd=\39be9dbeb3be4e40984ae6a70e2a07f7\6.1.15-1-lts\initrd root=UUID=a68cec36-3cda-4c58-b52a-b7b94bcf2ed5 rw cryptdevice=UUID=91386e25-1e7b-4df5-819d-96198a54eb82:luks-91386e25-1e7b-4df5-819d-96198a54eb82 root=/dev/mapper/luks-91386e25-1e7b-4df5-819d-96198a54eb82 loglevel=3 nowatchdog nvme_load=YES resume= resume_offset=119984128 resume=/dev/mapper/luks-91386e25-1e7b-4df5-819d-96198a54eb82 resume_offset=119984128 amd_pstate.shared_mem=1 systemd.machine_id=39be9dbeb3be4e40984ae6a70e2a07f7 systemd.machine_id=39be9dbeb3be4e40984ae6a70e2a07f7

EDIT 2:
I manually created the /etc/kernel/cmdline file from relevant parts of /prod/cmdline and reactivated resume/hibernation again.

Reboot results in the same error when USB-C docked. By magic something is pressing “Enter” twice and I end up here

I played around a bit…

Plugging USB devices / keyboards in and out with reboots. Tried a diffrent docking station. Plugged in an USB-C external disk while booting.

→ whenever there’s power supply via USB-C either via docking station or a dumb power charger then the automatic input happens and I end up in the above shown screenshot.

No clue what may cause this. Worked perfectly before migration to dracut and systemd-boot. And the other laptop I treated in the same manner (just not having a resum/hibernation) works perfectly.

I’ll try to install a completely diferent kernel (zen) to see if the recreation of initramfs etc works better then…

UPDATE: completely fresh installed other kernel did not help… Now clueless. :confused:

Does it make any difference if you boot the fallback image?

No, same behaviour.

What are the contents of /etc/dracut.conf.d/encryption.conf and /etc/kernel/cmdline. The latter should exist now since you installed a kernel.

 cat /etc/dracut.conf.d/encryption.conf
install_items+=" /etc/crypttab "
 cat /etc/kernel/cmdline
root=UUID=a68cec36-3cda-4c58-b52a-b7b94bcf2ed5 rw cryptdevice=UUID=91386e25-1e7b-4df5-819d-96198a54eb82:luks-91386e25-1e7b-4df5-819d-96198a54eb82 root=/dev/mapper/luks-91386e25-1e7b-4df5-819d-96198a54eb82 loglevel=3 nowatchdog nvme_load=YES resume= resume_offset=119984128 resume=/dev/mapper/luks-91386e25-1e7b-4df5-819d-96198a54eb82 resume_offset=119984128 amd_pstate.shared_mem=1

and it works fine if you don’t use your usb monitor?

Whenever there’s sth attached to USB-C that delivers power like my DELL Monitor with an integrated docking station or lenovo usb-c dock. Even a simple usb-c poweradapter connected let’s the boot-up fail. While a simple USB-C disc attached works. Just wonder if I need to add a “module” to the dracut build for boot to work. may be there’s sth missing in the kernel image file

Yes, it sounds more like a dracut issue than a systemd-boot one. That being said, it is pretty strange that usb power delivery would break the boot.

couldn’t believe it as well…

here’s the saved mkintcpio file (only active lines)

cat mkinitcpio.conf.old 
MODULES=(vmd)
BINARIES=()
FILES=(/crypto_keyfile.bin)
HOOKS=(base udev autodetect modconf block keyboard keymap consolefont encrypt filesystems fsck)
COMPRESSION="zstd"
$ cat encryption.conf 
install_items+=" /etc/crypttab "

$ cat eos-defaults.conf 
omit_dracutmodules+=" network cifs nfs brltty "
compress="zstd"

may be I need to add a module not inclueded by default that handles this…?

I don’t see anything interesting in mkinitcpio.conf.

You could try to figure out what is needed for usb-c power delivery and then add that. I am not exactly sure how you would locate that though.

Checking the logs?

@PeterRies you may want to try checking journal and Xorg logs when plugging/unplugging those devices:

  • Boot with single monitor
  • Run it terminal:
echo -e "\n   Starting test: Plug monitor\n" | sudo tee -a /var/log/Xorg.0.log
systemd-cat -t MyTests echo -e "\n   Starting test: Plug monitor\n"
  • Plug your monitor in
  • Run in terminal:
echo -e "\n   Starting test: UnPlug monitor\n" | sudo tee -a /var/log/Xorg.0.log
systemd-cat -t MyTests echo -e "\n   Starting test: UnPlug monitor\n"
  • Unplug your monitor

Repeat with other devices.
Adjust Xorg log file name, if the path/filename is different.

When finished, inspect the logs. The above commands write text in the logs, so you can separate starting/ending of related activity.

Tip for systemd-cat is from this source

Thanks Petsam, I’ll investigate on that, too.

Meanwhile I fount a refrence to my cryptop_keyfile.bin in /etc/crypttab. As I don’t need it anymore I replaced that line with “non” and rebuilt img files. Now I at least can boot with USB-C plugged in.

The difference with that change is that I’m now being askes a third time for thy cryptpassword. I additionally need to configure the laptop to display boot process on internal screen instead of USB-C.

So now I can see that after the 2 “ghost-inputs” the USB-C screen and attached Keyboard/Mous (built in dock) do not work anymore. After entering cryptpassword on laptop keyboard boot continues and all is working fine.

I suppose some module/driver in the image dracut is generating is missing or buggy and causes these ghost inputs. After decrypting root and booting up the correct drivers are loaded and screen/keyboard/monitor is working normally.

I saved journalctl from the second laptop as well and will now compare what’s different. The other laptop (lenovo x270) I treated similar works fine on the USB-C. Only significant difference between the lenovo t14 and the older x270 is that the t14 is an AMD…

Weird, but I’ll write more once I find the time.

Reading here:
https://wiki.archlinux.org/title/Dracut

Where’s the difference using
/etc/dracut.conf.d/cmdline.conf
in opposite to
/etc/kernel/cmdline?

Which one is used when yay updates a kernel or when I run dracut-rebuild or reinstall-kernels manually.

I guess without --force to dracut it will not affect existing kernel images in /efi/… anyway, correct?

@petsam - I’m still investigating / comparing journalctl in that early stage of bootup. No success yet…

will now check where

systemd-tty-ask-password-agent is being called during boot and if I can pass a parameter to play around with…

comparing the logs nothing seems wrong. on the machine with the error the /dev/tty seems to get an input via usb-c even if it’s a dumb power-plug…

Mär 08 12:15:38 t14 systemd-tty-ask-password-agent[531]: Starting password query on /dev/tty1.
Mär 08 12:15:38 t14 kernel: usb 2-2: new high-speed USB device number 2 using xhci_hcd
Mär 08 12:15:38 t14 kernel: usb 6-3: new full-speed USB device number 2 using xhci_hcd
Mär 08 12:15:38 t14 kernel: tsc: Refined TSC clocksource calibration: 1709.955 MHz
Mär 08 12:15:38 t14 kernel: clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x18a5e35963f, max_idle_ns: 440795253060 ns
Mär 08 12:15:38 t14 kernel: clocksource: Switched to clocksource tsc
Mär 08 12:15:38 t14 kernel: usb 2-2: New USB device found, idVendor=04f2, idProduct=b6d0, bcdDevice=59.18
Mär 08 12:15:38 t14 kernel: usb 2-2: New USB device strings: Mfr=3, Product=1, SerialNumber=2
Mär 08 12:15:38 t14 kernel: usb 2-2: Product: Integrated Camera
Mär 08 12:15:38 t14 kernel: usb 2-2: Manufacturer: Chicony Electronics Co.,Ltd.
Mär 08 12:15:38 t14 kernel: usb 2-2: SerialNumber: 0001
Mär 08 12:15:38 t14 kernel: usb 6-3: New USB device found, idVendor=06cb, idProduct=00bd, bcdDevice= 0.00
Mär 08 12:15:38 t14 kernel: usb 6-3: New USB device strings: Mfr=0, Product=0, SerialNumber=1
Mär 08 12:15:38 t14 kernel: usb 6-3: SerialNumber: 1802d1957c26
Mär 08 12:15:38 t14 kernel: usb 6-4: new full-speed USB device number 3 using xhci_hcd
Mär 08 12:15:38 t14 kernel: usb 6-4: New USB device found, idVendor=8087, idProduct=0029, bcdDevice= 0.01
Mär 08 12:15:38 t14 kernel: usb 6-4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Mär 08 12:15:39 t14 kernel: clocksource: timekeeping watchdog on CPU3: Marking clocksource 'tsc' as unstable because the skew is too large:
Mär 08 12:15:39 t14 kernel: clocksource:                       'hpet' wd_nsec: 510570407 wd_now: 1ba9d6c wd_last: 14b1105 mask: ffffffff
Mär 08 12:15:39 t14 kernel: clocksource:                       'tsc' cs_nsec: 506646417 cs_now: 4b39441c7 cs_last: 47ff0eb39 mask: ffffffffffffffff
Mär 08 12:15:39 t14 kernel: clocksource:                       'tsc' is current clocksource.
Mär 08 12:15:39 t14 kernel: tsc: Marking TSC unstable due to clocksource watchdog
Mär 08 12:15:39 t14 kernel: TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'.
Mär 08 12:15:39 t14 kernel: sched_clock: Marking unstable (2183226684, 343496)<-(2187891504, -4321624)
Mär 08 12:15:39 t14 kernel: clocksource: Checking clocksource tsc synchronization from CPU 13 to CPUs 0,2,5,9,11-12,14.
Mär 08 12:15:39 t14 kernel: clocksource: Switched to clocksource hpet
Mär 08 12:15:40 t14 systemd-tty-ask-password-agent[531]: Password query on /dev/tty1 finished successfully.
Mär 08 12:15:40 t14 systemd-cryptsetup[526]: Set cipher aes, mode xts-plain64, key size 512 bits for device /dev/disk/by-uuid/91386e25-1e7b-4df5-819d-96198a54eb82.
Mär 08 12:15:42 t14 systemd-cryptsetup[526]: Failed to activate with specified passphrase. (Passphrase incorrect?)
Mär 08 12:15:42 t14 systemd-tty-ask-password-agent[531]: Starting password query on /dev/tty1.
Mär 08 12:15:42 t14 systemd-tty-ask-password-agent[531]: Password query on /dev/tty1 finished successfully.
Mär 08 12:15:42 t14 systemd-cryptsetup[526]: Set cipher aes, mode xts-plain64, key size 512 bits for device /dev/disk/by-uuid/91386e25-1e7b-4df5-819d-96198a54eb82.
Mär 08 12:15:43 t14 systemd-cryptsetup[526]: Failed to activate with specified passphrase. (Passphrase incorrect?)
Mär 08 12:15:43 t14 systemd-tty-ask-password-agent[531]: Starting password query on /dev/tty1.
Mär 08 12:15:50 t14 systemd-tty-ask-password-agent[531]: Password query on /dev/tty1 finished successfully.
Mär 08 12:15:50 t14 systemd-cryptsetup[526]: Set cipher aes, mode xts-plain64, key size 512 bits for device /dev/disk/by-uuid/91386e25-1e7b-4df5-819d-96198a54eb82.
Mär 08 12:15:50 t14 kernel: Key type encrypted registered

Between 2nd and 3rd Starting password query on tty1 nothing happened in the log. But still there was an “Enter” on the screen that led to an empty passphrase input.

The former is for dracut, the latter is for systemd-boot. It depends where you want to manage that.

Those scripts expect you to use /etc/kernel/cmdline.

1 Like