Not booting after update, hanging on "create volatile files and directories"

General system Kernel & Hardware
1

Yesterday I ran a yay -Suy and everything updated nicely, it also sent a desktop notification that “a boot was recommended” but since I was in the middle of work I decided to reboot after finishing my shift. Today I found myself with a system that wont boot and it freezes right around the message “create volatile files and directories”, it is inconsistent when exactly it freezes but it’s always around there, sometimes it’s when it says starting sometimes it’s when it finishes and I had it one time while just mounting /boot.

  • I’ve booted into a live usb and chrooted into my system and ran another update, hoping that there was some broken update and a fix was already pushed for it, didn’t do anything.
  • chrooted again to add ibt=off and ran grub-mkconfig thinking it might be caused by this issue: Nvidia-inst make not boot my laptop - #12 by DawfukFR but had no luck, also trying to run nvidia-inst from chroot throws an error “2023-07-26 14:16:53: Error: This program must be started as a non-root user.”, using su to run it as my user throws libkmod and systemd errors and wont run.

PS: I believe I also ran nvidia-inst early in that day because I was having issues with my resolutions and refresh rates, but then I restarted and it worked fine if that’s the case. I ran nvidia-inst very recently that’s for sure.

here is the output of inxi -F in case it’s needed, I don’t know what’s the deal with all of those "12"s, I suspect it’s a badly formatted color code, I can’t see any color in my end, I see those 12s too:

[root@EndeavourOS /]# inxi -F
12System:
  12Kernel 6.3.4-arch1-1 12arch x86_64 12bits 64 12Console N/A 12Distro EndeavourOS
12Machine:
  12Type Desktop 12System Micro-Star 12product MS-7D91 12v 4.0 12serial N/A
  12Mobo Micro-Star 12model MAG Z790 TOMAHAWK WIFI (MS-7D91) 12v 4.0 12serial <filter> 12UEFI American
    Megatrends LLC. 12v H.60 12date 05/16/2023
12CPU:
  12Info 24-core (8-mt/16-st) 12model 13th Gen Intel Core i9-13900K 12bits 64 12type MST AMCP 12cache
    12L2 32 MiB
  12Speed (MHz) 12avg 2743 12min/max 800/5500:5800:4300 12cores 121 3000 122 3000 123 3000 124 3000 125 3000 126 3000
    127 1100 128 3000 129 3000 1210 3000 1211 1100 1212 3000 1213 3000 1214 3000 1215 3000 1216 3000 1217 800 1218 3000
    1219 3000 1220 3000 1221 3000 1222 3000 1223 3000 1224 3000 1225 3000 1226 3000 1227 3000 1228 3000 1229 3000 1230 3000
    1231 3000 1232 800
12Graphics:
  12Device-1 Intel Raptor Lake-S GT1 [UHD Graphics 770] 12driver i915 12v kernel
  12Device-2 NVIDIA GA102 [GeForce RTX 3090 Ti] 12driver nvidia 12v 530.41.03
  12Device-3 Logitech HD Pro Webcam C920 12driver snd-usb-audio,uvcvideo 12type USB
  12Display 12server X.org 12v 1.21.1.8 12driver 12X 12loaded N/A 12unloaded modesetting
    12gpu nvidia,nvidia-nvswitch 12resolution 3840x1080
  12API OpenGL 12Message GL data unavailable for root.
12Audio:
  12Device-1 Intel 12driver snd_hda_intel
  12Device-2 NVIDIA GA102 High Definition Audio 12driver snd_hda_intel
  12Device-3 Micro Star USB Audio 12driver hid-generic,snd-usb-audio,usbhid 12type USB
  12Device-4 SteelSeries ApS Arctis 7 12driver hid-generic,snd-usb-audio,usbhid 12type USB
  12Device-5 Logitech HD Pro Webcam C920 12driver snd-usb-audio,uvcvideo 12type USB
  12API ALSA 12v k6.3.4-arch1-1 12status kernel-api
12Network:
  12Device-1 Intel 12driver iwlwifi
  12IF wlan0 12state down 12mac <filter>
  12Device-2 Intel Ethernet I226-V 12driver igc
  12IF enp3s0 12state up 12speed 1000 Mbps 12duplex full 12mac <filter>
12Bluetooth:
  12Device-1 Intel 12driver btusb 12type USB
  12Report rfkill 12ID hci0 12rfk-id 1 12state down 12bt-service N/A 12rfk-block 12hardware no 12software no
    12address see --recommends
12Drives:
  12Local Storage 12total 4.58 TiB 12used 95.3 GiB (2.0%)
  12ID-1 /dev/nvme0n1 12vendor Samsung 12model SSD 980 PRO 2TB 12size 1.82 TiB
  12ID-2 /dev/sda 12vendor Samsung 12model SSD 850 EVO 500GB 12size 465.76 GiB
  12ID-3 /dev/sdb 12vendor Samsung 12model SSD 850 EVO 500GB 12size 465.76 GiB
  12ID-4 /dev/sdc 12vendor Seagate 12model ST2000DM001-1ER164 12size 1.82 TiB
  12ID-5 /dev/sdd 12vendor SanDisk 12model Ultra 12size 28.64 GiB 12type USB
12Partition:
  12ID-1 / 12size 456.41 GiB 12used 95.27 GiB (20.9%) 12fs ext4 12dev /dev/sda2
  12ID-2 /boot/efi 12size 499 MiB 12used 31.5 MiB (6.3%) 12fs vfat 12dev /dev/nvme0n1p3
12Swap:
  12Alert No swap data was found.
12Sensors:
  12System Temperatures 12cpu 38.0 C 12mobo N/A
  12Fan Speeds (RPM) N/A
12Info:
  12Processes 0 12Uptime 25m 12Memory 12total 64 GiB 12available 62.57 GiB 12used 2.89 GiB (4.6%)
  12Client Unknown Client: systemd 12inxi 3.3.28
2

you may want to arch-chroot instead of chroot. also to install the nvidia drives you only need the command
$nvidia-inst

also may be helpful to show use the boot log.
$journalctl -b 1

3

I did use arch-chroot, and as I said, nvidia-inst doesn’t work under chroot, it first complains about not being able to run as root and when I run it under su I get libkmod and systemd related errors:

[root@EndeavourOS /]# nvidia-inst
2023-07-26 14:37:17: Error: This program must be started as a non-root user.
[root@EndeavourOS /]# su illic
/home/illic/.zshrc:3: command not found: go
➜  / nvidia-inst
2023-07-26 14:37:23: Info: Running: nvidia-inst v23-6
2023-07-26 14:37:23: Info: Command line: nvidia-inst 
2023-07-26 14:37:23: Info: Selected mode: nvidia
lspci: Unable to load libkmod resources: error -2
lspci: Unable to load libkmod resources: error -2
NVIDIA card id: 2203
Fetching driver data from nvidia.com ...
2023-07-26 14:37:26: Info: Disabling bumblebeed.service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
COMMANDS TO RUN:
    systemctl disable bumblebeed.service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

==> NOTE: running the commands may take several minutes...

Root Password: 
Failed to disable unit, unit bumblebeed.service does not exist.

To use arch-chroot I mounted my system partition under /mnt and my efi partition under /mnt/boot/efi, then ran arch-chroot /mnt

4

gotcha my misunderstaning

5

journalctl -b 1 shows outdated logs from Jul 4th and it’s pretty long, but all seems to be from 4th of July. I can export them to a file and upload it here if needed

6

unfortunately the fourth wont tell us anything about whats going on today maybe 0 its been a while since I have had to use the command myself let me look at what gives the latest boot info or maybe someone will chime in before i return

1 Like
7

yes try
journalctl -b 0

8

0 has no entries, but I tried increasing the number and using 24-29 brings up logs from my boot attempts today, 23 shows logs from yesterday’s working boot

1 Like
9

ah using -b -1 shows last boot attempt

2 Likes
10

You can use sudo in the chroot if you want to run it as your user.

sudo -u illic nvidia-inst
2 Likes
11

let me export the last boot attempts and the one working from yesterday and I’ll upload it to some pastebin

1 Like
12

hi dalto, wouldn’t that be the same I did here? Not booting after update, hanging on "create volatile files and directories" - #3 by illic

13

here are the logs:

25th working: https://termbin.com/4nke8
26th 1st attempt: https://termbin.com/kul5
26th 2nd attempt: https://termbin.com/4m5a5
26th 3rd attempt: https://termbin.com/5fv9d
26th 4th attempt: https://termbin.com/10x2

14

couldn’t add more than 5 links on a single post:
26th 5th attempt: https://termbin.com/nkyf

15

As a side note, kernel parameter ibt=off should no more be needed with the latest Nvidia drivers.

And welcome to the forum! :smile:

16

thank you, good to know

17

sudo -u illic nvidia-inst -n
would revert to open source nouveau driver what hopefully will bring system back to boot into graphical mode.
In case you can also try reaching TTY from the non booting stage with pressing Ctrl+Alt+F4 what could bring up cli login … and you would be able to run commands as usual without the need to chroot…

2 Likes
18

I managed to run nvidia-inst -n and then rebooted and worked fine, although after that I ran nvidia-inst again to reinstall the drivers and now it freezes again at the same point

TTY consoles do not work, the screen is totally frozen and it wont switch. Ctr+alt+delete restarts the pc though, so it’s not a complete freeze

19

You should be able to arch-chroot by booting with the USB installer. Then run nvidia-inst -n again in order to explore the problem more.

By the way, you need a way to switch GPUs when having Nvidia drivers. There are lots of alternatives, but many use optimus manager or envycontrol. EndeavourOS and Arch wikis have good articles about them.

20

Sorry for the late response, isn’t that only for laptops with nvidia optimus setups? My machine is a desktop with a dedicated gpu, although it also has a cpu with integrated graphics. I think I posted the specs on a previous post but here they are again just in case: http://ix.io/4ByJ

PS: note that before I changed the driver for nouveau the output of the nvidia driver was:

12Device-2 NVIDIA GA102 [GeForce RTX 3090 Ti] 12driver nvidia 12v 530.41.03