I've had a hardware failure; how do I find the exact error message in the logs?

I’ve had my first Linux hardware failure. It’s almost certainly a GPU failure, which I’ve had before in Windows, but never in Linux.

How do I find the exact error message in the logs? Thanks.

which server are you using? wayland or xorg? @evh5150ni

It’s a standard Endeavour install with XFCE, so that’s Xorg, right?

for xorg less /var/log/Xorg.0.log you can use cat too

yes it is xorg

If you want to specifically grep for error messages related to the GPU, you can use a command like this grep -iE "EE|error|failed|fatal" /var/log/Xorg.0.log

Thank you @rudy-in. I don’t see timestamps in those logs. I know when the error occurred; how can I find the error details by the time it happened?

journalctl --since "START_TIME" --until "END_TIME" make sure to replace the start time and end time with the actual timestamps

Unfortunately journalctl skips over the time when the error occurred. eg. if my error was at 3pm, the journalctl log ends at 2.58pm and restarts at 3.02pm upon reboot.

I really appreciate your help. So if it’s not logged in journalctl, there is no log of the error at all? Arrghhhh!

grep -iE "error|gpu" /var/log/Xorg.0.log | grep "TIMESTAMP" grep -iE "error|gpu" /var/log/syslog | grep "TIMESTAMP"
replace the timestamp with the actual timestamp. i hope it helps

i really appreciate my brain is braining at the age of 14

Having never had this happen before on Linux (but a lot on Windows), it’s happened twice in two days! This time I’ve got a journalctl log. You can see it crashes at 20:39.

Dec 09 20:38:16 redacted-EndeavourOS systemd[1]: user-973.slice: Consumed 1.083s CPU time.
Dec 09 20:38:18 redacted-EndeavourOS systemd-timesyncd[444]: Contacted time server 176.58.109.199:123 (2.arch.pool.ntp.org).
Dec 09 20:38:18 redacted-EndeavourOS systemd-timesyncd[444]: Initial clock synchronization to Sat 2023-12-09 20:38:18.802261 GMT.
Dec 09 20:38:19 redacted-EndeavourOS systemd[1]: systemd-timedated.service: Deactivated successfully.
Dec 09 20:38:20 redacted-EndeavourOS systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Dec 09 20:38:25 redacted-EndeavourOS dbus-daemon[2858]: [session uid=1000 pid=2858] Activating via systemd: service name='org.freedesktop.Tracker3.Miner.Extract' unit='tracker-extract-3.service' requested by ':1.48' (uid=1000 pid=3000 com>
Dec 09 20:38:25 redacted-EndeavourOS systemd[2843]: Created slice User Background Tasks Slice.
Dec 09 20:38:25 redacted-EndeavourOS systemd[2843]: Starting Tracker metadata extractor...
Dec 09 20:38:25 redacted-EndeavourOS dbus-daemon[2858]: [session uid=1000 pid=2858] Successfully activated service 'org.freedesktop.Tracker3.Miner.Extract'
Dec 09 20:38:25 redacted-EndeavourOS systemd[2843]: Started Tracker metadata extractor.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: dev-disk-by\x2duuid-42f1660a\x2dba10\x2d40fc\x2dbea3\x2d73686c44f20f.device: Job dev-disk-by\x2duuid-42f1660a\x2dba10\x2d40fc\x2dbea3\x2d73686c44f20f.device/start timed out.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: Timed out waiting for device /dev/disk/by-uuid/42f1660a-ba10-40fc-bea3-73686c44f20f.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: Dependency failed for /mnt/42f1660a-ba10-40fc-bea3-73686c44f20f.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: mnt-42f1660a\x2dba10\x2d40fc\x2dbea3\x2d73686c44f20f.mount: Job mnt-42f1660a\x2dba10\x2d40fc\x2dbea3\x2d73686c44f20f.mount/start failed with result 'dependency'.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: dev-disk-by\x2duuid-42f1660a\x2dba10\x2d40fc\x2dbea3\x2d73686c44f20f.device: Job dev-disk-by\x2duuid-42f1660a\x2dba10\x2d40fc\x2dbea3\x2d73686c44f20f.device/start failed with result 'timeou>
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: dev-disk-by\x2duuid-50CC0F0BCC0EEB54.device: Job dev-disk-by\x2duuid-50CC0F0BCC0EEB54.device/start timed out.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: Timed out waiting for device /dev/disk/by-uuid/50CC0F0BCC0EEB54.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: Dependency failed for /mnt/50CC0F0BCC0EEB54.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: mnt-50CC0F0BCC0EEB54.mount: Job mnt-50CC0F0BCC0EEB54.mount/start failed with result 'dependency'.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: Startup finished in 3.390s (kernel) + 1min 30.601s (userspace) = 1min 33.992s.
Dec 09 20:39:18 redacted-EndeavourOS systemd[1]: dev-disk-by\x2duuid-50CC0F0BCC0EEB54.device: Job dev-disk-by\x2duuid-50CC0F0BCC0EEB54.device/start failed with result 'timeout'.
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#14 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#14 Sense Key : Not Ready [current] 
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#14 Add. Sense: Medium not present - tray closed
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#14 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
Dec 09 20:39:18 redacted-EndeavourOS kernel: I/O error, dev sr0, sector 0 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 2
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#15 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#15 Sense Key : Not Ready [current] 
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#15 Add. Sense: Medium not present - tray closed
Dec 09 20:39:18 redacted-EndeavourOS kernel: sr 5:0:0:0: [sr0] tag#15 CDB: Read(10) 28 00 00 00 00 00 00 00 02 00
Dec 09 20:39:18 redacted-EndeavourOS kernel: I/O error, dev sr0, sector 0 op 0x0:(READ) flags 0x0 phys_seg 8 prio class 2
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 0, async page read
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 1, async page read
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 2, async page read
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 3, async page read
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 4, async page read
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 5, async page read
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 6, async page read
Dec 09 20:39:18 redacted-EndeavourOS kernel: Buffer I/O error on dev sr0, logical block 7, async page read
Dec 09 20:39:22 redacted-EndeavourOS kernel: umip: fm.exe[4490] ip:160d59326 sp:11fb50: SGDT instruction cannot be used by applications.
Dec 09 20:39:22 redacted-EndeavourOS kernel: umip: fm.exe[4490] ip:160d59326 sp:11fb50: For now, expensive software emulation returns the result.
Dec 09 20:39:22 redacted-EndeavourOS kernel: umip: fm.exe[4490] ip:14b90c990 sp:11fb50: SGDT instruction cannot be used by applications.
Dec 09 20:39:22 redacted-EndeavourOS kernel: umip: fm.exe[4490] ip:14b90c990 sp:11fb50: For now, expensive software emulation returns the result.
Dec 09 20:39:22 redacted-EndeavourOS kernel: umip: fm.exe[4490] ip:14f6b4756 sp:11faa8: SGDT instruction cannot be used by applications.
Dec 09 20:39:24 redacted-EndeavourOS kernel: amdgpu 0000:08:00.0: amdgpu: Disabling VM faults because of PRT request!
Dec 09 20:41:55 redacted-EndeavourOS dbus-daemon[499]: [system] Activating via systemd: service name='org.bluez' unit='dbus-org.bluez.service' requested by ':1.64' (uid=1000 pid=5076 comm="/opt/brave-bin/brave")
Dec 09 20:41:55 redacted-EndeavourOS dbus-daemon[499]: [system] Activation via systemd failed for unit 'dbus-org.bluez.service': Unit dbus-org.bluez.service not found.
Dec 09 20:41:55 redacted-EndeavourOS gnome-keyring-daemon[2853]: asked to register item /org/freedesktop/secrets/collection/login/1, but it's already registered
Dec 09 20:41:55 redacted-EndeavourOS gnome-keyring-d[2853]: asked to register item /org/freedesktop/secrets/collection/login/1, but it's already registered
Dec 09 20:43:10 redacted-EndeavourOS systemd[2843]: Starting EOS update notifier service...
Dec 09 20:43:11 redacted-EndeavourOS dbus-daemon[2858]: [session uid=1000 pid=2858] Activating service name='org.xfce.Xfconf' requested by ':1.14' (uid=1000 pid=2950 comm="xfsettingsd")
Dec 09 20:43:11 redacted-EndeavourOS dbus-daemon[2858]: [session uid=1000 pid=2858] Successfully activated service 'org.xfce.Xfconf'
Dec 09 20:43:13 redacted-EndeavourOS polkitd[592]: Registered Authentication Agent for unix-process:5822:32899 (system bus name :1.66 [flatpak list], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale C)
Dec 09 20:43:13 redacted-EndeavourOS polkitd[592]: Unregistered Authentication Agent for unix-process:5822:32899 (system bus name :1.66, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale C)
Dec 09 20:43:15 redacted-EndeavourOS eos-update-notifier[5695]: Checking for updates .

Try showing these logs to your local repair shop he/she may help you further

First of all, please format the pasted text by highlighting it and pressing Ctrl-E

What is the output of:

cat /etc/fstab

and

sudo blkid

Copy the terminal output, paste it into your reply, highlight it and pres Ctrl-E to format.

2 Likes
UUID=4a97fe25-f62d-46dd-af23-051d7261d378	/	ext4	defaults,noatime	0	1
tmpfs	/tmp	tmpfs	defaults,noatime,mode=1777	0	0
/dev/disk/by-uuid/56CC9D22CC9CFD85	/mnt/56CC9D22CC9CFD85	auto	nosuid,nodev,nofail,x-gvfs-show	0	0
/dev/disk/by-uuid/50CC0F0BCC0EEB54	/mnt/50CC0F0BCC0EEB54	auto	nosuid,nodev,nofail,x-gvfs-show	0	0
/dev/disk/by-uuid/42f1660a-ba10-40fc-bea3-73686c44f20f	/mnt/42f1660a-ba10-40fc-bea3-73686c44f20f	auto	nosuid,nodev,nofail,x-gvfs-show	0	0
/dev/disk/by-id/usb-Seagate_BUP_Portable_00000000NAB9MJWT-0:0-part2 /mnt/usb-Seagate_BUP_Portable_00000000NAB9MJWT-0:0-part2 auto nosuid,nodev,nofail,x-gvfs-show 0 0
/dev/disk/by-uuid/882E87472E872D6E /mnt/882E87472E872D6E auto nosuid,nodev,nofail,x-gvfs-show 0 0
/dev/sdd1: UUID="6E28-1518" BLOCK_SIZE="512" TYPE="vfat"
/dev/sdb4: UUID="4a97fe25-f62d-46dd-af23-051d7261d378" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="a4128d27-308f-6e43-8c90-6e20cd32e14a"
/dev/sdb2: LABEL="Local Disk 2" BLOCK_SIZE="512" UUID="56CC9D22CC9CFD85" TYPE="ntfs" PARTLABEL="Basi" PARTUUID="97331999-ef38-4194-8f2a-b039aac34297"
/dev/sdb3: LABEL="Windows Backup" BLOCK_SIZE="512" UUID="3B04C7F4140CFD1B" TYPE="ntfs" PARTUUID="8198d59d-4798-4fa7-8741-3c6659802645"
/dev/sdb1: PARTLABEL="Micr" PARTUUID="606166d3-fcae-44aa-99b6-e3979a36df6e"
/dev/sdc2: LABEL="New FLAC (MAIN)" BLOCK_SIZE="512" UUID="3222B4BB22B484FF" TYPE="ntfs" PARTUUID="dd14f446-2562-4e1a-8035-fe54e902364e"
/dev/sdc1: LABEL_FATBOOT="EFI" LABEL="EFI" UUID="67E3-17ED" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="e23efc16-0b98-4bd5-984e-130e55254d8d"
/dev/sda4: BLOCK_SIZE="512" UUID="92DEA620DEA5FC9B" TYPE="ntfs" PARTUUID="d305c96e-0494-4e15-90b8-f124ebc85aed"
/dev/sda2: PARTLABEL="Microsoft reserved partition" PARTUUID="ebb328d1-c193-45c3-bcc4-4378b52b5da1"
/dev/sda5: LABEL="timeshift" UUID="048e0245-12e4-4f2e-98bb-fb67c0ba0050" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="74580260-1371-4f6b-a239-e9c372d2182b"
/dev/sda3: BLOCK_SIZE="512" UUID="882E87472E872D6E" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="d0d08d8d-f8c1-417a-b496-667ed77375a8"
/dev/sda1: UUID="8886-23D2" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="3b00111c-eed0-4bf5-a025-02f9f840ac18"

Your not showing your boot device in the fstab entry

/dev/sdd1: UUID=“6E28-1518” BLOCK_SIZE=“512” TYPE=“vfat”

i don’t see this UUID in the fstab

I don’t understand. Everything is working - I’m typing from this EndeavourOS install right now. It’s just that I’ve had two crashes while playing games, which has never happened to me before on Linux but has happened often on Windows. The screen crashes to black (or another colour) and the system reboots. Otherwise it is fine.

I’ve always assumed the error is GPU-related because it is having to work moderately hard when playing the game. The problem is I can’t replicate it in a stress test.