System locks up with kernel errors in journal

andersmmg · March 28, 2024, 7:37pm

Every once in a while lately, my whole system locks up and I have to force shutdown to use it again. This is what I find in the journal:

Mar 28 13:26:30 andersmmg-82k2 kernel: BUG: kernel NULL pointer dereference, address: 0000000000000082
Mar 28 13:26:30 andersmmg-82k2 kernel: #PF: supervisor read access in kernel mode
Mar 28 13:26:30 andersmmg-82k2 kernel: #PF: error_code(0x0000) - not-present page
Mar 28 13:26:30 andersmmg-82k2 kernel: PGD 0 P4D 0 
Mar 28 13:26:30 andersmmg-82k2 kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI
Mar 28 13:26:30 andersmmg-82k2 kernel: CPU: 3 PID: 116 Comm: kswapd0 Tainted: P           OE      6.8.2-arch1-1 #1 819344009c6dd5b594ee32401344754f6c37aaf0
Mar 28 13:26:30 andersmmg-82k2 kernel: Hardware name: LENOVO 82K2/LNVNB161216, BIOS H3CN30WW(V2.00) 06/28/2021
Mar 28 13:26:30 andersmmg-82k2 kernel: RIP: 0010:shrink_memcg_cb+0x110/0x2f0
Mar 28 13:26:30 andersmmg-82k2 kernel: Code: ff 48 89 ef 41 89 c6 e8 be 1e a2 00 45 85 f6 75 6f 48 83 05 d9 4a 4f 02 01 4d 8b 74 24 f8 4d 85 f6 74 1a 66 90 e8 30 82 dc ff <49> 8b 7e 10 be 6e 00 00 00 e8 d2 e0 ff ff e8 3d ba dc ff 65 48 ff
Mar 28 13:26:30 andersmmg-82k2 kernel: RSP: 0018:ffffafae805f39e8 EFLAGS: 00010202
Mar 28 13:26:30 andersmmg-82k2 kernel: RAX: 0000000000000001 RBX: ffff9a23c0b71ac0 RCX: 0000000000000021
Mar 28 13:26:30 andersmmg-82k2 kernel: RDX: ffff9a23c11c11c0 RSI: ffff9a23c2b51400 RDI: ffff9a23d3f7db68
Mar 28 13:26:30 andersmmg-82k2 kernel: RBP: ffff9a23d3f7db68 R08: 0000000000000000 R09: 0000000000000100
Mar 28 13:26:30 andersmmg-82k2 kernel: R10: 0000000000001000 R11: 0000000000001000 R12: ffff9a260fb4b8b0
Mar 28 13:26:30 andersmmg-82k2 kernel: R13: ffff9a260fb4b870 R14: 0000000000000072 R15: ffff9a23d3f7db60
Mar 28 13:26:30 andersmmg-82k2 kernel: FS:  0000000000000000(0000) GS:ffff9a265fec0000(0000) knlGS:0000000000000000
Mar 28 13:26:30 andersmmg-82k2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 28 13:26:30 andersmmg-82k2 kernel: CR2: 0000000000000082 CR3: 00000001ae820000 CR4: 0000000000f50ef0
Mar 28 13:26:30 andersmmg-82k2 kernel: PKRU: 55555554
Mar 28 13:26:30 andersmmg-82k2 kernel: Call Trace:
Mar 28 13:26:30 andersmmg-82k2 kernel:  <TASK>
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? __die+0x23/0x70
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? page_fault_oops+0x171/0x4e0
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? exc_page_fault+0x7f/0x180
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? asm_exc_page_fault+0x26/0x30
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? shrink_memcg_cb+0x110/0x2f0
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? shrink_memcg_cb+0x110/0x2f0
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? xa_load+0x8c/0xe0
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? __pfx_shrink_memcg_cb+0x10/0x10
Mar 28 13:26:30 andersmmg-82k2 kernel:  __list_lru_walk_one+0xbc/0x1d0
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? __pfx_shrink_memcg_cb+0x10/0x10
Mar 28 13:26:30 andersmmg-82k2 kernel:  list_lru_walk_one+0x5d/0x90
Mar 28 13:26:30 andersmmg-82k2 kernel:  zswap_shrinker_scan+0xc4/0x130
Mar 28 13:26:30 andersmmg-82k2 kernel:  do_shrink_slab+0x142/0x360
Mar 28 13:26:30 andersmmg-82k2 kernel:  shrink_slab+0x28e/0x3c0
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Mar 28 13:26:30 andersmmg-82k2 kernel:  shrink_one+0x123/0x1b0
Mar 28 13:26:30 andersmmg-82k2 kernel:  shrink_node+0x97e/0xbc0
Mar 28 13:26:30 andersmmg-82k2 kernel:  balance_pgdat+0x523/0x960
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? psi_task_switch+0xd6/0x230
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? finish_task_switch.isra.0+0x94/0x2f0
Mar 28 13:26:30 andersmmg-82k2 kernel:  kswapd+0x20d/0x400
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? __pfx_autoremove_wake_function+0x10/0x10
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? __pfx_kswapd+0x10/0x10
Mar 28 13:26:30 andersmmg-82k2 kernel:  kthread+0xe8/0x120
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? __pfx_kthread+0x10/0x10
Mar 28 13:26:30 andersmmg-82k2 kernel:  ret_from_fork+0x34/0x50
Mar 28 13:26:30 andersmmg-82k2 kernel:  ? __pfx_kthread+0x10/0x10
Mar 28 13:26:30 andersmmg-82k2 kernel:  ret_from_fork_asm+0x1b/0x30
Mar 28 13:26:30 andersmmg-82k2 kernel:  </TASK>

sempterobit · March 28, 2024, 10:32pm

Do you happen to have an AMD cpu? Reading about that type of freeze here. Sounds like the fix is to update kernel.

andersmmg · March 28, 2024, 11:25pm

I am running the latest kernel, 6.8.2 I believe

joekamprad · March 29, 2024, 12:08am

AMD cpu and used boards are known to may need latest Bios running latest kernels…
i see a newer one from 2024 on lenovo side…

IdeaPad Gaming 3 15ACH6

andersmmg · March 29, 2024, 12:45am

There is a newer bios, I don’t know how I would install it though. They only provide it as a windows installer which does not work and doesn’t really have a workaround that I have found besides installing Windows (which I can’t really do)

pebcak · March 29, 2024, 12:49am

You may be able to run the .exe file to update your BIOS using Hiren’s BootCD:

https://www.hirensbootcd.org/

joekamprad · March 29, 2024, 1:03am

i saw a lot users able to update lenovo Bios over Hiren

andersmmg · March 29, 2024, 4:51pm

I was able to update it using Hiren, we’ll see if it helps!

andersmmg · March 29, 2024, 5:26pm

Nope, just had another

Mar 29 11:20:31 andersmmg-82k2 kernel: BUG: kernel NULL pointer dereference, address: 00000000000000bf
Mar 29 11:20:31 andersmmg-82k2 kernel: #PF: supervisor read access in kernel mode
Mar 29 11:20:31 andersmmg-82k2 kernel: #PF: error_code(0x0000) - not-present page
Mar 29 11:20:31 andersmmg-82k2 kernel: PGD 0 P4D 0 
Mar 29 11:20:31 andersmmg-82k2 kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI
Mar 29 11:20:31 andersmmg-82k2 kernel: CPU: 1 PID: 3835 Comm: mullvad-gui Tainted: P           OE      6.8.2-arch1-1 #1 819344009c6dd5b594ee32401344754f6c37aaf0
Mar 29 11:20:31 andersmmg-82k2 kernel: Hardware name: LENOVO 82K2/LNVNB161216, BIOS H3CN46WW(V3.04) 01/12/2024
Mar 29 11:20:31 andersmmg-82k2 kernel: RIP: 0010:refill_obj_stock+0x4f/0x180
Mar 29 11:20:31 andersmmg-82k2 kernel: Code: c7 40 0b 03 00 65 4c 03 3d fe 05 23 44 49 8b 47 10 48 39 f8 0f 84 9c 00 00 00 4c 89 ff e8 49 f2 ff ff 49 89 c6 e8 21 6c d8 ff <48> 8b 45 00 a8 03 0f 85 c9 00 00 00 65 48 ff 00 e8 2c a4 d8 ff 49: sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Mar 29 11:20:31 andersmmg-82k2 kernel: RSP: 0000:ffffa8414ce9bba0 EFLAGS: 00010002
Mar 29 11:20:31 andersmmg-82k2 kernel: RAX: 0000000000000002 RBX: 0000000000000000 RCX: 0000000000000034
Mar 29 11:20:31 andersmmg-82k2 kernel: RDX: ffff9b6545fa0000 RSI: 0000000000000001 RDI: ffff9b6545fa0000
Mar 29 11:20:31 andersmmg-82k2 kernel: RBP: 00000000000000bf R08: 0000000000000001 R09: 0000000000000000
Mar 29 11:20:31 andersmmg-82k2 kernel: R10: ffff9b6592d83910 R11: 0000000000000000 R12: 0000000000000202
Mar 29 11:20:31 andersmmg-82k2 kernel: R13: 0000000000025c31 R14: ffff9b654f6a9ac0 R15: ffff9b683e670b40
Mar 29 11:20:31 andersmmg-82k2 kernel: FS:  0000791e542006c0(0000) GS:ffff9b683e640000(0000) knlGS:0000000000000000
Mar 29 11:20:31 andersmmg-82k2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 29 11:20:31 andersmmg-82k2 kernel: CR2: 00000000000000bf CR3: 0000000141bb8000 CR4: 0000000000f50ef0
Mar 29 11:20:31 andersmmg-82k2 kernel: PKRU: 55555554
Mar 29 11:20:31 andersmmg-82k2 kernel: Call Trace:
Mar 29 11:20:31 andersmmg-82k2 kernel:  <TASK>
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? __die+0x23/0x70
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? page_fault_oops+0x171/0x4e0
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? update_load_avg+0x7e/0x7e0
Mar 29 11:20:31 andersmmg-82k2 kernel: RAX: 0000000000000002 RBX: 0000000000000000 RCX: 0000000000000034
Mar 29 11:20:31 andersmmg-82k2 kernel: RDX: ffff9b6545fa0000 RSI: 0000000000000001 RDI: ffff9b6545fa0000
Mar 29 11:20:31 andersmmg-82k2 kernel: RBP: 00000000000000bf R08: 0000000000000001 R09: 0000000000000000
Mar 29 11:20:31 andersmmg-82k2 kernel: R10: ffff9b6592d83910 R11: 0000000000000000 R12: 0000000000000202
Mar 29 11:20:31 andersmmg-82k2 kernel: R13: 0000000000025c31 R14: ffff9b654f6a9ac0 R15: ffff9b683e670b40
Mar 29 11:20:31 andersmmg-82k2 kernel: FS:  0000791e542006c0(0000) GS:ffff9b683e640000(0000) knlGS:0000000000000000
Mar 29 11:20:31 andersmmg-82k2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 29 11:20:31 andersmmg-82k2 kernel: CR2: 00000000000000bf CR3: 0000000141bb8000 CR4: 0000000000f50ef0
Mar 29 11:20:31 andersmmg-82k2 kernel: PKRU: 55555554
Mar 29 11:20:31 andersmmg-82k2 kernel: Call Trace:
Mar 29 11:20:31 andersmmg-82k2 kernel:  <TASK>
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? __die+0x23/0x70
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? page_fault_oops+0x171/0x4e0
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? update_load_avg+0x7e/0x7e0
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? exc_page_fault+0x7f/0x180
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? asm_exc_page_fault+0x26/0x30
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? refill_obj_stock+0x4f/0x180
Mar 29 11:20:31 andersmmg-82k2 kernel:  obj_cgroup_uncharge_zswap+0x1d/0x60
Mar 29 11:20:31 andersmmg-82k2 kernel:  zswap_free_entry+0x29/0x110
Mar 29 11:20:31 andersmmg-82k2 kernel:  zswap_invalidate+0x73/0x80
Mar 29 11:20:31 andersmmg-82k2 kernel:  swap_range_free+0x7e/0xd0
Mar 29 11:20:31 andersmmg-82k2 kernel:  swapcache_free_entries+0x10e/0x270
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? __lruvec_stat_mod_folio+0x68/0xa0
Mar 29 11:20:31 andersmmg-82k2 kernel:  free_swap_slot+0xb4/0xc0
Mar 29 11:20:31 andersmmg-82k2 kernel:  put_swap_folio+0x1a7/0x3d0
Mar 29 11:20:31 andersmmg-82k2 kernel:  delete_from_swap_cache+0x63/0x80
Mar 29 11:20:31 andersmmg-82k2 kernel:  folio_free_swap+0x7a/0x90
Mar 29 11:20:31 andersmmg-82k2 kernel:  do_swap_page+0x9b8/0xd40
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Mar 29 11:20:31 andersmmg-82k2 kernel:  ? __pte_offset_map+0x1b/0x180
Mar 29 11:20:31 andersmmg-82k2 kernel:  __handle_mm_fault+0x883/0xe40
Mar 29 11:20:31 andersmmg-82k2 kernel:  handle_mm_fault+0x17f/0x360
Mar 29 11:20:31 andersmmg-82k2 kernel:  do_user_addr_fault+0x15b/0x670
Mar 29 11:20:31 andersmmg-82k2 kernel:  exc_page_fault+0x7f/0x180
Mar 29 11:20:31 andersmmg-82k2 kernel:  asm_exc_page_fault+0x26/0x30
Mar 29 11:20:31 andersmmg-82k2 kernel: RIP: 0033:0x6442293d9423
Mar 29 11:20:31 andersmmg-82k2 kernel: Code: cc cc cc cc cc cc cc 8b 46 08 48 85 c0 75 07 48 85 c0 0f 95 c0 c3 55 48 89 e5 48 63 0e 48 8b 57 18 48 8b 14 ca 48 85 d2 74 04 <48> 89 72 10 48 89 56 18 48 8b 57 18 48 89 34 ca 8b 4e 04 48 01 4f
Mar 29 11:20:31 andersmmg-82k2 kernel: RSP: 002b:0000791e541fe7b0 EFLAGS: 00010206
Mar 29 11:20:31 andersmmg-82k2 kernel: RAX: 0000000000c3a35d RBX: 0000000000000000 RCX: 0000000000000007
Mar 29 11:20:31 andersmmg-82k2 kernel: RDX: 0000056c00985e90 RSI: 0000056c00a9f960 RDI: 0000056c0012cc80
Mar 29 11:20:31 andersmmg-82k2 kernel: RBP: 0000791e541fe7b0 R08: 0000000000002000 R09: 0000000000000000
Mar 29 11:20:31 andersmmg-82k2 kernel: R10: 0000791e541fe810 R11: 00007ffc55db5080 R12: 0000000000000007
Mar 29 11:20:31 andersmmg-82k2 kernel: R13: 0000056c000c0078 R14: 0000056c000c0000 R15: 0000781600c00000
Mar 29 11:20:31 andersmmg-82k2 kernel:  </TASK>
Mar 29 11:20:31 andersmmg-82k2 kernel: Modules linked in: uinput nvidia_uvm(POE) nvidia_modeset(POE) rfcomm snd_seq_dummy snd_hrtimer snd_seq nvidia(POE) xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink xfrm_user xfrm_>
Mar 29 11:20:31 andersmmg-82k2 kernel:  snd_acp_legacy_common snd_hda_codec snd_ump gf128mul videobuf2_memops nf_reject_ipv6 btmtk ghash_clmulni_intel nft_reject snd_hda_core videobuf2_v4l2 snd_rawmidi sha512_ssse3 snd_pci_acp6x snd_seq_de>
Mar 29 11:20:31 andersmmg-82k2 kernel:  drm_suballoc_helper atkbd nvme drm_buddy libps2 vivaldi_fmap crc32c_intel drm_display_helper nvme_core sha256_ssse3 xhci_pci video i8042 cec nvme_auth xhci_pci_renesas wmi serio
Mar 29 11:20:31 andersmmg-82k2 kernel: CR2: 00000000000000bf
Mar 29 11:20:31 andersmmg-82k2 kernel: ---[ end trace 0000000000000000 ]---
Mar 29 11:20:31 andersmmg-82k2 kernel: RIP: 0010:refill_obj_stock+0x4f/0x180
Mar 29 11:20:31 andersmmg-82k2 kernel: Code: c7 40 0b 03 00 65 4c 03 3d fe 05 23 44 49 8b 47 10 48 39 f8 0f 84 9c 00 00 00 4c 89 ff e8 49 f2 ff ff 49 89 c6 e8 21 6c d8 ff <48> 8b 45 00 a8 03 0f 85 c9 00 00 00 65 48 ff 00 e8 2c a4 d8 ff 49
Mar 29 11:20:31 andersmmg-82k2 kernel: RSP: 0000:ffffa8414ce9bba0 EFLAGS: 00010002
Mar 29 11:20:31 andersmmg-82k2 kernel: RAX: 0000000000000002 RBX: 0000000000000000 RCX: 0000000000000034
Mar 29 11:20:31 andersmmg-82k2 kernel: RDX: ffff9b6545fa0000 RSI: 0000000000000001 RDI: ffff9b6545fa0000
Mar 29 11:20:31 andersmmg-82k2 kernel: RBP: 00000000000000bf R08: 0000000000000001 R09: 0000000000000000
Mar 29 11:20:31 andersmmg-82k2 kernel: R10: ffff9b6592d83910 R11: 0000000000000000 R12: 0000000000000202
Mar 29 11:20:31 andersmmg-82k2 kernel: R13: 0000000000025c31 R14: ffff9b654f6a9ac0 R15: ffff9b683e670b40
Mar 29 11:20:31 andersmmg-82k2 kernel: FS:  0000791e542006c0(0000) GS:ffff9b683e640000(0000) knlGS:0000000000000000
Mar 29 11:20:31 andersmmg-82k2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 29 11:20:31 andersmmg-82k2 kernel: CR2: 00000000000000bf CR3: 0000000141bb8000 CR4: 0000000000f50ef0
Mar 29 11:20:31 andersmmg-82k2 kernel: PKRU: 55555554

jake99 · March 31, 2024, 4:37pm

Looking at the short logs you posted, it seems that in both cases locks are caused by swap. There is a page fault, kswap kicks in, then zswap tries to rebalance/free pages and rips.

My guess is either a faulty hardware, or incorrect setup.
How did you setup your swap? Using hibernate? Can you post your full journal? Can you run memtest?

In the mean time you can try to disable zswap (add zswap.enabled=0 to your kernel command line).

andersmmg · March 31, 2024, 6:48pm

I am using a swapfile on btrfs right now, using the btrfs swap utility

andersmmg · April 1, 2024, 5:05am

I am not using hibernate. Also, I have run memtest86 for a while and it found no issues. I’m planning to replace one of my RAM cards soon so that may fix it? I’ll also try disabling swap altogether.

jake99 · April 4, 2024, 10:17pm

This seems to be bug with the NVIDIA drivers, see Issues with nvidia 550 drivers - General system / Kernel, boot, graphics & hardware - EndeavourOS