V6.13.1-arch1 and 2-1 kernel breaking win 11 compatibility modes in vm

When running 3d accelerated games in win11 using either vmware-workstation or virtualbox, that are also using compatibility mode for windows xp sp3, they crash almost immediately at/after launching. Programs that don’t use 3d acceleration appear to be unaffected, just programs that are accelerated. Repeatable on 2 different systems.

This bug(?) is pretty specific. Full system specs will be at the end. I was running the default kernel until 2/2, when I started experiencing issues. Did a full EOS reinstall, including wiping home, and still had issues. lts kernel fixes them. Looking at https://github.com/archlinux/linux/releases , v6.13.1-arch1 was when problems started. I compared it to previous kernel but nothing stood out to me https://github.com/archlinux/linux/compare/v6.13-arch1...v6.13.1-arch1 – latest zen kernel has same bug.

Specific game: Dungeon Siege 2 with Killa fix for Broken World + Legendary Mod
VM instead of Wine?: Playing with a friend, so requires gameranger

Steps to recreate: Install EOS with KDE, update, install vmware-workstation, start vmware-workstation services, install windows 11 in vmware, install Dungeon Siege 2 with Killa fix for Broken World + Legendary Mod, set EXE to use windows 98 sp3 compatibility mode, game crashes at/soon after launch. The longest I got a game to run was loading from the main menu into the actual game world.

Game will launch/run without compatibility mode active, but will have several visual artifacts such as non-transparent textures that have transparencies. This behavior is expected and why compatibility mode is used.

Desktop Specs works as expected with lts kernel
OS: EndeavourOS x86_64
Host: B650 AORUS ELITE AX
Kernel: Linux 6.12.12-1-lts
Shell: bash 5.2.37
Display (Acer K272HUL): 2560x1440 @ 60 Hz in 27" [External]
Display (RX-V385): 1920x1080 @ 60 Hz (as 2560x1440) [External]
Display (Dell S2716DG): 1440x2560 @ 144 Hz in 27" [External]
Display (DELA0FA): 3440x1440 @ 100 Hz in 34" [External] *
DE: KDE Plasma 6.2.5
WM: KWin (Wayland)
CPU: AMD Ryzen 5 9600X (12) @ 5.48 GHz
GPU: AMD Radeon RX 6800 [Discrete]
Memory: 7.65 GiB / 62.41 GiB (12%)
Swap: Disabled
Disk (/): 430.58 GiB / 1.82 TiB (23%) - btrfs
Locale: en_US.UTF-8

Laptop Specs crashes as expected with default kernel
OS: EndeavourOS x86_64
Host: 81YM (IdeaPad 5 14ARE05)
Kernel: Linux 6.13.1-arch1-1
Shell: bash 5.2.37
Display (AUO408D): 1920x1080 @ 60 Hz (as 1536x864) in 14" [Built-in]
DE: KDE Plasma 6.2.5
WM: KWin (Wayland)
CPU: AMD Ryzen 5 4600U (12) @ 2.10 GHz
GPU: AMD Radeon Vega Series / Radeon Vega Mobile Series [Integrated]
Memory: 1.74 GiB / 15.00 GiB (12%)
Swap: 0 B / 16.50 GiB (0%)
Disk (/): 209.03 GiB / 451.15 GiB (46%) - ext4
Locale: en_US.UTF-8

tried with linux-lts?

Lts works. My concern is that whatever is in the base kernel will eventually be merged to lts.

I’d tried restoring a system snapshot (timeshift) from Jan 5th, but had to update with arch-chroot on a live USB to have a bootable system again.

I’ve unfortunately reformatted since then, so spending part of today figuring out how to install the first 6.13 kernel from 2 weeks ago to confirm it didn’t have issues.

it stil can improve future version first two point releases has always some little bugs

6.13.2-arch1-1 still has the same issue

6.12.13-1-lts continues to be usable

it is not only the kernel…, but is vmware should make usable for the latest

Not intending to highjack this thread, so if I am, I apologise in advance.

OK…I have a similar weird experience with 6.13.x and KVM/Libvirtd on my Win10 gaming install. On Kernel 6.13.1 and 6.13.2 every time I attempt to do anything with Steam - it just silently crashes - nothing in the error logs there at all.

I re-installed Steam multiple times on that instance and it just kept crashing. I think tried a fresh install of Windows 10 on a different VM (with all of the same virtual hardware including a GPU passthrough) and that install started having issues with a different program. I finally decided to reboot back to LTS 6.12.x and when I started the original Win 10 VM - Steam was working just fine. All of the behaviour that caused crashing under 6.13.x worked fine in 6.12.x.

I updated when the kernel went from 6.13.1x to 6.13.2x and it didn’t help.

I want to dig into this more but I don’t know which log might show some errors. I trolled through journalctl looking for any sort of indicator and nothing showed up; I tried various forums and bug trackers for libvirtd, QEMU, Kernel.org and KVM - Nothing sounded even close to my type of issue.

So while this is different a different VM setup the kernel version issues sound similar.

I appear to be able to re-create the issue at will right now, so if someone has an idea on possible information gathering or troubleshooting I could do, I’m happy to do so.

Quick view of my specs:

System:

  • Kernel: 6.12.13-1-lts arch: x86_64 bits: 64 compiler: gcc v: 14.2.1*
  • Desktop: KDE Plasma v: 6.2.5 Distro: EndeavourOS base: Arch Linux*
    Machine:
  • Type: Desktop System: ASUS product: N/A v: N/A serial: *
  • Mobo: ASUSTeK model: ProArt X870E-CREATOR WIFI v: Rev 1.xx*
  • serial: UEFI: American Megatrends v: 0606*
  • date: 10/28/2024*
    Graphics:
  • Device-1: NVIDIA GA102 [GeForce RTX 3080] vendor: Gigabyte driver: vfio-pci*
  • v: N/A arch: Ampere bus-ID: 01:00.0*
  • Device-2: Advanced Micro Devices [AMD/ATI] Granite Ridge [Radeon Graphics]*
  • vendor: ASUSTeK driver: amdgpu v: kernel bus-ID: 78:00.0 temp: 42.0 C*
  • Display: x11 server: X.Org v: 21.1.15 with: Xwayland v: 24.1.5 driver: X:*
  • loaded: amdgpu unloaded: modesetting dri: radeonsi gpu: amdgpu resolution:*
  • 1: 3840x2160~30Hz 2: 3840x2160~60Hz*
  • API: EGL v: 1.5 drivers: kms_swrast,radeonsi,swrast platforms:*
  • active: gbm,x11,surfaceless,device inactive: wayland*
  • API: OpenGL v: 4.6 compat-v: 4.5 vendor: amd mesa v: 24.3.4-arch1.1*
  • glx-v: 1.4 direct-render: yes renderer: AMD Radeon Graphics (radeonsi*
  • raphael_mendocino LLVM 19.1.7 DRM 3.59 6.12.13-1-lts)*
  • API: Vulkan v: 1.4.303 drivers: N/A surfaces: xcb,xlib devices: 1*
  • Info: Tools: api: clinfo, eglinfo, glxinfo, vulkaninfo*
  • de: kscreen-console,kscreen-doctor gpu: corectrl,lact wl: wayland-info*
  • x11: xdpyinfo, xprop, xrandr*
    Audio:
  • Device-1: NVIDIA GA102 High Definition Audio vendor: Gigabyte*
  • driver: vfio-pci bus-ID: 01:00.1*
  • Device-2: Advanced Micro Devices [AMD/ATI] Rembrandt Radeon High*
  • Definition Audio vendor: ASUSTeK driver: snd_hda_intel v: kernel*
  • bus-ID: 78:00.1*
  • Device-3: Advanced Micro Devices [AMD] Family 17h/19h/1ah HD Audio*
  • vendor: ASUSTeK driver: snd_hda_intel v: kernel bus-ID: 78:00.6*
  • Device-4: C-Media USB Modi Device driver: hid-generic,snd-usb-audio,usbhid*
  • type: USB bus-ID: 1-2:2*
  • API: ALSA v: k6.12.13-1-lts status: kernel-api*
  • Server-1: PipeWire v: 1.2.7 status: active*

virsh --version=long
Virsh command line tool of libvirt 11.0.0
See web site at https://libvirt.org/
Compiled with support for:

  • Hypervisors: QEMU/KVM LXC OpenVZ Cloud-Hypervisor VMware VirtualBox ESX Test*
  • Networking: Remote Network Bridging Interface udev Nwfilter*
  • Storage: Dir Disk Filesystem SCSI Multipath iSCSI iSCSI-direct LVM Gluster ZFS*
  • Miscellaneous: Daemon Nodedev Secrets Debug Readline*

*virsh -c qemu:///system version *
Compiled against library: libvirt 11.0.0
Using library: libvirt 11.0.0
Using API: QEMU 11.0.0
Running hypervisor: QEMU 9.2.0

Sorry, didn’t mean to reply specifically to this post and apparently deleting it leaves a “deleted post” message. Still getting used to this forum.

New Kernels, same behavior. Attempting to use compatibility mode on a win11 guest with a 3d accelerated game causes various crashes. Specific test game is Dungeon Siege 2

6.13.3.arch1-1 crashes nearly immediately. Always crashes by the time it loads in to the game world.

lts 6.12.15-1 still works fine.

I don’t know when lts bumps to the 6.13 branch, but hopefully it doesn’t include whatever broke things.

hijack lol…is qemu with 3d accel?

next january 2026 next lts… so round december you can check progress next lts

If I understand your QEMU question - I’m using GPU Passthrough (RTX 3080) so 3D accel isn’t applicable. If I’m misunderstanding, let me know.

hah i see… lts does not crash?

Correct - Steam just closes - Doesn’t generate any errors in it’s logs or any Window’s logs. On the underlying Endeavour layer - Nothing in Journalctl that looks like it would be the problem and I didn’t see anything in the systemclt status logs for Libvirtd right after. Happy to look at other logs if you have an idea where.

idk lol

check https://steamcommunity.com/discussions/forum/1/135509758529538311/#c135509758529700366

Heh - Already done that. Nothing there. Mentioned that twice - I’m talking on the Endeavour side.

1 Like