Spice performance HORRIBLE

General system Newbie
1

Hi all,

I am running a KVM with EndeavourOS as a guest (host is ProxmoxVE) and tried using SPICE to connect to it, but the performance is horrible (and by that I mean dreadful). Using VNC (both from gnome sharing but also by running Xvnc) works much much better. I do not understand this, as I expected SPICE to be much better-performing.

All I did in the EndeavourOS guest is install guest utilities with: yay -Sy qemu-agent-utils spice-vdagentd. I can see them running:

$ systemctl status spice-vdagentd qemu-guest-agent
● spice-vdagentd.service - Agent daemon for Spice guests
     Loaded: loaded (/usr/lib/systemd/system/spice-vdagentd.service; indirect; vendor preset: disabled)
     Active: active (running) since Wed 2022-01-12 18:22:59 GMT; 6s ago
TriggeredBy: ● spice-vdagentd.socket
    Process: 883 ExecStart=/usr/bin/spice-vdagentd $SPICE_VDAGENTD_EXTRA_ARGS (code=exited, status=0/SUCCESS)
   Main PID: 886 (spice-vdagentd)
      Tasks: 2 (limit: 9476)
     Memory: 1.0M
        CPU: 6ms
     CGroup: /system.slice/spice-vdagentd.service
             └─886 /usr/bin/spice-vdagentd

Jan 12 18:22:59 ironyman systemd[1]: Starting Agent daemon for Spice guests...
Jan 12 18:22:59 ironyman systemd[1]: Started Agent daemon for Spice guests.
Jan 12 18:22:59 ironyman spice-vdagentd[886]: opening vdagent virtio channel

● qemu-guest-agent.service - QEMU Guest Agent
     Loaded: loaded (/usr/lib/systemd/system/qemu-guest-agent.service; enabled; vendor preset: disabled)
     Active: active (running) since Wed 2022-01-12 18:21:02 GMT; 2min 3s ago
   Main PID: 434 (qemu-ga)
      Tasks: 2 (limit: 9476)
     Memory: 1.3M
        CPU: 63ms
     CGroup: /system.slice/qemu-guest-agent.service
             └─434 /usr/bin/qemu-ga

Jan 12 18:21:17 ironyman qemu-ga[434]: info: guest-ping called
Jan 12 18:21:29 ironyman qemu-ga[434]: info: guest-ping called
Jan 12 18:21:40 ironyman qemu-ga[434]: info: guest-ping called

I am using virt-viewer version 10.0.256 to connect.

What can I do to improve performance? So far I have managed to make things better by:

  1. Using X.org instead of Wayland (SIGNIFICANT speed-up from this)
  2. Reduced resolution to 1680x1050

I have not found a way to reduce color-depth (both on the virt-viewer client and the display settings).

Does anyone know if it is possible to reduce color depth and/or anything else that might help?

2

What virtual video driver are you using?

3

I am using QXL with 16MB of memory:

$ sudo dmesg | grep qxl
[   12.632738] fb0: switching to qxl from EFI VGA
[   12.640088] qxl 0000:00:01.0: vgaarb: deactivate vga console
[   12.640129] [drm] qxl: 16M of VRAM memory size
[   12.640130] [drm] qxl: 63M of IO pages memory ready (VRAM domain)
[   12.640130] [drm] qxl: 64M of Surface memory size
[   12.648590] [drm] Initialized qxl 0.1.0 20120117 for 0000:00:01.0 on minor 0
[   12.649184] fbcon: qxldrmfb (fb0) is primary device
[   12.668886] qxl 0000:00:01.0: [drm] fb0: qxldrmfb frame buffer device
4

OK, good. Did you install xf86-video-qxl on the guest? If this doesn’t work, try using virtio.

5

The VirGL driver does wonders on KVM/Qemu for linux guests over the QXL driver.

6

So, no that package was missing. I just installed it now and the situation is as follows:

I can no longer log in to X.org (it must be crashing as soon I enter the password and throws me back into the login prompt).

I can still log in to Wayland (an xf86 driver has not impact here).

Just trying to find X.org logs to see why it borks…

7

Hmmm, found the log in .local/share/xorg/Xorg.0.log:

[   855.232] (II) LoadModule: "fbdev"
[   855.232] (WW) Warning, couldn't open module fbdev
[   855.232] (EE) Failed to load module "fbdev" (module does not exist, 0)
[   855.232] (II) LoadModule: "vesa"
[   855.232] (WW) Warning, couldn't open module vesa
[   855.232] (EE) Failed to load module "vesa" (module does not exist, 0)
[   855.232] (II) qxl: Driver for QXL virtual graphics: QXL 1
[   855.232] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[   855.232] xf86EnableIO: failed to enable I/O ports 0000-03ff (Operation not permitted)
[   855.232] (WW) Falling back to old probe method for modesetting
[   855.232] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
[   855.232] (II) qxl(0): Creating default Display subsection in Screen section
        "Default Screen Section" for depth/fbbpp 24/32
[   855.232] (==) qxl(0): Depth 24, (--) framebuffer bpp 32
[   855.232] (==) qxl(0): RGB weight 888
[   855.232] (==) qxl(0): Default visual is TrueColor
[   855.232] (==) qxl(0): Using gamma correction (1.0, 1.0, 1.0)
[   855.232] (II) qxl(0): Deferred Frames: Disabled
[   855.232] (II) qxl(0): Offscreen Surfaces: Disabled
[   855.232] (II) qxl(0): Image Cache: Disabled
[   855.232] (II) qxl(0): Fallback Cache: Disabled
[   855.232] (II) UnloadModule: "qxl"
[   855.232] (EE) Screen(s) found, but none have a usable configuration.
[   855.232] (EE)
Fatal server error:
[   855.232] (EE) no screens found(EE)
[   855.232] (EE)
Please consult the The X.Org Foundation support
         at http://wiki.x.org
 for help.
[   855.232] (EE) Please also check the log file at "/home/user/.local/share/xorg/Xorg.0.log" for additional information.
[   855.232] (EE)
[   855.243] (EE) Server terminated with error (1). Closing log file.
8

thats not unusual, both my systems have that when running Xorg and work just fine.

9

So, I find this weird:

[   855.232] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support

Why does it thing I have multiple GPUs? I only have a single SPICE display in the VM configuration.

I am also now wondering what xf86 driver was used before I added the qxl one. I will remove it and log in again to see what gets loaded in the log…

10

that doesnt necessarily mean it thinks you have multi gpu, just that it was checking and found its not supported.

I would switch your video model to Virtio as it should be faster than QXL unless you have some specific reason to refuse using Virtio.

11

So, yes that (arbiter) line is present even when I remove xf86-video-qxl and it still works. It also seems to use the modesetting driver:

[   122.015] (==) Matched qxl as autoconfigured driver 0
[   122.015] (==) Matched modesetting as autoconfigured driver 1
[   122.015] (==) Matched fbdev as autoconfigured driver 2
[   122.015] (==) Matched vesa as autoconfigured driver 3
[   122.015] (==) Assigned the driver to the xf86ConfigLayout
[   122.015] (II) LoadModule: "qxl"
[   122.015] (WW) Warning, couldn't open module qxl
[   122.015] (EE) Failed to load module "qxl" (module does not exist, 0)
[   122.015] (II) LoadModule: "modesetting"
[   122.015] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[   122.017] (II) Module modesetting: vendor="X.Org Foundation"
[   122.017]    compiled for 1.21.1.3, module version = 1.21.1
[   122.017]    Module class: X.Org Video Driver
[   122.017]    ABI class: X.Org Video Driver, version 25.2
[   122.017] (II) LoadModule: "fbdev"
[   122.017] (WW) Warning, couldn't open module fbdev
[   122.017] (EE) Failed to load module "fbdev" (module does not exist, 0)
[   122.017] (II) LoadModule: "vesa"
[   122.017] (WW) Warning, couldn't open module vesa
[   122.017] (EE) Failed to load module "vesa" (module does not exist, 0)
[   122.017] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[   122.017] (II) modeset(0): using drv /dev/dri/card0
[   122.017] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
[   122.017] (II) modeset(0): Creating default Display subsection in Screen section
        "Default Screen Section" for depth/fbbpp 24/32
[   122.017] (==) modeset(0): Depth 24, (==) framebuffer bpp 32
[   122.017] (==) modeset(0): RGB weight 888
[   122.017] (==) modeset(0): Default visual is TrueColor
[   122.017] (II) Loading sub module "glamoregl"
[   122.017] (II) LoadModule: "glamoregl"
[   122.017] (II) Loading /usr/lib/xorg/modules/libglamoregl.so
[   122.020] (II) Module glamoregl: vendor="X.Org Foundation"
[   122.020]    compiled for 1.21.1.3, module version = 1.0.1
[   122.020]    ABI class: X.Org ANSI C Emulation, version 0.4
[   122.036] (II) modeset(0): Refusing to try glamor on llvmpipe
[   122.037] (II) modeset(0): glamor initialization failed
[   122.037] (II) modeset(0): ShadowFB: preferred NO, enabled NO
[   122.037] (II) modeset(0): Output Virtual-1 has no monitor section
[   122.037] (II) modeset(0): Output Virtual-2 has no monitor section
[   122.037] (II) modeset(0): Output Virtual-3 has no monitor section
[   122.037] (II) modeset(0): Output Virtual-4 has no monitor section
[   122.037] (II) modeset(0): EDID for output Virtual-1
[   122.037] (II) modeset(0): Printing probed modes for output Virtual-1
[   122.037] (II) modeset(0): Modeline "1024x768"x60.0   65.00  1024 1048 1184 1344  768 771 777 806 -hsync -vsync (48.4 kHz eP)
...
12

So, this also looks suspicious when using QXL driver:

[   639.936] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[   639.936] xf86EnableIO: failed to enable I/O ports 0000-03ff (Operation not permitted)
[   639.936] (WW) Falling back to old probe method for modesetting

I wonder whether this is the reason QXL subsequently fails?

13

maybe but have you tried Virtio instead? considering Virtio/Virtio+Virgl is substantially faster than QXL. If it resolves your issue you get faster performance and problem solved.

14

So, firstly I’ve given up on xf86-video-qxl. I found this post which seems similar and the advice given there was:

Did you install xf86-video-qxl? If so uninstall it, linux has a modesetting driver for QXL.

Indeed my original situation was to use modesetting driver with QXL and that works.

Regarding VirtIo: will I be able to connect with SPICE client then? I am accessing this VM remotely. If I go with VirtIo won’t I need VNC anyway?

15

Didnt realize it was remote :stuck_out_tongue_closed_eyes:

I use spice for my VM using Virtio locally but im not sure about remotely. AFAIK using Virtio should work fine in that way but im not 100%.

I wonder if its just a spice thing in this case, do you have any image compression or anything enabled? Is it just EOS or any other distros? Spice i believe can get pretty sluggish over WAN at times or at least thats what ive heard ive never tried it.

16

So, when I switch Proxmox to use VirtIO for the VM, it stops providing the SPICE option in the console button (only VNC is available).

I like your suggestion about trying with anoter distro. I’ll create another VM with something non-Arch (maybe Fedora?) and see how SPICE performs there.

P.S. I use this over very fast WiFi, so not worried about WAN performance, but more about usability. TigerVNC with Xvnc so far give the best performance.

17

So, interestingly, Fedora 35 seems to work fine. I pre-installed qemu-guest-agent, spice-vdagent, and the xf86-qxl driver and worked out of the box.

Here’s the respective relevant output from the running Fedora:

$ sudo dmesg | grep qxl
[    1.106468] fb0: switching to qxl from EFI VGA
[    1.109491] qxl 0000:00:01.0: vgaarb: deactivate vga console
[    1.110004] [drm] qxl: 16M of VRAM memory size
[    1.110005] [drm] qxl: 63M of IO pages memory ready (VRAM domain)
[    1.110006] [drm] qxl: 64M of Surface memory size
[    1.115026] [drm] Initialized qxl 0.1.0 20120117 for 0000:00:01.0 on minor 0
[    1.116872] fbcon: qxldrmfb (fb0) is primary device
[    1.116876] qxl 0000:00:01.0: [drm] fb0: qxldrmfb frame buffer device

$ sudo rpm -qa | grep qxl
xorg-x11-drv-qxl-0.1.5-20.fc35.x86_64
qemu-device-display-qxl-6.1.0-10.fc35.x86_64

Interestingly it was printing similar errors:

[    23.247] xf86EnableIOPorts: failed to set IOPL for I/O (Operation not permitted)
[    23.247] (II) [KMS] Kernel modesetting enabled.
[    23.247] (WW) Falling back to old probe method for modesetting

The main difference was that it has all drivers installed: it loaded qxl, modesetting, fbdev, and vesa successfully. I think I’ll try adding these to EndeavourOS and see if maybe they are relevant…

18

So, installing fbdev/vesa drivers didn’t make a difference. I guess I’ll just have to accept QXL video simply does not work in EOS…

19

it should, thats a really strange issue and TBH im not sure what might be causing it :man_shrugging: maybe someone with more experience with it can help.

20

Thanks for your help though @Echoa. At least quite a few things were ruled out…