KVM/Quemu - > GPU & Windows NVME Passthrough

Title: Dual Boot with Windows and Linux Using KVM/QEMU

Hello dear community,

Lately, Windows has been getting quite a bit of negative press. I took the opportunity to thoroughly research and decided to finally switch to Linux. I’m very satisfied with this decision. However, due to my profession, I cannot completely abandon Windows. I’ve read in various small posts and other sources that I can have a dual Windows boot system and simultaneously access my Windows partition via KVM/QEMU. The biggest problem for me right now is that I have to restart my computer for every little thing I need to do in Windows, which doesn’t work under Linux.

So, I’m currently desperate and just don’t know exactly how to implement this. My current setup is as follows:

My EndeavourOS System:

nvme2n1     259:7    0   1.8T  0 disk 
├─nvme2n1p1 259:8    0     1G  0 part /boot/efi
└─nvme2n1p2 259:9    0   1.8T  0 part /var
                                      /home
                                      /.snapshots
                                      /

My Windows System:

nvme1n1     259:0    0 931.5G  0 disk 
├─nvme1n1p1 259:1    0   100M  0 part 
├─nvme1n1p2 259:2    0    16M  0 part 
├─nvme1n1p3 259:3    0 930.6G  0 part 
└─nvme1n1p4 259:4    0   783M  0 part 

First, I installed a fresh copy of Windows (standalone). Then, I installed EndeavourOS on my other drive. Both systems can be booted via Grub (btrfs). So far, so good.

Of course, I have also installed and set up KVM and QEMU with User/Group, etc.:

sudo pacman -S virt-manager virt-viewer qemu vde2 ebtables iptables-nft nftables dnsmasq bridge-utils ovmf swtpm

I can install virtual machines. Currently, I’m working on getting IOMMU to run properly because I also want to pass through my GPU.

Now, the problems are starting:
All the tutorials I’ve found so far are based on Arch Linux and therefore on:

mkinitcpio

However, EndeavourOS uses dracut. Yes, I found this: Arch Linux PCI Passthrough via OVMF, and there are various solutions, but they are often just snippets and therefore hard for an inexperienced user like me to understand.

But back to the main topic:
How exactly can I integrate my Windows partition?
For example, I found this statement: Reddit VFIO NVMe Partition Passthrough

However, I don’t understand exactly what it means. Where do I enter what? Do I need to have IOMMU running? Can my partition be mounted? The drive consists of several partitions as shown above. Standard Windows partitioning. Which partition will it be?

Of course, I tried it myself. I created a new VM. Chose an existing system, then as drive/directory: /dev/nvme1n1p3, and booted. The only thing that pops up is a window with a shell.

Then I also found other sources, such as: Reddit VFIO Partition Passthrough. Here, for example, the XML file of the machine itself is edited.

I’m slowly getting desperate.

Can someone please provide something more coherent than just snippets?
I don’t expect a full tutorial, but something more cohesive would be great.
I’m still learning.

Thanks in advance!

you have two gpu ?

Yes, I have two GPUs. Currently, I’m running my system using the IGPU as I want to use my GeForce for the VM.

GeForce:
IOMMU Group 19:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA102 [GeForce RTX 3090 Ti] [10de:2203] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GA102 High Definition Audio Controller [10de:1aef] (rev a1)

IGPU:
IOMMU Group 0:

00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] [8086:a780] (rev 04)

lspci | grep -i "vga"
00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] (rev 04)
01:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3090 Ti] (rev a1)

HDD Groups:
IOMMU Group 2:

00:01.0 PCI bridge [0604]: Intel Corporation Raptor Lake PCI Express 5.0 Graphics Port (PEG010) [8086:a70d] (rev 01)

IOMMU Group 20:

02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal] [144d:a80c]

IOMMU Group 21:

03:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal] [144d:a80c]

IOMMU Group 22:

04:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]

IOMMU Group 23:

06:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO [144d:a80a]

My Windows system, including the Windows Boot Partition, is on one of these hard drives. Currently, I only have IOMMU activated.

GRUB_CMDLINE_LINUX_DEFAULT='rd.driver.pre=vfio-pci intel_iommu=on iommu=pt nowatchdog nvme_load=YES loglevel=3'

In theory, I should now configure dracut:

Create or edit the Dracut configuration file:

Create or edit the file /etc/dracut.conf.d/vfio.conf:

sudo nano /etc/dracut.conf.d/vfio.conf

Add the following lines to load the necessary modules:

add_dracutmodules+=" vfio vfio_pci vfio_iommu_type1 "

Recreate the initramfs:

sudo dracut --force

However, I’m not sure if that’s correct, and I’m still researching, as these commands are from ChatGPT.

On the other hand, in this post “https://forum.level1techs.com/t/gpu-pass-through-vfio-pci-not-claiming-device-nvidia-intel/120843”, it’s mentioned that a file needs to be edited:

"I managed to get IOMMU working after setting the kernel parameters in GRUB

GRUB_CMDLINE_LINUX="[...] iommu=1 intel_iommu=on rd.driver.pre=vfio-pci"
and I was able to identify the grouping of my PCI devices. My Nvidia GTX 770 video card is in a separate IOMMU group together with the PCIe x16 controller that it is seated in.

IOMMU group 1
	00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
	01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104 [GeForce GTX 770] [10de:1184] (rev a1)
	01:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio Controller [10de:0e0a] (rev a1)
Since I remember Wendell once mentioning that all devices in an IOMMU group have to be passed-through, I wrote down all three IDs for the next steps.

I added the options option to my /etc/modprobe.d/vfio.conf file as follows:

options vfio-pci ids=8086:0c01,10de:1184,10de:0e0a
To write the changes to the initram-image I ran dracut as per the tutorial:

dracut -f --kver `uname –r`

However, the vfio.conf doesn’t exist, and I’m not sure if that’s correct. Also, I’m not exactly sure which hard drive I need to “isolate” in the grub.cfg. See snippet above.

That’s it for now.

reinstall-kernels as eos drivers is sufficient ? i am not so proficiant with passtrough

i have a desktop theoretical i can use my apu vega but i dont … some config need to be added… but make sure you have a working liveusb

grub config is edit /etc/default/grub and regen…

Somehow, I don’t know what you’re getting at. My problem is not that my kernel, etc. isn’t working. I just don’t know exactly what settings I need to configure.

My system, EndevourOS, works wonderfully. I’m just working on preparing the system for passthrough and how to integrate it into KVM/QEMU. That’s why I have IOMMU enabled.

As you can see, the kernel, etc. works perfectly.

However, it may also be that I simply don’t understand what you’re trying to tell me. English is not my native language; it’s German.

i mean you did grub-mkconfig -o /boot/grub/grub.cfg , when you editted the /etc/default/grub ?