Why does linking a gpu freeze QEMU in Ubuntu

Question

I am running a Ubuntu on a workstation with 2 GTX 1070 and I5 7400 CPU, I have QEMU installed and I have an old windows 10 vm I tried to link the GPU by adding the PCI link hardware in virt-manager GUI, and restored windows vm (from a saved state) it detects a PCI hardware but does not install any drivers, I powered the VM completely then tried to run it again but I got this error

  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1384, in startup
    self._backend.create()
  File "/usr/lib/python3/dist-packages/libvirt.py", line 1353, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: unsupported configuration: host doesn't support passthrough of host PCI devices

so I check if VT is enable and it is, since sudo rdmsr 0x3A returns a 5

next step I did was to check kvm-ok and I got

 INFO: /dev/kvm exists
 KVM acceleration can be used

everything seems okay so far so I used

virt-host-validate and realized IOMMU is not enabled so I did the following, edited GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" to GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on" followed by update-grub

now IOMMU is enabled since virt-host-validate outputs

  QEMU: Checking if IOMMU is enabled by kernel                               : PASS

now when I try to run the same vm without the pci link it works fine but adding the pci link causes the libvirtd to timeout and freeze

авг 28 00:45:22 WORKSTATION systemd[1]: libvirtd.service: Unit process 1814 (dnsmasq) remains running after unit stopped.
авг 28 00:45:22 WORKSTATION systemd[1]: libvirtd.service: Unit process 1815 (dnsmasq) remains running after unit stopped.
авг 28 00:45:22 WORKSTATION systemd[1]: libvirtd.service: Unit process 4018 (libvirtd) remains running after unit stopped.
авг 28 00:45:22 WORKSTATION systemd[1]: Failed to start Virtualization daemon.

The output pf dmesg

[32729.791574] [drm:nv_drm_master_set [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000500] Failed to grab modeset ownership
[32729.792216] [drm:nv_drm_master_set [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000900] Failed to grab modeset ownership

am I missing something, do I need to enable any extra features or maybe disable any?

Have you excluded the other GPU from your Linux driver? The Linux GPU driver must not manage the GPU you want to pass-through. Please add `dmesg` output right after boot so we can see the status. — Tero Kilkanen, Aug 28 '22 at 08:15
This is only part of `dmesg` output, please add the full output after system boot. — Tero Kilkanen, Aug 28 '22 at 15:22
@TeroKilkanen please check the output in the pastebin [link](https://pastebin.com/raw/yU53b3v2) — Weed Cookie, Aug 28 '22 at 18:04

score 1 · Accepted Answer · answered Aug 28 '22 at 21:04

From the dmesg output it is visible that the Linux kernel loads drivers for both cards.

In order to free the device for your VM, you need to unbind the driver from it. https://www.iram.fr/~blanchet/ethercat/unbind_manually_pci_device.html tells how to unbind the driver from a device.

First, find PCI slots and actual drivers:

lspci -Dk

The output might look like:

0000:01:06.0 Ethernet controller: Intel Corporation 8255xER/82551IT Fast

Ethernet Controller (rev 10) Kernel driver in use: e100 0000:01:07.0 Ethernet controller: Intel Corporation 8255xER/82551IT Fast Ethernet Controller (rev 10) Kernel driver in use: e100

You need to find the NVidia entries there. You might need to guess which device is the one you are using.

You need to note the PCI ID (0000:01:07.0) and driver name (e100) for your GPU.

Then you can issue

echo -n "<pci id>" > /sys/bus/pci/drivers/<driver>/unbind

You need to replace <pci_id> and <driver> with the values you got from lspci output. This will unbind the NVidia driver from the specified device, freeing it up for PCI passthrough.

That is the most likely blocker here. I cannot definitely say it is the only reason. — Tero Kilkanen, Aug 29 '22 at 15:58
So i used the command `echo " | sudo tee /sys/bus/pci/drivers/nvidia/unbind"` but this never returns to the terminal and the only option is to reboot the system even using `kill ` does not help I already asked a question [here](https://unix.stackexchange.com/questions/715246/how-to-unbind-a-gpu-sys-bus-pci?noredirect=1#comment1355276_715246) but thought i would mention it here — Weed Cookie, Aug 29 '22 at 16:24

Why does linking a gpu freeze QEMU in Ubuntu

1 Answers1