Mauricio Tavares
2020-Apr-24 20:09 UTC
Not able to add pcie card to guest: Operation not permitted
Let's say I have libvirt [root@vmhost2 ~]# virsh version Compiled against library: libvirt 4.5.0 Using library: libvirt 4.5.0 Using API: QEMU 4.5.0 Running hypervisor: QEMU 2.12.0 [root@vmhost2 ~]# running on centos 8 and then I have this card [root@vmhost2 ~]# virsh nodedev-dumpxml pci_0000_01_00_0 <device> <name>pci_0000_01_00_0</name> <path>/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0</path> <parent>pci_0000_00_01_0</parent> <driver> <name>mlx4_core</name> </driver> <capability type='pci'> <domain>0</domain> <bus>1</bus> <slot>0</slot> <function>0</function> <product id='0x1003'>MT27500 Family [ConnectX-3]</product> <vendor id='0x15b3'>Mellanox Technologies</vendor> <iommuGroup number='1'> <address domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </iommuGroup> <pci-express> <link validity='cap' port='8' speed='8' width='8'/> <link validity='sta' speed='5' width='8'/> </pci-express> </capability> </device> which I added to the guest (arch='x86_64' machine='pc-q35-rhel7.6.0') as <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </hostdev> When I try to start the guest I get the following error message: [root@vmhost2 ~]# virsh start testfedora error: Failed to start domain testfedora error: internal error: qemu unexpectedly closed the monitor: 2020-04-24T20:01:35.341020Z qemu-kvm: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.8,addr=0x0: vfio error: 0000:01:00.0: failed to setup INTx fd: Operation not permitted [root@vmhost2 ~]# Why is it telling me that is not permitted?
Peter Crowther
2020-Apr-24 20:34 UTC
Re: Not able to add pcie card to guest: Operation not permitted
On Fri, 24 Apr 2020 at 21:10, Mauricio Tavares <raubvogel@gmail.com> wrote:> Let's say I have libvirt > > [root@vmhost2 ~]# virsh version > [...]Running hypervisor: QEMU 2.12.0> [root@vmhost2 ~]# > [...] >When I try to start the guest I get the following error message:> > [root@vmhost2 ~]# virsh start testfedora > error: Failed to start domain testfedora > error: internal error: qemu unexpectedly closed the monitor: > 2020-04-24T20:01:35.341020Z qemu-kvm: -device > vfio-pci,host=01:00.0,id=hostdev0,bus=pci.8,addr=0x0: vfio error: > 0000:01:00.0: failed to setup INTx fd: Operation not permitted > > [root@vmhost2 ~]# > > Why is it telling me that is not permitted? > > The guest will be running as qemu on the host. Does qemu have appropriatepermissions in the host, and does that include in any hardening like SElinux that you're running? Cheers, - Peter
Mauricio Tavares
2020-Apr-25 01:30 UTC
Re: Not able to add pcie card to guest: Operation not permitted
On Fri, Apr 24, 2020 at 4:35 PM Peter Crowther <peter.crowther@melandra.com> wrote:> > On Fri, 24 Apr 2020 at 21:10, Mauricio Tavares <raubvogel@gmail.com> wrote: >> >> Let's say I have libvirt >> >> [root@vmhost2 ~]# virsh version >> [...] >> >> Running hypervisor: QEMU 2.12.0 >> [root@vmhost2 ~]# >> [...] > > When I try to start the guest I get the following error message: >> >> >> [root@vmhost2 ~]# virsh start testfedora >> error: Failed to start domain testfedora >> error: internal error: qemu unexpectedly closed the monitor: >> 2020-04-24T20:01:35.341020Z qemu-kvm: -device >> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.8,addr=0x0: vfio error: >> 0000:01:00.0: failed to setup INTx fd: Operation not permitted >> >> [root@vmhost2 ~]# >> >> Why is it telling me that is not permitted? >> > The guest will be running as qemu on the host. Does qemu have appropriate permissions in the host, and does that include in any hardening like SElinux that you're running? >I tried with selinux in permissive mode to see if it made a difference. Not much. [root@vmhost2 ~]# getenforce Permissive [root@vmhost2 ~]# virsh start testfedora error: Failed to start domain testfedora error: internal error: qemu unexpectedly closed the monitor: 2020-04-25T00:43:36.621246Z qemu-kvm: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.8,addr=0x0: vfio error: 0000:01:00.0: failed to setup INTx fd: Operation not permitted [root@vmhost2 ~]# For the fun of it, I swapped that card with another one (same speed, number of ports, diff brand), so it is on th every sam epci slot: [root@vmhost2 ~]# virsh nodedev-dumpxml pci_0000_01_00_0 <device> <name>pci_0000_01_00_0</name> <path>/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0</path> <parent>pci_0000_00_01_0</parent> <driver> <name>vfio-pci</name> </driver> <capability type='pci'> <domain>0</domain> <bus>1</bus> <slot>0</slot> <function>0</function> <product id='0x4000' /> <vendor id='0x19ee'>Netronome Systems, Inc.</vendor> <capability type='virt_functions' maxCount='64'/> <iommuGroup number='1'> <address domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </iommuGroup> <pci-express> <link validity='cap' port='0' speed='8' width='8'/> <link validity='sta' speed='2.5' width='8'/> </pci-express> </capability> </device> [root@vmhost2 ~]# And it starts without an issue: [root@vmhost2 ~]# virsh start testfedora Domain testfedora started [root@vmhost2 ~]# Inside the guest: [root@testfedora ~]# dmesg |grep -i netronome [ 12.327316] nfp: NFP PCIe Driver, Copyright (C) 2014-2017 Netronome Systems [ 12.335036] nfp 0000:07:00.0: Netronome Flow Processor NFP4000/NFP5000/NFP6000 PCIe Card Probe [root@testfedora ~]# so I do not know what is going on.> Cheers, > > - Peter >
Apparently Analagous Threads
- Re: Not able to add pcie card to guest: Operation not permitted
- Not able to add pcie card to guest: Operation not permitted
- systemctl behaves like it is being piped to less in centos 8?
- systemctl behaves like it is being piped to less in centos 8?
- NOTIFYCMD not running