I have been investigating pci pass-through for virtualized
guests and the documentation I have found seems to me to
lack a certain consistency in its example. This may be
due to my not understanding what it is trying to inform
me.
What I wish to do is to configure a pci multi-port serial
i/o card for use by a single virtual host.
I start by running lspci -v on the host to identify the
serial card:
03:00.0 Serial controller: Oxford Semiconductor Ltd
OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06
[16950])
Subsystem: Oxford Semiconductor Ltd Device 0000
Flags: medium devsel, IRQ 17
I/O ports at d040 [size=32]
Memory at d0702000 (32-bit, non-prefetchable)
[size=4K]
I/O ports at d020 [size=32]
Memory at d0701000 (32-bit, non-prefetchable)
[size=4K]
Capabilities: [40] Power Management version 2
Kernel driver in use: serial
I then check for possible multiple IRQ assignment:
lspci -v | grep ' IRQ 17'
Flags: medium devsel, IRQ 17
I next use lspci -n to identify the vendor codes:
lspci -n | grep '00:03.0'
00:03.0 0780: 8086:2e24 (rev 03)
So this is an Intel chipset and the device id is 2e24.
Now this is the point in the example where the
documentation and I part company. In the examples I have
found, although the pci device ids listed from virsh
nodedev-list are uniformly of the form pci_0000_00_03_0
those used in the examples then switch and use the form
pci_8086_3a6c for the subsequent steps.
This pattern appears to be the prefix pci followed by the
manufacturer's code followed by the device id. There is no
other mapping to the pci device ids previously reported by
virsh nodedev-list and lspci in the examples that I can
discern.
However, if I attempt to use the manufacturer and device
ids in the next step of the example, substituting those
used in the example with those reported on my own system,
then I get a device not found reported:
virsh nodedev-dumpxml pci_8086_2e24
error: Could not find matching device 'pci_8086_2e24'
error: Node device not found
If instead I use the pci device ids exactly as reported by
virsh nodedev-list then I get what I expect:
virsh nodedev-dumpxml pci_0000_00_03_0
<device>
<name>pci_0000_00_03_0</name>
<parent>computer</parent>
<capability type='pci'>
<domain>0</domain>
<bus>0</bus>
<slot>3</slot>
<function>0</function>
<product id='0x2e24'>4 Series Chipset HECI
Controller</product>
<vendor id='0x8086'>Intel Corporation</vendor>
<capability type='virt_functions'>
</capability>
</capability>
</device>
My question is: Why does the documentation change the form
of the pci identifiers used in the second half of the
example from those reported previously in the same
example? Is this change significant? What does it mean?
Am I missing something important here?
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:ByrneJB at Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3
On Fri, February 3, 2012 10:34, James B. Byrne wrote:> I have been investigating pci pass-through for virtualized > guests and the documentation I have found seems to me to > lack a certain consistency in its example. This may be > due to my not understanding what it is trying to inform > me. >Following along in the example and working on the basis that what is reported by virsh nodedev-dumpxml pci_0000_00_03_0 is what I should use I get to the point where I try the readline command, and discover there is no such link. Proceeding past this point I detach the pci device from the host. I then edit the guest config file as given in the example only to discover that the changes are not saved. it reported that the configuration was edted by opening the configuration in virtsh edit a second time shows that the added <hostdev></hostdev> section was not saved. Perhaps this is because the example provides no context as to where it is nested within the xml file and my placing it directly within the <domain> </domain> tags is invalid. Further, the requirement to set setsebool -P virt_manage_sysfs 1 cannot be met since an SELinux boolean of that name does not exist (apparently it was renamed to virt_use_nfs). The reference I am using is found at: http://www.linuxtopia.org/online_books/rhel6/rhel_6_virtualization/rhel_6_virtualization_chap-Virtualization-PCI_passthrough.html However, I do not think that I can recommend it based on my experience. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB at Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
On Sat, February 4, 2012 10:39, Nenad Opsenica wrote:> > > On 02/03/2012 05:32 PM, James B. Byrne wrote: >> Where does this go inside the rest of the guest >> configuration? > > virt-manager GUI places PCI device pass-through inside > <devices> ...http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/chap-Virtualization_Host_Configuration_and_Guest_Installation_Guide-PCI_Assignment.html I have followed the instructions in the RedHat reference above to the best of my ability to understand them. I add the pci multi-port serial io card through virt-manager and it was indeed entered into the virtual machine's configuration file inside the <devices> tag: <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </hostdev> However, when I try and start the virtual machine I get this error: Error starting domain: internal error Unable to reset PCI device 0000:00:03.0: no FLR, PM reset or bus reset available Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 44, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/asyncjob.py", line 65, in tmpcb callback(*args, **kwargs) File "/usr/share/virt-manager/virtManager/domain.py", line 1050, in startup self._backend.create() File "/usr/lib64/python2.6/site-packages/libvirt.py", line 511, in create if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self) libvirtError: internal error Unable to reset PCI device 0000:00:03.0: no FLR, PM reset or bus reset available The steps I followed were: 1. Check VT-D extensions available and enabled in BIOS - yes 2. Restart virtual host - yes 3. Identify device - yes <address domain='0x000' bus='0x00' slot='0x03' function='0x0'/> 4. Add device to virtual machine configuration - yes . . . <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> </domain> 5. Enable SELinux boolean - yes getsebool virt_use_sysfs virt_use_sysfs --> on 6. Start virtual machine - fails Am I making any obvious errors? Has anyone here configured and managed to get a multi-port serial card working with a virtual guest? -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB at Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
On Mon, February 6, 2012 18:05, Ken Bass wrote:> Take a look at http://wiki.xensource.com/xenwiki/VTdHowTo > > Two things in particular about PCI passthrough: > > - Only devices with FLR capabilities are supported. > - Some motherboards are buggy. They advertised that they > support Vt-d > but do not correctly handle it (those with a broken ACPI > DMAR table) > > I think lspci -vv will tell you if the device supports > FLR. It will show > 'FLReset+' I believe. >03:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950]) Subsystem: Oxford Semiconductor Ltd Device 0000 Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 17 Region 0: I/O ports at d040 [size=32] Region 1: Memory at d0702000 (32-bit, non-prefetchable) [size=4K] Region 2: I/O ports at d020 [size=32] Region 3: Memory at d0701000 (32-bit, non-prefetchable) [size=4K] Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: serial No FLR string is present. So pci pass through is a dead end I take it? I increased the number of uarts available to the host system at boot with the 8250.nr_uarts=10 option. This gives me the following: # setserial -g /dev/ttyS* /dev/ttyS0, UART: 16550A, Port: 0x03f8, IRQ: 4 /dev/ttyS1, UART: unknown, Port: 0x02f8, IRQ: 3 /dev/ttyS2, UART: unknown, Port: 0x03e8, IRQ: 4 /dev/ttyS3, UART: unknown, Port: 0x02e8, IRQ: 3 /dev/ttyS4, UART: 16950/954, Port: 0xd040, IRQ: 17 /dev/ttyS5, UART: 16950/954, Port: 0xd048, IRQ: 17 /dev/ttyS6, UART: 16950/954, Port: 0xd050, IRQ: 17 /dev/ttyS7, UART: 16950/954, Port: 0xd058, IRQ: 17 /dev/ttyS8, UART: unknown, Port: 0x0000, IRQ: 0 /dev/ttyS9, UART: unknown, Port: 0x0000, IRQ: 0 With this change I now can add one serial port (/dev/ttyS4) to a virtual guest using virt-manager and have the guest start, but no more than one. Any more that one and the guest fails to run with the same irq conflict error as before. I still have not tried to see if the serial port actually works in this case, just that the system starts. I ran across this thread relating to serial devices in qemu from some time ago: http://www.mail-archive.com/qemu-devel at nongnu.org/msg27354.html Which seems to me to imply that it is not possible for a qemu guest to have more than 2 serial ports, one of which I gather has to be the console. However, this statement attracted my attention:> This is wrong. Two devices should never be manipulating > the same qemu_irq object. If you want multiple devices > connected to the same IRQ then you need an explicit > multiplexer. e.g. arm_timer.c:sp804_set_irq.And in a later message in the same thread:> Two devices have the same s->irq. Give each on its own > qemu_irq, and feed it into a multiplexer that ORs them > together and sends the result to the interrupt > controller's qemu_irq:Is there a way to set irqs in quem to map to specific ports on a pci card as this seems to imply? How is it done? -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB at Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3