Hi, where can I find out about the experimental PCI pass-through in XCP? I do not seem to be able to find any documentation. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, The only documentation is currently in the code ☹ Try a command like: xe vm-param-set uuid=<uuid> other-config:pci=0/0000:00:00:0 and then start the VM. (Change that PCI bus path to be the one you actually want to use) http://xenbits.xen.org/xapi/xen-api.hg?file/a8decf9d3bee/ocaml/xapi/vmops.ml (line 483, pcidevs_of_vm) Cheers, Dave From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of James Troup Sent: 24 February 2010 07:18 To: xen-users Subject: [Xen-users] XCP: PCI pass through Howto Hi, where can I find out about the experimental PCI pass-through in XCP? I do not seem to be able to find any documentation. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Dave, is there any more to it than that? Do I have to hide the device from, dom0. I tried as you suggested but I cannot see the device in domU Am I missing something? Does this rely on VT-d ? Does it only apply to HVM? Thanks, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: "xen-users@lists.xensource.com" <xen-users@lists.xensource.com> Sent: Wednesday, 24 February, 2010 8:33:22 PM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi, The only documentation is currently in the code L Try a command like: xe vm-param-set uuid=<uuid> other-config:pci=0/0000:00:00:0 and then start the VM. (Change that PCI bus path to be the one you actually want to use) http://xenbits.xen.org/xapi/xen-api.hg?file/a8decf9d3bee/ocaml/xapi/vmops.ml (line 483, pcidevs_of_vm) Cheers, Dave From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of James Troup Sent: 24 February 2010 07:18 To: xen-users Subject: [Xen-users] XCP: PCI pass through Howto Hi, where can I find out about the experimental PCI pass-through in XCP? I do not seem to be able to find any documentation. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi James, That should be it… but it is a bit experimental ☺ The device should be hidden from dom0 when the VM is started. I imagine this might fail if the device is in use. I last tested it with HVM (SLES10 IIRC) on a VT-d system, after adding ioemmu=1 onto the xen commandline (it’s not the default). It ought to work with PV as-is. Could you give more details about what you tried? You might find useful information in /var/log/xensource.log, especially if you grep for “PCI”. Cheers, Dave From: James Troup [mailto:james@linuxterminal.com] Sent: 24 February 2010 12:58 To: Dave Scott Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] XCP: PCI pass through Howto Hi Dave, is there any more to it than that? Do I have to hide the device from, dom0. I tried as you suggested but I cannot see the device in domU Am I missing something? Does this rely on VT-d ? Does it only apply to HVM? Thanks, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: "xen-users@lists.xensource.com" <xen-users@lists.xensource.com> Sent: Wednesday, 24 February, 2010 8:33:22 PM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi, The only documentation is currently in the code ☹ Try a command like: xe vm-param-set uuid=<uuid> other-config:pci=0/0000:00:00:0 and then start the VM. (Change that PCI bus path to be the one you actually want to use) http://xenbits.xen.org/xapi/xen-api.hg?file/a8decf9d3bee/ocaml/xapi/vmops.ml (line 483, pcidevs_of_vm) Cheers, Dave From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of James Troup Sent: 24 February 2010 07:18 To: xen-users Subject: [Xen-users] XCP: PCI pass through Howto Hi, where can I find out about the experimental PCI pass-through in XCP? I do not seem to be able to find any documentation. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Dave, Here is the output when I attempt PCI passthrough... [20100224T21:39:31.973Z|debug|lts-xen1|5228 unix-RPC|VM.start R:6a8f8407bbc4|xenops] pci: unbounding device 0000:00:12.0 from driver ohci_hcd [20100224T21:39:31.975Z|debug|lts-xen1|5228 unix-RPC|VM.start R:6a8f8407bbc4|helpers] Ignoring exception: INTERNAL_ERROR: [ Unix.Unix_error(20, "open", "/sys/bus/pci/drivers/pciback/new_slot") ] while attach_pcis Noted that "/sys/bus/pci/drivers/pciback" directory did not exist. Is it supposed to be created by this action? ...after poking around a bit I realized you would need the pciback module loaded. So the xensource.log now reveals... [20100224T22:21:20.182Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] Doing FLR on pci device: 0000:00:12.0 [20100224T22:21:20.193Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] pci add mem bar fe02e000-fe02efff [20100224T22:21:20.193Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] adding device B0[/local/domain/0/backend/pci/23/0] F23[/local/domain/23/device/pci/0] H[/xapi/23/hotplug/pci/0] There is nothing attached to the PCI device. Inside the domU dmesg reveals... [ 0.004928] Booting paravirtualized kernel on Xen [ 0.004930] Xen version: 3.4.2 (preserve-AD) [ 0.005132] Grant table initialized [ 0.005135] regulator: core version 0.5 [ 0.005214] NET: Registered protocol family 16 [ 0.005425] PCI: Fatal: No config space access function found [ 0.005425] bio: create slab <bio-0> at 0 [ 0.005425] ACPI: Interpreter disabled. [ 0.005425] xen_balloon: Initialising balloon driver. [ 0.008183] PCI: System does not support PCI [ 0.008185] PCI: System does not support PCI [ 0.008374] pnp: PnP ACPI: disabled [ 0.008376] PnPBIOS: Disabled [ 0.008799] NET: Registered protocol family 2 [ 0.008869] IP route cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.009068] TCP established hash table entries: 8192 (order: 4, 65536 bytes) [ 0.009094] TCP bind hash table entries: 8192 (order: 4, 65536 bytes) [ 0.009119] TCP: Hash tables configured (established 8192 bind 8192) [ 0.009122] TCP reno registered [ 0.009180] NET: Registered protocol family 1 [ 0.039675] XENBUS: Device with no driver: device/vbd/51776 [ 0.039678] XENBUS: Device with no driver: device/vbd/51712 [ 0.039680] XENBUS: Device with no driver: device/vif/0 [ 0.039682] XENBUS: Device with no driver: device/pci/0 Is there USB passthrough in the pipeline? I am actually trying to pass the USB controller. Thanks for the help, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: xen-users@lists.xensource.com Sent: Thursday, 25 February, 2010 5:00:10 AM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi James, That should be it… but it is a bit experimental J The device should be hidden from dom0 when the VM is started. I imagine this might fail if the device is in use. I last tested it with HVM (SLES10 IIRC) on a VT-d system, after adding ioemmu=1 onto the xen commandline (it’s not the default). It ought to work with PV as-is. Could you give more details about what you tried? You might find useful information in /var/log/xensource.log, especially if you grep for “PCI”. Cheers, Dave From: James Troup [mailto:james@linuxterminal.com] Sent: 24 February 2010 12:58 To: Dave Scott Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] XCP: PCI pass through Howto Hi Dave, is there any more to it than that? Do I have to hide the device from, dom0. I tried as you suggested but I cannot see the device in domU Am I missing something? Does this rely on VT-d ? Does it only apply to HVM? Thanks, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: "xen-users@lists.xensource.com" <xen-users@lists.xensource.com> Sent: Wednesday, 24 February, 2010 8:33:22 PM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi, The only documentation is currently in the code L Try a command like: xe vm-param-set uuid=<uuid> other-config:pci=0/0000:00:00:0 and then start the VM. (Change that PCI bus path to be the one you actually want to use) http://xenbits.xen.org/xapi/xen-api.hg?file/a8decf9d3bee/ocaml/xapi/vmops.ml (line 483, pcidevs_of_vm) Cheers, Dave From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of James Troup Sent: 24 February 2010 07:18 To: xen-users Subject: [Xen-users] XCP: PCI pass through Howto Hi, where can I find out about the experimental PCI pass-through in XCP? I do not seem to be able to find any documentation. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi James, Loading the pciback module in dom0 was the right thing to do. I think this is automatically loaded in newer builds, so when I next refresh the .isos that step shouldn’t be necessary. Looking at your domU kernel dmesg it looks like you’re missing a set of PV drivers: netfront, blkfront and pcifront. What domU kernel version are you using? There are no immediate plans to add USB passthrough to XCP. It think passing through the controller (as you’re attempting to do) is your best bet. Cheers, Dave From: James Troup [mailto:james@linuxterminal.com] Sent: 24 February 2010 23:25 To: Dave Scott Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] XCP: PCI pass through Howto Hi Dave, Here is the output when I attempt PCI passthrough... [20100224T21:39:31.973Z|debug|lts-xen1|5228 unix-RPC|VM.start R:6a8f8407bbc4|xenops] pci: unbounding device 0000:00:12.0 from driver ohci_hcd [20100224T21:39:31.975Z|debug|lts-xen1|5228 unix-RPC|VM.start R:6a8f8407bbc4|helpers] Ignoring exception: INTERNAL_ERROR: [ Unix.Unix_error(20, "open", "/sys/bus/pci/drivers/pciback/new_slot") ] while attach_pcis Noted that "/sys/bus/pci/drivers/pciback" directory did not exist. Is it supposed to be created by this action? ...after poking around a bit I realized you would need the pciback module loaded. So the xensource.log now reveals... [20100224T22:21:20.182Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] Doing FLR on pci device: 0000:00:12.0 [20100224T22:21:20.193Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] pci add mem bar fe02e000-fe02efff [20100224T22:21:20.193Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] adding device B0[/local/domain/0/backend/pci/23/0] F23[/local/domain/23/device/pci/0] H[/xapi/23/hotplug/pci/0] There is nothing attached to the PCI device. Inside the domU dmesg reveals... [ 0.004928] Booting paravirtualized kernel on Xen [ 0.004930] Xen version: 3.4.2 (preserve-AD) [ 0.005132] Grant table initialized [ 0.005135] regulator: core version 0.5 [ 0.005214] NET: Registered protocol family 16 [ 0.005425] PCI: Fatal: No config space access function found [ 0.005425] bio: create slab <bio-0> at 0 [ 0.005425] ACPI: Interpreter disabled. [ 0.005425] xen_balloon: Initialising balloon driver. [ 0.008183] PCI: System does not support PCI [ 0.008185] PCI: System does not support PCI [ 0.008374] pnp: PnP ACPI: disabled [ 0.008376] PnPBIOS: Disabled [ 0.008799] NET: Registered protocol family 2 [ 0.008869] IP route cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.009068] TCP established hash table entries: 8192 (order: 4, 65536 bytes) [ 0.009094] TCP bind hash table entries: 8192 (order: 4, 65536 bytes) [ 0.009119] TCP: Hash tables configured (established 8192 bind 8192) [ 0.009122] TCP reno registered [ 0.009180] NET: Registered protocol family 1 [ 0.039675] XENBUS: Device with no driver: device/vbd/51776 [ 0.039678] XENBUS: Device with no driver: device/vbd/51712 [ 0.039680] XENBUS: Device with no driver: device/vif/0 [ 0.039682] XENBUS: Device with no driver: device/pci/0 Is there USB passthrough in the pipeline? I am actually trying to pass the USB controller. Thanks for the help, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: xen-users@lists.xensource.com Sent: Thursday, 25 February, 2010 5:00:10 AM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi James, That should be it… but it is a bit experimental ☺ The device should be hidden from dom0 when the VM is started. I imagine this might fail if the device is in use. I last tested it with HVM (SLES10 IIRC) on a VT-d system, after adding ioemmu=1 onto the xen commandline (it’s not the default). It ought to work with PV as-is. Could you give more details about what you tried? You might find useful information in /var/log/xensource.log, especially if you grep for “PCI”. Cheers, Dave From: James Troup [mailto:james@linuxterminal.com] Sent: 24 February 2010 12:58 To: Dave Scott Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] XCP: PCI pass through Howto Hi Dave, is there any more to it than that? Do I have to hide the device from, dom0. I tried as you suggested but I cannot see the device in domU Am I missing something? Does this rely on VT-d ? Does it only apply to HVM? Thanks, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: "xen-users@lists.xensource.com" <xen-users@lists.xensource.com> Sent: Wednesday, 24 February, 2010 8:33:22 PM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi, The only documentation is currently in the code ☹ Try a command like: xe vm-param-set uuid=<uuid> other-config:pci=0/0000:00:00:0 and then start the VM. (Change that PCI bus path to be the one you actually want to use) http://xenbits.xen.org/xapi/xen-api.hg?file/a8decf9d3bee/ocaml/xapi/vmops.ml (line 483, pcidevs_of_vm) Cheers, Dave From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of James Troup Sent: 24 February 2010 07:18 To: xen-users Subject: [Xen-users] XCP: PCI pass through Howto Hi, where can I find out about the experimental PCI pass-through in XCP? I do not seem to be able to find any documentation. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Dave, I am using the kernel on the xs-tools.iso that installs via the install.sh bash script. # dpkg -l |grep linux-image ii linux-image-2.6-686-bigmem 2.6.26+17+lenny1 Linux 2.6 image on PPro/Celeron/PII/PIII/P4 ii linux-image-2.6-xen 2.6.30+0.xs219 Linux 2.6 image on PPro/Celeron/PII/PIII/P4 ii linux-image-2.6.26-2-686-bigmem 2.6.26-19lenny2 Linux 2.6.26 image on PPro/Celeron/PII/PIII/ ii linux-image-2.6.30-2-686-bigmem 2.6.30-8.xs0.1.1.219 Linux 2.6.30 image on PPro/Celeron/PII/PIII/ # uname -a Linux faithx 2.6.30-2-686-bigmem #1 SMP Fri Jan 15 18:11:08 UTC 2010 i686 GNU/Linux ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: xen-users@lists.xensource.com Sent: Friday, 26 February, 2010 12:11:01 AM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi James, Loading the pciback module in dom0 was the right thing to do. I think this is automatically loaded in newer builds, so when I next refresh the .isos that step shouldn’t be necessary. Looking at your domU kernel dmesg it looks like you’re missing a set of PV drivers: netfront, blkfront and pcifront. What domU kernel version are you using? There are no immediate plans to add USB passthrough to XCP. It think passing through the controller (as you’re attempting to do) is your best bet. Cheers, Dave From: James Troup [mailto:james@linuxterminal.com] Sent: 24 February 2010 23:25 To: Dave Scott Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] XCP: PCI pass through Howto Hi Dave, Here is the output when I attempt PCI passthrough... [20100224T21:39:31.973Z|debug|lts-xen1|5228 unix-RPC|VM.start R:6a8f8407bbc4|xenops] pci: unbounding device 0000:00:12.0 from driver ohci_hcd [20100224T21:39:31.975Z|debug|lts-xen1|5228 unix-RPC|VM.start R:6a8f8407bbc4|helpers] Ignoring exception: INTERNAL_ERROR: [ Unix.Unix_error(20, "open", "/sys/bus/pci/drivers/pciback/new_slot") ] while attach_pcis Noted that "/sys/bus/pci/drivers/pciback" directory did not exist. Is it supposed to be created by this action? ...after poking around a bit I realized you would need the pciback module loaded. So the xensource.log now reveals... [20100224T22:21:20.182Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] Doing FLR on pci device: 0000:00:12.0 [20100224T22:21:20.193Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] pci add mem bar fe02e000-fe02efff [20100224T22:21:20.193Z|debug|lts-xen1|5857 unix-RPC|VM.start R:49d16bf207f8|xenops] adding device B0[/local/domain/0/backend/pci/23/0] F23[/local/domain/23/device/pci/0] H[/xapi/23/hotplug/pci/0] There is nothing attached to the PCI device. Inside the domU dmesg reveals... [ 0.004928] Booting paravirtualized kernel on Xen [ 0.004930] Xen version: 3.4.2 (preserve-AD) [ 0.005132] Grant table initialized [ 0.005135] regulator: core version 0.5 [ 0.005214] NET: Registered protocol family 16 [ 0.005425] PCI: Fatal: No config space access function found [ 0.005425] bio: create slab <bio-0> at 0 [ 0.005425] ACPI: Interpreter disabled. [ 0.005425] xen_balloon: Initialising balloon driver. [ 0.008183] PCI: System does not support PCI [ 0.008185] PCI: System does not support PCI [ 0.008374] pnp: PnP ACPI: disabled [ 0.008376] PnPBIOS: Disabled [ 0.008799] NET: Registered protocol family 2 [ 0.008869] IP route cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.009068] TCP established hash table entries: 8192 (order: 4, 65536 bytes) [ 0.009094] TCP bind hash table entries: 8192 (order: 4, 65536 bytes) [ 0.009119] TCP: Hash tables configured (established 8192 bind 8192) [ 0.009122] TCP reno registered [ 0.009180] NET: Registered protocol family 1 [ 0.039675] XENBUS: Device with no driver: device/vbd/51776 [ 0.039678] XENBUS: Device with no driver: device/vbd/51712 [ 0.039680] XENBUS: Device with no driver: device/vif/0 [ 0.039682] XENBUS: Device with no driver: device/pci/0 Is there USB passthrough in the pipeline? I am actually trying to pass the USB controller. Thanks for the help, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: xen-users@lists.xensource.com Sent: Thursday, 25 February, 2010 5:00:10 AM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi James, That should be it… but it is a bit experimental J The device should be hidden from dom0 when the VM is started. I imagine this might fail if the device is in use. I last tested it with HVM (SLES10 IIRC) on a VT-d system, after adding ioemmu=1 onto the xen commandline (it’s not the default). It ought to work with PV as-is. Could you give more details about what you tried? You might find useful information in /var/log/xensource.log, especially if you grep for “PCI”. Cheers, Dave From: James Troup [mailto:james@linuxterminal.com] Sent: 24 February 2010 12:58 To: Dave Scott Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] XCP: PCI pass through Howto Hi Dave, is there any more to it than that? Do I have to hide the device from, dom0. I tried as you suggested but I cannot see the device in domU Am I missing something? Does this rely on VT-d ? Does it only apply to HVM? Thanks, James ----- Original Message ----- From: "Dave Scott" <Dave.Scott@eu.citrix.com> To: "James Troup" <james@linuxterminal.com> Cc: "xen-users@lists.xensource.com" <xen-users@lists.xensource.com> Sent: Wednesday, 24 February, 2010 8:33:22 PM Subject: RE: [Xen-users] XCP: PCI pass through Howto Hi, The only documentation is currently in the code L Try a command like: xe vm-param-set uuid=<uuid> other-config:pci=0/0000:00:00:0 and then start the VM. (Change that PCI bus path to be the one you actually want to use) http://xenbits.xen.org/xapi/xen-api.hg?file/a8decf9d3bee/ocaml/xapi/vmops.ml (line 483, pcidevs_of_vm) Cheers, Dave From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of James Troup Sent: 24 February 2010 07:18 To: xen-users Subject: [Xen-users] XCP: PCI pass through Howto Hi, where can I find out about the experimental PCI pass-through in XCP? I do not seem to be able to find any documentation. Thanks. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users