Hello!!! Does pci hotplug work with pci passthrough in xcp? Currently, I have 2 VMs and do the following to transfer pci device from one domU to another 1. shutdown vm1 2. shutdown vm2 2. in dom0, xe vm-param-set uuid=<vm1-uuid> other-config:pci="" 3. in dom0, xe vm-param-set uuid=<vm2-uuid> other-config:pci=<pci-dev> 4. start vm1 5. start vm2 I want to avoid VM shutdown/start. is there a way to hot unplug from vm1 and hot plug into vm2 in xcp? Thanks _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, Currently the only way to hotplug is via the ''xenops'' commandline tool. Try: # /opt/xensource/debug/xenops plug_pci -domid .. -pci .. -devid 0 And similarly for unplug_pci Cheers, Dave From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Ritu kaur Sent: 06 April 2010 18:04 To: xen-devel@lists.xensource.com Subject: [Xen-devel] pci hotplug in xcp Hello!!! Does pci hotplug work with pci passthrough in xcp? Currently, I have 2 VMs and do the following to transfer pci device from one domU to another 1. shutdown vm1 2. shutdown vm2 2. in dom0, xe vm-param-set uuid=<vm1-uuid> other-config:pci="" 3. in dom0, xe vm-param-set uuid=<vm2-uuid> other-config:pci=<pci-dev> 4. start vm1 5. start vm2 I want to avoid VM shutdown/start. is there a way to hot unplug from vm1 and hot plug into vm2 in xcp? Thanks _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Currently the only way to hotplug is via the ''xenops'' commandline tool. > Try: > > # /opt/xensource/debug/xenops plug_pci -domid .. -pci .. -devid 0 > > And similarly for unplug_pciIt would be a good ''starter'' project for someone to add hotplug support to the API -- I''m sure there are folk on the list that would be happy to help with the design. Any takers? Thanks, Ian _______________________________________________ xen-api mailing list xen-api@lists.xensource.com http://lists.xensource.com/mailman/listinfo/xen-api
On Tue, 6 Apr 2010, Dave Scott wrote:> > Hi, > > Currently the only way to hotplug is via the ‘xenops’ commandline tool. Try: > > # /opt/xensource/debug/xenops plug_pci –domid .. –pci .. –devid 0 > > And similarly for unplug_pcixl is also present is XCP so the following should also work: xl pci-attach <domid> <BDF> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
xl pci-attach doesn''t seem to work unless vm is rebooted. Steps followed 1. modprobe pciback on dom0 2. execute xl pci-attach 1 0000:08:01.0, following msgs seen... [-1] libxl.c:2061:libxl_device_pci_add: xc_assign_device failed: Success [3] libxl.c:1763:libxl_create_pci_backend: Creating pci backend 3. xl pci-list 1 VFn domain bus slot func 0x00 0x0000 0x08 0x01 0x0 4. login into vm, lspci doesn''t show anything. 5. vm:~# lspci vm:~# 6. reboot vm, do lspci again it works. vm:~# lspci 00:00.0 Ethernet controller: Device feed:face I want to avoid vm reboot? Thanks On Wed, Apr 7, 2010 at 4:09 AM, Stefano Stabellini < stefano.stabellini@eu.citrix.com> wrote:> On Tue, 6 Apr 2010, Dave Scott wrote: > > > > Hi, > > > > Currently the only way to hotplug is via the ‘xenops’ commandline tool. > Try: > > > > # /opt/xensource/debug/xenops plug_pci –domid .. –pci .. –devid 0 > > > > And similarly for unplug_pci > > xl is also present is XCP so the following should also work: > > xl pci-attach <domid> <BDF> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 7 Apr 2010, Ritu kaur wrote:> xl pci-attach doesn''t seem to work unless vm is rebooted. Steps followed > > 1. modprobe pciback on dom0did you bind the device to pciback before executing "xl pci-attach"? xl doesn''t do it for you. Beside you need to add -priv to the qemu command line options and to do that you need to edit /opt/xensource/libexec/qemu-dm-wrapper for example adding qemu_args.append("-priv") right after qemu_args.append("pty") sorry for not mentioning this before :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On Wed, 7 Apr 2010, Ritu kaur wrote: > > xl pci-attach doesn''t seem to work unless vm is rebooted. Steps followed > > > > 1. modprobe pciback on dom0 > > did you bind the device to pciback before executing "xl pci-attach"? > xl doesn''t do it for you. > Beside you need to add -priv to the qemu command line options and to do > that you need to edit /opt/xensource/libexec/qemu-dm-wrapper for example > adding > > qemu_args.append("-priv") > > right after > > qemu_args.append("pty") > > sorry for not mentioning this before :)If you want to use hotplug rather than static assignment, you should make sure that the guest has a "pci" key (even an empty one) as this causes xapi to take care of setting the guest up for PCI passthrough, so you don''t have to manually edit anything. E.g.: xe vm-param-set other-config:pci="" uuid=$VMUUID Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks Stefano. I am having problems binding device to pciback(this is from day1). Driver for this device is not loaded in dom0 so this device is not bound to any driver. #echo $pcidev 0000:08:01.0 # echo -n $pcidev > /sys/bus/pci/drivers/pciback/bind -bash: echo: write error: No such device ls -l /sys/bus/pci/drivers/pciback/bind --w------- 1 root root 4096 Apr 8 07:40 /sys/bus/pci/drivers/pciback/bind Any inputs? Thanks On Thu, Apr 8, 2010 at 3:55 AM, Stefano Stabellini < stefano.stabellini@eu.citrix.com> wrote:> On Wed, 7 Apr 2010, Ritu kaur wrote: > > xl pci-attach doesn''t seem to work unless vm is rebooted. Steps followed > > > > 1. modprobe pciback on dom0 > > did you bind the device to pciback before executing "xl pci-attach"? > xl doesn''t do it for you. > Beside you need to add -priv to the qemu command line options and to do > that you need to edit /opt/xensource/libexec/qemu-dm-wrapper for example > adding > > qemu_args.append("-priv") > > right after > > qemu_args.append("pty") > > sorry for not mentioning this before :) > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks Ian. I have other-config:pci="" set in both my VM''s. Once I have pci device binding issue solved I will try this out. Thanks On Thu, Apr 8, 2010 at 5:35 AM, Ian Pratt <Ian.Pratt@eu.citrix.com> wrote:> > On Wed, 7 Apr 2010, Ritu kaur wrote: > > > xl pci-attach doesn''t seem to work unless vm is rebooted. Steps > followed > > > > > > 1. modprobe pciback on dom0 > > > > did you bind the device to pciback before executing "xl pci-attach"? > > xl doesn''t do it for you. > > Beside you need to add -priv to the qemu command line options and to do > > that you need to edit /opt/xensource/libexec/qemu-dm-wrapper for example > > adding > > > > qemu_args.append("-priv") > > > > right after > > > > qemu_args.append("pty") > > > > sorry for not mentioning this before :) > > If you want to use hotplug rather than static assignment, you should make > sure that the guest has a "pci" key (even an empty one) as this causes xapi > to take care of setting the guest up for PCI passthrough, so you don''t have > to manually edit anything. E.g.: > xe vm-param-set other-config:pci="" uuid=$VMUUID > > > > Ian > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 8 Apr 2010, Ritu kaur wrote:> Thanks Stefano. > > I am having problems binding device to pciback(this is from day1). Driver for this device is not loaded in dom0 so this > device is not bound to any driver. > > #echo $pcidev > 0000:08:01.0 > > # echo -n $pcidev > /sys/bus/pci/drivers/pciback/bind > -bash: echo: write error: No such device > > ls -l /sys/bus/pci/drivers/pciback/bind > --w------- 1 root root 4096 Apr 8 07:40 /sys/bus/pci/drivers/pciback/bind > > Any inputs? >you need to do this: echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/new_slot echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/bind _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks Stefano. It did work fine first time however subsequent attach/detach doesn''t work. What I did 1. (VM has other-config:pci="" as suggested by Ian) 2. modprobe pciback 3. echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/new_slot echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/bind 4. start VM 5. xl pci-attach 1 0000:08:01.0(Cross checked in VM can see device via lspci) 6. xl pci-detach 1 0000:08:01.0(Cross checked in VM, no device via lspci) 7. xl pci-attach 1 0000:08:01.0(lspci in VM doesn''t list the device) 8. xl pci-list 1 in dom0 lists the device 9. When try to detach I get xl pci-detach 1 0000:08:01.0 [0] libxl_device.c:444:libxl_wait_for_backend: Backend /local/domain/0/backend/pci/1/0 not ready [3] libxl.c:1881:libxl_device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/1/0 is not ready [-1] libxl.c:2151:libxl_device_pci_remove: xc_deassign_device failed: Success dmesg in dom0 has ... pciback: vpci: 0000:08:01.0: assign to virtual slot 0 pciback: vpci: 0000:08:01.0: assign to virtual slot 0 nothing related to detach. Do I need to run any other command prior to subsequent attach? Thanks On Thu, Apr 8, 2010 at 7:47 AM, Stefano Stabellini < stefano.stabellini@eu.citrix.com> wrote:> On Thu, 8 Apr 2010, Ritu kaur wrote: > > Thanks Stefano. > > > > I am having problems binding device to pciback(this is from day1). Driver > for this device is not loaded in dom0 so this > > device is not bound to any driver. > > > > #echo $pcidev > > 0000:08:01.0 > > > > # echo -n $pcidev > /sys/bus/pci/drivers/pciback/bind > > -bash: echo: write error: No such device > > > > ls -l /sys/bus/pci/drivers/pciback/bind > > --w------- 1 root root 4096 Apr 8 07:40 > /sys/bus/pci/drivers/pciback/bind > > > > Any inputs? > > > > you need to do this: > > echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/new_slot > echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/bind >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Please post the qemu logs (in XCP they go to syslog, so /var/log/messages and /tmp/qemu.$domid), they''ll probably reveal an error. What is the guest operating system? If it is Linux make sure acpiphp is loaded. On Thu, 8 Apr 2010, Ritu kaur wrote:> Thanks Stefano. It did work fine first time however subsequent attach/detach doesn''t work. What I did > > 1. (VM has other-config:pci="" as suggested by Ian) > > 2. modprobe pciback > > 3. echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/new_slot echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/bind > > 4. start VM > > 5. xl pci-attach 1 0000:08:01.0(Cross checked in VM can see device via lspci) > > 6. xl pci-detach 1 0000:08:01.0(Cross checked in VM, no device via lspci) > > 7. xl pci-attach 1 0000:08:01.0(lspci in VM doesn''t list the device) > > 8. xl pci-list 1 in dom0 lists the device > > 9. When try to detach I get > xl pci-detach 1 0000:08:01.0 > > [0] libxl_device.c:444:libxl_wait_for_backend: Backend /local/domain/0/backend/pci/1/0 not ready > [3] libxl.c:1881:libxl_device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/1/0 is not ready > [-1] libxl.c:2151:libxl_device_pci_remove: xc_deassign_device failed: Success > > dmesg in dom0 has > > ... > pciback: vpci: 0000:08:01.0: assign to virtual slot 0 > pciback: vpci: 0000:08:01.0: assign to virtual slot 0 > > nothing related to detach. Do I need to run any other command prior to subsequent attach? > > Thanks > > On Thu, Apr 8, 2010 at 7:47 AM, Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote: > On Thu, 8 Apr 2010, Ritu kaur wrote: > > Thanks Stefano. > > > > I am having problems binding device to pciback(this is from day1). Driver for this device is not loaded in > dom0 so this > > device is not bound to any driver. > > > > #echo $pcidev > > 0000:08:01.0 > > > > # echo -n $pcidev > /sys/bus/pci/drivers/pciback/bind > > -bash: echo: write error: No such device > > > > ls -l /sys/bus/pci/drivers/pciback/bind > > --w------- 1 root root 4096 Apr 8 07:40 /sys/bus/pci/drivers/pciback/bind > > > > Any inputs? > > > > you need to do this: > > echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/new_slot > echo -n "$pcidev" > /sys/bus/pci/drivers/pciback/bind > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
[This email is either empty or too large to be displayed at this time]
On Thu, 8 Apr 2010, Ritu kaur wrote:> 3. in my xcp dom0 I do not see /tmp/qemu.<domid> files, what I have > > # ls -l > total 8 > drwx------ 3 root root 4096 Mar 11 10:21 lvm > -rw-r--r-- 1 root root 0 Apr 8 09:25 master > -rw-r--r-- 1 root root 848 Apr 8 09:27 xenguest.log > > xenguest.log doesn''t reveal anything. >they are deleted when the VM shut down> 4. attached /var/log/messages. >unfortunately the interesting stuff was already rotated somewhere else _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Guest VM''s are still running, they are not shutdown. I am just doing multiple attach/detach sequence. Thanks On Thu, Apr 8, 2010 at 10:59 AM, Stefano Stabellini < stefano.stabellini@eu.citrix.com> wrote:> On Thu, 8 Apr 2010, Ritu kaur wrote: > > 3. in my xcp dom0 I do not see /tmp/qemu.<domid> files, what I have > > > > # ls -l > > total 8 > > drwx------ 3 root root 4096 Mar 11 10:21 lvm > > -rw-r--r-- 1 root root 0 Apr 8 09:25 master > > -rw-r--r-- 1 root root 848 Apr 8 09:27 xenguest.log > > > > xenguest.log doesn''t reveal anything. > > > > they are deleted when the VM shut down > > > > 4. attached /var/log/messages. > > > > unfortunately the interesting stuff was already rotated somewhere > else > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel