Masaki Kanno
2009-May-14 12:59 UTC
[Xen-devel] [PATCH] Fix xm pci-attach/detach for inactive managed domains
Hi, Currently, xm pci-attach and xm pci-detach don''t work for inactive managed domains. This patch solves the problems. # xm new vm1 Using config file "/etc/xen/vm1". # xm pci-list vm1 VSlt domain bus slot func 0x6 0x0 0x01 0x00 0x0 0x7 0x0 0x01 0x00 0x1 # xm pci-attach vm1 00:1a.1 8 Error: ''vslot'' Usage: xm pci-attach [-o|--options=<opt>] <Domain> <domain:bus:slot.func> [virtual slot] Insert a new pass-through pci device. # xm pci-detach vm1 01:00.1 Error: ''vslot'' Usage: xm pci-detach <Domain> <domain:bus:slot.func> Remove a domain''s pass-through pci device. Cc: Simon Horman <horms@verge.net.au> Signed-off-by: Masaki Kanno <kanno.masaki@jp.fujitsu.com> Best regards, Kan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-May-15 00:30 UTC
[Xen-devel] Re: [PATCH] Fix xm pci-attach/detach for inactive managed domains
On Thu, May 14, 2009 at 09:59:55PM +0900, Masaki Kanno wrote: Content-Description: Mail message body> Hi, > > Currently, xm pci-attach and xm pci-detach don''t work for inactive > managed domains. This patch solves the problems. > > # xm new vm1 > Using config file "/etc/xen/vm1". > # xm pci-list vm1 > VSlt domain bus slot func > 0x6 0x0 0x01 0x00 0x0 > 0x7 0x0 0x01 0x00 0x1 > # xm pci-attach vm1 00:1a.1 8 > Error: ''vslot'' > Usage: xm pci-attach [-o|--options=<opt>] <Domain> <domain:bus:slot.func> [virtual slot] > > Insert a new pass-through pci device. > # xm pci-detach vm1 01:00.1 > Error: ''vslot'' > Usage: xm pci-detach <Domain> <domain:bus:slot.func> > > Remove a domain''s pass-through pci device.Good morning Kanno-san, unfortunately this change does not seem to be correct to me. In the case of pci-attach if I follow the folliwing steps: $ xm new hvm.conf $ xm pci-attach hvm 00:1a.0 $ xm start hvm [ wait a bit ] $ xm pci-list hvm Then pci-list never indicates that 00:1a.0 has been assigned a vslot. In the case of pci-detach, I am wondering if it makes any sense to detach a function from a paused domain as the guest os really needs to release the device for pci-detach to complete successfully. On an semi-related note, I am seeing an error when trying to connect to the console when I start a domain. I will investigate further. $ xm new hvm.conf $ xm start -c hvm /usr/lib64/xen/bin/xenconsole: invalid option -- 1 Invalid number of arguments Try `/usr/lib64/xen/bin/xenconsole --help'' for more information. and $ xm new hvm.conf $ xm start hvm $ sudo xm console debian [ after a few seconds ] xenconsole: Could not read tty from store: No such file or directory _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-May-15 00:58 UTC
[Xen-devel] Re: [PATCH] Fix xm pci-attach/detach for inactive managed domains
On Fri, May 15, 2009 at 10:30:15AM +1000, Simon Horman wrote:> On Thu, May 14, 2009 at 09:59:55PM +0900, Masaki Kanno wrote: > Content-Description: Mail message body > > Hi, > > > > Currently, xm pci-attach and xm pci-detach don''t work for inactive > > managed domains. This patch solves the problems. > > > > # xm new vm1 > > Using config file "/etc/xen/vm1". > > # xm pci-list vm1 > > VSlt domain bus slot func > > 0x6 0x0 0x01 0x00 0x0 > > 0x7 0x0 0x01 0x00 0x1 > > # xm pci-attach vm1 00:1a.1 8 > > Error: ''vslot'' > > Usage: xm pci-attach [-o|--options=<opt>] <Domain> <domain:bus:slot.func> [virtual slot] > > > > Insert a new pass-through pci device. > > # xm pci-detach vm1 01:00.1 > > Error: ''vslot'' > > Usage: xm pci-detach <Domain> <domain:bus:slot.func> > > > > Remove a domain''s pass-through pci device. > > Good morning Kanno-san, > > unfortunately this change does not seem to be correct to me. > > In the case of pci-attach if I follow the folliwing steps: > > $ xm new hvm.conf > $ xm pci-attach hvm 00:1a.0 > $ xm start hvm > [ wait a bit ] > $ xm pci-list hvm > > Then pci-list never indicates that 00:1a.0 has been assigned a vslot. > > In the case of pci-detach, I am wondering if it makes any > sense to detach a function from a paused domain as the guest os > really needs to release the device for pci-detach to complete successfully. > > > On an semi-related note, I am seeing an error when trying to connect > to the console when I start a domain. I will investigate further. > > $ xm new hvm.conf > $ xm start -c hvm > /usr/lib64/xen/bin/xenconsole: invalid option -- 1 > Invalid number of arguments > Try `/usr/lib64/xen/bin/xenconsole --help'' for more information.Sorry, this was not quite correct. The console seems to work but runs into trouble if the domain is restarted after pci-attach was performed while the guest was running. The following is without any patches applied. $ xm new hvm.conf $ xm start -c hvm [ wait a bit, guest boots] ^] [ back to host console ] $ xm pci-attach hvm 00:1a.0 $ xm pci-list hgm [ looks ok ] $ xm destroy debian $ xm start -c debian /usr/lib64/xen/bin/xenconsole: invalid option -- 1 Invalid number of arguments Try `/usr/lib64/xen/bin/xenconsole --help'' for more information. $ xm list ame ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 413.1 hvm 128 1 0.0 $ xm start -c debian xenconsole: Could not read tty from store: No such file or directory $ xm list xm list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 413.9 debian 17 128 1 ----c- 0.0 *sigh*, it feels like there are many bugs in there. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Masaki Kanno
2009-May-15 07:01 UTC
Re: [Xen-devel] Re: [PATCH] Fix xm pci-attach/detach for inactivemanaged domains
Hi Simon, Thanks for your review and test.>In the case of pci-attach if I follow the folliwing steps: > >$ xm new hvm.conf >$ xm pci-attach hvm 00:1a.0 >$ xm start hvm >[ wait a bit ] >$ xm pci-list hvm > >Then pci-list never indicates that 00:1a.0 has been assigned a vslot.Maybe, the domain start failed. Are there the following messages in a qemu-dm-hvm.log file? register_real_device: Assigning real physical device 00:1a.0 ... register_real_device: Error: no free virtual PCI slot pt_init: Error: Registration failed (00:1a.0) Error: Initialization failed for pass-through devices If so, the problem is solved by new patches. Could you review and test the patches? Signed-off-by: Masaki Kanno <kanno.masaki@jp.fujitsu.com> Best regards, Kan Current xen-unstable + my patches (xm_pci_attach_v2.patch + xm_pci_list_v3.patch): The test results are as follows. # cat /etc/xen/vm1 | grep pci pci = [ ''01:00.0@6'' ] # xm new vm1 Using config file "/etc/xen/vm1". # xm pci-list vm1 VSlt domain bus slot func 0x6 0x0 0x01 0x00 0x0 # xm pci-attach vm1 00:1a.0 8 # xm pci-list vm1 VSlt domain bus slot func 0x6 0x0 0x01 0x00 0x0 0x8 0x0 0x00 0x1a 0x0 # xm pci-detach vm1 01:00.0 # xm pci-list vm1 VSlt domain bus slot func 0x8 0x0 0x00 0x1a 0x0 # xm pci-detach vm1 00:1a.0 # xm pci-list vm1 # xm pci-attach vm1 01:00.0 6 # xm pci-attach vm1 00:1a.0 8 # xm pci-list vm1 VSlt domain bus slot func 0x6 0x0 0x01 0x00 0x0 0x8 0x0 0x00 0x1a 0x0 # xm pci-detach vm1 00:1a.0 # xm pci-list vm1 VSlt domain bus slot func 0x6 0x0 0x01 0x00 0x0 # xm pci-attach vm1 00:1a.0 # xm pci-list vm1 VSlt domain bus slot func 0x6 0x0 0x01 0x00 0x0 - 0x0 0x00 0x1a 0x0 # xm start vm1 # xm pci-list vm1 VSlt domain bus slot func 0x06 0x0 0x01 0x00 0x0 0x03 0x0 0x00 0x1a 0x0 Current xen-unstable: I got the following error messages from console and xend.log. # xm new vm1 Using config file "/etc/xen/vm1". # xm pci-list vm1 domain bus slot func 0x0 0x01 0x00 0x0 # xm pci-attach vm1 00:1a.0 8 Error: ''vslot'' Usage: xm pci-attach [-o|--options=<opt>] <Domain> <domain:bus:slot.func> [virtual slot] Insert a new pass-through pci device. # xm pci-detach vm1 01:00.0 Error: ''vslot'' Usage: xm pci-detach <Domain> <domain:bus:slot.func> Remove a domain''s pass-through pci device. [2009-05-15 13:26:09 4307] DEBUG (XendDomainInfo:779) XendDomainInfo.pci_device_configure: [\047pci\047, [\047dev\047, [\047domain\047, \0470x0\047], [\047bus\047, \0470x00\047], [\047slot\047, \0470x1a\047], [\047func\047, \0470x0\047], [\047vslot\047, \0470x8\047]], [\047state\047, \047Initialising\047]] [2009-05-15 13:26:09 4307] DEBUG (XendDomainInfo:606) XendDomainInfo.hvm_pci_device_create: {\047devs\047: [{\047slot\047: \0470x1a\047, \047domain\047: \0470x0\047, \047uuid\047: \04787122168-057f-dda6-a56d-1f37b8ddc466\047, \047vslot\047: \0470x8\047, \047bus\047: \0470x00\047, \047func\047: \0470x0\047}], \047states\047: [\047Initialising\047]} [2009-05-15 13:26:09 4307] ERROR (xmlrpclib2:178) Internal error handling xend.domain.device_configure Traceback (most recent call last): File "usr/lib64/python2.4/site-packages/xen/util/xmlrpclib2.py", line 131, in _marshaled_dispatch response = self._dispatch(method, params) File "/usr/lib64/python2.4/SimpleXMLRPCServer.py", line 406, in _dispatch return func(*params) File "<string>", line 1, in <lambda> File "usr/lib64/python2.4/site-packages/xen/xend/server/XMLRPCServer.py", line 55, in dispatch return getattr(info, fn)(*args) File "usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1040, in device_configure return self.pci_device_configure(dev_sxp) File "usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 801, in pci_device_configure vslot = self.hvm_pci_device_create(dev_config) File "usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 624, in hvm_pci_device_create if (int(x[\047vslot\047], 16) == int(new_dev[\047vslot\047], 16) and KeyError: \047vslot\047 [2009-05-15 13:26:39 4307] DEBUG (XendDomainInfo:779) XendDomainInfo.pci_device_configure: [\047pci\047, [\047dev\047, [\047domain\047, \0470x0\047], [\047bus\047, \0470x01\047], [\047slot\047, \0470x00\047], [\047func\047, \0470x0\047], [\047vslot\047, \0470x20\047]], [\047state\047, \047Closing\047]] [2009-05-15 13:26:39 4307] ERROR (xmlrpclib2:178) Internal error handling xend.domain.device_configure Traceback (most recent call last): File "usr/lib64/python2.4/site-packages/xen/util/xmlrpclib2.py", line 131, in _marshaled_dispatch response = self._dispatch(method, params) File "/usr/lib64/python2.4/SimpleXMLRPCServer.py", line 406, in _dispatch return func(*params) File "<string>", line 1, in <lambda> File "usr/lib64/python2.4/site-packages/xen/xend/server/XMLRPCServer.py", line 55, in dispatch return getattr(info, fn)(*args) File "usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1040, in device_configure return self.pci_device_configure(dev_sxp) File "usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 818, in pci_device_configure vslot = x[\047vslot\047] KeyError: \047vslot\047 Fri, 15 May 2009 10:30:15 +1000, Simon Horman wrote:>On Thu, May 14, 2009 at 09:59:55PM +0900, Masaki Kanno wrote: >Content-Description: Mail message body >> Hi, >> >> Currently, xm pci-attach and xm pci-detach don''t work for inactive >> managed domains. This patch solves the problems. >> >> # xm new vm1 >> Using config file "/etc/xen/vm1". >> # xm pci-list vm1 >> VSlt domain bus slot func >> 0x6 0x0 0x01 0x00 0x0 >> 0x7 0x0 0x01 0x00 0x1 >> # xm pci-attach vm1 00:1a.1 8 >> Error: ''vslot'' >> Usage: xm pci-attach [-o|--options=<opt>] <Domain> <domain:bus:slot.func> >> [virtual slot] >> >> Insert a new pass-through pci device. >> # xm pci-detach vm1 01:00.1 >> Error: ''vslot'' >> Usage: xm pci-detach <Domain> <domain:bus:slot.func> >> >> Remove a domain''s pass-through pci device. > >Good morning Kanno-san, > >unfortunately this change does not seem to be correct to me. > >In the case of pci-attach if I follow the folliwing steps: > >$ xm new hvm.conf >$ xm pci-attach hvm 00:1a.0 >$ xm start hvm >[ wait a bit ] >$ xm pci-list hvm > >Then pci-list never indicates that 00:1a.0 has been assigned a vslot. > >In the case of pci-detach, I am wondering if it makes any >sense to detach a function from a paused domain as the guest os >really needs to release the device for pci-detach to complete successfully. > > >On an semi-related note, I am seeing an error when trying to connect >to the console when I start a domain. I will investigate further. > >$ xm new hvm.conf >$ xm start -c hvm >/usr/lib64/xen/bin/xenconsole: invalid option -- 1 >Invalid number of arguments >Try `/usr/lib64/xen/bin/xenconsole --help'' for more information. > >and > >$ xm new hvm.conf >$ xm start hvm >$ sudo xm console debian >[ after a few seconds ] >xenconsole: Could not read tty from store: No such file or directory > > >_______________________________________________ >Xen-devel mailing list >Xen-devel@lists.xensource.com >http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-May-15 16:34 UTC
Re: [Xen-devel] Re: [PATCH] Fix xm pci-attach/detach for inactivemanaged domains
On Fri, May 15, 2009 at 04:01:30PM +0900, Masaki Kanno wrote: Content-Description: Mail message body> Hi Simon, > > Thanks for your review and test. > > >In the case of pci-attach if I follow the folliwing steps: > > > >$ xm new hvm.conf > >$ xm pci-attach hvm 00:1a.0 > >$ xm start hvm > >[ wait a bit ] > >$ xm pci-list hvm > > > >Then pci-list never indicates that 00:1a.0 has been assigned a vslot. > > Maybe, the domain start failed. > Are there the following messages in a qemu-dm-hvm.log file? > > register_real_device: Assigning real physical device 00:1a.0 ... > register_real_device: Error: no free virtual PCI slot > pt_init: Error: Registration failed (00:1a.0) > Error: Initialization failed for pass-through devices > > If so, the problem is solved by new patches. > Could you review and test the patches?Yes, I cam to the same conclusion. Though my solution was a little different. I think that the change below be an alternate solution to both of your patches. Though there is still a problem that the assigned vslot isn''t reported by pci-list if a function is attached while the domain isn''t running. I will review your patches once I have had some sleep. In the mean time, here is the change that have been thinking about. Index: xen-unstable.hg/tools/python/xen/xend/server/pciif.py ==================================================================--- xen-unstable.hg.orig/tools/python/xen/xend/server/pciif.py 2009-05-16 01:40:42.000000000 +1000 +++ xen-unstable.hg/tools/python/xen/xend/server/pciif.py 2009-05-16 01:41:14.000000000 +1000 @@ -73,13 +73,15 @@ class PciController(DevController): for pci_config in config.get(''devs'', []): vslot = pci_config.get(''vslot'') if vslot is not None: - vslots = vslots + vslot + ";" + vslot = parse_hex(vslot) + vslots = "%s0x%02x;" % (vslots, vslot) + else: + vslot = parse_hex(pci_config.get(''requested_vslot'')) domain = parse_hex(pci_config.get(''domain'', 0)) bus = parse_hex(pci_config.get(''bus'', 0)) slot = parse_hex(pci_config.get(''slot'', 0)) func = parse_hex(pci_config.get(''func'', 0)) - requested_vslot = parse_hex(pci_config.get(''requested_vslot'', 0)) opts = pci_config.get(''opts'', '''') if len(opts) > 0: @@ -90,7 +92,7 @@ class PciController(DevController): back[''dev-%i'' % pcidevid] = "%04x:%02x:%02x.%01x" % \ (domain, bus, slot, func) back[''uuid-%i'' % pcidevid] = pci_config.get(''uuid'', '''') - back[''vslot-%i'' % pcidevid] = "%02x" % requested_vslot + back[''vslot-%i'' % pcidevid] = "%02x" % vslot pcidevid += 1 if vslots != "": _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Simon Horman
2009-May-17 08:15 UTC
Re: [Xen-devel] Re: [PATCH] Fix xm pci-attach/detach for inactivemanaged domains
On Sat, May 16, 2009 at 02:34:30AM +1000, Simon Horman wrote:> On Fri, May 15, 2009 at 04:01:30PM +0900, Masaki Kanno wrote: > Content-Description: Mail message body > > Hi Simon, > > > > Thanks for your review and test. > > > > >In the case of pci-attach if I follow the folliwing steps: > > > > > >$ xm new hvm.conf > > >$ xm pci-attach hvm 00:1a.0 > > >$ xm start hvm > > >[ wait a bit ] > > >$ xm pci-list hvm > > > > > >Then pci-list never indicates that 00:1a.0 has been assigned a vslot. > > > > Maybe, the domain start failed. > > Are there the following messages in a qemu-dm-hvm.log file? > > > > register_real_device: Assigning real physical device 00:1a.0 ... > > register_real_device: Error: no free virtual PCI slot > > pt_init: Error: Registration failed (00:1a.0) > > Error: Initialization failed for pass-through devices > > > > If so, the problem is solved by new patches. > > Could you review and test the patches? > > Yes, I cam to the same conclusion. > Though my solution was a little different. > > I think that the change below be an alternate solution > to both of your patches. Though there is still a problem > that the assigned vslot isn''t reported by pci-list if > a function is attached while the domain isn''t running. > > I will review your patches once I have had some sleep. > In the mean time, here is the change that have been thinking about.Hi Kano-san, it turns out that my idea doesn''t work so well :-( But your patches seem to work for all the cases that I can dream up relating to starting and destroying domains while attaching and detaching pci devices. Acked-by: Simon Horman <horms@verge.net.au> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel