thr3ads.net - Xen devel - [Xen-devel] Re: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time [Jun 2009]

If this information is useful, please help other people find it:
Share via:

Cui, Dexuan

2009-Jun-12 05:51 UTC

[Xen-devel] Re: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Hi Simon,
After this changeset, I find there are some new issues in the xend:
I noticed in xend.log, setupOneDevice() is invoked twice, but actually I only
statically assign 1 device to hvm guest.

After looking into the xend code, I find in XendDomainInfo.py: _initDomain()
-> _createDevices(), we invoke self._createDevice(devclass, config) that
eventually invokes setupOneDevice() -- this is the first time;
And later, still in  _createDevices(), we invoke pci_device_configure_boot()
-> pci_device_configure() -> dev_control.reconfigureDevice(devid,
dev_config) -> xend/server/pciif.py:reconfigureDevice() ->
setupOneDevice()  --  this is the second time.  Can you remove the duplicate
invocation?

Thanks,
-- Dexuan



-----Original Message-----
From: xen-changelog-bounces@lists.xensource.com
[mailto:xen-changelog-bounces@lists.xensource.com] On Behalf Of Xen
patchbot-unstable
Sent: 2009?5?30? 17:30
To: xen-changelog@lists.xensource.com
Subject: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

# HG changeset patch
# User Keir Fraser <keir.fraser@citrix.com>
# Date 1243585922 -3600
# Node ID ec2bc4b9fa326048fefa81e3399e519e3902e15d
# Parent  ef6911934b6f7c85d51417455156466ff0507a56
xend: hot-plug PCI devices at boot-time

Currently there are two interfaces to pass-through PCI devices:
1. A method driven through per-device xenstore entries that is used at
boot-time
2. An event-based method used for hot-plug.

This seems somewhat redundant and makes extending the code cumbersome
and prone to error - often the change needs to be made twice, in
two different ways.

This patch unifies PCI pass-through by using the existing event-based
method at boot-time.

Signed-off-by: Simon Horman <horms@verge.net.au>
---
 tools/python/xen/xend/XendConfig.py     |   14 +++-
 tools/python/xen/xend/XendDomainInfo.py |   67 +++++++++++-----------
 tools/python/xen/xend/image.py          |   16 +++++
 tools/python/xen/xend/server/pciif.py   |   94 +++++++++++---------------------
 4 files changed, 96 insertions(+), 95 deletions(-)

diff -r ef6911934b6f -r ec2bc4b9fa32 tools/python/xen/xend/XendConfig.py
--- a/tools/python/xen/xend/XendConfig.py       Fri May 29 09:29:58 2009 +0100
+++ b/tools/python/xen/xend/XendConfig.py       Fri May 29 09:32:02 2009 +0100
@@ -1669,6 +1669,13 @@ class XendConfig(dict):

         return ''''

+    def pci_convert_dict_to_sxp(self, dev, state, sub_state = None):
+        sxp =  [''pci'', [''dev''] + map(lambda
(x, y): [x, y], dev.items()),
+                [''state'', state]]
+        if sub_state != None:
+            sxp.append([''sub_state'', sub_state])
+        return sxp
+
     def pci_convert_sxp_to_dict(self, dev_sxp):
         """Convert pci device sxp to dict
         @param dev_sxp: device configuration
@@ -1723,9 +1730,10 @@ class XendConfig(dict):
                     pci_dev_info[opt] = val
                 except (TypeError, ValueError):
                     pass
-            # append uuid for each pci device.
-            dpci_uuid = pci_dev_info.get(''uuid'',
uuid.createString())
-            pci_dev_info[''uuid''] = dpci_uuid
+            # append uuid to each pci device that does''t already have
one.
+            if not pci_dev_info.has_key(''uuid''):
+                dpci_uuid = pci_dev_info.get(''uuid'',
uuid.createString())
+                pci_dev_info[''uuid''] = dpci_uuid
             pci_devs.append(pci_dev_info)
         dev_config[''devs''] = pci_devs

diff -r ef6911934b6f -r ec2bc4b9fa32 tools/python/xen/xend/XendDomainInfo.py
--- a/tools/python/xen/xend/XendDomainInfo.py   Fri May 29 09:29:58 2009 +0100
+++ b/tools/python/xen/xend/XendDomainInfo.py   Fri May 29 09:32:02 2009 +0100
@@ -601,7 +601,7 @@ class XendDomainInfo:
         asserts.isCharConvertible(key)
         self.storeDom("control/sysrq", ''%c'' % key)

-    def sync_pcidev_info(self):
+    def pci_device_configure_boot(self):

         if not self.info.is_hvm():
             return
@@ -615,33 +615,12 @@ class XendDomainInfo:
         dev_uuid = sxp.child_value(dev_info, ''uuid'')
         pci_conf = self.info[''devices''][dev_uuid][1]
         pci_devs = pci_conf[''devs'']
-
-        count = 0
-        vslots = None
-        while vslots is None and count < 20:
-            vslots =
xstransact.Read("/local/domain/0/backend/pci/%u/%s/vslots"
-                              % (self.getDomid(), devid))
-            time.sleep(0.1)
-            count += 1
-        if vslots is None:
-            log.error("Device model didn''t tell the vslots for
PCI device")
-            return
-
-        #delete last delim
-        if vslots[-1] == ";":
-            vslots = vslots[:-1]
-
-        slot_list = vslots.split('';'')
-        if len(slot_list) != len(pci_devs):
-            log.error("Device model''s pci dev num dismatch")
-            return
-
-        #update the vslot info
-        count = 0;
-        for x in pci_devs:
-            x[''vslot''] = slot_list[count]
-            count += 1
-
+        request = map(lambda x:
+                      self.info.pci_convert_dict_to_sxp(x,
''Initialising'',
+                                                       
''Booting''), pci_devs)
+
+        for i in request:
+                self.pci_device_configure(i)

     def hvm_pci_device_create(self, dev_config):
         log.debug("XendDomainInfo.hvm_pci_device_create: %s"
@@ -741,6 +720,23 @@ class XendDomainInfo:
                     " assigned to other domain." \
                     )% (pci_device.name,
self.info[''name_label''], pci_str))

+        return self.hvm_pci_device_insert_dev(new_dev)
+
+    def hvm_pci_device_insert(self, dev_config):
+        log.debug("XendDomainInfo.hvm_pci_device_insert: %s"
+                  % scrub_password(dev_config))
+
+        if not self.info.is_hvm():
+            raise VmError("hvm_pci_device_create called on non-HVM
guest")
+
+        new_dev = dev_config[''devs''][0]
+
+        return self.hvm_pci_device_insert_dev(new_dev)
+
+    def hvm_pci_device_insert_dev(self, new_dev):
+        log.debug("XendDomainInfo.hvm_pci_device_insert_dev: %s"
+                  % scrub_password(new_dev))
+
         if self.domid is not None:
             opts = ''''
             if ''opts'' in new_dev and
len(new_dev[''opts'']) > 0:
@@ -752,7 +748,10 @@ class XendDomainInfo:
                 new_dev[''bus''],
                 new_dev[''slot''],
                 new_dev[''func''],
-                new_dev[''requested_vslot''],
+                # vslot will be used when activating a
+                # previously activated domain.
+                # Otherwise requested_vslot will be used.
+                assigned_or_requested_vslot(new_dev),
                 opts)
             self.image.signalDeviceModel(''pci-ins'',
''pci-inserted'', bdf_str)

@@ -827,6 +826,7 @@ class XendDomainInfo:
             return False

         pci_state = sxp.child_value(dev_sxp, ''state'')
+        pci_sub_state = sxp.child_value(dev_sxp, ''sub_state'')
         existing_dev_info = self._getDeviceInfo_pci(devid)

         if existing_dev_info is None and pci_state !=
''Initialising'':
@@ -840,7 +840,10 @@ class XendDomainInfo:
         if self.info.is_hvm():
             if pci_state == ''Initialising'':
                 # HVM PCI device attachment
-                vslot = self.hvm_pci_device_create(dev_config)
+                if pci_sub_state == ''Booting'':
+                    vslot = self.hvm_pci_device_insert(dev_config)
+                else:
+                    vslot = self.hvm_pci_device_create(dev_config)
                 # Update vslot
                 dev[''vslot''] = vslot
                 for n in sxp.children(pci_dev):
@@ -907,7 +910,7 @@ class XendDomainInfo:
                         continue
                 new_dev_sxp.append(cur_dev)

-            if pci_state == ''Initialising'':
+            if pci_state == ''Initialising'' and pci_sub_state
!= ''Booting'':
                 for new_dev in sxp.children(dev_sxp, ''dev''):
                     new_dev_sxp.append(new_dev)

@@ -2246,7 +2249,7 @@ class XendDomainInfo:
             self.image.createDeviceModel()

         #if have pass-through devs, need the virtual pci slots info from qemu
-        self.sync_pcidev_info()
+        self.pci_device_configure_boot()

     def _releaseDevices(self, suspend = False):
         """Release all domain''s devices.  Nothrow
guarantee."""
diff -r ef6911934b6f -r ec2bc4b9fa32 tools/python/xen/xend/image.py
--- a/tools/python/xen/xend/image.py    Fri May 29 09:29:58 2009 +0100
+++ b/tools/python/xen/xend/image.py    Fri May 29 09:32:02 2009 +0100
@@ -454,8 +454,22 @@ class ImageHandler:
         if cmd is '''' or ret is '''':
             raise VmError(''need valid command and result when signal
device model'')

-        orig_state =
xstransact.Read("/local/domain/0/device-model/%i/state"
+        count = 0
+        while True:
+            orig_state =
xstransact.Read("/local/domain/0/device-model/%i/state"
                                 % self.vm.getDomid())
+            # This can occur right after start-up
+            if orig_state != None:
+                break
+
+            log.debug(''signalDeviceModel: orig_state is None,
retrying'')
+
+            time.sleep(0.1)
+            count += 1
+            if count < 100:
+                continue
+
+            VmError(''Device model isn\''t ready for
commands'')

         if par is not None:
             xstransact.Store("/local/domain/0/device-model/%i"
diff -r ef6911934b6f -r ec2bc4b9fa32 tools/python/xen/xend/server/pciif.py
--- a/tools/python/xen/xend/server/pciif.py     Fri May 29 09:29:58 2009 +0100
+++ b/tools/python/xen/xend/server/pciif.py     Fri May 29 09:32:02 2009 +0100
@@ -69,12 +69,7 @@ class PciController(DevController):
         """@see DevController.getDeviceDetails"""
         back = {}
         pcidevid = 0
-        vslots = ""
         for pci_config in config.get(''devs'', []):
-            attached_vslot = pci_config.get(''vslot'')
-            if attached_vslot is not None:
-                vslots = vslots + attached_vslot + ";"
-
             domain = parse_hex(pci_config.get(''domain'', 0))
             bus = parse_hex(pci_config.get(''bus'', 0))
             slot = parse_hex(pci_config.get(''slot'', 0))
@@ -92,9 +87,6 @@ class PciController(DevController):
             back[''uuid-%i'' % pcidevid] =
pci_config.get(''uuid'', '''')
             back[''vslot-%i'' % pcidevid] = "%02x" %
vslot
             pcidevid += 1
-
-        if vslots != "":
-            back[''vslots''] = vslots

         back[''num_devs'']=str(pcidevid)
         back[''uuid''] =
config.get(''uuid'','''')
@@ -105,16 +97,17 @@ class PciController(DevController):

         return (0, back, {})

+    def reconfigureDevice_find(self, devid, nsearch_dev, match_dev):
+        for j in range(nsearch_dev):
+            if match_dev == self.readBackend(devid, ''dev-%i''
% j):
+                return j
+        return None

     def reconfigureDevice(self, _, config):
         """@see
DevController.reconfigureDevice"""
         (devid, back, front) = self.getDeviceDetails(config)
         num_devs = int(back[''num_devs''])
         states = config.get(''states'', [])
-
-        old_vslots = self.readBackend(devid, ''vslots'')
-        if old_vslots is None:
-            old_vslots = ''''
         num_olddevs = int(self.readBackend(devid,
''num_devs''))

         for i in range(num_devs):
@@ -129,11 +122,15 @@ class PciController(DevController):
                 raise XendError(''Error reading config'')

             if state == ''Initialising'':
-                # PCI device attachment
-                for j in range(num_olddevs):
-                    if dev == self.readBackend(devid,
''dev-%i'' % j):
-                        raise XendError(''Device %s is already
connected.'' % dev)
-                log.debug(''Attaching PCI device %s.'' % dev)
+                devno = self.reconfigureDevice_find(devid, num_olddevs, dev)
+                if devno == None:
+                    devno = num_olddevs + i
+                    log.debug(''Attaching PCI device %s.'' %
dev)
+                    attaching = True
+                else:
+                    log.debug(''Reconfiguring PCI device %s.''
% dev)
+                    attaching = False
+
                 (domain, bus, slotfunc) = dev.split('':'')
                 (slot, func) = slotfunc.split(''.'')
                 domain = parse_hex(domain)
@@ -142,41 +139,28 @@ class PciController(DevController):
                 func = parse_hex(func)
                 self.setupOneDevice(domain, bus, slot, func)

-                self.writeBackend(devid, ''dev-%i'' %
(num_olddevs + i), dev)
-                self.writeBackend(devid, ''state-%i'' %
(num_olddevs + i),
+                self.writeBackend(devid, ''dev-%i'' % devno,
dev)
+                self.writeBackend(devid, ''state-%i'' % devno,
                                  
str(xenbusState[''Initialising'']))
-                self.writeBackend(devid, ''uuid-%i'' %
(num_olddevs + i), uuid)
+                self.writeBackend(devid, ''uuid-%i'' % devno,
uuid)
                 if len(opts) > 0:
-                    self.writeBackend(devid, ''opts-%i'' %
(num_olddevs + i), opts)
-                self.writeBackend(devid, ''num_devs'',
str(num_olddevs + i + 1))
-
-                # Update vslots
-                if back[''vslots''] is not None:
-                    vslots = old_vslots + back[''vslots'']
-                    self.writeBackend(devid, ''vslots'',
vslots)
+                    self.writeBackend(devid, ''opts-%i'' %
devno, opts)
+                if back.has_key(''vslot-%i'' % i):
+                    self.writeBackend(devid, ''vslot-%i'' %
devno,
+                                      back[''vslot-%i'' % i])
+
+                # If a device is being attached then num_devs will grow
+                if attaching:
+                    self.writeBackend(devid, ''num_devs'',
str(devno + 1))

             elif state == ''Closing'':
                 # PCI device detachment
-                found = False
-                for j in range(num_olddevs):
-                    if dev == self.readBackend(devid,
''dev-%i'' % j):
-                        found = True
-                        log.debug(''Detaching device %s'' %
dev)
-                        self.writeBackend(devid, ''state-%i'' %
j,
-                                         
str(xenbusState[''Closing'']))
-                if not found:
+                devno = self.reconfigureDevice_find(devid, num_olddevs, dev)
+                if devno == None:
                     raise XendError(''Device %s is not
connected'' % dev)
-
-                # Update vslots
-                if back.get(''vslots'') is not None:
-                    vslots = old_vslots
-                    for vslot in
back[''vslots''].split('';''):
-                        if vslot != '''':
-                            vslots = vslots.replace(vslot +
'';'', '''', 1)
-                    if vslots == '''':
-                        self.removeBackend(devid, ''vslots'')
-                    else:
-                        self.writeBackend(devid, ''vslots'',
vslots)
+                log.debug(''Detaching device %s'' % dev)
+                self.writeBackend(devid, ''state-%i'' % devno,
+                                 
str(xenbusState[''Closing'']))

             else:
                 raise XendError(''Error configuring device %s: invalid
state %s''
@@ -191,12 +175,6 @@ class PciController(DevController):
         result = DevController.getDeviceConfiguration(self, devid, transaction)
         num_devs = self.readBackend(devid, ''num_devs'')
         pci_devs = []
-
-        vslots = self.readBackend(devid, ''vslots'')
-        if vslots is not None:
-            if vslots[-1] == ";":
-                vslots = vslots[:-1]
-            slot_list = vslots.split('';'')

         for i in range(int(num_devs)):
             dev_config = self.readBackend(devid, ''dev-%d'' %
i)
@@ -215,13 +193,11 @@ class PciController(DevController):

                 # Per device uuid info
                 dev_dict[''uuid''] = self.readBackend(devid,
''uuid-%d'' % i)
-
-                #append vslot info
-                if vslots is not None:
-                    try:
-                        dev_dict[''vslot''] = slot_list[i]
-                    except IndexError:
-                        dev_dict[''vslot''] = AUTO_PHP_SLOT_STR
+                vslot = self.readBackend(devid, ''vslot-%d'' %
i)
+                if vslot != None:
+                    dev_dict[''vslot''] =
self.readBackend(devid, ''vslot-%d'' % i)
+                else:
+                    dev_dict[''vslot''] = AUTO_PHP_SLOT_STR

                 #append opts info
                 opts = self.readBackend(devid, ''opts-%d'' % i)

_______________________________________________
Xen-changelog mailing list
Xen-changelog@lists.xensource.com
http://lists.xensource.com/xen-changelog

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Simon Horman

2009-Jun-12 06:33 UTC

head link

[Xen-devel] Re: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

On Fri, Jun 12, 2009 at 01:51:10PM +0800, Cui, Dexuan
wrote:> Hi Simon,
> After this changeset, I find there are some new issues in the xend:
> I noticed in xend.log, setupOneDevice() is invoked twice, but actually I
only statically assign 1 device to hvm guest.
> 
> After looking into the xend code, I find in XendDomainInfo.py:
_initDomain() -> _createDevices(), we invoke self._createDevice(devclass,
config) that eventually invokes setupOneDevice() -- this is the first time;
> And later, still in  _createDevices(), we invoke
pci_device_configure_boot() -> pci_device_configure() ->
dev_control.reconfigureDevice(devid, dev_config) ->
xend/server/pciif.py:reconfigureDevice() -> setupOneDevice()  --  this is the
second time.  Can you remove the duplicate invocation?
Sure, I will look into it ASAP.

Can I confirm which version of xen-unstable.hg and qemu-xen-unstable.git
you are using?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan

2009-Jun-12 06:35 UTC

head link

[Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

I''m using the latest xen-unstable 19740, Dom0 898, ioemu
e0bb6b8df60863bca0163a1688baf4854e931e55.

Thanks,
-- Dexuan



-----Original Message-----
From: Simon Horman [mailto:horms@verge.net.au] 
Sent: 2009?6?12? 14:34
To: Cui, Dexuan
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at
boot-time

On Fri, Jun 12, 2009 at 01:51:10PM +0800, Cui, Dexuan
wrote:> Hi Simon,
> After this changeset, I find there are some new issues in the xend:
> I noticed in xend.log, setupOneDevice() is invoked twice, but actually I
only statically assign 1 device to hvm guest.
> 
> After looking into the xend code, I find in XendDomainInfo.py:
_initDomain() -> _createDevices(), we invoke self._createDevice(devclass,
config) that eventually invokes setupOneDevice() -- this is the first time;
> And later, still in  _createDevices(), we invoke
pci_device_configure_boot() -> pci_device_configure() ->
dev_control.reconfigureDevice(devid, dev_config) ->
xend/server/pciif.py:reconfigureDevice() -> setupOneDevice()  --  this is the
second time.  Can you remove the duplicate invocation?
Sure, I will look into it ASAP.

Can I confirm which version of xen-unstable.hg and qemu-xen-unstable.git
you are using?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Simon Horman

2009-Jun-12 06:45 UTC

head link

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

On Fri, Jun 12, 2009 at 02:35:02PM +0800, Cui, Dexuan
wrote:> I''m using the latest xen-unstable 19740, Dom0 898, ioemu
e0bb6b8df60863bca0163a1688baf4854e931e55.
Thanks, I''ll look into it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Simon Horman

2009-Jun-15 01:53 UTC

head link

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

On Fri, Jun 12, 2009 at 02:35:02PM +0800, Cui, Dexuan
wrote:> On Fri, Jun 12, 2009 at 14:34, Simon Horman wrote:
> > On Fri, Jun 12, 2009 at 01:51:10PM +0800, Cui, Dexuan wrote:
> > > Hi Simon,
> > > After this changeset, I find there are some new issues in the
xend:
> > > I noticed in xend.log, setupOneDevice() is invoked twice,
> > > but actually I only statically assign 1 device to hvm guest.
> > > 
> > > After looking into the xend code, I find in XendDomainInfo.py:
> > > _initDomain() -> _createDevices(), we invoke
> > > self._createDevice(devclass, config) that eventually invokes
> > > setupOneDevice() -- this is the first time;
> > > And later, still in  _createDevices(), we invoke
> > > pci_device_configure_boot() -> pci_device_configure() ->
> > > dev_control.reconfigureDevice(devid, dev_config) ->
> > > xend/server/pciif.py:reconfigureDevice() -> setupOneDevice()
> > > --  this is the second time.  Can you remove the duplicate
invocation?
> > 
> > Sure, I will look into it ASAP.
> 
> > Can I confirm which version of xen-unstable.hg and
qemu-xen-unstable.git
> > you are using?
> I''m using the latest xen-unstable 19740, Dom0 898, ioemu
> e0bb6b8df60863bca0163a1688baf4854e931e55.
Hi Dexuan,

I think that a simple solution to this is to just remove the
first invocation.

-----------------------------------------------------------------------

xend: pass-through: Only call setupOneDevice() once per device

As observed by Dexuan Cui, when PCI devices are passed through at
domain-creation-time setupOneDevice() will be called twice.

Once via setupDevice() and once via econfigureDevice() which
is called in pci_device_configure().

This patch removes the first of these.

Cc: Dexuan Cui <dexuan.cui@intel.com>
Cc: Masaki Kanno <kanno.masaki@jp.fujitsu.com>
Signed-off-by: Simon Horman <horms@verge.net.au>

Index: xen-unstable.hg/tools/python/xen/xend/server/pciif.py
==================================================================---
xen-unstable.hg.orig/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:00.000000000 +1000
+++ xen-unstable.hg/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:02.000000000 +1000
@@ -436,8 +436,6 @@ class PciController(DevController):
                                     '' same guest with %s''
                                 raise VmError(err_msg % (s, dev.name))
 
-        for (domain, bus, slot, func) in pci_dev_list:
-            self.setupOneDevice(domain, bus, slot, func)
         wPath = ''/local/domain/0/backend/pci/%u/0/aerState'' %
(self.getDomid())
         self.aerStateWatch = xswatch(wPath, self._handleAerStateWatch)
         log.debug(''pci: register aer watch %s'', wPath)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan

2009-Jun-15 02:01 UTC

head link

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Yes, this can avoid the duplication invocation. 

Thanks,
-- Dexuan



-----Original Message-----
From: Simon Horman [mailto:horms@verge.net.au] 
Sent: 2009?6?15? 9:53
To: Cui, Dexuan
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI
devices at boot-time

On Fri, Jun 12, 2009 at 02:35:02PM +0800, Cui, Dexuan
wrote:> On Fri, Jun 12, 2009 at 14:34, Simon Horman wrote:
> > On Fri, Jun 12, 2009 at 01:51:10PM +0800, Cui, Dexuan wrote:
> > > Hi Simon,
> > > After this changeset, I find there are some new issues in the
xend:
> > > I noticed in xend.log, setupOneDevice() is invoked twice,
> > > but actually I only statically assign 1 device to hvm guest.
> > > 
> > > After looking into the xend code, I find in XendDomainInfo.py:
> > > _initDomain() -> _createDevices(), we invoke
> > > self._createDevice(devclass, config) that eventually invokes
> > > setupOneDevice() -- this is the first time;
> > > And later, still in  _createDevices(), we invoke
> > > pci_device_configure_boot() -> pci_device_configure() ->
> > > dev_control.reconfigureDevice(devid, dev_config) ->
> > > xend/server/pciif.py:reconfigureDevice() -> setupOneDevice()
> > > --  this is the second time.  Can you remove the duplicate
invocation?
> > 
> > Sure, I will look into it ASAP.
> 
> > Can I confirm which version of xen-unstable.hg and
qemu-xen-unstable.git
> > you are using?
> I''m using the latest xen-unstable 19740, Dom0 898, ioemu
> e0bb6b8df60863bca0163a1688baf4854e931e55.
Hi Dexuan,

I think that a simple solution to this is to just remove the
first invocation.

-----------------------------------------------------------------------

xend: pass-through: Only call setupOneDevice() once per device

As observed by Dexuan Cui, when PCI devices are passed through at
domain-creation-time setupOneDevice() will be called twice.

Once via setupDevice() and once via econfigureDevice() which
is called in pci_device_configure().

This patch removes the first of these.

Cc: Dexuan Cui <dexuan.cui@intel.com>
Cc: Masaki Kanno <kanno.masaki@jp.fujitsu.com>
Signed-off-by: Simon Horman <horms@verge.net.au>

Index: xen-unstable.hg/tools/python/xen/xend/server/pciif.py
==================================================================---
xen-unstable.hg.orig/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:00.000000000 +1000
+++ xen-unstable.hg/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:02.000000000 +1000
@@ -436,8 +436,6 @@ class PciController(DevController):
                                     '' same guest with %s''
                                 raise VmError(err_msg % (s, dev.name))
 
-        for (domain, bus, slot, func) in pci_dev_list:
-            self.setupOneDevice(domain, bus, slot, func)
         wPath = ''/local/domain/0/backend/pci/%u/0/aerState'' %
(self.getDomid())
         self.aerStateWatch = xswatch(wPath, self._handleAerStateWatch)
         log.debug(''pci: register aer watch %s'', wPath)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan

2009-Jul-28 06:47 UTC

head link

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Hi Simon, > I think that a simple solution to this is to just remove the first
invocation.This was checked in as c/s 19754.
Unluckily, this breaks device assignment for pv guest: xend would not invoke
setupOneDevice() for pv guest at all.

The attached patch fixes the issue.  Please have a look.

Thanks,
-- Dexuan



-----Original Message-----
From: Simon Horman [mailto:horms@verge.net.au] 
Sent: 2009?6?15? 9:53
To: Cui, Dexuan
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI
devices at boot-time

On Fri, Jun 12, 2009 at 02:35:02PM +0800, Cui, Dexuan
wrote:> On Fri, Jun 12, 2009 at 14:34, Simon Horman wrote:
> > On Fri, Jun 12, 2009 at 01:51:10PM +0800, Cui, Dexuan wrote:
> > > Hi Simon,
> > > After this changeset, I find there are some new issues in the
xend:
> > > I noticed in xend.log, setupOneDevice() is invoked twice,
> > > but actually I only statically assign 1 device to hvm guest.
> > > 
> > > After looking into the xend code, I find in XendDomainInfo.py:
> > > _initDomain() -> _createDevices(), we invoke
> > > self._createDevice(devclass, config) that eventually invokes
> > > setupOneDevice() -- this is the first time;
> > > And later, still in  _createDevices(), we invoke
> > > pci_device_configure_boot() -> pci_device_configure() ->
> > > dev_control.reconfigureDevice(devid, dev_config) ->
> > > xend/server/pciif.py:reconfigureDevice() -> setupOneDevice()
> > > --  this is the second time.  Can you remove the duplicate
invocation?
> > 
> > Sure, I will look into it ASAP.
> 
> > Can I confirm which version of xen-unstable.hg and
qemu-xen-unstable.git
> > you are using?
> I''m using the latest xen-unstable 19740, Dom0 898, ioemu
> e0bb6b8df60863bca0163a1688baf4854e931e55.
Hi Dexuan,

I think that a simple solution to this is to just remove the
first invocation.

-----------------------------------------------------------------------

xend: pass-through: Only call setupOneDevice() once per device

As observed by Dexuan Cui, when PCI devices are passed through at
domain-creation-time setupOneDevice() will be called twice.

Once via setupDevice() and once via econfigureDevice() which
is called in pci_device_configure().

This patch removes the first of these.

Cc: Dexuan Cui <dexuan.cui@intel.com>
Cc: Masaki Kanno <kanno.masaki@jp.fujitsu.com>
Signed-off-by: Simon Horman <horms@verge.net.au>

Index: xen-unstable.hg/tools/python/xen/xend/server/pciif.py
==================================================================---
xen-unstable.hg.orig/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:00.000000000 +1000
+++ xen-unstable.hg/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:02.000000000 +1000
@@ -436,8 +436,6 @@ class PciController(DevController):
                                     '' same guest with %s''
                                 raise VmError(err_msg % (s, dev.name))
 
-        for (domain, bus, slot, func) in pci_dev_list:
-            self.setupOneDevice(domain, bus, slot, func)
         wPath = ''/local/domain/0/backend/pci/%u/0/aerState'' %
(self.getDomid())
         self.aerStateWatch = xswatch(wPath, self._handleAerStateWatch)
         log.debug(''pci: register aer watch %s'', wPath)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Simon Horman

2009-Jul-28 07:15 UTC

head link

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Hi,

is an alternative to just revert 19754 and allow
duplicate calls to setupOneDevice() in the HVM case?

On Tue, Jul 28, 2009 at 02:47:16PM +0800, Cui, Dexuan
wrote:> Hi Simon, 
> > I think that a simple solution to this is to just remove the first
invocation.
> This was checked in as c/s 19754.
> Unluckily, this breaks device assignment for pv guest: xend would not
invoke setupOneDevice() for pv guest at all.
> 
> The attached patch fixes the issue.  Please have a look.
> 
> Thanks,
> -- Dexuan
> 
> 
> 
> -----Original Message-----
> From: Simon Horman [mailto:horms@verge.net.au] 
> Sent: 2009?6?15? 9:53
> To: Cui, Dexuan
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug
PCI devices at boot-time
> 
> On Fri, Jun 12, 2009 at 02:35:02PM +0800, Cui, Dexuan wrote:
> > On Fri, Jun 12, 2009 at 14:34, Simon Horman wrote:
> > > On Fri, Jun 12, 2009 at 01:51:10PM +0800, Cui, Dexuan wrote:
> > > > Hi Simon,
> > > > After this changeset, I find there are some new issues in
the xend:
> > > > I noticed in xend.log, setupOneDevice() is invoked twice,
> > > > but actually I only statically assign 1 device to hvm guest.
> > > > 
> > > > After looking into the xend code, I find in
XendDomainInfo.py:
> > > > _initDomain() -> _createDevices(), we invoke
> > > > self._createDevice(devclass, config) that eventually invokes
> > > > setupOneDevice() -- this is the first time;
> > > > And later, still in  _createDevices(), we invoke
> > > > pci_device_configure_boot() -> pci_device_configure()
->
> > > > dev_control.reconfigureDevice(devid, dev_config) ->
> > > > xend/server/pciif.py:reconfigureDevice() ->
setupOneDevice()
> > > > --  this is the second time.  Can you remove the duplicate
invocation?
> > > 
> > > Sure, I will look into it ASAP.
> > 
> > > Can I confirm which version of xen-unstable.hg and
qemu-xen-unstable.git
> > > you are using?
> > I''m using the latest xen-unstable 19740, Dom0 898, ioemu
> > e0bb6b8df60863bca0163a1688baf4854e931e55.
> 
> Hi Dexuan,
> 
> I think that a simple solution to this is to just remove the
> first invocation.
> 
> -----------------------------------------------------------------------
> 
> xend: pass-through: Only call setupOneDevice() once per device
> 
> As observed by Dexuan Cui, when PCI devices are passed through at
> domain-creation-time setupOneDevice() will be called twice.
> 
> Once via setupDevice() and once via econfigureDevice() which
> is called in pci_device_configure().
> 
> This patch removes the first of these.
> 
> Cc: Dexuan Cui <dexuan.cui@intel.com>
> Cc: Masaki Kanno <kanno.masaki@jp.fujitsu.com>
> Signed-off-by: Simon Horman <horms@verge.net.au>
> 
> Index: xen-unstable.hg/tools/python/xen/xend/server/pciif.py
> ==================================================================> ---
xen-unstable.hg.orig/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:00.000000000 +1000
> +++ xen-unstable.hg/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:02.000000000 +1000
> @@ -436,8 +436,6 @@ class PciController(DevController):
>                                      '' same guest with
%s''
>                                  raise VmError(err_msg % (s, dev.name))
>  
> -        for (domain, bus, slot, func) in pci_dev_list:
> -            self.setupOneDevice(domain, bus, slot, func)
>          wPath =
''/local/domain/0/backend/pci/%u/0/aerState'' %
(self.getDomid())
>          self.aerStateWatch = xswatch(wPath, self._handleAerStateWatch)
>          log.debug(''pci: register aer watch %s'', wPath)
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan

2009-Jul-28 08:42 UTC

head link

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Duplicate calls to setupOneDevice() in the case of HVM  are really odd to me. 

BTW, actually I suspect setupOneDevice() is unnecessary for HVM guest since
looks all of the actual work of setupOneDevice() is done by ioemu-dm.

Thanks,
-- Dexuan

-----Original Message-----
From: Simon Horman [mailto:horms@verge.net.au] 
Sent: 2009?7?28? 15:16
To: Cui, Dexuan
Cc: Keir Fraser; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI
devices at boot-time

Hi,

is an alternative to just revert 19754 and allow
duplicate calls to setupOneDevice() in the HVM case?

On Tue, Jul 28, 2009 at 02:47:16PM +0800, Cui, Dexuan
wrote:> Hi Simon, 
> > I think that a simple solution to this is to just remove the first
invocation.
> This was checked in as c/s 19754.
> Unluckily, this breaks device assignment for pv guest: xend would not
invoke setupOneDevice() for pv guest at all.
> 
> The attached patch fixes the issue.  Please have a look.
> 
> Thanks,
> -- Dexuan
> 
> 
> 
> -----Original Message-----
> From: Simon Horman [mailto:horms@verge.net.au] 
> Sent: 2009?6?15? 9:53
> To: Cui, Dexuan
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug
PCI devices at boot-time
> 
> On Fri, Jun 12, 2009 at 02:35:02PM +0800, Cui, Dexuan wrote:
> > On Fri, Jun 12, 2009 at 14:34, Simon Horman wrote:
> > > On Fri, Jun 12, 2009 at 01:51:10PM +0800, Cui, Dexuan wrote:
> > > > Hi Simon,
> > > > After this changeset, I find there are some new issues in
the xend:
> > > > I noticed in xend.log, setupOneDevice() is invoked twice,
> > > > but actually I only statically assign 1 device to hvm guest.
> > > > 
> > > > After looking into the xend code, I find in
XendDomainInfo.py:
> > > > _initDomain() -> _createDevices(), we invoke
> > > > self._createDevice(devclass, config) that eventually invokes
> > > > setupOneDevice() -- this is the first time;
> > > > And later, still in  _createDevices(), we invoke
> > > > pci_device_configure_boot() -> pci_device_configure()
->
> > > > dev_control.reconfigureDevice(devid, dev_config) ->
> > > > xend/server/pciif.py:reconfigureDevice() ->
setupOneDevice()
> > > > --  this is the second time.  Can you remove the duplicate
invocation?
> > > 
> > > Sure, I will look into it ASAP.
> > 
> > > Can I confirm which version of xen-unstable.hg and
qemu-xen-unstable.git
> > > you are using?
> > I''m using the latest xen-unstable 19740, Dom0 898, ioemu
> > e0bb6b8df60863bca0163a1688baf4854e931e55.
> 
> Hi Dexuan,
> 
> I think that a simple solution to this is to just remove the
> first invocation.
> 
> -----------------------------------------------------------------------
> 
> xend: pass-through: Only call setupOneDevice() once per device
> 
> As observed by Dexuan Cui, when PCI devices are passed through at
> domain-creation-time setupOneDevice() will be called twice.
> 
> Once via setupDevice() and once via econfigureDevice() which
> is called in pci_device_configure().
> 
> This patch removes the first of these.
> 
> Cc: Dexuan Cui <dexuan.cui@intel.com>
> Cc: Masaki Kanno <kanno.masaki@jp.fujitsu.com>
> Signed-off-by: Simon Horman <horms@verge.net.au>
> 
> Index: xen-unstable.hg/tools/python/xen/xend/server/pciif.py
> ==================================================================> ---
xen-unstable.hg.orig/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:00.000000000 +1000
> +++ xen-unstable.hg/tools/python/xen/xend/server/pciif.py	2009-06-15
11:24:02.000000000 +1000
> @@ -436,8 +436,6 @@ class PciController(DevController):
>                                      '' same guest with
%s''
>                                  raise VmError(err_msg % (s, dev.name))
>  
> -        for (domain, bus, slot, func) in pci_dev_list:
> -            self.setupOneDevice(domain, bus, slot, func)
>          wPath =
''/local/domain/0/backend/pci/%u/0/aerState'' %
(self.getDomid())
>          self.aerStateWatch = xswatch(wPath, self._handleAerStateWatch)
>          log.debug(''pci: register aer watch %s'', wPath)
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Simon Horman

2009-Jul-28 08:56 UTC

head link

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

On Tue, Jul 28, 2009 at 04:42:17PM +0800, Cui, Dexuan
wrote:> Duplicate calls to setupOneDevice() in the case of HVM  are really odd to
me.
Personally I''d rather call the code twice than have extra if logic.
But your patch is ok with me.
> BTW, actually I suspect setupOneDevice() is unnecessary for HVM guest
> since looks all of the actual work of setupOneDevice() is done by
> ioemu-dm.
I suspect you are right there.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan

2009-Jul-28 09:04 UTC

head link

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Hi Simon,
Actually I think you have done a great work to clean up the related xend code,
however looks we should do more. :-)

Thanks,
-- Dexuan


-----Original Message-----
From: Simon Horman [mailto:horms@verge.net.au] 
Sent: 2009?7?28? 16:57
To: Cui, Dexuan
Cc: Keir Fraser; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI
devices at boot-time

On Tue, Jul 28, 2009 at 04:42:17PM +0800, Cui, Dexuan
wrote:> Duplicate calls to setupOneDevice() in the case of HVM  are really odd to
me.
Personally I''d rather call the code twice than have extra if logic.
But your patch is ok with me.
> BTW, actually I suspect setupOneDevice() is unnecessary for HVM guest
> since looks all of the actual work of setupOneDevice() is done by
> ioemu-dm.
I suspect you are right there.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Jun 2009 - Re: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

[Xen-devel] Re: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

[Xen-devel] Re: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

[Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

Re: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time

RE: [Xen-devel] RE: [Xen-changelog] [xen-unstable] xend: hot-plug PCI devices at boot-time