Xen devel - Jul 2009 - [PATCH][ioemu] support the assignment of the VF of Intel 82599 10GbE Controller

The datasheet is available at
http://download.intel.com/design/network/datashts/82599_datasheet.pdf

See 'Table 9.7. VF PCIe Configuration Space' of the datasheet, the PCI
Express Capability Structure of the VF of Intel 82599 10GbE Controller looks
trivial, e.g., the PCI Express Capabilities Register is 0, so the Capability
Version is 0 and pt_pcie_size_init() would fail.

We should not try to expose the PCIe cap of the device to guest.

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>

diff --git a/hw/pass-through.c b/hw/pass-through.c
index f3d033b..360f2a1 100644
--- a/hw/pass-through.c
+++ b/hw/pass-through.c
@@ -2408,6 +2408,25 @@ out:
     return err;
 }
 
+/* A return value of 1 means the capability should NOT be exposed to guest. */
+static int pt_hide_dev_cap(const struct pci_dev *dev, uint8_t grp_id)
+{
+    switch (grp_id)
+    {
+    case PCI_CAP_ID_EXP:
+        /* The PCI Express Capability Structure of the VF of Intel 82599 10GbE
+         * Controller looks trivial, e.g., the PCI Express Capabilities
+         * Register is 0. We should not try to expose it to guest.
+         */
+        if (dev->vendor_id == PCI_VENDOR_ID_INTEL &&
+                dev->device_id == PCI_DEVICE_ID_INTEL_82599_VF)
+            return 1;
+        break;
+    }
+
+    return 0;
+}
+
 /* initialize emulate register group */
 static int pt_config_init(struct pt_dev *ptdev)
 {
@@ -2424,6 +2443,9 @@ static int pt_config_init(struct pt_dev *ptdev)
     {
         if (pt_emu_reg_grp_tbl[i].grp_id != 0xFF)
         {
+            if (pt_hide_dev_cap(ptdev->pci_dev,
pt_emu_reg_grp_tbl[i].grp_id))
+                continue;
+
             reg_grp_offset = (uint32_t)find_cap_offset(ptdev->pci_dev,
                                  pt_emu_reg_grp_tbl[i].grp_id);
             if (!reg_grp_offset)
@@ -2556,6 +2578,8 @@ static uint32_t pt_ptr_reg_init(struct pt_dev *ptdev,
             /* check whether the next capability
              * should be exported to guest or not
              */
+            if (pt_hide_dev_cap(ptdev->pci_dev,
pt_emu_reg_grp_tbl[i].grp_id))
+                continue;
             if (pt_emu_reg_grp_tbl[i].grp_id ==
ptdev->dev.config[reg_field])
             {
                 if (pt_emu_reg_grp_tbl[i].grp_type == GRP_TYPE_EMU)
diff --git a/hw/pci.h b/hw/pci.h
index 30bcb04..de5a4e1 100644
--- a/hw/pci.h
+++ b/hw/pci.h
@@ -112,6 +112,7 @@ extern target_phys_addr_t pci_mem_base;
 #define PCI_DEVICE_ID_INTEL_82371AB      0x7111
 #define PCI_DEVICE_ID_INTEL_82371AB_2    0x7112
 #define PCI_DEVICE_ID_INTEL_82371AB_3    0x7113
+#define PCI_DEVICE_ID_INTEL_82599_VF     0x10ed
 
 #define PCI_VENDOR_ID_FSL                0x1957
 #define PCI_DEVICE_ID_FSL_E500           0x0030

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Jul 31, 2009 at 10:16:46PM +0800, Cui, Dexuan
wrote:
> The datasheet is available at
> http://download.intel.com/design/network/datashts/82599_datasheet.pdf
> 
> See ''Table 9.7. VF PCIe Configuration Space'' of the
datasheet, the PCI
> Express Capability Structure of the VF of Intel 82599 10GbE Controller
looks
> trivial, e.g., the PCI Express Capabilities Register is 0, so the
Capability
> Version is 0 and pt_pcie_size_init() would fail.
> 
> We should not try to expose the PCIe cap of the device to guest.
> 
> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
Hi Dexuan,

is a similar change needed for the 82576?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Cui, Dexuan writes ("[PATCH][ioemu] support the assignment of the VF of
Intel 82599 10GbE Controller"):
> The datasheet is available at
> http://download.intel.com/design/network/datashts/82599_datasheet.pdf
> 
> See ''Table 9.7. VF PCIe Configuration Space'' of the
datasheet, the PCI
> Express Capability Structure of the VF of Intel 82599 10GbE Controller
looks
> trivial, e.g., the PCI Express Capabilities Register is 0, so the
Capability
> Version is 0 and pt_pcie_size_init() would fail.
> 
> We should not try to expose the PCIe cap of the device to guest.
I''ve applied this for now but it seems a bit ugly to code in the
device IDs in this way.  Couldn''t we instead detect this situation
programmatically ?  Alternatively, if this is going to happen for
other devices, do we want to provide a proper whitelist in the form of
a table ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Simon,
I didn''t test 82576 myself, but I think it doesn''t need such a
change per its datasheet.

Thanks,
-- Dexuan



-----Original Message-----
From: Simon Horman [mailto:horms@verge.net.au] 
Sent: 2009?8?4? 8:04
To: Cui, Dexuan
Cc: Ian Jackson; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] [PATCH][ioemu] support the assignment of the VF of
Intel 82599 10GbE Controller

On Fri, Jul 31, 2009 at 10:16:46PM +0800, Cui, Dexuan
wrote:
> The datasheet is available at
> http://download.intel.com/design/network/datashts/82599_datasheet.pdf
> 
> See ''Table 9.7. VF PCIe Configuration Space'' of the
datasheet, the PCI
> Express Capability Structure of the VF of Intel 82599 10GbE Controller
looks
> trivial, e.g., the PCI Express Capabilities Register is 0, so the
Capability
> Version is 0 and pt_pcie_size_init() would fail.
> 
> We should not try to expose the PCIe cap of the device to guest.
> 
> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
Hi Dexuan,

is a similar change needed for the 82576?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Aug 05, 2009 at 09:56:21AM +0800, Cui, Dexuan
wrote:
> Hi Simon,
> I didn''t test 82576 myself, but I think it doesn''t need
such a change per
> its datasheet.
Thanks.

I''m actually having a bit of trouble getting SR-IOV working at all
under Xen due to no MMCONFIG in my 2.6.18.8 dom0 :-(


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson wrote:
> Cui, Dexuan writes ("[PATCH][ioemu] support the assignment of the VF
> of Intel 82599 10GbE Controller"): 
>> The datasheet is available at
>> http://download.intel.com/design/network/datashts/82599_datasheet.pdf
>> 
>> See ''Table 9.7. VF PCIe Configuration Space'' of the
datasheet, the
>> PCI Express Capability Structure of the VF of Intel 82599 10GbE
>> Controller looks trivial, e.g., the PCI Express Capabilities
>> Register is 0, so the Capability Version is 0 and
>> pt_pcie_size_init() would fail. 
>> 
>> We should not try to expose the PCIe cap of the device to guest.
> 
> I''ve applied this for now but it seems a bit ugly to code in the
> device IDs in this way.
Thanks.
>  Couldn''t we instead detect this situation
> programmatically ?  Alternatively, if this is going to happen for
> other devices, do we want to provide a proper whitelist in the form of
> a table ?
Using the patch (yes, that''s a quirk) I think we can have the least
change.
I''m not sure if there are other devices having the same issue.
I think we can wait to see if there are so many such devices that we can make a
table. :-)

Thanks,
-- Dexuan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

The 82576 does not need this patch.  It''s PCIe capability structure in
the Virtual Function devices is set up correctly.

- Greg 

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Cui, Dexuan
Sent: Tuesday, August 04, 2009 6:56 PM
To: Simon Horman
Cc: xen-devel@lists.xensource.com; Ian Jackson
Subject: RE: [Xen-devel] [PATCH][ioemu] support the assignment of the VF of
Intel 82599 10GbE Controller

Hi Simon,
I didn''t test 82576 myself, but I think it doesn''t need such a
change per its datasheet.

Thanks,
-- Dexuan



-----Original Message-----
From: Simon Horman [mailto:horms@verge.net.au] 
Sent: 2009?8?4? 8:04
To: Cui, Dexuan
Cc: Ian Jackson; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] [PATCH][ioemu] support the assignment of the VF of
Intel 82599 10GbE Controller

On Fri, Jul 31, 2009 at 10:16:46PM +0800, Cui, Dexuan
wrote:
> The datasheet is available at
> http://download.intel.com/design/network/datashts/82599_datasheet.pdf
> 
> See ''Table 9.7. VF PCIe Configuration Space'' of the
datasheet, the PCI
> Express Capability Structure of the VF of Intel 82599 10GbE Controller
looks
> trivial, e.g., the PCI Express Capabilities Register is 0, so the
Capability
> Version is 0 and pt_pcie_size_init() would fail.
> 
> We should not try to expose the PCIe cap of the device to guest.
> 
> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
Hi Dexuan,

is a similar change needed for the 82576?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Future generations of Intel 10Gig controllers that support SR-IOV will not have
this problem.  It is a design flaw unique to the 82599.  It''s unlikely
that other manufacturer''s controllers would have this same issue. 
I''d agree with Dexuan that it would be better to wait and see if the
table becomes necessary.  At this time I think it probably won''t be.

- Greg 

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Cui, Dexuan
Sent: Tuesday, August 04, 2009 10:17 PM
To: Ian Jackson
Cc: xen-devel@lists.xensource.com
Subject: [Xen-devel] RE: [PATCH][ioemu] support the assignment of the VF of
Intel 82599 10GbE Controller

Ian Jackson wrote:
> Cui, Dexuan writes ("[PATCH][ioemu] support the assignment of the VF
> of Intel 82599 10GbE Controller"): 
>> The datasheet is available at
>> http://download.intel.com/design/network/datashts/82599_datasheet.pdf
>> 
>> See ''Table 9.7. VF PCIe Configuration Space'' of the
datasheet, the
>> PCI Express Capability Structure of the VF of Intel 82599 10GbE
>> Controller looks trivial, e.g., the PCI Express Capabilities
>> Register is 0, so the Capability Version is 0 and
>> pt_pcie_size_init() would fail. 
>> 
>> We should not try to expose the PCIe cap of the device to guest.
> 
> I''ve applied this for now but it seems a bit ugly to code in the
> device IDs in this way.
Thanks.
>  Couldn''t we instead detect this situation
> programmatically ?  Alternatively, if this is going to happen for
> other devices, do we want to provide a proper whitelist in the form of
> a table ?
Using the patch (yes, that''s a quirk) I think we can have the least
change.
I''m not sure if there are other devices having the same issue.
I think we can wait to see if there are so many such devices that we can make a
table. :-)

Thanks,
-- Dexuan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Rose, Gregory V writes ("RE: [Xen-devel] RE: [PATCH][ioemu] support the
assignment of the VF of Intel 82599 10GbE
Controller"):
> Future generations of Intel 10Gig controllers that support SR-IOV
> will not have this problem.  It is a design flaw unique to the
> 82599.  It''s unlikely that other manufacturer''s
controllers would
> have this same issue.  I''d agree with Dexuan that it would be
better
> to wait and see if the table becomes necessary.  At this time I
> think it probably won''t be.
Right, that''s fine.  If it''s just a quirk it''s no
problem (and it
might have been worth putting `quirk'' in a comment nearby ...)

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel