Dexuan, Stefan, from my past experience, I can tell you that these boards with nForce chipsets like the one Stefan uses have all 3/4 PCI slots behind a PCI bridge. I own a Gigabyte GA-M56S-S3. Assuming that Stefan has only PV domains running, I can confirm that the patch, which I included manually (due to some offsets) still works. I still would vote for some kind of option to disable the necessety of co-assignment in cases where they are not necessary or wanted. I am missing the knowledge to help, but I can imagine that there are cases in which it is mandatory (when I understood right, it''s when VT-d is used), but maybe it is usefull to have FLR support also, when you have HVM Domains running without VT-d support. Is it usefull to have FLR implemented in a pure-PV environment, too? I attached the outputs of the commands, just in case it makes sense to have a look at them. BR, Carsten. ----- Originalnachricht ----- Von: "Cui, Dexuan" <dexuan.cui@intel.com> Gesendet: Don, 11.6.2009 04:33 An: xen-devel@lists.xensource.com Betreff: RE: [Xen-devel] Xen-3.4 and FLR Hi Stefan, Are you assigning device to PV guest or HVM guest? Can you attach the log files of "lspci -tv" and "lspci -xxx -vvv" on your host? Thanks, -- Dexuan -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Kuhne Sent: 2009?6?11? 8:25 To: xen-devel@lists.xensource.com Subject: [Xen-devel] Xen-3.4 and FLR Hi, i''ve upgrade from xen-3.3.1 (with flr disable patch) to xen-3.4. How i''ve my old FLR problem, when i try to give an PCI-Card from a real slot to a domU i get: VmError: pci: 0000:01:0e.0 must be co-assigned to the same guest with 0000:01:09.0 When i''ve more than one PCI-Card in system xen will all PCI-Cards give this domU till xen is by an onBoard device. When i give an onBoard device to a domU i''ve no problem. I''ve a Gigabyte GA-M52S-S3P in this system. I''am involved in a Project (EisXen), there are many with the same Problem on xen-3.3.1 without patch. So, where does it come from? Regads, Stefan Kuhne _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Carsten, In some cases (especially assigning device to PV guest without iommu), the current code in xend does have some known limitations/issues. I even think there is not a "generic and clean" solution... In the long run, new BIOS/device should support the standard PCIe FLR or PCI FLR so we can get rid of the issues. At present I''m busy on something else and I appreciate somebody could help to try to improve the current code. :-) Thanks, -- Dexuan -----Original Message----- From: Carsten Schiers [mailto:carsten@schiers.de] Sent: 2009?6?11? 16:20 To: Cui, Dexuan; xen-devel@lists.xensource.com Subject: AW: [Xen-devel] Xen-3.4 and FLR Dexuan, Stefan, from my past experience, I can tell you that these boards with nForce chipsets like the one Stefan uses have all 3/4 PCI slots behind a PCI bridge. I own a Gigabyte GA-M56S-S3. Assuming that Stefan has only PV domains running, I can confirm that the patch, which I included manually (due to some offsets) still works. I still would vote for some kind of option to disable the necessety of co-assignment in cases where they are not necessary or wanted. I am missing the knowledge to help, but I can imagine that there are cases in which it is mandatory (when I understood right, it''s when VT-d is used), but maybe it is usefull to have FLR support also, when you have HVM Domains running without VT-d support. Is it usefull to have FLR implemented in a pure-PV environment, too? I attached the outputs of the commands, just in case it makes sense to have a look at them. BR, Carsten. ----- Originalnachricht ----- Von: "Cui, Dexuan" <dexuan.cui@intel.com> Gesendet: Don, 11.6.2009 04:33 An: xen-devel@lists.xensource.com Betreff: RE: [Xen-devel] Xen-3.4 and FLR Hi Stefan, Are you assigning device to PV guest or HVM guest? Can you attach the log files of "lspci -tv" and "lspci -xxx -vvv" on your host? Thanks, -- Dexuan -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Kuhne Sent: 2009?6?11? 8:25 To: xen-devel@lists.xensource.com Subject: [Xen-devel] Xen-3.4 and FLR Hi, i''ve upgrade from xen-3.3.1 (with flr disable patch) to xen-3.4. How i''ve my old FLR problem, when i try to give an PCI-Card from a real slot to a domU i get: VmError: pci: 0000:01:0e.0 must be co-assigned to the same guest with 0000:01:09.0 When i''ve more than one PCI-Card in system xen will all PCI-Cards give this domU till xen is by an onBoard device. When i give an onBoard device to a domU i''ve no problem. I''ve a Gigabyte GA-M52S-S3P in this system. I''am involved in a Project (EisXen), there are many with the same Problem on xen-3.3.1 without patch. So, where does it come from? Regads, Stefan Kuhne _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Dexuan, if I would be able to, I would. But unfortunately I only understood only 10% of what is happening. As long as it''s ok to patch it away if you don''t need it, why not. I had the feeling already that it''s quite tricky. Thanks, Carsten. ----- Originalnachricht ----- Von: "Cui, Dexuan" <dexuan.cui@intel.com> Gesendet: Don, 11.6.2009 11:39 An: Carsten Schiers <carsten@schiers.de> ; xen-devel@lists.xensource.com Betreff: RE: [Xen-devel] Xen-3.4 and FLR Hi Carsten, In some cases (especially assigning device to PV guest without iommu), the current code in xend does have some known limitations/issues. I even think there is not a "generic and clean" solution... In the long run, new BIOS/device should support the standard PCIe FLR or PCI FLR so we can get rid of the issues. At present I''m busy on something else and I appreciate somebody could help to try to improve the current code. :-) Thanks, -- Dexuan -----Original Message----- From: Carsten Schiers [mailto:carsten@schiers.de] Sent: 2009?6?11? 16:20 To: Cui, Dexuan; xen-devel@lists.xensource.com Subject: AW: [Xen-devel] Xen-3.4 and FLR Dexuan, Stefan, from my past experience, I can tell you that these boards with nForce chipsets like the one Stefan uses have all 3/4 PCI slots behind a PCI bridge. I own a Gigabyte GA-M56S-S3. Assuming that Stefan has only PV domains running, I can confirm that the patch, which I included manually (due to some offsets) still works. I still would vote for some kind of option to disable the necessety of co-assignment in cases where they are not necessary or wanted. I am missing the knowledge to help, but I can imagine that there are cases in which it is mandatory (when I understood right, it''s when VT-d is used), but maybe it is usefull to have FLR support also, when you have HVM Domains running without VT-d support. Is it usefull to have FLR implemented in a pure-PV environment, too? I attached the outputs of the commands, just in case it makes sense to have a look at them. BR, Carsten. ----- Originalnachricht ----- Von: "Cui, Dexuan" <dexuan.cui@intel.com> Gesendet: Don, 11.6.2009 04:33 An: xen-devel@lists.xensource.com Betreff: RE: [Xen-devel] Xen-3.4 and FLR Hi Stefan, Are you assigning device to PV guest or HVM guest? Can you attach the log files of "lspci -tv" and "lspci -xxx -vvv" on your host? Thanks, -- Dexuan -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Kuhne Sent: 2009?6?11? 8:25 To: xen-devel@lists.xensource.com Subject: [Xen-devel] Xen-3.4 and FLR Hi, i''ve upgrade from xen-3.3.1 (with flr disable patch) to xen-3.4. How i''ve my old FLR problem, when i try to give an PCI-Card from a real slot to a domU i get: VmError: pci: 0000:01:0e.0 must be co-assigned to the same guest with 0000:01:09.0 When i''ve more than one PCI-Card in system xen will all PCI-Cards give this domU till xen is by an onBoard device. When i give an onBoard device to a domU i''ve no problem. I''ve a Gigabyte GA-M52S-S3P in this system. I''am involved in a Project (EisXen), there are many with the same Problem on xen-3.3.1 without patch. So, where does it come from? Regads, Stefan Kuhne _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
In some cases (e.g., assigning device to HVM guest with iommu), we do need the co-assignment checking/constraint, while for PV guest, we may not need it if iommu is not used. The trouble is: to give a generic solution, we must consider both PV guest and HVM guest, and actually I think it''s not easy to make the code clean to fit every case. Thanks, -- Dexuan -----Original Message----- From: Carsten Schiers [mailto:carsten@schiers.de] Sent: 2009?6?11? 18:03 To: Cui, Dexuan; xen-devel@lists.xensource.com Subject: AW: [Xen-devel] Xen-3.4 and FLR Hi Dexuan, if I would be able to, I would. But unfortunately I only understood only 10% of what is happening. As long as it''s ok to patch it away if you don''t need it, why not. I had the feeling already that it''s quite tricky. Thanks, Carsten. ----- Originalnachricht ----- Von: "Cui, Dexuan" <dexuan.cui@intel.com> Gesendet: Don, 11.6.2009 11:39 An: Carsten Schiers <carsten@schiers.de> ; xen-devel@lists.xensource.com Betreff: RE: [Xen-devel] Xen-3.4 and FLR Hi Carsten, In some cases (especially assigning device to PV guest without iommu), the current code in xend does have some known limitations/issues. I even think there is not a "generic and clean" solution... In the long run, new BIOS/device should support the standard PCIe FLR or PCI FLR so we can get rid of the issues. At present I''m busy on something else and I appreciate somebody could help to try to improve the current code. :-) Thanks, -- Dexuan -----Original Message----- From: Carsten Schiers [mailto:carsten@schiers.de] Sent: 2009?6?11? 16:20 To: Cui, Dexuan; xen-devel@lists.xensource.com Subject: AW: [Xen-devel] Xen-3.4 and FLR Dexuan, Stefan, from my past experience, I can tell you that these boards with nForce chipsets like the one Stefan uses have all 3/4 PCI slots behind a PCI bridge. I own a Gigabyte GA-M56S-S3. Assuming that Stefan has only PV domains running, I can confirm that the patch, which I included manually (due to some offsets) still works. I still would vote for some kind of option to disable the necessety of co-assignment in cases where they are not necessary or wanted. I am missing the knowledge to help, but I can imagine that there are cases in which it is mandatory (when I understood right, it''s when VT-d is used), but maybe it is usefull to have FLR support also, when you have HVM Domains running without VT-d support. Is it usefull to have FLR implemented in a pure-PV environment, too? I attached the outputs of the commands, just in case it makes sense to have a look at them. BR, Carsten. ----- Originalnachricht ----- Von: "Cui, Dexuan" <dexuan.cui@intel.com> Gesendet: Don, 11.6.2009 04:33 An: xen-devel@lists.xensource.com Betreff: RE: [Xen-devel] Xen-3.4 and FLR Hi Stefan, Are you assigning device to PV guest or HVM guest? Can you attach the log files of "lspci -tv" and "lspci -xxx -vvv" on your host? Thanks, -- Dexuan -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Kuhne Sent: 2009?6?11? 8:25 To: xen-devel@lists.xensource.com Subject: [Xen-devel] Xen-3.4 and FLR Hi, i''ve upgrade from xen-3.3.1 (with flr disable patch) to xen-3.4. How i''ve my old FLR problem, when i try to give an PCI-Card from a real slot to a domU i get: VmError: pci: 0000:01:0e.0 must be co-assigned to the same guest with 0000:01:09.0 When i''ve more than one PCI-Card in system xen will all PCI-Cards give this domU till xen is by an onBoard device. When i give an onBoard device to a domU i''ve no problem. I''ve a Gigabyte GA-M52S-S3P in this system. I''am involved in a Project (EisXen), there are many with the same Problem on xen-3.3.1 without patch. So, where does it come from? Regads, Stefan Kuhne _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel