Hello I''m working on XEN since one month now. I want create four domU with 2 PCI device per dom. I''ve bought a backplane with 3 bridges PCIe 1x to PCI and 4 PCI directly connected to the PCH (Q67). (Moreover PCI card (ethernet card) are composed with a PCI bridge) I''ve succefuly created 4 domU with bakend devices, with option "pci-backend-strict-check" disable. I''ve try with it but even if all PCI ports behind PCIe to PCI bridge appears in pci-list-assignable-devices, domU won''t start, with an error message "device XX:XX.X need to be co-assigned with XX:XX.X, but it''s not owned by pci-backend The probleme is : even if my domU are configuer with 1 VCPUs for each of them, when domU acces to I/O devices, VCPUs aren''t separated (I''ve notice this using virt-manager charts). During a basic CPU benchmark on a domU, VCPUs are isolated. I Have attached "lspci-v", "xm dmesg", "xm pci-list-assignable-devices", "default-grub", "xm info" and a "domU cfg file" A share core is very problematic my RT application thank you for your help in advance _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hi, I''ve never been successful with VT-d-style pci delegation, but I read devices behind pci bridges could not be allocated to different VMs in earlier times. Make sure this is supposed to work before you tackle the harder issues. 2012/6/22 amiard brice <perso@brice-amiard.fr>:> Hello > > > > I''m working on XEN since one month now. > > > > I want create four domU with 2 PCI device per dom. > > I''ve bought a backplane with 3 bridges PCIe 1x to PCI and 4 PCI directly > connected to the PCH (Q67). > > (Moreover PCI card (ethernet card) are composed with a PCI bridge) > > > > I''ve succefuly created 4 domU with bakend devices, with option > "pci-backend-strict-check" disable. > > I’ve try with it but even if all PCI ports behind PCIe to PCI bridge appears > in pci-list-assignable-devices, domU won’t start, with an error message > "device XX:XX.X need to be co-assigned with XX:XX.X, but it''s not owned by > pci-backend > > > > The probleme is : even if my domU are configuer with 1 VCPUs for each of > them, when domU acces to I/O devices, VCPUs aren''t separated (I''ve notice > this using virt-manager charts). During a basic CPU benchmark on a domU, > VCPUs are isolated. > > > > I Have attached "lspci-v", "xm dmesg", "xm pci-list-assignable-devices", > "default-grub", "xm info" and a “domU cfg file” > > > > A share core is very problematic my RT application > > > > thank you for your help in advance > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users-- the purpose of libvirt is to provide an abstraction layer hiding all xen features added since 2006 until they were finally understood and copied by the kvm devs.
But as I said, my backplane ("motherboard extension") is composed with
4
bridge "PCI-e to PCI". And I have one domU for each "PCI-e to PCI
bridge",
so according to the intel_VT-d pdf, it could be possible in my case to have
4 domU (one for each PCIe to PCI bridges).
In my opinion it could be due to the EPT table. In my "xm dmesg" I
could see
"EPT not enable" and "CPU capabilities : EPT table 2Mo",
but I can''t active
it (I''ve try different options in default/grub without succes)
(if devices behind the same pci bridges could not be allocated to different
VMs, it'' simply because VT-d use specific interruption command call
MSI-X in
aim to identify a domU, and those interruptions commands could be possible
only with PCIe, you can have more information in the pdf intel VT-d).
thank you for your help in advance
Hi,
I''ve never been successful with VT-d-style pci delegation, but I read
devices behind pci bridges could not be allocated to different VMs in
earlier times.
Make sure this is supposed to work before you tackle the harder issues.
2012/6/22 amiard brice <perso@brice-amiard.fr>:> Hello
>
>
>
> I''m working on XEN since one month now.
>
>
>
> I want create four domU with 2 PCI device per dom.
>
> I''ve bought a backplane with 3 bridges PCIe 1x to PCI and 4 PCI
> directly connected to the PCH (Q67).
>
> (Moreover PCI card (ethernet card) are composed with a PCI bridge)
>
>
>
> I''ve succefuly created 4 domU with bakend devices, with option
> "pci-backend-strict-check" disable.
>
> Ive try with it but even if all PCI ports behind PCIe to PCI bridge
> appears in pci-list-assignable-devices, domU wont start, with an
> error message "device XX:XX.X need to be co-assigned with XX:XX.X, but
> it''s not owned by pci-backend
>
>
>
> The probleme is : even if my domU are configuer with 1 VCPUs for each
> of them, when domU acces to I/O devices, VCPUs aren''t separated
(I''ve
> notice this using virt-manager charts). During a basic CPU benchmark
> on a domU, VCPUs are isolated.
>
>
>
> I Have attached "lspci-v", "xm dmesg", "xm
> pci-list-assignable-devices", "default-grub", "xm
info" and a domU cfg
file>
>
>
> A share core is very problematic my RT application
>
>
>
> thank you for your help in advance
>
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
--
the purpose of libvirt is to provide an abstraction layer hiding all xen
features added since 2006 until they were finally understood and copied by
the kvm devs.
On Thu, Jun 28, 2012 at 8:52 AM, amiard brice <perso@brice-amiard.fr> wrote:> But as I said, my backplane ("motherboard extension") is composed with 4 > bridge "PCI-e to PCI". And I have one domU for each "PCI-e to PCI bridge", > so according to the intel_VT-d pdf, it could be possible in my case to have > 4 domU (one for each PCIe to PCI bridges).This sounds like a question it would be useful to ask on xen-devel. (Unfortuantely I''m not familiar enough with PCIe-to-PCI bridges to give you a good answer.) -George> > In my opinion it could be due to the EPT table. In my "xm dmesg" I could see > "EPT not enable" and "CPU capabilities : EPT table 2Mo", but I can''t active > it (I''ve try different options in default/grub without succes) > > (if devices behind the same pci bridges could not be allocated to different > VMs, it'' simply because VT-d use specific interruption command call MSI-X in > aim to identify a domU, and those interruptions commands could be possible > only with PCIe, you can have more information in the pdf intel VT-d). > > thank you for your help in advance > > > > > > > Hi, > > I''ve never been successful with VT-d-style pci delegation, but I read > devices behind pci bridges could not be allocated to different VMs in > earlier times. > > Make sure this is supposed to work before you tackle the harder issues. > > 2012/6/22 amiard brice <perso@brice-amiard.fr>: >> Hello >> >> >> >> I''m working on XEN since one month now. >> >> >> >> I want create four domU with 2 PCI device per dom. >> >> I''ve bought a backplane with 3 bridges PCIe 1x to PCI and 4 PCI >> directly connected to the PCH (Q67). >> >> (Moreover PCI card (ethernet card) are composed with a PCI bridge) >> >> >> >> I''ve succefuly created 4 domU with bakend devices, with option >> "pci-backend-strict-check" disable. >> >> I’ve try with it but even if all PCI ports behind PCIe to PCI bridge >> appears in pci-list-assignable-devices, domU won’t start, with an >> error message "device XX:XX.X need to be co-assigned with XX:XX.X, but >> it''s not owned by pci-backend >> >> >> >> The probleme is : even if my domU are configuer with 1 VCPUs for each >> of them, when domU acces to I/O devices, VCPUs aren''t separated (I''ve >> notice this using virt-manager charts). During a basic CPU benchmark >> on a domU, VCPUs are isolated. >> >> >> >> I Have attached "lspci-v", "xm dmesg", "xm >> pci-list-assignable-devices", "default-grub", "xm info" and a “domU cfg > file” >> >> >> >> A share core is very problematic my RT application >> >> >> >> thank you for your help in advance >> >> >> >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xen.org >> http://lists.xen.org/xen-users > > > > -- > the purpose of libvirt is to provide an abstraction layer hiding all xen > features added since 2006 until they were finally understood and copied by > the kvm devs. > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users
Hey, On Fri, Jun 22, 2012 at 3:58 PM, amiard brice <perso@brice-amiard.fr> wrote:> Hello > > > > I'm working on XEN since one month now. > > > > I want create four domU with 2 PCI device per dom. > > I've bought a backplane with 3 bridges PCIe 1x to PCI and 4 PCI directly > connected to the PCH (Q67). > > (Moreover PCI card (ethernet card) are composed with a PCI bridge)Why do you still use PCI ethernet cards when your system provides PCIe connectivity?> I've succefuly created 4 domU with bakend devices, with option > "pci-backend-strict-check" disable. > > I’ve try with it but even if all PCI ports behind PCIe to PCI bridge appears > in pci-list-assignable-devices, domU won’t start, with an error message > "device XX:XX.X need to be co-assigned with XX:XX.X, but it's not owned by > pci-backendDoes ist work when you only delegate the one ethernet card directly connected via PCI to a domU?> > > > The probleme is : even if my domU are configuer with 1 VCPUs for each of > them, when domU acces to I/O devices, VCPUs aren't separated (I've notice > this using virt-manager charts). During a basic CPU benchmark on a domU, > VCPUs are isolated.Can you elaborate on "when domU acces to I/O devices, VCPUs aren't separated[..]"? Regards, Linus _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
For example when I transfer some file between a other PC and a domU through the Ethernet card connected using VT-d, VCPU's charts are identical between each domU (it's like if I/O instruction aren't separated between CPU cores, I think XEN can't determine CPU cores assignation for d'I/O, so it execute instruction on each CPU, and after DMA remapping transfer data to the correct domU). It could be a problem between LoAPIC and APIC. I've notice something : My hardware support EPT table, but I can active it (I've try different parameters in default grub without succes) -----Message d'origine----- De : Linus van Geuns [mailto:linus@vangeuns.name] Envoyé : samedi 30 juin 2012 12:51 À : amiard brice Cc : xen-users@lists.xen.org Objet : Re: [Xen-users] VT-d CPU resources Hey, On Fri, Jun 22, 2012 at 3:58 PM, amiard brice <perso@brice-amiard.fr> wrote:> Hello > > > > I'm working on XEN since one month now. > > > > I want create four domU with 2 PCI device per dom. > > I've bought a backplane with 3 bridges PCIe 1x to PCI and 4 PCI > directly connected to the PCH (Q67). > > (Moreover PCI card (ethernet card) are composed with a PCI bridge)Why do you still use PCI ethernet cards when your system provides PCIe connectivity?> I've succefuly created 4 domU with bakend devices, with option > "pci-backend-strict-check" disable. > > I’ve try with it but even if all PCI ports behind PCIe to PCI bridge > appears in pci-list-assignable-devices, domU won’t start, with an > error message "device XX:XX.X need to be co-assigned with XX:XX.X, but > it's not owned by pci-backendDoes ist work when you only delegate the one ethernet card directly connected via PCI to a domU?> > > > The probleme is : even if my domU are configuer with 1 VCPUs for each > of them, when domU acces to I/O devices, VCPUs aren't separated (I've > notice this using virt-manager charts). During a basic CPU benchmark > on a domU, VCPUs are isolated.Can you elaborate on "when domU acces to I/O devices, VCPUs aren't separated[..]"? Regards, Linus _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users