Masroor Vettuparambil
2008-Jan-15 08:39 UTC
[Xen-devel] Xen support for PCI-SIG I/O Virtualization
Hi, what is the current support of Xen for PCI-SIG IOV SR/MR spec? I have a PCIe device which supports multiple virtual functions(VF), and want to enable direct assignment of each of these VFs to different domUs. So is it possible to assign a PCIe virtual channel/function to domU directly? Can anybody explain me how could I do this. thank you, Masroor _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Masroor Vettuparambil wrote:> Hi, > > what is the current support of Xen for PCI-SIG IOV SR/MR spec? > I have a PCIe device which supports multiple virtual functions(VF), and > want to enable direct assignment of each of these VFs to different domUs. > So is it possible to assign a PCIe virtual channel/function to domU > directly?Have you seen the work we did recently to allow smart NIC''s to be accessed directly from the guest? Essentially your "accelerated" driver is a plugin to netfront (which knows how to send/receive packets with safe direct access hardware to the NIC), and a plugin for netback which deals with the privileged part of programming the NIC (e.g. creating virtual devices and handing them out to the guests as appropriate). Among other things, this architecture means you can support migrating to machines with no smart hardware, or a different smart NIC. Let me know if you want to know more. As far as programming the IOMMU goes, I''ve seen various patches flying around so I know work has been done here, but I don''t know any more than that. Cheers, Greg -- Greg Law +44 1223 518 040 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Woller, Thomas
2008-Jan-17 00:20 UTC
RE: [Xen-devel] Xen support for PCI-SIG I/O Virtualization
> Let me know if you want to know more.Curious if your patches have any multi-root logic.. Or if only SR. Do you have specific smart NICs that you recommend that work with your patches/Xen? thanks, Tom> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Greg Law > Sent: Wednesday, January 16, 2008 11:56 AM > To: Masroor Vettuparambil > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Xen support for PCI-SIG I/O Virtualization > > Masroor Vettuparambil wrote: > > Hi, > > > > what is the current support of Xen for PCI-SIG IOV SR/MR spec? > > I have a PCIe device which supports multiple virtual functions(VF), > > and want to enable direct assignment of each of these VFs > to different domUs. > > So is it possible to assign a PCIe virtual channel/function to domU > > directly? > > Have you seen the work we did recently to allow smart NIC''s > to be accessed directly from the guest? Essentially your > "accelerated" driver is a plugin to netfront (which knows how > to send/receive packets with safe direct access hardware to > the NIC), and a plugin for netback which deals with the > privileged part of programming the NIC (e.g. creating virtual > devices and handing them out to the guests as appropriate). > Among other things, this architecture means you can support > migrating to machines with no smart hardware, or a different > smart NIC. > > Let me know if you want to know more. > > As far as programming the IOMMU goes, I''ve seen various > patches flying around so I know work has been done here, but > I don''t know any more than that. > > Cheers, > > Greg > -- > Greg Law +44 1223 518 040 > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Woller, Thomas wrote:>> Let me know if you want to know more. > Curious if your patches have any multi-root logic.. Or if only SR.Actually, our patches do nothing with IOV (SR or MR). They are a framework which allows one to write plugin drivers for netfront/netback for accelerated network tx/rx. The framework does not mandate how this is implemented. We expect it to be used by smart devices which can be accessed directly from the guest, but even that is not strictly necessary to use the framework. If the NIC does allow safe access directly from the guest, it might be that the silicon uses PCI IOV, or some other mechanism (e.g. our smart NIC uses a proprietary mechanism which pre-dates IOV).> > Do you have specific smart NICs that you recommend that work with your > patches/Xen?Yes, ours :) You can currently buy them as "SMC Tigertron" cards. Our plugin drivers will be submitted to this list soon. Cheers, Greg -- Greg Law Solarflare Communications +44 1223 518 040 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Masroor Vettuparambil
2008-Jan-17 12:02 UTC
RE: [Xen-devel] Xen support for PCI-SIG I/O Virtualization
hi Greg, thanks for the comment. I have yet to go through ur work. Please provide a quick link. I would like to know whether your proprietary mechanism supports virtual functions just like in IOV. If so, can you brief me on how this is accessed from the host and assigned to guest domains? Infact I am more interested how this is happening in IOV method. regards Masroor -----Original Message----- From: Greg Law [mailto:glaw@solarflare.com] Sent: Thursday, January 17, 2008 4:59 PM To: Woller, Thomas Cc: Masroor Vettuparambil; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Xen support for PCI-SIG I/O Virtualization Woller, Thomas wrote:>> Let me know if you want to know more. > Curious if your patches have any multi-root logic.. Or if only SR.Actually, our patches do nothing with IOV (SR or MR). They are a framework which allows one to write plugin drivers for netfront/netback for accelerated network tx/rx. The framework does not mandate how this is implemented. We expect it to be used by smart devices which can be accessed directly from the guest, but even that is not strictly necessary to use the framework. If the NIC does allow safe access directly from the guest, it might be that the silicon uses PCI IOV, or some other mechanism (e.g. our smart NIC uses a proprietary mechanism which pre-dates IOV).> > Do you have specific smart NICs that you recommend that work with your> patches/Xen?Yes, ours :) You can currently buy them as "SMC Tigertron" cards. Our plugin drivers will be submitted to this list soon. Cheers, Greg -- Greg Law Solarflare Communications +44 1223 518 040 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Masroor, Masroor Vettuparambil wrote:> hi Greg, > > thanks for the comment. > I have yet to go through ur work. Please provide a quick link. > > I would like to know whether your proprietary mechanism supports virtual > functions just like in IOV. > If so, can you brief me on how this is accessed from the host and > assigned to guest domains? > Infact I am more interested how this is happening in IOV method.The framework is deliberately very generic. Essentially it is just a way for netfront to have a plugin which can handle tx/rx of packets directly. How this works (IOV, proprietary, whatever) is not the concern of the framework: you write netfront and netback plugins to implement it as required by your hardware. The framework is not really intended to avoid you needing to write lots of code - it''s not a "library". Rather it''s the minimum amount of infrastructure required so that VMs can be migrated between different hosts with different types of smart NICs (or no smart NICs). No one has written an implementation of plugins that use IOV yet, but I''m pretty confident it will work without major changes. As for the code, a good place to start is the definition of struct netfront_accel_hooks in linux/drivers/xen/netfront/netfront.h The set of patches were posted to xen-devel by Kieran Mansley Fri, 18 May 2007. Sadly, I''m not sure what the change set number is, but you can get more info here: http://lwn.net/Articles/235124/ g -- Greg Law Solarflare Communications +44 1223 518 040 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel