Hi, I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for Xen-IB source codes, but it did''t exist anymore. Are there other ways to get the Xen-IB source codes? Thanks. Regards, Yi-Man _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, May 19, 2011 at 1:27 PM, maus <maustudent@gmail.com> wrote:> Hi, > > I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for Xen-IB > source codes, but it did''t exist anymore. > Are there other ways to get the Xen-IB source codes? > Thanks.Probably http://xenbits.xensource.com/ext/xen-smartio/ -- Fajar _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 2011-05-23 at 11:03 +0100, Fajar A. Nugraha wrote:> On Thu, May 19, 2011 at 1:27 PM, maus <maustudent@gmail.com> wrote: > > Hi, > > > > I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for Xen-IB > > source codes, but it did''t exist anymore. > > Are there other ways to get the Xen-IB source codes? > > Thanks. > > Probably http://xenbits.xensource.com/ext/xen-smartio/Note that this tree hasn''t been touched since 2006 and appears to be based on a Xen 3.0.0 era hypervisor and the changes mostly consist of patches to the 2.6.16 kernel used in that era. IOW I doubt you will find much useful code in that repository for further development and you most certainly should not be deploying anything based on it. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, There many file in http://xenbits.xensource.com/ext/xen-smartio/ , which should I use for Xen-IB? Use the following three? Updated defconfig to include infiniband driver as modules for dom0/domU<http://xenbits.xensource.com/ext/xen-smartio/rev/b796b92c0cbe> Updating drivers/infiniband to latest. Should shrink size of xen_ib.patch in<http://xenbits.xensource.com/ext/xen-smartio/rev/13ee554d2b7c> Add support for Infiniband into Xen<http://xenbits.xensource.com/ext/xen-smartio/rev/610fe641e6e8> Yi-Man 2011/5/23 Ian Campbell <Ian.Campbell@citrix.com>> On Mon, 2011-05-23 at 11:03 +0100, Fajar A. Nugraha wrote: > > On Thu, May 19, 2011 at 1:27 PM, maus <maustudent@gmail.com> wrote: > > > Hi, > > > > > > I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for > Xen-IB > > > source codes, but it did''t exist anymore. > > > Are there other ways to get the Xen-IB source codes? > > > Thanks. > > > > Probably http://xenbits.xensource.com/ext/xen-smartio/ > > Note that this tree hasn''t been touched since 2006 and appears to be > based on a Xen 3.0.0 era hypervisor and the changes mostly consist of > patches to the 2.6.16 kernel used in that era. > > IOW I doubt you will find much useful code in that repository for > further development and you most certainly should not be deploying > anything based on it. > > Ian. > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Please do not top post. On Tue, 2011-05-24 at 11:07 +0100, maus wrote:> Hi, > > There many file in http://xenbits.xensource.com/ext/xen-smartio/ , > which should I use for Xen-IB? > Use the following three? > Updated defconfig to include infiniband driver as modules for > dom0/domU > Updating drivers/infiniband to latest. Should shrink size of > xen_ib.patch in > Add support for Infiniband into XenAgain, these changesets are >5 years old and have not been looked at or maintained since -- I am very sceptical that you will find anything of use on a modern Xen system in there. I expect that IB support in the kernels we currently use for dom0 is vastly improved over what was available in the 2.6.16 tree referenced in the above repository. Perhaps you don''t even need anything special outside of what is available in the upstream dom0 kernel. Perhaps if you explain what you are actually trying to achieve then people will be able to better advise you? Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, I have a project which is make KVM use InfiniBand. However, since Xen can use InfiniBand through Xen-IB, I want to try to make Xen use InfiniBand first. Through that way, I think it may give me some idea about how to make KVM use InfiniBand. So, as you said, I just use "Updated defconfig to include infiniband driver as modules for dom0/domU" to do? And since I don''t know how to I do first for Xen-IB, is there any tutorial for Xen-IB? Or does Xen build-in support IB currently? Yi-Man 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>> Please do not top post. > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: > > Hi, > > > > There many file in http://xenbits.xensource.com/ext/xen-smartio/ , > > which should I use for Xen-IB? > > Use the following three? > > Updated defconfig to include infiniband driver as modules for > > dom0/domU > > Updating drivers/infiniband to latest. Should shrink size of > > xen_ib.patch in > > Add support for Infiniband into Xen > > Again, these changesets are >5 years old and have not been looked at or > maintained since -- I am very sceptical that you will find anything of > use on a modern Xen system in there. > > I expect that IB support in the kernels we currently use for dom0 is > vastly improved over what was available in the 2.6.16 tree referenced in > the above repository. Perhaps you don''t even need anything special > outside of what is available in the upstream dom0 kernel. > > Perhaps if you explain what you are actually trying to achieve then > people will be able to better advise you? > > Ian. > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 2011-05-25 at 14:40 +0100, maus wrote:> Hi, > > I have a project which is make KVM use InfiniBand. > However, since Xen can use InfiniBand through Xen-IB, I want to try to > make Xen use InfiniBand first. > Through that way, I think it may give me some idea about how to make > KVM use InfiniBand. > So, as you said, I just use "Updated defconfig to include infiniband > driver as modules for dom0/domU" to do? > And since I don''t know how to I do first for Xen-IB, is there any > tutorial for Xen-IB? > Or does Xen build-in support IB currently?I''m not sure what IB support you expect Xen to have over and above the driver and utils support, which are functions of the dom0 kernel and distro and not the hypervisor. Have you got IB working on a native system? I expect that if you do that then Xen or KVM will be able to make use of it basically for free, i.e. configure a VIF on an IP-over-IB bridge or use a local IP-over-IB address in dom0 for migration etc etc. Ian.> > Yi-Man > > > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> > Please do not top post. > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: > > Hi, > > > > There many file in > http://xenbits.xensource.com/ext/xen-smartio/ , > > which should I use for Xen-IB? > > Use the following three? > > Updated defconfig to include infiniband driver as modules > for > > dom0/domU > > Updating drivers/infiniband to latest. Should shrink size of > > xen_ib.patch in > > Add support for Infiniband into Xen > > > Again, these changesets are >5 years old and have not been > looked at or > maintained since -- I am very sceptical that you will find > anything of > use on a modern Xen system in there. > > I expect that IB support in the kernels we currently use for > dom0 is > vastly improved over what was available in the 2.6.16 tree > referenced in > the above repository. Perhaps you don''t even need anything > special > outside of what is available in the upstream dom0 kernel. > > Perhaps if you explain what you are actually trying to achieve > then > people will be able to better advise you? > > Ian. > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, I know use IP-over-IB can let KVM or Xen use IB, but what I have to do is like Xen-IB. Not just use IB. So, there is no toturial for Xen-IB,right? Yi-Man 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>> On Wed, 2011-05-25 at 14:40 +0100, maus wrote: > > Hi, > > > > I have a project which is make KVM use InfiniBand. > > However, since Xen can use InfiniBand through Xen-IB, I want to try to > > make Xen use InfiniBand first. > > Through that way, I think it may give me some idea about how to make > > KVM use InfiniBand. > > So, as you said, I just use "Updated defconfig to include infiniband > > driver as modules for dom0/domU" to do? > > And since I don''t know how to I do first for Xen-IB, is there any > > tutorial for Xen-IB? > > Or does Xen build-in support IB currently? > > I''m not sure what IB support you expect Xen to have over and above the > driver and utils support, which are functions of the dom0 kernel and > distro and not the hypervisor. > > Have you got IB working on a native system? I expect that if you do that > then Xen or KVM will be able to make use of it basically for free, i.e. > configure a VIF on an IP-over-IB bridge or use a local IP-over-IB > address in dom0 for migration etc etc. > > Ian. > > > > > Yi-Man > > > > > > > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> > > Please do not top post. > > > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: > > > Hi, > > > > > > There many file in > > http://xenbits.xensource.com/ext/xen-smartio/ , > > > which should I use for Xen-IB? > > > Use the following three? > > > Updated defconfig to include infiniband driver as modules > > for > > > dom0/domU > > > Updating drivers/infiniband to latest. Should shrink size of > > > xen_ib.patch in > > > Add support for Infiniband into Xen > > > > > > Again, these changesets are >5 years old and have not been > > looked at or > > maintained since -- I am very sceptical that you will find > > anything of > > use on a modern Xen system in there. > > > > I expect that IB support in the kernels we currently use for > > dom0 is > > vastly improved over what was available in the 2.6.16 tree > > referenced in > > the above repository. Perhaps you don''t even need anything > > special > > outside of what is available in the upstream dom0 kernel. > > > > Perhaps if you explain what you are actually trying to achieve > > then > > people will be able to better advise you? > > > > Ian. > > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2011-05-26 at 15:04 +0100, maus wrote:> Hi, > > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do > is like Xen-IB. Not just use IB.What is Xen-IB exactly?> So, there is no toturial for Xen-IB,right?That''s right, there is no documentation and anyone who would have been involved with this stuff is long gone. Ian.> > > Yi-Man > > > > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote: > > Hi, > > > > I have a project which is make KVM use InfiniBand. > > However, since Xen can use InfiniBand through Xen-IB, I want > to try to > > make Xen use InfiniBand first. > > Through that way, I think it may give me some idea about how > to make > > KVM use InfiniBand. > > So, as you said, I just use "Updated defconfig to include > infiniband > > driver as modules for dom0/domU" to do? > > And since I don''t know how to I do first for Xen-IB, is > there any > > tutorial for Xen-IB? > > Or does Xen build-in support IB currently? > > > I''m not sure what IB support you expect Xen to have over and > above the > driver and utils support, which are functions of the dom0 > kernel and > distro and not the hypervisor. > > Have you got IB working on a native system? I expect that if > you do that > then Xen or KVM will be able to make use of it basically for > free, i.e. > configure a VIF on an IP-over-IB bridge or use a local > IP-over-IB > address in dom0 for migration etc etc. > > Ian. > > > > > > Yi-Man > > > > > > > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> > > Please do not top post. > > > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: > > > Hi, > > > > > > There many file in > > http://xenbits.xensource.com/ext/xen-smartio/ , > > > which should I use for Xen-IB? > > > Use the following three? > > > Updated defconfig to include infiniband driver as > modules > > for > > > dom0/domU > > > Updating drivers/infiniband to latest. Should > shrink size of > > > xen_ib.patch in > > > Add support for Infiniband into Xen > > > > > > Again, these changesets are >5 years old and have > not been > > looked at or > > maintained since -- I am very sceptical that you > will find > > anything of > > use on a modern Xen system in there. > > > > I expect that IB support in the kernels we currently > use for > > dom0 is > > vastly improved over what was available in the > 2.6.16 tree > > referenced in > > the above repository. Perhaps you don''t even need > anything > > special > > outside of what is available in the upstream dom0 > kernel. > > > > Perhaps if you explain what you are actually trying > to achieve > > then > > people will be able to better advise you? > > > > Ian. > > > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:> On Thu, 2011-05-26 at 15:04 +0100, maus wrote: > > Hi, > > > > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do > > is like Xen-IB. Not just use IB. > > What is Xen-IB exactly? >Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ? I can''t really think of other uses.. -- Pasi> > So, there is no toturial for Xen-IB,right? > > That''s right, there is no documentation and anyone who would have been > involved with this stuff is long gone. > > Ian. > > > > > > > Yi-Man > > > > > > > > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com> > > On Wed, 2011-05-25 at 14:40 +0100, maus wrote: > > > Hi, > > > > > > I have a project which is make KVM use InfiniBand. > > > However, since Xen can use InfiniBand through Xen-IB, I want > > to try to > > > make Xen use InfiniBand first. > > > Through that way, I think it may give me some idea about how > > to make > > > KVM use InfiniBand. > > > So, as you said, I just use "Updated defconfig to include > > infiniband > > > driver as modules for dom0/domU" to do? > > > And since I don''t know how to I do first for Xen-IB, is > > there any > > > tutorial for Xen-IB? > > > Or does Xen build-in support IB currently? > > > > > > I''m not sure what IB support you expect Xen to have over and > > above the > > driver and utils support, which are functions of the dom0 > > kernel and > > distro and not the hypervisor. > > > > Have you got IB working on a native system? I expect that if > > you do that > > then Xen or KVM will be able to make use of it basically for > > free, i.e. > > configure a VIF on an IP-over-IB bridge or use a local > > IP-over-IB > > address in dom0 for migration etc etc. > > > > Ian. > > > > > > > > > > Yi-Man > > > > > > > > > > > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> > > > Please do not top post. > > > > > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: > > > > Hi, > > > > > > > > There many file in > > > http://xenbits.xensource.com/ext/xen-smartio/ , > > > > which should I use for Xen-IB? > > > > Use the following three? > > > > Updated defconfig to include infiniband driver as > > modules > > > for > > > > dom0/domU > > > > Updating drivers/infiniband to latest. Should > > shrink size of > > > > xen_ib.patch in > > > > Add support for Infiniband into Xen > > > > > > > > > Again, these changesets are >5 years old and have > > not been > > > looked at or > > > maintained since -- I am very sceptical that you > > will find > > > anything of > > > use on a modern Xen system in there. > > > > > > I expect that IB support in the kernels we currently > > use for > > > dom0 is > > > vastly improved over what was available in the > > 2.6.16 tree > > > referenced in > > > the above repository. Perhaps you don''t even need > > anything > > > special > > > outside of what is available in the upstream dom0 > > kernel. > > > > > > Perhaps if you explain what you are actually trying > > to achieve > > > then > > > people will be able to better advise you? > > > > > > Ian. > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, I happen to have done a huge amount of research into this as our infrastructure is all based on converged Infiniband. Xen-IB and SmartIO were two seperate attempts at building a paravirtualised Infinband driver for Xen that allowed guests to use native Infinband protocols at close to native speeds. Being paravirtualised they were live-migration cappable and many domU could share a single Infiniband adapter. As far as I am aware both of the projects are dead, however SR-IOV isn''t long off for Infiniband, Mellanox is on target to push theirs upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be quick to follow I would imagine. SR-IOV will allow pretty similar functionality but less portable in terms of live-migrate etc. It is worth noting here that you can''t bridge IPoIB, IPoIB is not a layer 2 device and doesn''t understand Ethernet. You can use EoIB (Ethernet over IB) or routing to provide network access to domUs. Feel free to email me if you need any help with Infiniband and Xen. Joseph. On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote: >> On Thu, 2011-05-26 at 15:04 +0100, maus wrote: >> > Hi, >> > >> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do >> > is like Xen-IB. Not just use IB. >> >> What is Xen-IB exactly? >> > > Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ? > I can''t really think of other uses.. > > -- Pasi > > >> > So, there is no toturial for Xen-IB,right? >> >> That''s right, there is no documentation and anyone who would have been >> involved with this stuff is long gone. >> >> Ian. >> >> > >> > >> > Yi-Man >> > >> > >> > >> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com> >> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote: >> > > Hi, >> > > >> > > I have a project which is make KVM use InfiniBand. >> > > However, since Xen can use InfiniBand through Xen-IB, I want >> > to try to >> > > make Xen use InfiniBand first. >> > > Through that way, I think it may give me some idea about how >> > to make >> > > KVM use InfiniBand. >> > > So, as you said, I just use "Updated defconfig to include >> > infiniband >> > > driver as modules for dom0/domU" to do? >> > > And since I don''t know how to I do first for Xen-IB, is >> > there any >> > > tutorial for Xen-IB? >> > > Or does Xen build-in support IB currently? >> > >> > >> > I''m not sure what IB support you expect Xen to have over and >> > above the >> > driver and utils support, which are functions of the dom0 >> > kernel and >> > distro and not the hypervisor. >> > >> > Have you got IB working on a native system? I expect that if >> > you do that >> > then Xen or KVM will be able to make use of it basically for >> > free, i.e. >> > configure a VIF on an IP-over-IB bridge or use a local >> > IP-over-IB >> > address in dom0 for migration etc etc. >> > >> > Ian. >> > >> > >> > > >> > > Yi-Man >> > > >> > > >> > > >> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> >> > > Please do not top post. >> > > >> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: >> > > > Hi, >> > > > >> > > > There many file in >> > > http://xenbits.xensource.com/ext/xen-smartio/ , >> > > > which should I use for Xen-IB? >> > > > Use the following three? >> > > > Updated defconfig to include infiniband driver as >> > modules >> > > for >> > > > dom0/domU >> > > > Updating drivers/infiniband to latest. Should >> > shrink size of >> > > > xen_ib.patch in >> > > > Add support for Infiniband into Xen >> > > >> > > >> > > Again, these changesets are >5 years old and have >> > not been >> > > looked at or >> > > maintained since -- I am very sceptical that you >> > will find >> > > anything of >> > > use on a modern Xen system in there. >> > > >> > > I expect that IB support in the kernels we currently >> > use for >> > > dom0 is >> > > vastly improved over what was available in the >> > 2.6.16 tree >> > > referenced in >> > > the above repository. Perhaps you don''t even need >> > anything >> > > special >> > > outside of what is available in the upstream dom0 >> > kernel. >> > > >> > > Perhaps if you explain what you are actually trying >> > to achieve >> > > then >> > > people will be able to better advise you? >> > > >> > > Ian. >> > > >> > > >> > > >> > >> > >> > >> > >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- Kind regards, Joseph. Founder | Director Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56 99 52 | Mobile: 0428 754 846 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:> On Thu, 2011-05-26 at 15:04 +0100, maus wrote: > > Hi, > > > > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do > > is like Xen-IB. Not just use IB. > > What is Xen-IB exactly?OS and hypervisor bypass, i.e., direct access from a user application running in a domU to the Infiniband adapter (HCA), bypassing both the domU kernel and Xen for frequent data-path operations. Configuration and privileged operations still go through the kernel and the hypervisor.> > So, there is no toturial for Xen-IB,right? > > That''s right, there is no documentation and anyone who would have > been involved with this stuff is long gone.Actually, there is a very nice paper describing the approach: "High Performance VMM-Bypass I/O in Virtual Machines", Jiuxing Liu, Wei Huang, Bulent Abali, Dhabaleswar K. Panda, USENIX ATC ''06 Cheers, Muli _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, I happen to have done a huge amount of research into this as our infrastructure is all based on converged Infiniband. Xen-IB and SmartIO were two seperate attempts at building a paravirtualised Infinband driver for Xen that allowed guests to use native Infinband protocols at close to native speeds. Being paravirtualised they were live-migration cappable and many domU could share a single Infiniband adapter. As far as I am aware both of the projects are dead, however SR-IOV isn''t long off for Infiniband, Mellanox is on target to push theirs upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be quick to follow I would imagine. SR-IOV will allow pretty similar functionality but less portable in terms of live-migrate etc. It is worth noting here that you can''t bridge IPoIB, IPoIB is not a layer 2 device and doesn''t understand Ethernet. You can use EoIB (Ethernet over IB) or routing to provide network access to domUs. Feel free to email me if you need any help with Infiniband and Xen. Joseph. On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote: >> On Thu, 2011-05-26 at 15:04 +0100, maus wrote: >> > Hi, >> > >> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do >> > is like Xen-IB. Not just use IB. >> >> What is Xen-IB exactly? >> > > Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ? > I can''t really think of other uses.. > > -- Pasi > > >> > So, there is no toturial for Xen-IB,right? >> >> That''s right, there is no documentation and anyone who would have been >> involved with this stuff is long gone. >> >> Ian. >> >> > >> > >> > Yi-Man >> > >> > >> > >> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com> >> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote: >> > > Hi, >> > > >> > > I have a project which is make KVM use InfiniBand. >> > > However, since Xen can use InfiniBand through Xen-IB, I want >> > to try to >> > > make Xen use InfiniBand first. >> > > Through that way, I think it may give me some idea about how >> > to make >> > > KVM use InfiniBand. >> > > So, as you said, I just use "Updated defconfig to include >> > infiniband >> > > driver as modules for dom0/domU" to do? >> > > And since I don''t know how to I do first for Xen-IB, is >> > there any >> > > tutorial for Xen-IB? >> > > Or does Xen build-in support IB currently? >> > >> > >> > I''m not sure what IB support you expect Xen to have over and >> > above the >> > driver and utils support, which are functions of the dom0 >> > kernel and >> > distro and not the hypervisor. >> > >> > Have you got IB working on a native system? I expect that if >> > you do that >> > then Xen or KVM will be able to make use of it basically for >> > free, i.e. >> > configure a VIF on an IP-over-IB bridge or use a local >> > IP-over-IB >> > address in dom0 for migration etc etc. >> > >> > Ian. >> > >> > >> > > >> > > Yi-Man >> > > >> > > >> > > >> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> >> > > Please do not top post. >> > > >> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: >> > > > Hi, >> > > > >> > > > There many file in >> > > http://xenbits.xensource.com/ext/xen-smartio/ , >> > > > which should I use for Xen-IB? >> > > > Use the following three? >> > > > Updated defconfig to include infiniband driver as >> > modules >> > > for >> > > > dom0/domU >> > > > Updating drivers/infiniband to latest. Should >> > shrink size of >> > > > xen_ib.patch in >> > > > Add support for Infiniband into Xen >> > > >> > > >> > > Again, these changesets are >5 years old and have >> > not been >> > > looked at or >> > > maintained since -- I am very sceptical that you >> > will find >> > > anything of >> > > use on a modern Xen system in there. >> > > >> > > I expect that IB support in the kernels we currently >> > use for >> > > dom0 is >> > > vastly improved over what was available in the >> > 2.6.16 tree >> > > referenced in >> > > the above repository. Perhaps you don''t even need >> > anything >> > > special >> > > outside of what is available in the upstream dom0 >> > kernel. >> > > >> > > Perhaps if you explain what you are actually trying >> > to achieve >> > > then >> > > people will be able to better advise you? >> > > >> > > Ian. >> > > >> > > >> > > >> > >> > >> > >> > >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- Kind regards, Joseph. Founder | Director Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56 99 52 | Mobile: 0428 754 846 -- Kind regards, Joseph. Founder | Director Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56 99 52 | Mobile: 0428 754 846 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Sun, 2011-05-29 at 10:08 +0100, Muli Ben-Yehuda wrote:> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote: > > On Thu, 2011-05-26 at 15:04 +0100, maus wrote: > > > Hi, > > > > > > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do > > > is like Xen-IB. Not just use IB. > > > > What is Xen-IB exactly? > > OS and hypervisor bypass, i.e., direct access from a user application > running in a domU to the Infiniband adapter (HCA), bypassing both the > domU kernel and Xen for frequent data-path operations. Configuration > and privileged operations still go through the kernel and the > hypervisor. > > > > So, there is no toturial for Xen-IB,right? > > > > That''s right, there is no documentation and anyone who would have > > been involved with this stuff is long gone. > > Actually, there is a very nice paper describing the approach: > > "High Performance VMM-Bypass I/O in Virtual Machines", Jiuxing > Liu, Wei Huang, Bulent Abali, Dhabaleswar K. Panda, USENIX ATC ''06Thanks for the reference. Are any of these folks still active? It seems like they could be far more helpful to the OP than anyone here (certainly more useful than me ;-))... Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Sat, 2011-05-28 at 19:15 +0100, Joseph Glanville wrote:> Hi, > > I happen to have done a huge amount of research into this as our > infrastructure is all based on converged Infiniband. > > Xen-IB and SmartIO were two seperate attempts at building a > paravirtualised Infinband driver for Xen that allowed guests to use > native Infinband protocols at close to native speeds. Being > paravirtualised they were live-migration cappable and many domU could > share a single Infiniband adapter. > > As far as I am aware both of the projects are dead,Ah, thanks for the info, sounds like there was some interesting ideas here once upon a time. I''m not 100% sure that Xen-IB and SmartIO were actually different projects, http://nowlab.cse.ohio-state.edu/projects/xen/index.html seems to suggest they are actually the same. So I guess I stand corrected: the xen-smartio repo does contain some PV IB drivers but since they are for 2.6.16 and for the Xen PV architecture (not KVM) I don''t know how much use they will be to the OP. FWIW I also found http://wiki.xensource.com/xenwiki/XenSmartIO and http://www.openfabrics.org/archives/nov2006sc/xen-ib-presentation.pdf while further researching. Ian.> however SR-IOV > isn''t long off for Infiniband, Mellanox is on target to push theirs > upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be > quick to follow I would imagine. > > SR-IOV will allow pretty similar functionality but less portable in > terms of live-migrate etc. > > It is worth noting here that you can''t bridge IPoIB, IPoIB is not a > layer 2 device and doesn''t understand Ethernet. You can use EoIB > (Ethernet over IB) or routing to provide network access to domUs. > > Feel free to email me if you need any help with Infiniband and Xen. > > Joseph. > > On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote: > > On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote: > >> On Thu, 2011-05-26 at 15:04 +0100, maus wrote: > >> > Hi, > >> > > >> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do > >> > is like Xen-IB. Not just use IB. > >> > >> What is Xen-IB exactly? > >> > > > > Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ? > > I can''t really think of other uses.. > > > > -- Pasi > > > > > >> > So, there is no toturial for Xen-IB,right? > >> > >> That''s right, there is no documentation and anyone who would have been > >> involved with this stuff is long gone. > >> > >> Ian. > >> > >> > > >> > > >> > Yi-Man > >> > > >> > > >> > > >> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com> > >> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote: > >> > > Hi, > >> > > > >> > > I have a project which is make KVM use InfiniBand. > >> > > However, since Xen can use InfiniBand through Xen-IB, I want > >> > to try to > >> > > make Xen use InfiniBand first. > >> > > Through that way, I think it may give me some idea about how > >> > to make > >> > > KVM use InfiniBand. > >> > > So, as you said, I just use "Updated defconfig to include > >> > infiniband > >> > > driver as modules for dom0/domU" to do? > >> > > And since I don''t know how to I do first for Xen-IB, is > >> > there any > >> > > tutorial for Xen-IB? > >> > > Or does Xen build-in support IB currently? > >> > > >> > > >> > I''m not sure what IB support you expect Xen to have over and > >> > above the > >> > driver and utils support, which are functions of the dom0 > >> > kernel and > >> > distro and not the hypervisor. > >> > > >> > Have you got IB working on a native system? I expect that if > >> > you do that > >> > then Xen or KVM will be able to make use of it basically for > >> > free, i.e. > >> > configure a VIF on an IP-over-IB bridge or use a local > >> > IP-over-IB > >> > address in dom0 for migration etc etc. > >> > > >> > Ian. > >> > > >> > > >> > > > >> > > Yi-Man > >> > > > >> > > > >> > > > >> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> > >> > > Please do not top post. > >> > > > >> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: > >> > > > Hi, > >> > > > > >> > > > There many file in > >> > > http://xenbits.xensource.com/ext/xen-smartio/ , > >> > > > which should I use for Xen-IB? > >> > > > Use the following three? > >> > > > Updated defconfig to include infiniband driver as > >> > modules > >> > > for > >> > > > dom0/domU > >> > > > Updating drivers/infiniband to latest. Should > >> > shrink size of > >> > > > xen_ib.patch in > >> > > > Add support for Infiniband into Xen > >> > > > >> > > > >> > > Again, these changesets are >5 years old and have > >> > not been > >> > > looked at or > >> > > maintained since -- I am very sceptical that you > >> > will find > >> > > anything of > >> > > use on a modern Xen system in there. > >> > > > >> > > I expect that IB support in the kernels we currently > >> > use for > >> > > dom0 is > >> > > vastly improved over what was available in the > >> > 2.6.16 tree > >> > > referenced in > >> > > the above repository. Perhaps you don''t even need > >> > anything > >> > > special > >> > > outside of what is available in the upstream dom0 > >> > kernel. > >> > > > >> > > Perhaps if you explain what you are actually trying > >> > to achieve > >> > > then > >> > > people will be able to better advise you? > >> > > > >> > > Ian. > >> > > > >> > > > >> > > > >> > > >> > > >> > > >> > > >> > >> > >> > >> _______________________________________________ > >> Xen-devel mailing list > >> Xen-devel@lists.xensource.com > >> http://lists.xensource.com/xen-devel > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The Xen-IB I said means VMM-bypass I/O virtualization of InfiniBand on Xen. At the beginning, the link: http://xenbits.xensource.com/ext/xen-smartio.hgI found is for that. However, it looks like Xen-IB(as I said in first line) doesn''t exist and no one would know how to build up it, right? Yi-Man 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>> On Thu, 2011-05-26 at 15:04 +0100, maus wrote: > > Hi, > > > > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do > > is like Xen-IB. Not just use IB. > > What is Xen-IB exactly? > > > So, there is no toturial for Xen-IB,right? > > That''s right, there is no documentation and anyone who would have been > involved with this stuff is long gone. > > Ian. > > > > > > > Yi-Man > > > > > > > > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com> > > On Wed, 2011-05-25 at 14:40 +0100, maus wrote: > > > Hi, > > > > > > I have a project which is make KVM use InfiniBand. > > > However, since Xen can use InfiniBand through Xen-IB, I want > > to try to > > > make Xen use InfiniBand first. > > > Through that way, I think it may give me some idea about how > > to make > > > KVM use InfiniBand. > > > So, as you said, I just use "Updated defconfig to include > > infiniband > > > driver as modules for dom0/domU" to do? > > > And since I don''t know how to I do first for Xen-IB, is > > there any > > > tutorial for Xen-IB? > > > Or does Xen build-in support IB currently? > > > > > > I''m not sure what IB support you expect Xen to have over and > > above the > > driver and utils support, which are functions of the dom0 > > kernel and > > distro and not the hypervisor. > > > > Have you got IB working on a native system? I expect that if > > you do that > > then Xen or KVM will be able to make use of it basically for > > free, i.e. > > configure a VIF on an IP-over-IB bridge or use a local > > IP-over-IB > > address in dom0 for migration etc etc. > > > > Ian. > > > > > > > > > > Yi-Man > > > > > > > > > > > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com> > > > Please do not top post. > > > > > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote: > > > > Hi, > > > > > > > > There many file in > > > http://xenbits.xensource.com/ext/xen-smartio/ , > > > > which should I use for Xen-IB? > > > > Use the following three? > > > > Updated defconfig to include infiniband driver as > > modules > > > for > > > > dom0/domU > > > > Updating drivers/infiniband to latest. Should > > shrink size of > > > > xen_ib.patch in > > > > Add support for Infiniband into Xen > > > > > > > > > Again, these changesets are >5 years old and have > > not been > > > looked at or > > > maintained since -- I am very sceptical that you > > will find > > > anything of > > > use on a modern Xen system in there. > > > > > > I expect that IB support in the kernels we currently > > use for > > > dom0 is > > > vastly improved over what was available in the > > 2.6.16 tree > > > referenced in > > > the above repository. Perhaps you don''t even need > > anything > > > special > > > outside of what is available in the upstream dom0 > > kernel. > > > > > > Perhaps if you explain what you are actually trying > > to achieve > > > then > > > people will be able to better advise you? > > > > > > Ian. > > > > > > > > > > > > > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel