Hi, I installed xen3.4.2 running kernel 2.6.18.8 as dom0. I would like to create multiple domu, all of them having access to my infiniband card. I looked at the documentation and search the web, but I cannot find how to set this up!!! I need that for many reason, 1) one of out filesystem is lustre on infiniband 2) using parallel application with infiniband First of all, is it possible? Thanks, _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I have gotten as far as booting the dom0 kernel 2.6.18-164.11.1.el5xen that comes with redhat/centos and verifying that in dom0 I can see and talk to the infiniband card. next step is to try to do pci-passthrough to a domU. As far as passthrough to multiple domU it is supposed to depend on what hardware version of the infiniband card you have. I am going out for bid for the latest version from mellanox, (connectx) which is supposed to support multiple virtual channels under virtualization, right now vmware and soon also xen. We are still early in the process. I think the kind we have in our test machine, infiniscale, will work with at most one domU at a time but I haven''t even got that far yet. Steve Timm On Thu, 11 Feb 2010, Jonathan Ferland wrote:> Hi, > > I installed xen3.4.2 running kernel 2.6.18.8 as dom0. I would like to create > multiple domu, all of them having access to my infiniband card. I looked at > the documentation and search the web, but I cannot find how to set this up!!! > > I need that for many reason, > 1) one of out filesystem is lustre on infiniband > 2) using parallel application with infiniband > > First of all, is it possible? > > Thanks, > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- ------------------------------------------------------------------ Steven C. Timm, Ph.D (630) 840-8525 timm@fnal.gov http://home.fnal.gov/~timm/ Fermilab Computing Division, Scientific Computing Facilities, Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Steve, Thanks for your prompt answer. I also manage to run ofed stuff on my dom0. My infiniband is mellanox connectx, so maybe the latest driver will support what I want. The issue I have right now concern the compilation of the latest ofed driver. If I boot a clean node with no ofed installed, and try to build OFED-1.4-mlnx8, it failed under xen... It search for iscsi, and found the one in used by xen, then fail to uninstall it, and stop the build. How did you manage to install ofed driver properly? I was able to work around by keeping the openib driver from the kernel and install the tools from ofed to make infiniband work, but I would prefer to have the real thing.. Thanks, Jonathan Steven Timm wrote:> I have gotten as far as booting the dom0 kernel > 2.6.18-164.11.1.el5xen that comes with redhat/centos > and verifying that in dom0 I can see and talk to the infiniband > card. next step is to try to do pci-passthrough to a domU. > > As far as passthrough to multiple domU it is supposed to depend > on what hardware version of the infiniband card you have. I > am going out for bid for the latest version from mellanox, (connectx) > which is supposed to support multiple virtual channels under > virtualization, > right now vmware and soon also xen. We are still early in the process. > I think the kind we have in our test machine, infiniscale, will work > with at most one domU at a time but I haven''t even got that far yet. > > Steve Timm > > > > On Thu, 11 Feb 2010, Jonathan Ferland wrote: > >> Hi, >> >> I installed xen3.4.2 running kernel 2.6.18.8 as dom0. I would like to >> create multiple domu, all of them having access to my infiniband >> card. I looked at the documentation and search the web, but I cannot >> find how to set this up!!! >> >> I need that for many reason, >> 1) one of out filesystem is lustre on infiniband >> 2) using parallel application with infiniband >> >> First of all, is it possible? >> >> Thanks, >> >> >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xensource.com >> http://lists.xensource.com/xen-users >> >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users