Hi, I have a Lustre filesystem where all servers and clients are using DDR IB interconnects. Customer wants to add an additional client which has a 10 GigE interface that they want to use to connect to the filesystem. The Lustre documentation says that iWarp interfaces are supported using OFED drivers. What would the syntax of the modprobe.conf.local file look like? I have tried this: options lnet networks=o2ib(eth2) Is that correct? The client is connected to the IB fabric using a Voltaire 10 GigE line card on a IB switch. Has anyone tested such a configuration? Should I expect it to work? Thanks,
On Fri, 2009-05-29 at 08:57 -0500, Dennis Nelson wrote:> I have tried this: > > options lnet networks=o2ib(eth2) > > Is that correct?I would think so, but someone more experienced in the Lustre o2ib LND and iWarp should chime in on that -- I''ve never used iWarp, only IB.> The client is connected to the IB fabric using a Voltaire 10 GigE line card > on a IB switch. Has anyone tested such a configuration? Should I expect it > to work?I don''t expect that to work, unless the Voltaire is converting iWarp to IB -- and I highly doubt that. It is more likely using EoIB to transport the ethernet frames over the IB fabric. You will likely need a Lustre router to sit between the two fabrics. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office
David Dillow wrote:> On Fri, 2009-05-29 at 08:57 -0500, Dennis Nelson wrote: > >> I have tried this: >> >> options lnet networks=o2ib(eth2) >> >> Is that correct? >> > > I would think so, but someone more experienced in the Lustre o2ib LND > and iWarp should chime in on that -- I''ve never used iWarp, only IB. >we also need this for iWarp: options o2iblnd map_on_demand=64 or map_on_demand=32 if you failed to startup Regards Liang> >> The client is connected to the IB fabric using a Voltaire 10 GigE line card >> on a IB switch. Has anyone tested such a configuration? Should I expect it >> to work? >> > > I don''t expect that to work, unless the Voltaire is converting iWarp to > IB -- and I highly doubt that. It is more likely using EoIB to transport > the ethernet frames over the IB fabric. You will likely need a Lustre > router to sit between the two fabrics. >