Lustre discuss - May 2008 - Adding additional ethernet ports to OSS

I have a Lustre (1.6.4.3) system consisting of an MGS/MGT node and 4 OSS nodes. 
I have just added an additional dual port Ethernet board to each of the OSSs, so
they now have 3 Ethernet ports (eth0,eth1,eth2) instead of just one.
 
My question is how do I get lnet to use them all.  I have added options
tcp0(eth0,eth1,eth2) to the modprobe.conf file and "lctl
interface_list" shows all three, but it appears that all the traffic is
still going over eth0 and eth1 and eth2 are not being used.
 
Do I need to do a writeconf on the MGS to get these new boards recognised?
 
Thanks in advance,
 
Ron Jerome,
National Research Council Canada
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080524/2232a2dd/attachment.html

On Sat, 2008-05-24 at 21:29 -0400, Jerome, Ron wrote:
> I have a Lustre (1.6.4.3) system consisting of an MGS/MGT node and 4
> OSS nodes.  I have just added an additional dual port Ethernet board
> to each of the OSSs, so they now have 3 Ethernet ports
> (eth0,eth1,eth2) instead of just one.  
>  
> My question is how do I get lnet to use them all.
Pleaes refer to section 12 of the Lustre Operations Manual:
http://manual.lustre.org/manual/LustreManual16_HTML/Bonding.html#50446395_pgfId-5529

b.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080526/d986f66c/attachment.bin

Hi Brian,

                 From the manual - " Lustre can use multiple NICs 
without bonding"
How does that work from an ip point of view, normally it is not possible 
to have more than one interface on the same machine
on the same subnet. Does this work similar to the round robin bonding mode?

Thanks
Balagopal

Brian J. Murrell wrote:
> On Sat, 2008-05-24 at 21:29 -0400, Jerome, Ron wrote:
>   
>> I have a Lustre (1.6.4.3) system consisting of an MGS/MGT node and 4
>> OSS nodes.  I have just added an additional dual port Ethernet board
>> to each of the OSSs, so they now have 3 Ethernet ports
>> (eth0,eth1,eth2) instead of just one.  
>>  
>> My question is how do I get lnet to use them all.
>>     
>
> Pleaes refer to section 12 of the Lustre Operations Manual:
>
http://manual.lustre.org/manual/LustreManual16_HTML/Bonding.html#50446395_pgfId-5529
>
> b.
>
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

On Mon, May 26, 2008 at 01:04:46PM -0300, Balagopal Pillai
wrote:
> Hi Brian,
> 
>                  From the manual - " Lustre can use multiple NICs 
> without bonding"
This referred to the now obsolete socklnd bonding feature, developed when
Linux Ethernet bonding wasn''t as stable and competent as it is today.

Isaac