While looking at lctl I wanted to know what and how to use show_route/route_list but when I run it I get nothing back. I am doing, lctl network tcp0 show_routes Can someone please confirm if this is a way to do it? TIA
On Oct 07, 2008 07:00 -0400, Mag Gam wrote:> While looking at lctl I wanted to know what and how to use > show_route/route_list but when I run it I get nothing back. I am > doing, > > lctl > network tcp0 > show_routes > > Can someone please confirm if this is a way to do it?Very few, large, sites use Lustre routing. This has nothing to do with TCP routing. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
Great. Thanks for the response Andreas. My intention was I wanted to see if my lustre connection is being routed thru other interfaces. I have 4 interfaces on my server: eth0 thru eth4. eth0 is used for Lustre but it seems lustre picks the best route. I wanted to be 100% sure its not using anything else but eth0. Is there another test for this? I know editing /etc/modules.conf will fix it and I already have that, but I want a realtime test. TIA On Tue, Oct 7, 2008 at 7:14 AM, Andreas Dilger <adilger at sun.com> wrote:> On Oct 07, 2008 07:00 -0400, Mag Gam wrote: >> While looking at lctl I wanted to know what and how to use >> show_route/route_list but when I run it I get nothing back. I am >> doing, >> >> lctl >> network tcp0 >> show_routes >> >> Can someone please confirm if this is a way to do it? > > Very few, large, sites use Lustre routing. This has nothing to do > with TCP routing. > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. > >
On Oct 07, 2008 22:58 -0400, Mag Gam wrote:> My intention was I wanted to see if my lustre connection is being > routed thru other interfaces. I have 4 interfaces on my server: eth0 > thru eth4. eth0 is used for Lustre but it seems lustre picks the best > route. I wanted to be 100% sure its not using anything else but eth0. > Is there another test for this? I know editing /etc/modules.conf will > fix it and I already have that, but I want a realtime test.You can use "lctl list_nids" to show the interfaces that lustre is using. "lctl ping {nid}" will show the connections between the nodes. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
On Tue, Oct 07, 2008 at 11:00:20PM -0600, Andreas Dilger wrote:> On Oct 07, 2008 22:58 -0400, Mag Gam wrote: > > My intention was I wanted to see if my lustre connection is being > > routed thru other interfaces. I have 4 interfaces on my server: eth0 > > thru eth4. eth0 is used for Lustre but it seems lustre picks the best > > route. I wanted to be 100% sure its not using anything else but eth0. > > Is there another test for this? I know editing /etc/modules.conf will > > fix it and I already have that, but I want a realtime test. > > You can use "lctl list_nids" to show the interfaces that lustre is > using. "lctl ping {nid}" will show the connections between the nodes.While "lctl list_nids" certainly shows the LNet interfaces Lustre is using, it''s probably not the most accurate way to find out which Ethernet interface Lustre traffic really goes on. When socklnd bonding is in use (e.g. networks="tcp0(eth0,eth1)"), the LNet NID would only pickup IP of the 1st interface in the networks specification (e.g. ip_of_eth0 at tcp), despite the fact that LNet actually tries to make use of both interfaces. Moreover, the Ethernet interface in use is solely determined by the Linux IP routing. For example, you have eth0 and eth1 and asks lnet to use eth0 only (e.g. networks="tcp(eth0)"), the traffic could still end up going through eth1 if Linux IP routing chooses so (e.g. both interfaces are in the same IP network, perhaps by mistake, and the routing table entry for eth1 comes earlier). Such misconfigured routing and the accompanying confusion happened a couple of times at some customer sites as I recall. I usually rely on watching TX/RX bytes of ''ifconfig'' and a bit tcpdumping to find out which interfaces are really being used. Thanks, Isaac