On Mon, Nov 09, 2009 at 10:55:34PM -0800, Eric Adint
wrote:> OK i have read the manual and i have read the boards and done as much
> research as i can, but i cant seem to bend my head around this, what i
> want to do is create a router so that i can keep my OST and MGS/MDT on
> the IB network and have another computer handle the ib to ip routing.
> here is the configuration.
>
> nas-0-0, MGS/MDT , OST1 - o2ib 192.168.1.2, tcp0 10.1.254.3
> nas-0-1, OST2 - o2ib 192.168.1.3, tcp0 10.1.254.4
> nas-0-2, OST3 - o2ib 192.168.1.4, tcp0 10.1.254.5
> compute-0-0 - o2ib 192.168.1.10, tcp0 10.1.254.10
> compute-0-1 - o2ib 192.168.1.11, tcp0 10.1.254.11
> compute-0-* -o2ib 192.168.1.*,tcp0 10.1.254.*
> compute-1-* -tcp0 10.1.253.*
It appears that all nodes reside in the tcp0 network, and thus no
router would be needed for them to communicate with each other over
@tcp0 NIDs.
> What i think i need is
> in modprob.conf
> options lnet ''ip2nets="o2ib0(ib0) 192.168.*.*; tcp(eth0)
10.1.*.*"'' \
> ''routes="tcp 192.168.1.[3-5]@o2ib0; o2ib
10.1.254.[10-11]@tcp0"
The ''routes'' would be ignored by all node groups except
''compute-1-*''
because ''tcp'' and ''o2ib'' are local networks
to them. The nodes in
''compute-1-*'' would use routers 10.1.254.[10-11]@tcp0 to reach
the
''o2ib'' network, however servers in the
''o2ib'' network would NOT try to
use any of the routers in order to talk to clients in ''tcp0''
network
because they are already connected to the ''tcp0'' network -
this
probably is not what you wanted.
If the 10.1.254.* network is separate from the 10.1.253.* network, you
might consider assigning two ''tcp'' networks to them.
Thanks,
Isaac