Hi we would like the lustre servers available to two networks, one on eth0 (192.168.1.0/24) the other on eth1(192.168.8.0/22), they are on different networks. I''ve tried the following:- 1. Create /etc/modprobe.d/lustre that contains options lnet networks=tcp0,tcp1 2. Used tunefs.lustre to specify the ip address of both interfaces of the MGS/MDT is on, i.e. tunefs.lustre --eraseparams --mgsnode=<net1>@tcp0,<net2>@tcp1 /dev/md5 3. Started up lustre The clients could connect on network tcp0, but the clients that would connect via the IP address of network tcp1 could not. What am I missing ? Thanks Phil.
On Sep 29, 2009 16:05 +1000, Philip Manuel wrote:> Hi we would like the lustre servers available to two networks, one on > eth0 (192.168.1.0/24) the other on eth1(192.168.8.0/22), they are on > different networks. > > I''ve tried the following:- > > 1. Create /etc/modprobe.d/lustre that contains > options lnet networks=tcp0,tcp1This probably needs to be like: options lnet networks=tcp0(eth0),tcp1(eth1)> 2. Used tunefs.lustre to specify the ip address of both interfaces of > the MGS/MDT is on, i.e. tunefs.lustre --eraseparams > --mgsnode=<net1>@tcp0,<net2>@tcp1 /dev/md5 > 3. Started up lustre > > The clients could connect on network tcp0, but the clients that would > connect via the IP address of network tcp1 could not. > > What am I missing ? > > Thanks > > Phil. > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discussCheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
Hi, Thanks for your quick response. My /etc/modprobe.d/lustre now contains options lnet networks=tcp0(eth0),tcp1(eth1) I updated the partitions with the following:- mgsnode=172.16.0.57 at tcp,172.17.11.250 at tcp1 On both of the MDT''s lctl list_nids reports (note they have the above lnet entry in their /etc/modprobe.d/lustre file):- 172.16.0.51 at tcp 172.17.11.252 at tcp1 On clients, for either network, they do not have the lnet entry and lctl list_nids reports on the local machine ip address. However, clients on the tcp0 network can mount the lustre partition. Clients on the tcp1 network cannot mount via the tcp1 IP address but can via the tcp network (with a another machine routing the traffic). Do I need the same lnet module entry everywhere to make this work ? Regards Phil. Andreas Dilger wrote:> On Sep 29, 2009 16:05 +1000, Philip Manuel wrote: > >> Hi we would like the lustre servers available to two networks, one on >> eth0 (192.168.1.0/24) the other on eth1(192.168.8.0/22), they are on >> different networks. >> >> I''ve tried the following:- >> >> 1. Create /etc/modprobe.d/lustre that contains >> options lnet networks=tcp0,tcp1 >> > > This probably needs to be like: > > options lnet networks=tcp0(eth0),tcp1(eth1) > > >> 2. Used tunefs.lustre to specify the ip address of both interfaces of >> the MGS/MDT is on, i.e. tunefs.lustre --eraseparams >> --mgsnode=<net1>@tcp0,<net2>@tcp1 /dev/md5 >> 3. Started up lustre >> >> The clients could connect on network tcp0, but the clients that would >> connect via the IP address of network tcp1 could not. >> >> What am I missing ? >> >> Thanks >> >> Phil. >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090930/e2511dc4/attachment.html
On Sep 30, 2009 16:13 +1000, Philip Manuel wrote:> My /etc/modprobe.d/lustre now contains > > options lnet networks=tcp0(eth0),tcp1(eth1) > > I updated the partitions with the following:- > > mgsnode=172.16.0.57 at tcp,172.17.11.250 at tcp1 > > On both of the MDT''s lctl list_nids reports (note they have the above > lnet entry in their /etc/modprobe.d/lustre file):- > > 172.16.0.51 at tcp > 172.17.11.252 at tcp1 > > On clients, for either network, they do not have the lnet entry and lctl > list_nids reports on the local machine ip address. However, clients on > the tcp0 network can mount the lustre partition. Clients on the tcp1 > network cannot mount via the tcp1 IP address but can via the tcp network > (with a another machine routing the traffic). > > Do I need the same lnet module entry everywhere to make this work ?Yes, all of the clients need to be told whether they are in the tcp0 network or the tcp1 network. Otherwise they will default to tcp0 (which will cause the ones on the 172.17.11.* subnet to be rejected). The manual describes different ways you can make this line the same for all nodes so they can automatically pick the right LNET network based on their subnet address.> Andreas Dilger wrote: >> On Sep 29, 2009 16:05 +1000, Philip Manuel wrote: >> >>> Hi we would like the lustre servers available to two networks, one on >>> eth0 (192.168.1.0/24) the other on eth1(192.168.8.0/22), they are on >>> different networks. >>> >>> I''ve tried the following:- >>> >>> 1. Create /etc/modprobe.d/lustre that contains >>> options lnet networks=tcp0,tcp1 >>> >> >> This probably needs to be like: >> >> options lnet networks=tcp0(eth0),tcp1(eth1) >> >> >>> 2. Used tunefs.lustre to specify the ip address of both interfaces of >>> the MGS/MDT is on, i.e. tunefs.lustre --eraseparams >>> --mgsnode=<net1>@tcp0,<net2>@tcp1 /dev/md5 >>> 3. Started up lustre >>> >>> The clients could connect on network tcp0, but the clients that would >>> connect via the IP address of network tcp1 could not. >>> >>> What am I missing ? >>> >>> Thanks >>> >>> Phil. >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >>> >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Sr. Staff Engineer, Lustre Group >> Sun Microsystems of Canada, Inc. >> >>Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.