Eric Mauger
2010-Jan-13 10:19 UTC
[Lustre-discuss] New to lustre, client connection question
Hi,
I''ve built a small lustre system with 2 OST nodes, 1 MDS/MGT node and
one client.
Each node has two nic. The first one (eth0) is used for administration
purpose and the second (eth1)
is used for lustre interconnection.
I''ve installed the latest version (1.8.1) on x86_64 platforms running
centos 5.4.
When I try to moun the fs on the client, I use the following command :
mount -t lustre 10.7.0.127 at tcp0:/lustre /mnt/lustre/
The lustre network is 10.7.0.0/24
On the MDS/MGT logs, I have :
mds kernel: LustreError: 120-3: Refusing connection from 195.220.11.120
for 195.220.11.127 at tcp: No matching NI
The address 195.220.11.120 is on eth0 and I really don''t understand why
the connection seems to come from
this network instead of 10.7.0.0
I tried the lctl list_nids on all machines an here is the result :
[root at ost1 ~]# lctl list_nids
10.7.0.126 at tcp
[root at ost2 ~]# lctl list_nids
10.7.0.125 at tcp
[root at mds ~]# lctl list_nids
10.7.0.127 at tcp
[root at lustre-client ~]# lctl list_nids
10.7.0.120 at tcp
Any suggestions ?
--
Eric Mauger | IGF UMR5203
Administrateur systeme et reseau | 141, rue de la Cardonille
04 67 14 29 79 | 34094 Montpellier cedex 5
--
Passerelle antivirus IGF
--
Brian J. Murrell
2010-Jan-13 13:30 UTC
[Lustre-discuss] New to lustre, client connection question
On Wed, 2010-01-13 at 11:19 +0100, Eric Mauger wrote:> Hi,Hi,> Each node has two nic. The first one (eth0) is used for administration > purpose and the second (eth1) > is used for lustre interconnection.> mount -t lustre 10.7.0.127 at tcp0:/lustre /mnt/lustre/ > > The lustre network is 10.7.0.0/24 > > On the MDS/MGT logs, I have : > > mds kernel: LustreError: 120-3: Refusing connection from 195.220.11.120 > for 195.220.11.127 at tcp: No matching NI > > The address 195.220.11.120 is on eth0 and I really don''t understand why > the connection seems to come from > this network instead of 10.7.0.0You need to review the section in the manual about LNET networking and specifically how to bind NIDs to specific interfaces. You are getting the default behaviour but given your description above, that''s not what you want. You need to explicitly configure your bindings to reflect your "non-default" configuration. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100113/2c226806/attachment.bin
Eric Mauger
2010-Jan-13 15:30 UTC
[Lustre-discuss] New to lustre, client connection question
Le 13/01/2010 14:30, Brian J. Murrell a ?crit :> On Wed, 2010-01-13 at 11:19 +0100, Eric Mauger wrote: > >> Hi, >> > Hi, > > >> Each node has two nic. The first one (eth0) is used for administration >> purpose and the second (eth1) >> is used for lustre interconnection. >> > >> mount -t lustre 10.7.0.127 at tcp0:/lustre /mnt/lustre/ >> >> The lustre network is 10.7.0.0/24 >> >> On the MDS/MGT logs, I have : >> >> mds kernel: LustreError: 120-3: Refusing connection from 195.220.11.120 >> for 195.220.11.127 at tcp: No matching NI >> >> The address 195.220.11.120 is on eth0 and I really don''t understand why >> the connection seems to come from >> this network instead of 10.7.0.0 >> > You need to review the section in the manual about LNET networking and > specifically how to bind NIDs to specific interfaces. You are getting > the default behaviour but given your description above, that''s not what > you want. You need to explicitly configure your bindings to reflect > your "non-default" configuration. > > b. >Brian, I forgot to tell that I''ve added this line in modprobe.conf of all machines : options lnet networks=tcp0(eth1) I haven''t seen anything else to bind NIDs to eth1. Eric> > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-- Eric Mauger | IGF UMR5203 Administrateur systeme et reseau | 141, rue de la Cardonille 04 67 14 29 79 | 34094 Montpellier cedex 5 -- Passerelle antivirus IGF --
Aaron Knister
2010-Jan-14 01:00 UTC
[Lustre-discuss] New to lustre, client connection question
Did you change modprobe.conf before or after you loaded the lustre modules? IIRC you need to reload the lustre modules to pick up any lnet networks changes in the modprobe.conf file. I think there''s a way to change it on the fly but I''m not sure how to do it. To see which NIDS are currently configured on a given machine you can run an "lctl list_nids". On Jan 13, 2010, at 10:30 AM, Eric Mauger wrote:> Le 13/01/2010 14:30, Brian J. Murrell a ?crit : >> On Wed, 2010-01-13 at 11:19 +0100, Eric Mauger wrote: >> >>> Hi, >>> >> Hi, >> >> >>> Each node has two nic. The first one (eth0) is used for administration >>> purpose and the second (eth1) >>> is used for lustre interconnection. >>> >> >>> mount -t lustre 10.7.0.127 at tcp0:/lustre /mnt/lustre/ >>> >>> The lustre network is 10.7.0.0/24 >>> >>> On the MDS/MGT logs, I have : >>> >>> mds kernel: LustreError: 120-3: Refusing connection from 195.220.11.120 >>> for 195.220.11.127 at tcp: No matching NI >>> >>> The address 195.220.11.120 is on eth0 and I really don''t understand why >>> the connection seems to come from >>> this network instead of 10.7.0.0 >>> >> You need to review the section in the manual about LNET networking and >> specifically how to bind NIDs to specific interfaces. You are getting >> the default behaviour but given your description above, that''s not what >> you want. You need to explicitly configure your bindings to reflect >> your "non-default" configuration. >> >> b. >> > Brian, > > I forgot to tell that I''ve added this line in modprobe.conf of all > machines : > > options lnet networks=tcp0(eth1) > > I haven''t seen anything else to bind NIDs to eth1. > > Eric >> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > > > -- > Eric Mauger | IGF UMR5203 > Administrateur systeme et reseau | 141, rue de la Cardonille > 04 67 14 29 79 | 34094 Montpellier cedex 5 > > > -- > Passerelle antivirus IGF > -- > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss