Lustre discuss - Feb 2011 - Help! Newbie trying to set up Lustre network

I am completely new to this and have a simple question.

The lustre version I am using is 1.8.5 on  SLES10 x86_64 machines.

 

I''ve set up a server with combined MGS and OST on "eth2" with
an lnet
entry in modprobe.conf:

options lnet networks=tcp0(eth2)

 

and the ip address for eth2 is: 192.168.0.2

 

when I do "lctl list_nids" on the server, it correctly responds by:

192.168.0.2 at tcp

And on a client, I only installed lustre-client-modules-<ver> and
lustre-client-<ver> rpms. And I can ping the server.

 

However, when I tried to mount the lustre on the client by doing:

mount -t lustre 192.168.0.2 at tcp0:/temp /lustre

 

It fails and complains about:

mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
device

Are the lustre modules loaded?

Check /etc/modprobe.conf and /proc/filesystems

Note ''alias lustre llite'' should be removed from modprobe.conf

 

 

Any idea what''s going on there?

 

Please help. I am reading the LNET configuration chapter and could not
find anything relating to client side NID correlation.

 

Thanks,

 

Yang

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/cb684ea1/attachment.html

On 11-02-22 02:00 PM, Xiang, Yang wrote:
> It fails and complains about:
> 
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
> device
> 
> Are the lustre modules loaded?
> 
> Check /etc/modprobe.conf and /proc/filesystems
> 
> Note ''alias lustre llite'' should be removed from
modprobe.conf
The first thing to check in such situations is the client''s syslog
and/or dmesg.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/e562625a/attachment.bin

Quite often, sunrpc takes the port needed by Lustre, before Lustre can 
get to it.  That results in the messages below.   No recourse but to 
reboot.  Put the mount in your /etc/fstab as the simplest approach.  
This may not be the ONLY reason why this happens, but it is the one that 
has most often bitten me.

10.10.1.140 at tcp0:/myLustre /lustre/myLustre lustre _netdev,localflock 0 0

bob

On 2/22/2011 2:00 PM, Xiang, Yang wrote:
>
> I am completely new to this and have a simple question.
>
> The lustre version I am using is 1.8.5 on  SLES10 x86_64 machines.
>
> I''ve set up a server with combined MGS and OST on "eth2"
with an lnet
> entry in modprobe.conf:
>
> options lnet networks=tcp0(eth2)
>
> and the ip address for eth2 is: 192.168.0.2
>
> when I do "lctl list_nids" on the server, it correctly responds
by:
>
> 192.168.0.2 at tcp
>
> And on a client, I only installed lustre-client-modules-<ver> and 
> lustre-client-<ver> rpms. And I can ping the server.
>
> However, when I tried to mount the lustre on the client by doing:
>
> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
>
> It fails and complains about:
>
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such 
> device
>
> Are the lustre modules loaded?
>
> Check /etc/modprobe.conf and /proc/filesystems
>
> Note ''alias lustre llite'' should be removed from
modprobe.conf
>
> Any idea what''s going on there?
>
> Please help. I am reading the LNET configuration chapter and could not 
> find anything relating to client side NID correlation.
>
> Thanks,
>
> Yang
>
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/adc7b0ca/attachment.html

Dmesg and syslog are clean and has no entries about lustre client. And
on the server side, there is no entry on the client side connection
attempt either. Tcpdump shows no trace of incoming client connection
request. I think it is the syntax of the MGSNID of the mount.lustre is
not interpreted by the client correctly or I am missing some
configuration on the client side to turn the "192.168.0.2 at tcp0" to
a
valid NID, ie lustre device for the client to make an outwards
connection request to the server.

Thanks,

Yang


-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org
[mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
Murrell
Sent: Tuesday, February 22, 2011 11:04 AM
To: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
network

On 11-02-22 02:00 PM, Xiang, Yang wrote:
> It fails and complains about:
> 
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
> device
> 
> Are the lustre modules loaded?
> 
> Check /etc/modprobe.conf and /proc/filesystems
> 
> Note ''alias lustre llite'' should be removed from
modprobe.conf
The first thing to check in such situations is the client''s syslog
and/or dmesg.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

Run ''lctl list_nids'' on the client also.
Then you can
# lctl ping <other nid>
from both server and client to verify your LNET is functioning.

Also, use tunefs.lustre --print on your MDS/MGT and OST devices to verify
that mgsnid
is set correctly there.
cliffw


On Tue, Feb 22, 2011 at 11:10 AM, Xiang, Yang <Yang.Xiang at
teradata.com>wrote:
> Dmesg and syslog are clean and has no entries about lustre client. And
> on the server side, there is no entry on the client side connection
> attempt either. Tcpdump shows no trace of incoming client connection
> request. I think it is the syntax of the MGSNID of the mount.lustre is
> not interpreted by the client correctly or I am missing some
> configuration on the client side to turn the "192.168.0.2 at
tcp0" to a
> valid NID, ie lustre device for the client to make an outwards
> connection request to the server.
>
> Thanks,
>
> Yang
>
>
> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
> Murrell
> Sent: Tuesday, February 22, 2011 11:04 AM
> To: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
> network
>
> On 11-02-22 02:00 PM, Xiang, Yang wrote:
> > It fails and complains about:
> >
> > mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No
such
> > device
> >
> > Are the lustre modules loaded?
> >
> > Check /etc/modprobe.conf and /proc/filesystems
> >
> > Note ''alias lustre llite'' should be removed from
modprobe.conf
>
> The first thing to check in such situations is the client''s syslog
> and/or dmesg.
>
> b.
>
> --
> Brian J. Murrell
> Senior Software Engineer
> Whamcloud, Inc.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>


-- 
cliffw
Support Guy
WhamCloud, Inc.
www.whamcloud.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/237a5da3/attachment-0001.html

>Dmesg and syslog are clean and has no entries about lustre client.
... are you _sure_?  Even /var/log/messages?

I ask because this sure seems like the Lustre modules are not loaded (you
can check that with the lsmod command).  If they aren''t loaded, then
the
core issue will be buried somewhere in the messages file (the trick is
to look at the earliest related Lustre messages).  For example, if you
run into the problem that Bob Ball mentioned where one of the RPC services
is using the Lustre port, you can find the "Address already in use"
error
message if you look at the right spot.

If the lustre modules are loaded, then it''s beats me what is causing
this
problem.

--Ken

On 11-02-22 02:10 PM, Xiang, Yang wrote:
> Dmesg and syslog are clean and has no entries about lustre client.
That seems suspect.  I don''t recall a situation where I have failed a
client mount and there hasn''t been something in the client log.  Lustre
is quite verbose, especially when something goes wrong.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/3083ad94/attachment.bin

you should do a "lsmod" and check what modules are loaded :)


-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces
at lists.lustre.org] On Behalf Of Brian J. Murrell
Sent: Tuesday, February 22, 2011 11:17 AM
To: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre network

On 11-02-22 02:10 PM, Xiang, Yang wrote:
> Dmesg and syslog are clean and has no entries about lustre client.
That seems suspect.  I don''t recall a situation where I have failed a
client mount and there hasn''t been something in the client log.  Lustre
is quite verbose, especially when something goes wrong.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

On Tue, Feb 22, 2011 at 2:00 PM, Xiang, Yang <Yang.Xiang at teradata.com>
wrote:
> I am completely new to this and have a simple question.
>
...
>
> However, when I tried to mount the lustre on the client by doing:
>
> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
>
>
>
> It fails and complains about:
>
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
device
>
Given the other answers, it might make sense to doublecheck some
basics: does the /lustre directory exist on the client?

Steven

On the client, I only did:
"rpm -ivh
lustre-client-modules-1.8.52.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
And
"rpm -ivh
lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"

Lsmod does not show any lustre modules getting loaded.

Do I need to do an "insmod" of some top level client ko? The
installation guide does not say anything.

Thanks so much,

Yang


-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org
[mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
Murrell
Sent: Tuesday, February 22, 2011 11:17 AM
To: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
network

On 11-02-22 02:10 PM, Xiang, Yang wrote:
> Dmesg and syslog are clean and has no entries about lustre client.
That seems suspect.  I don''t recall a situation where I have failed a
client mount and there hasn''t been something in the client log.  Lustre
is quite verbose, especially when something goes wrong.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

On 11-02-22 02:55 PM, Xiang, Yang wrote:
> On the client, I only did:
> "rpm -ivh
>
lustre-client-modules-1.8.52.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
> And
> "rpm -ivh
> lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
That''s fine.
> Lsmod does not show any lustre modules getting loaded.
Even after you run "mount -t lustre ..."?
> Do I need to do an "insmod" of some top level client ko? The
> installation guide does not say anything.
On a properly configured and working Linux system, no.  The "mount -t
lustre" command should load the lustre modules.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/cbc25fad/attachment.bin

But my mount command is failing and that''s the issue:
mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
device
Are the lustre modules loaded?
Check /etc/modprobe.conf and /proc/filesystems
Note ''alias lustre llite'' should be removed from modprobe.conf

And yes I created the /lustre mount point.

And I tried to add to the fstab to avoid "sunrpc" to take over on
boot.

This is frustrating...

Thanks,

Yang


-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org
[mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
Murrell
Sent: Tuesday, February 22, 2011 11:58 AM
To: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
network

On 11-02-22 02:55 PM, Xiang, Yang wrote:
> On the client, I only did:
> "rpm -ivh
>
lustre-client-modules-1.8.52.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
> And
> "rpm -ivh
> lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
That''s fine.
> Lsmod does not show any lustre modules getting loaded.
Even after you run "mount -t lustre ..."?
> Do I need to do an "insmod" of some top level client ko? The
> installation guide does not say anything.
On a properly configured and working Linux system, no.  The "mount -t
lustre" command should load the lustre modules.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

On 11-02-22 03:03 PM, Xiang, Yang wrote:
> But my mount command is failing and that''s the issue:
> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
> device
> Are the lustre modules loaded?
Right.  So, if after such a command, lsmod still shows no lustre modules
loaded, then it would seem your system is not loading the modules on
demand, as it should.  Or that there is an issue in which they are
failing to install correctly.

You can try to manually load the modules before issuing the mount
command with:

# modprobe lustre

Of course, if that fails, the syslog kernel target and/or dmesg should
be informative of why.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/4d6ef057/attachment-0001.bin

Make sure that the kernel you are running matches up with the rpms you 
installed then?

[ball at umt3int01:gate01_b]$ rpm -qa|grep lustre
lustre-client-1.8.4-2.6.18_194.17.4.el5.x86_64
lustre-client-modules-1.8.4-2.6.18_194.17.4.el5.x86_64
[ball at umt3int01:gate01_b]$ uname -r
2.6.18-194.17.4.el5

bob

On 2/22/2011 3:03 PM, Xiang, Yang wrote:
> But my mount command is failing and that''s the issue:
> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
> device
> Are the lustre modules loaded?
> Check /etc/modprobe.conf and /proc/filesystems
> Note ''alias lustre llite'' should be removed from
modprobe.conf
>
> And yes I created the /lustre mount point.
>
> And I tried to add to the fstab to avoid "sunrpc" to take over on
boot.
>
> This is frustrating...
>
> Thanks,
>
> Yang
>
>
> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
> Murrell
> Sent: Tuesday, February 22, 2011 11:58 AM
> To: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
> network
>
> On 11-02-22 02:55 PM, Xiang, Yang wrote:
>> On the client, I only did:
>> "rpm -ivh
>>
>
lustre-client-modules-1.8.52.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
>> And
>> "rpm -ivh
>> lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
> That''s fine.
>
>> Lsmod does not show any lustre modules getting loaded.
> Even after you run "mount -t lustre ..."?
>
>> Do I need to do an "insmod" of some top level client ko? The
>> installation guide does not say anything.
> On a properly configured and working Linux system, no.  The "mount -t
> lustre" command should load the lustre modules.
>
> b.
>

>But my mount command is failing and that''s the issue:
>mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
>mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
>device
>Are the lustre modules loaded?
Right, and every time I''ve seen the mount command fail like this (with
ENODEV as the error), the _root cause_ is the kernel modules are not
loading; that can happen for a variety of reasons.  The fact you''re
getting _nothing_ in the logs is itself rather suspicious; as Brian has
already pointed out Lustre is normally very chatty, even when it is
working correctly.

You could try loading the modules yourself with insmod; if that''s not
working, then you should start from there.

--Ken

Hi Yang,

on SLES11 I get this message, if the loading of unsupported modules is 
disabled (default for SLES11). To enable the loading of such modules you 
have to set the value of allow_unsupported_modules to 1 in 
/etc/modprobe.d/unsupported-modules

Maybe Novell has added this "feature" to later versions of SLES 10? I 
suppose you are using service pack 3?

best regards,
Christian

Xiang, Yang wrote:
> But my mount command is failing and that''s the issue:
> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
> device
> Are the lustre modules loaded?
> Check /etc/modprobe.conf and /proc/filesystems
> Note ''alias lustre llite'' should be removed from
modprobe.conf
> 
> And yes I created the /lustre mount point.
> 
> And I tried to add to the fstab to avoid "sunrpc" to take over on
boot.
> 
> This is frustrating...
> 
> Thanks,
> 
> Yang
> 
> 
> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
> Murrell
> Sent: Tuesday, February 22, 2011 11:58 AM
> To: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
> network
> 
> On 11-02-22 02:55 PM, Xiang, Yang wrote:
>> On the client, I only did:
>> "rpm -ivh
>>
>
lustre-client-modules-1.8.52.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
>> And
>> "rpm -ivh
>> lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
> 
> That''s fine.
> 
>> Lsmod does not show any lustre modules getting loaded.
> 
> Even after you run "mount -t lustre ..."?
> 
>> Do I need to do an "insmod" of some top level client ko? The
>> installation guide does not say anything.
> 
> On a properly configured and working Linux system, no.  The "mount -t
> lustre" command should load the lustre modules.
> 
> b.
>

At this point, you need to  verify your installation actually worked, and
kernel deps are correct.
Also be sure the running kernel matches the version of lustre-client-modules
you installed
(2.6.16_60_0.69.1, from your above i think)
Run
# modprobe -v lustre
And you should have an error in syslog that will help.
Haven''t seen this in awhile, but if the lustre modules are in the
proper
directory under /lib/modules
for your running kernel, you might  need ''depmod -a''


On Tue, Feb 22, 2011 at 12:03 PM, Xiang, Yang <Yang.Xiang at
teradata.com>wrote:
> But my mount command is failing and that''s the issue:
> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
> device
> Are the lustre modules loaded?
> Check /etc/modprobe.conf and /proc/filesystems
> Note ''alias lustre llite'' should be removed from
modprobe.conf
>
> And yes I created the /lustre mount point.
>
> And I tried to add to the fstab to avoid "sunrpc" to take over on
boot.
>
> This is frustrating...
>
> Thanks,
>
> Yang
>
>
> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
> Murrell
> Sent: Tuesday, February 22, 2011 11:58 AM
> To: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
> network
>
> On 11-02-22 02:55 PM, Xiang, Yang wrote:
> > On the client, I only did:
> > "rpm -ivh
> >
>
lustre-client-modules-1.8.52.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
> > And
> > "rpm -ivh
> > lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp.x86_64.rpm"
>
> That''s fine.
>
> > Lsmod does not show any lustre modules getting loaded.
>
> Even after you run "mount -t lustre ..."?
>
> > Do I need to do an "insmod" of some top level client ko? The
> > installation guide does not say anything.
>
> On a properly configured and working Linux system, no.  The "mount -t
> lustre" command should load the lustre modules.
>
> b.
>
> --
> Brian J. Murrell
> Senior Software Engineer
> Whamcloud, Inc.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>


-- 
cliffw
Support Guy
WhamCloud, Inc.
www.whamcloud.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110222/c25ba115/attachment.html

I think my problem is that the client machine is also multihomed and has
5 network interfaces. When it tries to mount, it uses the eth0 and can
not reach the server.  I need to force it to use eth2 to talk to the
server.  I don''t know how to do that from the client configuration
prospective.

Thanks for any hint,

Yang


-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org
[mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
Murrell
Sent: Tuesday, February 22, 2011 12:09 PM
To: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
network

On 11-02-22 03:03 PM, Xiang, Yang wrote:
> But my mount command is failing and that''s the issue:
> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No such
> device
> Are the lustre modules loaded?
Right.  So, if after such a command, lsmod still shows no lustre modules
loaded, then it would seem your system is not loading the modules on
demand, as it should.  Or that there is an issue in which they are
failing to install correctly.

You can try to manually load the modules before issuing the mount
command with:

# modprobe lustre

Of course, if that fails, the syslog kernel target and/or dmesg should
be informative of why.

b.

-- 
Brian J. Murrell
Senior Software Engineer
Whamcloud, Inc.

Client LNET config is the same as the servers.  Add:

lnet networks="tcp0(eth2)"

to your client modprobe.conf.  That will tell the client to ONLY use  
eth2.  By default, lustre will use all network interfaces.

-Marc

----
D. Marc Stearman
Lustre Operations Lead
marc at llnl.gov
925.423.9670
Pager: 1.888.203.0641




On Feb 22, 2011, at 3:45 PM, Xiang, Yang wrote:
> I think my problem is that the client machine is also multihomed and  
> has
> 5 network interfaces. When it tries to mount, it uses the eth0 and can
> not reach the server.  I need to force it to use eth2 to talk to the
> server.  I don''t know how to do that from the client configuration
> prospective.
>
> Thanks for any hint,
>
> Yang
>
>
> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
> Murrell
> Sent: Tuesday, February 22, 2011 12:09 PM
> To: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
> network
>
> On 11-02-22 03:03 PM, Xiang, Yang wrote:
>> But my mount command is failing and that''s the issue:
>> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
>> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No
such
>> device
>> Are the lustre modules loaded?
>
> Right.  So, if after such a command, lsmod still shows no lustre  
> modules
> loaded, then it would seem your system is not loading the modules on
> demand, as it should.  Or that there is an issue in which they are
> failing to install correctly.
>
> You can try to manually load the modules before issuing the mount
> command with:
>
> # modprobe lustre
>
> Of course, if that fails, the syslog kernel target and/or dmesg should
> be informative of why.
>
> b.
>
> -- 
> Brian J. Murrell
> Senior Software Engineer
> Whamcloud, Inc.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

I tried that but on the server side, it is still refusing connection
because it came from the network on eth0:
LustreError: 120-3: Refusing connection from 39.80.8.8 for
39.80.8.9 at tcp: No matching NI

The server is actually running on eth2 on 192.168.0.2.

I added "lnet networks="tcp0(eth2)" to both /etc/modprobe.conf
files on
the server and the client and rebooted.

Strange...

Yang


-----Original Message-----
From: D. Marc Stearman [mailto:marc at llnl.gov] 
Sent: Tuesday, February 22, 2011 3:54 PM
To: Xiang, Yang
Cc: Brian J. Murrell; lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
network

Client LNET config is the same as the servers.  Add:

lnet networks="tcp0(eth2)"

to your client modprobe.conf.  That will tell the client to ONLY use  
eth2.  By default, lustre will use all network interfaces.

-Marc

----
D. Marc Stearman
Lustre Operations Lead
marc at llnl.gov
925.423.9670
Pager: 1.888.203.0641




On Feb 22, 2011, at 3:45 PM, Xiang, Yang wrote:
> I think my problem is that the client machine is also multihomed and  
> has
> 5 network interfaces. When it tries to mount, it uses the eth0 and can
> not reach the server.  I need to force it to use eth2 to talk to the
> server.  I don''t know how to do that from the client configuration
> prospective.
>
> Thanks for any hint,
>
> Yang
>
>
> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J.
> Murrell
> Sent: Tuesday, February 22, 2011 12:09 PM
> To: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Help! Newbie trying to set up Lustre
> network
>
> On 11-02-22 03:03 PM, Xiang, Yang wrote:
>> But my mount command is failing and that''s the issue:
>> mount -t lustre 192.168.0.2 at tcp0:/temp /lustre
>> mount.lustre: mount 192.168.0.2 at tcp0:/temp at /lustre failed: No
such
>> device
>> Are the lustre modules loaded?
>
> Right.  So, if after such a command, lsmod still shows no lustre  
> modules
> loaded, then it would seem your system is not loading the modules on
> demand, as it should.  Or that there is an issue in which they are
> failing to install correctly.
>
> You can try to manually load the modules before issuing the mount
> command with:
>
> # modprobe lustre
>
> Of course, if that fails, the syslog kernel target and/or dmesg should
> be informative of why.
>
> b.
>
> -- 
> Brian J. Murrell
> Senior Software Engineer
> Whamcloud, Inc.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss