Lustre discuss - Sep 2008 - Lustre 1.6.5.1 on the box with 2 interfaces

Hello,

I have some difficulties to setup a server acting as metadata server and OST
server. The box has two interfaces, eth0 and eth2. eth0 is the primary
interface and has the ip which is associated with the hostname.

However, I would like to use the eth2 interface for Lustre.
In an ideal case, I would like to use Lustre on both interfaces. But for now,
it would be sufficient to use just the non-primary one.

I use
options lnet networks="tcp0(eth2),tcp1(eth0)"

I created the metadata:
/usr/local/lustre/sbin/mkfs.lustre --fsname=l_smaug2 --reformat --mdt --mgs
/dev/Scratch_VG/Scratch_1

I mounted the mdt:
mount /dev/Scratch_VG/Scratch_1 /mnt/lustre/mdt

Then I created the OST
/usr/local/lustre/sbin/mkfs.lustre --fsname=l_smaug2 --reformat --ost 
--mgsnode=192.168.1.1 at tcp0 /dev/Scratch_VG/Scratch_2

I mounted it:
mount /dev/Scratch_VG/Scratch_2 /mnt/lustre/ost0

However, cat /proc/fs/lustre/devices shows that no OST is attached at all.

What am I missing?

eth0 has a public address,
eth2 has the 192.168.1.1 address.

-- 
Luk?? Hejtm?nek

On Wed, 2008-09-24 at 21:41 +0200, Lukas Hejtmanek
wrote:
> Hello,
Hi,
> I have some difficulties to setup a server acting as metadata server and
OST
> server.
Let''s start by saying putting an MDS and OSS on the same box is not
good
from both a performance and recovery point of view.  If that box fails,
recovery can fail.
> I use
> options lnet networks="tcp0(eth2),tcp1(eth0)"
> Then I created the OST
> /usr/local/lustre/sbin/mkfs.lustre --fsname=l_smaug2 --reformat --ost 
> --mgsnode=192.168.1.1 at tcp0 /dev/Scratch_VG/Scratch_2
> 
> I mounted it:
> mount /dev/Scratch_VG/Scratch_2 /mnt/lustre/ost0
Do you get any errors from this mount?
> However, cat /proc/fs/lustre/devices shows that no OST is attached at all.
What does it show?
> What am I missing?
You haven''t really said what the problem is beyond the device list. 
Did
you get errors?  Are there any problems with clients, etc.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080924/9be75370/attachment.bin

Hi Lukas,

Do you want to use these interfaces together as a logical unit? The syntax
below would construct two separate LNET networks.

Additionally, you would need to qualify the mount path on the client side to
bind Lustre to a specific LNET instance, i.e. ''mount /mnt/lustre
192.168.1.1 at tcp0:/lustre'', or else it will default to the first
available
LNET instance. The order doesn''t really matter in the module
declaration,
though.

Looking at my OSS nodes, I can see entries in /proc/fs/lustre/devices, as
well as in the output of ''df''. Did you specify a file system
type? Normally
when mounting either an OST or an MDT, you need to provide a ''-t
lustre''
argument to ''mount'', or list that as the fstype in /etc/fstab.
Otherwise,
I''m assuming it mounts the device as ext3, which works, but
isn''t usable by
Lustre.

hth,
Klaus

On 9/24/08 12:41 PM, "Lukas Hejtmanek" <xhejtman at
ics.muni.cz>did etch on
stone tablets:
> Hello,
> 
> I have some difficulties to setup a server acting as metadata server and
OST
> server. The box has two interfaces, eth0 and eth2. eth0 is the primary
> interface and has the ip which is associated with the hostname.
> 
> However, I would like to use the eth2 interface for Lustre.
> In an ideal case, I would like to use Lustre on both interfaces. But for
now,
> it would be sufficient to use just the non-primary one.
> 
> I use
> options lnet networks="tcp0(eth2),tcp1(eth0)"
> 
> I created the metadata:
> /usr/local/lustre/sbin/mkfs.lustre --fsname=l_smaug2 --reformat --mdt --mgs
> /dev/Scratch_VG/Scratch_1
> 
> I mounted the mdt:
> mount /dev/Scratch_VG/Scratch_1 /mnt/lustre/mdt
> 
> Then I created the OST
> /usr/local/lustre/sbin/mkfs.lustre --fsname=l_smaug2 --reformat --ost
> --mgsnode=192.168.1.1 at tcp0 /dev/Scratch_VG/Scratch_2
> 
> I mounted it:
> mount /dev/Scratch_VG/Scratch_2 /mnt/lustre/ost0
> 
> However, cat /proc/fs/lustre/devices shows that no OST is attached at all.
> 
> What am I missing?
> 
> eth0 has a public address,
> eth2 has the 192.168.1.1 address.

On Wed, Sep 24, 2008 at 03:58:36PM -0400, Brian J. Murrell
wrote:
> Let''s start by saying putting an MDS and OSS on the same box is
not good
> from both a performance and recovery point of view.  If that box fails,
> recovery can fail.
Well, I guess that for the first tests it could be ok :)
> 
> > I use
> > options lnet networks="tcp0(eth2),tcp1(eth0)"
> 
> > Then I created the OST
> > /usr/local/lustre/sbin/mkfs.lustre --fsname=l_smaug2 --reformat --ost 
> > --mgsnode=192.168.1.1 at tcp0 /dev/Scratch_VG/Scratch_2
> > 
> > I mounted it:
> > mount /dev/Scratch_VG/Scratch_2 /mnt/lustre/ost0
> 
> Do you get any errors from this mount?
none at all.
> 
> > However, cat /proc/fs/lustre/devices shows that no OST is attached at
all.
> 
> What does it show?
# cat /proc/fs/lustre/devices 
  0 UP mgs MGS MGS 5
  1 UP mgc MGC195.113.235.225 at tcp 38be3e11-3501-85e4-da0a-4b5efb287c2b 5
  2 UP mdt MDS MDS_uuid 3
  3 UP lov l_smaug2-mdtlov l_smaug2-mdtlov_UUID 4
  4 UP mds l_smaug2-MDT0000 l_smaug2-MDT0000_UUID 3
#
> 
> > What am I missing?
> 
> You haven''t really said what the problem is beyond the device
list.  Did
> you get errors?  Are there any problems with clients, etc.
Yes, the client complains that mdt does not have any osts.

However, I tried it all again and it mysteriously works now. Somewhat
strange...

So sorry for the buzz.

-- 
Luk?? Hejtm?nek