Lustre discuss - Feb 2008 - how do you mount mountconf (i.e. 1.6) lustre on your servers?

As any of you using version 1.6 of Lustre knows, Lustre servers can now
be started simply my mounting the devices it is using.  Even
an /etc/fstab entry can be used if you can have the mount delayed until
the network is started.

Given this change, you have also notices that we have eliminated the
initscript for Lustre that used to exist for releases prior to 1.6.

I''d like to take a small survey on how those of you using mountconf
(1.6) are managing the mounting of your Lustre devices on the servers.
The obvious choices are:

      * /etc/fstab
      * home-grown initscript
      * heartbeat managed
      * other

If you are using /etc/fstab I''d be interested in seeing your /etc/fstab
entries (you can omit obvious near-duplication in the case of many OSTs
though) and knowing what distribution (make, version) you are using.

/etc/fstab entries provides an interesting wrinkle in that you cannot
mount the devices until the network is up and yet most distributions
do /etc/fstab (i.e. "local") mounting before the network unless some
mechanism is used to filter out entries that need network connectivity
first.  The traditional way of doing this has been to add _netdev to
those entries and they are then delayed until the network is up.

Cheers,
b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080214/19fc3ec8/attachment-0002.bin

Hi Brian,

On Thursday 14 February 2008 10:38:45 am Brian J. Murrell
wrote:
> I''d like to take a small survey on how those of you using
mountconf
> (1.6) are managing the mounting of your Lustre devices on the
> servers. 
We do use /etc/fstab, with the _netdev option (RHEL4):

on a client:
  ibmds2 at o2ib:/home   /home  lustre  defaults,flock,_netdev  0 0

on an OSS:
  /dev/sdb  /lustre/ost-home  lustre  defaults,_netdev 0 0


Cheers,
-- 
Kilian

On Thu, 2008-02-14 at 10:43 -0800, Kilian CAVALOTTI
wrote:
> Hi Brian,
Hi Kilian,
> on an OSS:
>   /dev/sdb  /lustre/ost-home  lustre  defaults,_netdev 0 0
No heartbeat or failover then?

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080214/3626ad4e/attachment-0002.bin

On Thursday 14 February 2008 10:44:33 am Brian J. Murrell
wrote:
> > on an OSS:
> >   /dev/sdb  /lustre/ost-home  lustre  defaults,_netdev 0 0
>
> No heartbeat or failover then?
Nope. We initially planned to implement failover on our MDS, but I never 
managed to get Heartbeat working reliabily on our shared-bus 
configuration. It caused more downtime than it provides 
high-availability. 

We also had hardware issues, which likely caused the problems, but now 
that our cluster is in production, I can''t really bring it down to 
reimplement failover. Users would probably begin to throw things at 
me... :)

Cheers,
-- 
Kilian

Hi Brian,

Here''s a mount line from our first OSS node:

LABEL=lustre-OST0000    /mnt/lustreost0         lustre  defaults        0 0
LABEL=lustre-OST0001    /mnt/lustreost1         lustre  defaults,noauto 0 0

It has a partner, and the lines in that fstab swap the
''noauto'' flag.

And from one of the client nodes:

hm0-0 at tcp0:hm0-1 at tcp0:/lustre   /mnt/lustre     lustre  defaults,_netdev 0
0

cheers,
Klaus

On 2/14/08 10:38 AM, "Brian J. Murrell" <Brian.Murrell at
Sun.COM>did etch on
stone tablets:
> As any of you using version 1.6 of Lustre knows, Lustre servers can now
> be started simply my mounting the devices it is using.  Even
> an /etc/fstab entry can be used if you can have the mount delayed until
> the network is started.
> 
> Given this change, you have also notices that we have eliminated the
> initscript for Lustre that used to exist for releases prior to 1.6.
> 
> I''d like to take a small survey on how those of you using
mountconf
> (1.6) are managing the mounting of your Lustre devices on the servers.
> The obvious choices are:
> 
>       * /etc/fstab
>       * home-grown initscript
>       * heartbeat managed
>       * other
> 
> If you are using /etc/fstab I''d be interested in seeing your
/etc/fstab
> entries (you can omit obvious near-duplication in the case of many OSTs
> though) and knowing what distribution (make, version) you are using.
> 
> /etc/fstab entries provides an interesting wrinkle in that you cannot
> mount the devices until the network is up and yet most distributions
> do /etc/fstab (i.e. "local") mounting before the network unless
some
> mechanism is used to filter out entries that need network connectivity
> first.  The traditional way of doing this has been to add _netdev to
> those entries and they are then delayed until the network is up.
> 
> Cheers,
> b.
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

Brian J. Murrell wrote:
> As any of you using version 1.6 of Lustre knows, Lustre servers can now
> be started simply my mounting the devices it is using.  Even
> an /etc/fstab entry can be used if you can have the mount delayed until
> the network is started.
> 
> Given this change, you have also notices that we have eliminated the
> initscript for Lustre that used to exist for releases prior to 1.6.
> 
> I''d like to take a small survey on how those of you using
mountconf
> (1.6) are managing the mounting of your Lustre devices on the servers.
> The obvious choices are:
> 
>       * /etc/fstab
>       * home-grown initscript
>       * heartbeat managed
>       * other

We are using a hacked up lustre_config (see bug 14095 & friends).
I''m
predicting that we''ll continue on that path, adding features that were 
in lconf as we go. The centralized command & control are important 
features for us.

We have a separate platform specific heartbeat. We will be adding a 
''lconf --service'' type approach for lustre_config in the near
future to
enable automatic failover on Lustre 1.6

Nic

On Feb 14, 2008  11:17 -0800, Klaus Steden wrote:
> Here''s a mount line from our first OSS node:
> 
> LABEL=lustre-OST0000    /mnt/lustreost0         lustre  defaults        0 0
> LABEL=lustre-OST0001    /mnt/lustreost1         lustre  defaults,noauto 0 0
> 
> It has a partner, and the lines in that fstab swap the
''noauto'' flag.
Klaus, if you have the backup node mounting the filesystem because of
primary server failure, how do you prevent the primary server from mounting
the filesystem again as soon as it boots?

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

On Feb 14, 2008  13:38 -0500, Brian J. Murrell wrote:
> As any of you using version 1.6 of Lustre knows, Lustre servers can now
> be started simply my mounting the devices it is using.  Even
> an /etc/fstab entry can be used if you can have the mount delayed until
> the network is started.
> 
> Given this change, you have also notices that we have eliminated the
> initscript for Lustre that used to exist for releases prior to 1.6.
> 
> I''d like to take a small survey on how those of you using
mountconf
> (1.6) are managing the mounting of your Lustre devices on the servers.
> The obvious choices are:
> 
>       * /etc/fstab
My home system has no failover (only a single node), so it mounts and
also runs e2fsck at boot:

LABEL=myth-OST0000  /mnt/ost/myth-ost0  lustre  _netdev                   1 3
: (etc...)
LABEL=myth-MDT0000  /mnt/mdt/myth-mdt0  lustre  _netdev,acl               1 3

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

At this point, recovery of the primary server is a manual process -- so in
the case of a failure, the secondary would assume service for the failed
node, which gets powered off; an administrator is required to intervene to
recover the primary.

Klaus

On 2/14/08 2:51 PM, "Andreas Dilger" <adilger at sun.com>did
etch on stone
tablets:
> On Feb 14, 2008  11:17 -0800, Klaus Steden wrote:
>> Here''s a mount line from our first OSS node:
>> 
>> LABEL=lustre-OST0000    /mnt/lustreost0         lustre  defaults       
0 0
>> LABEL=lustre-OST0001    /mnt/lustreost1         lustre  defaults,noauto
0 0
>> 
>> It has a partner, and the lines in that fstab swap the
''noauto'' flag.
> 
> Klaus, if you have the backup node mounting the filesystem because of
> primary server failure, how do you prevent the primary server from mounting
> the filesystem again as soon as it boots?
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

We are using a home-grown set of init scripts.  There is an "lnet"
init
script to bring up the lnet networking, and a "lustre" init script to
start lustre services.

We tend to be paranoid about the possibility of double mounting a
multi-homed LUNs, so failover for us requires a sysadmin to manually
issue the command to move a LUN to the failover node.

We have a custom udev script to detect any available LUNs on a node,
identify them as normal or backup lustre devices, and name them
accordingly under /dev/lustre.

The lustre init script runs an optional journal-replay-only fsck on each
device before issuing a mount command to start the lustre service.

Chris

Brian J. Murrell wrote:
> As any of you using version 1.6 of Lustre knows, Lustre servers can now
> be started simply my mounting the devices it is using.  Even
> an /etc/fstab entry can be used if you can have the mount delayed until
> the network is started.
> 
> Given this change, you have also notices that we have eliminated the
> initscript for Lustre that used to exist for releases prior to 1.6.
> 
> I''d like to take a small survey on how those of you using
mountconf
> (1.6) are managing the mounting of your Lustre devices on the servers.
> The obvious choices are:
> 
>       * /etc/fstab
>       * home-grown initscript
>       * heartbeat managed
>       * other
> 
> If you are using /etc/fstab I''d be interested in seeing your
/etc/fstab
> entries (you can omit obvious near-duplication in the case of many OSTs
> though) and knowing what distribution (make, version) you are using.
> 
> /etc/fstab entries provides an interesting wrinkle in that you cannot
> mount the devices until the network is up and yet most distributions
> do /etc/fstab (i.e. "local") mounting before the network unless
some
> mechanism is used to filter out entries that need network connectivity
> first.  The traditional way of doing this has been to add _netdev to
> those entries and they are then delayed until the network is up.
> 
> Cheers,
> b.
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHvjcyg40IeHfy1xIRCjSaAKCOVBqNlxhpdYghV30aTIUqUoITFQCggCcr
VRoSFlEIhSDTqwMsCML0dKs=GJRV
-----END PGP SIGNATURE-----