Lustre discuss - Dec 2006 - Some questions about Lustre FS and building SAN.

Hello!

I have several questions about Lustre.

First of all, ClusterFS has commercial support and on the page
http://www.lustre.org/architecture.html I see following:
"On the download pages you can find a ***sample experimental
implementation***
of some the components above. We hope to enhance many of these and many
other components as time goes along. Much documentation can be found
on this site."

Does it mean that there are two separate versions of Lustre file system:
one commercially supported and another one commercially supported?

Second, I want to build system consisting of several frontend-nodes
and two-four backend (storage) nodes. I don''t want to use hardware
shared
storage for some reasons. So, I can use some networking block device
to connect two nodes (e.g. DRBD v0.8 in primary/primary mode) if needed.
The question, is it possible to setup Lustre in such configuration:
1) over DRBD or local hard drives if possible on two or four storage nodes;
2) to be able mount Lustre file system at frontend nodes which have no
direct access to shared storage device, i.g. is it possible to mount
Lustre over network?;
3) to have failover and load-balancing storage nodes, so each node
will hold entire copy of stored data, so storage nodes can be accessed
in parallel and if one node goes down, system will function correctly.

Is Lustre file system suitable for this?

Thank you.

Igor Zhbanov wrote:
> Hello!
>
> I have several questions about Lustre.
>
> First of all, ClusterFS has commercial support and on the page
> http://www.lustre.org/architecture.html I see following:
> "On the download pages you can find a ***sample experimental 
> implementation***
> of some the components above. We hope to enhance many of these and many
> other components as time goes along. Much documentation can be found
> on this site."
>
> Does it mean that there are two separate versions of Lustre file system:
> one commercially supported and another one commercially supported?
>
The lustre.org website is probably out of date.  ClusterFS maintains 
just 1 GPLed
Lustre now.
> Second, I want to build system consisting of several frontend-nodes
> and two-four backend (storage) nodes. I don''t want to use hardware
shared
> storage for some reasons. So, I can use some networking block device
> to connect two nodes (e.g. DRBD v0.8 in primary/primary mode) if needed.
> The question, is it possible to setup Lustre in such configuration:
> 1) over DRBD or local hard drives if possible on two or four storage 
> nodes;
We have heard of some success using network block devices, although
your performance will probably be bad.
> 2) to be able mount Lustre file system at frontend nodes which have no
> direct access to shared storage device, i.g. is it possible to mount
> Lustre over network?;
That''s the whole point of Lustre.  Clients just need network
connections
to the servers.
> 3) to have failover and load-balancing storage nodes, so each node
> will hold entire copy of stored data, so storage nodes can be accessed
> in parallel and if one node goes down, system will function correctly.
>
To do this you need a true shared storage device, because you need 
access to
the data if either node goes down - a device network-shared from a down
node does no good.  Lustre does not currently provide a RAID-type of data
backup, although there are future plans to do this.
> Is Lustre file system suitable for this?
>
> Thank you.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>

On 12/15/06, Nathaniel Rutman <nathan@clusterfs.com>
wrote:
> Igor Zhbanov wrote:
> >
> > Second, I want to build system consisting of several frontend-nodes
> > and two-four backend (storage) nodes. I don''t want to use
hardware shared
> > storage for some reasons. So, I can use some networking block device
> > to connect two nodes (e.g. DRBD v0.8 in primary/primary mode) if
needed.
Discussed earlier in a thread as well. DRBD in primary/primary mode
isn''t required and won''t work given Lustre''s design.
Something like
GFS can make use of this, however, that doesn''t work beyond a two node
configuration. Even using gndb to export drives to other systems and
creating layer upon layer to construct redundant shared storage gets
complicated far too quickly. Using LVM mirroring is an option,
however, that''s a discussion for the relevant mailing list.
> > The question, is it possible to setup Lustre in such configuration:
> > 1) over DRBD or local hard drives if possible on two or four storage
> > nodes;
> We have heard of some success using network block devices, although
> your performance will probably be bad.
We''re running such an implementation and also moving towards medium
sized deployments for client sites. Some deployments are already in
place and waiting to enter a production environment.

There is obviously the overhead of DRBD which comes into play if one
wants to run an active/active failover for storage, however, quite a
bit is dependent on the network, deployment scenario, client base,
etc.

The point is: it works quite well and integrates with HA services for
failover as is most often required.
> > 2) to be able mount Lustre file system at frontend nodes which have no
> > direct access to shared storage device, i.g. is it possible to mount
> > Lustre over network?;
> That''s the whole point of Lustre.  Clients just need network
connections
> to the servers.
> > 3) to have failover and load-balancing storage nodes, so each node
> > will hold entire copy of stored data, so storage nodes can be accessed
> > in parallel and if one node goes down, system will function correctly.
> >
> To do this you need a true shared storage device, because you need
> access to
> the data if either node goes down - a device network-shared from a down
> node does no good.  Lustre does not currently provide a RAID-type of data
> backup, although there are future plans to do this.
Again, this is where DRBD comes in if using local storage. Works
splendidly well.
-- 
Mustafa A. Hashmi
mahashmi@gmail.com