Gluster users - Oct 2010 - Question about Volume Type when bricks are on SAN

Howdy,

I'm in the process of setting up GlusterFS for our users to test. I'd
like some opinions about which volume type makes sense for our configuration.

Here's our hardware config:
2 x Gluster servers with 4Gbit FC and 10Gbit Ethernet (both FC and 10GigE are
dual path to their respective switches)

Each Gluster server has been allocated a single 9TB LUN (if it makes any sense
to do so, I could have the SAN admin provide many smaller LUNs per server and
use LVM to combine into single brick)

The servers are attached to the SAN via 4Gbit fiber channel. The SAN uses RAID
5'like to create the LUNs (or more likely the volume from which the LUNs are
carved).

So, here's my dilemma. Part of me says, 18TB of raw storage is a whole lot
better than 9TB, and given that the LUNs are on enterprise hardware, why not
create a Distributed volume? It should be safe pretty safe, right?

But then I think, well if server2 goes down, then all of the files on its LUN
are not accessible to the clients, so maybe Distributed Replicated makes more
sense, at the cost of only providing 9TB raw to the end users.

Would a Distributed Stripe (or Distributed Replicate for that matter) make any
sense in this configuration, given that both bricks are physically on the same
SAN?

One final question, is there were a way in Gluster to have a Distributed with
failover, where if server2 dies, server1 can mount server2's LUN, once
server2 was back online, server1 could be told to stop hosting the brick and
return it to server2.

We do this with Lustre and it's worked well so far.

Thanks for any insight,

Mike

Hi mike

-----Original Message-----
From: gluster-users-bounces at gluster.org
[mailto:gluster-users-bounces at gluster.org] On Behalf Of Mike Hanby
Sent: Friday, October 22, 2010 12:23 PM
To: gluster-users at gluster.org
Subject: [Gluster-users] Question about Volume Type when bricks are on SAN

>One final question, is there were a way in Gluster to have a Distributed
>with failover, where if server2 dies, server1 can mount server2's LUN,
once
>server2 was back online, server1 could be told to stop hosting the brick
>and return it to server2.
In gluster (bye it's self) ... no,  but through corosync/pacemaker yes.


I am currently doing just that but with ISCSI.  
In my case:

	2 Gluster servers A & B
	5 Gluster clients 1 to 5

A and B each attach individual ISCSI targets, mount them and then server
them with gluster

The clients 1 to 5 then mount a replicated gluster share made from servers A
& B

If A should go down, then B will attach to the ISCSI target A was using and
then re-server it for the clients.  When A comes backup up, B stops
servering A's resources and disconnects from A's ISCSI target so A can
bring
it all back online as normal.

As a note I am using corosync/pacemaker to control starting, stopping and
moving of the required resource.

I hope this helps

Pat

On Sat, Oct 23, 2010 at 5:22 AM, Mike Hanby <mhanby at uab.edu>
wrote:
> Each Gluster server has been allocated a single 9TB LUN (if it makes any
sense to do so, I could have the SAN admin provide many smaller LUNs per server
and use LVM to combine into single brick)
Generally speaking you should LVM over many smaller LUNs.  With
multiport FC HBAs being the standard, you will get better performance
as separate requests hit separate LUNs.  Multiple LUNs mean multiple
SCSI queues, which can reduce I/O latency substantially.

We try to cap our LUNs at around 100GB.  It does make for a lot of
LUNs per server, but it's worth if for the performance gain.  Talk to
your storage admin and ask them what their recommendation is based on
your vendor and their best practices.

-Dan