Hi, all,
We're looking to replace our DRBD/Ext4/NFS file storage configuration,
using RHEL cluster w/ GlusterFS 3.2.2 (or greater, depending on the
timeline). Currently, our configuration includes 2 four node clusters
at different sites with
1. 8 @ 1.5TB LVM LVs on top of an HP-P2000 storage array
2. Each LV is mirrored to an identical LV on a cluster at a remote site
using DRBD
3. Each LV/DRBD is an Ext4 volume and an NFS mount-point
4. DRBD is started as primary in one site and secondary in the remote
site and can be switched easily in the event of a failure.
5. RHEL cluster w/ some patches runs DRBD, floating IP, ext4, NFS as a
service for each of the 8 mounts.
We went this route because it was the only way to get POSIX-ACL support
and a working Quota implementation w/ replication of a large-ish volume
without spending incredible amounts of money. With GlusterFS 3.2.2,
these are both supported features. My proposed layout for the new
configuration would look like so:
1. 8 @ 1.5TB LVM LVs on top of an HP-P2000 storage array
2. Each LV is an Ext4 FS
3. RHEL cluster runs a glusterd instance, floating IP and, ext4 mount
for each of the 8 LVs.
4. Each of the 8 LVs is configured with a replicated pair in our remote
site while they distribute across the local site. For instance:
site1-node1: site2-node1:
gluster1:/glusterfs <------> gluster9:/glusterfs
gluster2:/glusterfs <------> gluster10:/glusterfs
site1-node2:
gluster3:/glusterfs <------> gluster11:/glusterfs
gluster4:/glusterfs <------> gluster12:/glusterfs
site1-node3:
gluster5:/glusterfs <------> gluster13:/glusterfs
gluster6:/glusterfs <------> gluster14:/glusterfs
site1-node4:
gluster7:/glusterfs <------> gluster15:/glusterfs
gluster8:/glusterfs <------> gluster16:/glusterfs
||||||||
||||||||
Distributed
||||||||
||||||||
----------------------
| | | |
client client client client
We will still use RHEL cluster to facilitate HA and failover of the
glusterd/ip/fs instances on each cluster site.
What say the experts about this approach and what caveats/issues should
I be looking out for? I'll be building a test environment, but was
wondering, before I start, whether this is a supportable configuration
in the event we decide to get support, etc.
Many thanks in advance!
-Brian
--
Brian Smith
Senior Systems Administrator
IT Research Computing, University of South Florida
4202 E. Fowler Ave. ENB308
Office Phone: +1 813 974-1467
Organization URL: http://rc.usf.edu