Hello, I'm very new to GlusterFS and would like to see if it may fit into a storage solution I'm working on. I have a dual server system with 95TB of data (10 x 9.5TB GFS2 partitions; each partition being a 10+2 RAID6 under the covers) connected via Fibre Channel. So each server can see all the data and provide server failover (turn one server off, the other server picks up the load). To complicate matters, there will be a need to expand this system to 190TB's in about a year (20 x 9.5TB). A new customer requirement came in where I need to provide access to these partitions as an NFS NAS and the existing files in the 10 GFS2 partitions need to be presented as one very large 95TB file system. The client is unaware there is any server side directory hierarchy. All the client will do is read from the root of the 95GB partition and write new files to the root where the server must transparently write the file data to one of the 10 GFS2 partitions in some distributed fashion (e.g. round-robin). GlusterFS provides the exact functionality I'm looking for - NAS + a union file system with a distributed write capability. However I get the idea that just laying GlusterFS on the existing system will probably not work. I'm mixing a SAN with a NAS and it seems that GlusterFS only can handle server level fault tolerance through data replication and that it cannot deal with the server level fault tolerance offered by the existing SAN. Data replication is not an option as the customer will not pay for twice the storage. So... - Can GlusterFS even work with GFS2 partitions or must it be XFS / EXT4 / (et al)? - Can GlusterFS be configured in to accommodate the existing SAN? - Even if I reconfigure such that these are XFS partitions and configure 5 arrays on one server and 5 on the other and without replication, I believe a server failover would make the storage on the failed server inaccessible. Is this correct? Thanks very much, Bob Ellison -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131111/ad97b685/attachment.html>
On Mon, Nov 11, 2013 at 10:49 AM, Ellison, Bob <bob.ellison at ccur.com> wrote:> Hello, > > I?m very new to GlusterFS and would like to see if it may fit into a storage > solution I?m working on. > > > > I have a dual server system with 95TB of data (10 x 9.5TB GFS2 partitions; > each partition being a 10+2 RAID6 under the covers) connected via Fibre > Channel. So each server can see all the data and provide server failover > (turn one server off, the other server picks up the load). To complicate > matters, there will be a need to expand this system to 190TB?s in about a > year (20 x 9.5TB). > > > > A new customer requirement came in where I need to provide access to these > partitions as an NFS NAS and the existing files in the 10 GFS2 partitions > need to be presented as one very large 95TB file system. The client is > unaware there is any server side directory hierarchy. All the client will do > is read from the root of the 95GB partition and write new files to the root > where the server must transparently write the file data to one of the 10 > GFS2 partitions in some distributed fashion (e.g. round-robin). > > > > GlusterFS provides the exact functionality I?m looking for - NAS + a union > file system with a distributed write capability. However I get the idea that > just laying GlusterFS on the existing system will probably not work. I?m > mixing a SAN with a NAS and it seems that GlusterFS only can handle server > level fault tolerance through data replication and that it cannot deal with > the server level fault tolerance offered by the existing SAN. Data > replication is not an option as the customer will not pay for twice the > storage. > > > > So? > > - Can GlusterFS even work with GFS2 partitions or must it be XFS / EXT4 / > (et al)?In theory sure but you wouldn't want it to. Under the hood XFS and GFS2 are very similar so what works well on one will generally work well on the other. Gluster normally handles its own replication so the idea of putting it on a shared filesystem is not all that useful> > > > - Can GlusterFS be configured in to accommodate the existing SAN?Not in your current configuration but it can very effectively be used as a NAS head for redundant SANS> > > > - Even if I reconfigure such that these are XFS partitions and configure 5 > arrays on one server and 5 on the other and without replication, I believe a > server failover would make the storage on the failed server inaccessible. Is > this correct?That is essentially correct. there is a possible way to get it to work in an active passive mode but its a messy kludge so I wont get into it. In the scenarios you described simple NFS would probably be your best bet. NFS 4 would be best but NFS 3 will work too as long as the lock file is also on a shared volume.then you could use Keepalived or Parana as a loadbalancer any you are done.> > > > Thanks very much, > > > > Bob Ellison > > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users
On 11/11/2013 10:49 AM, Ellison, Bob wrote:> - Can GlusterFS even work with GFS2 partitions or must it be XFS / > EXT4 / (et al)?GlusterFS can work with any filesystem that supports extended attributes. As far as I know this would include GFS2, though I don't know of anybody doing that and I suspect it might be a bad idea (because having two levels of heartbeats and HA is always a a bad idea).> - Can GlusterFS be configured in to accommodate the existing SAN?We - upstream/community, not Red Hat Storage - neither support nor preclude it. In other words, we don't have any code to set it up for you, but if you want to set up heartbeat and failover between paired servers instead of using our replication then you can. You'll probably even get better performance, at the cost of greater administrative and operational complexity. On the other hand, if you only have two servers then the other stuff is sufficient and you don't need us. ;) Such a configuration would only make sense if you're using it instead of that one specific replication component, but also need distribution/striping or other GlusterFS features in addition to that. For example, if you had ten servers and you used GlusterFS to create a "union" across five pairs, that's not a bad way to do things.