[Gluster-users] GlusterFS with iSCSI and PaceMaker

Jon Heese
2015-04-21T11:49:02Z

Justin,


I do something similar, but not quite as complex.


I have a replicated (x2) Gluster volume where I drop thin-provisioned iSCSI
volume files to be served up via tgtd on CentOS6.  I mount the Gluster volume
locally on both Gluster servers (FUSE driver), then point the tgtd daemons to
the image files.  I use this as a back-end for a VMware ESXi datastore, so
I'm using the multipath iSCSI functionality of ESXi to handle the failover
between the nodes.


I was previously doing this with DRBD/Pacemaker/Corosync, but ESXi freaks out
when all paths to a datastore go down, and it takes ~2-5 seconds for the entire
cluster stack to go down and come back up during an orderly failover (to say
nothing of a catastrophic failover), so that model just didn't work for me.


I've since done some testing with just a simple VIP in Keepalived on top of
Gluster using LIO and the libgfapi stuff on CentOS7, and that seemed to work
great -- but I have some other incompatibilities with CentOS7, so I decided not
to pursue that for this project -- maybe another one on the horizon.


I briefly tried testing the libgfapi driver on CentOS6 with my current
production setup, but when I started the rebuild tgtd instance it gave my iSCSI
LUNs different LUN numbers so my ESXi cluster didn't recognize them as
different paths to the same LUN.  I couldn't be bothered to work out the
reason for this change, so I just switched back in the meantime.  I will
probably play with it a bit in a test environment when I have time -- the
libgfapi stuff should be faster and more efficient than going through the FUSE
stuff.


What would the Pacemaker CRM handle for you, besides a shared VIP?  Would you
want it to start/stop the iSCSI target daemon as well?  (If so, why?)  Is there
any reason to use a full CRM for this versus a simple VIP in something like
keepalived?


Good luck, and let us know how you get on!


Regards,

Jon Heese



________________________________
From: gluster-users-bounces at gluster.org <gluster-users-bounces at
gluster.org> on behalf of Justin Chin-You <justin.chinyou at gmail.com>
Sent: Monday, April 20, 2015 10:14 PM
To: gluster-users at gluster.org
Subject: [Gluster-users] GlusterFS with iSCSI and PaceMaker

Hi All,

I'm new to Gluster and I'm trying to work through some test
configurations.

What I am trying to do is use Gluster to create a mirrored and scalable storage
array. My plan is once I setup Gluster is to then export the GlusterFS via an
iSCSI interface which will have an Active/Standby failover managed via
Pacemaker.

I am curious if anyone has tried anything similar and if you have what iSCSI
driver are you using.

Thanks,

Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150421/4967f560/attachment.html>