Herb Burnswell
2018-Aug-07 02:40 UTC
[Gluster-users] Active-Active HA ganesha-nfs configuration
All, I would like to set up HA NFS (Active/Active) on our 2 node gluster environment using NFS-Ganesha. Specs: - RHEL 7 - glusterfs 3.8.15 built on Aug 16 2017 14:48:01 I am following this process in this documentation, however it is confusing to me: https://docs.gluster.org/en/v3/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/ I already had Pacemaker/Corosync up and running on the our 2 node gluster environment with fence resources on each. After the package installs and confirming "Pre-requisites to run NFS-Ganesha", here is what I've done: 1. # gluster volume set all cluster.enable-shared-storage enable 2. Create /etc/ganesha/ganesha-ha.conf file (scrubbed. I'm also assuming that HA_NAME should be equal to the already created pacemaker cluster name): # # HA File HA_NAME="clustername" HA_CLUSTER_NODES="server1,server2" VIP_server1="10.19.3.66" VIP_server2="10.19.3.67" 3. # gluster nfs-ganesha enable Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue? (y/n) y This will take a few minutes to complete. Please wait .. nfs-ganesha : success 4. # gluster volume set vol1 ganesha.enable on volume set: success At this point I can see the export available: # showmount -e Export list for server1: /vol1 (everyone) And I can successfully mount the export from another server. However, nothing appears to be done regarding HA. nfs-ganesha is not started on server2 and no additional resources are created in pacemaker. Can anyone provide guidance as to what I may be doing incorrectly? Thanks, HB -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180806/4de97386/attachment.html>
David Spisla
2018-Aug-07 07:51 UTC
[Gluster-users] Active-Active HA ganesha-nfs configuration
Hello Herbs, this setup is not as easy as it sounds. Here you can find additional setup instructions (look for the chapter of NFS-Ganesha HA): https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.2/pdf/administration_guide/Red_Hat_Gluster_Storage-3.2-Administration_Guide-en-US.pdf Maybe you should use Gluster3.10 because the setup for 3.9 and 3.8 is not stable. With Gluster 3.10 I had a stable HA-Cluster. Also you should be aware of the fact, that since Gluster 3.11 this setup may not work because the Developers switching to storhaug: https://github.com/linux-ha-storage/storhaug/blob/master/README.md At the moment there is confusing in the coummunity about that issue. Storhaug is not complete and still under development. I don't know if my informations are up-to-date. Because of that fact I was switching to Samba (SMB). You can also have a HA Cluster with Samba/CTDB. Regards David Spisla 2018-08-07 4:40 GMT+02:00 Herb Burnswell <herbert.burnswell at gmail.com>:> All, > > I would like to set up HA NFS (Active/Active) on our 2 node gluster > environment using NFS-Ganesha. > > Specs: > > - RHEL 7 > - glusterfs 3.8.15 built on Aug 16 2017 14:48:01 > > I am following this process in this documentation, however it is confusing > to me: > > https://docs.gluster.org/en/v3/Administrator%20Guide/NFS- > Ganesha%20GlusterFS%20Integration/ > > I already had Pacemaker/Corosync up and running on the our 2 node gluster > environment with fence resources on each. > > After the package installs and confirming "Pre-requisites to run > NFS-Ganesha", here is what I've done: > > 1. # gluster volume set all cluster.enable-shared-storage enable > > 2. Create /etc/ganesha/ganesha-ha.conf file (scrubbed. I'm also assuming > that HA_NAME should be equal to the already created pacemaker cluster name): > > # > # HA File > > HA_NAME="clustername" > HA_CLUSTER_NODES="server1,server2" > > VIP_server1="10.19.3.66" > VIP_server2="10.19.3.67" > > 3. # gluster nfs-ganesha enable > Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the > trusted pool. Do you still want to continue? > (y/n) y > This will take a few minutes to complete. Please wait .. > nfs-ganesha : success > > > 4. # gluster volume set vol1 ganesha.enable on > volume set: success > > At this point I can see the export available: > > # showmount -e > Export list for server1: > /vol1 (everyone) > > And I can successfully mount the export from another server. > > However, nothing appears to be done regarding HA. nfs-ganesha is not > started on server2 and no additional resources are created in pacemaker. > > Can anyone provide guidance as to what I may be doing incorrectly? > > Thanks, > > HB > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180807/2c665090/attachment.html>
Kaleb S. KEITHLEY
2018-Aug-07 15:20 UTC
[Gluster-users] Active-Active HA ganesha-nfs configuration
Pacemaker and corosync running on both nodes? ports in the firewall open for pacemaker and corosync? selinux ? `pcs status` shows what? You should have a pair of ganesha_mon, ganesha_grace, ganesha_nfsd, and VIP resource agents more or less matching your two hosts and the two VIPs. Look for errors in /var/log/messages and /var/log/cluster/* for any anomalies. Might want to check with the real experts in #clusterlabs on freenode too. Because when the stuff works, it's great, but when it doesn't work, they're the experts. -- Kaleb On 08/06/2018 10:40 PM, Herb Burnswell wrote:> All, > > I would like to set up HA NFS (Active/Active) on our 2 node gluster > environment using NFS-Ganesha. > > Specs: > > - RHEL 7 > - glusterfs 3.8.15 built on Aug 16 2017 14:48:01 > > I am following this process in this documentation, however it is > confusing to me: > > https://docs.gluster.org/en/v3/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/ > > I already had Pacemaker/Corosync up and running on the our 2 node > gluster environment with fence resources on each. > > After the package installs and confirming "Pre-requisites to run > NFS-Ganesha", here is what I've done: > > 1. # gluster volume set all cluster.enable-shared-storage enable > > 2. Create /etc/ganesha/ganesha-ha.conf file (scrubbed.? I'm also > assuming that HA_NAME should be equal to the already created pacemaker > cluster name): > > # > # HA File > > HA_NAME="clustername" > HA_CLUSTER_NODES="server1,server2" > > VIP_server1="10.19.3.66" > VIP_server2="10.19.3.67" > > 3. # gluster nfs-ganesha enable > Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the > trusted pool. Do you still want to continue? > ?(y/n) y > This will take a few minutes to complete. Please wait .. > nfs-ganesha : success? > > > 4. # gluster volume set vol1 ganesha.enable on > volume set: success > > At this point I can see the export available: > > # showmount -e > Export list for server1: > /vol1 (everyone) > > And I can successfully mount the export from another server. > > However, nothing appears to be done regarding HA.? nfs-ganesha is not > started on server2 and no additional resources are created in pacemaker. > > Can anyone provide guidance as to what I may be doing incorrectly? > > Thanks, > > HB > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users >