Surya K Ghatty
2015-Nov-05 15:13 UTC
[Gluster-users] Question on HA Active-Active Ganesha setup
All... I need your help! I am trying to setup Highly available Active-Active Ganesha configuration on two glusterfs nodes based on instructions here: https://gluster.readthedocs.org/en/latest/Administrator%20Guide/Configuring%20HA%20NFS%20Server/ and http://www.slideshare.net/SoumyaKoduri/high-49117846 and https://www.youtube.com/watch?v=Z4mvTQC-efM. My questions: 1. what is the expected behvaior? Is the cluster.enable-shared-storage command expected to create shared storage? It seems odd to return a success message without creating the shared volume. 2. Any suggestions on how to get past this problem? Details: I am using glusterfs 3.7.5 and Ganesha 2.2.0.6 installable packages. I'm installing Also, I am using the following command gluster volume set all cluster.enable-shared-storage enable that would automatically setup the shared_storage directory under /run/gluster/ and automounts the shared volume for HA. This command was working perfectly fine, and I was able to setup ganesha HA successfully on cent OS 7.0 running on bare metals - until now. [root at qint-tor01-c7 gluster]# gluster vol set all cluster.enable-shared-storage enable volume set: success [root at qint-tor01-c7 gluster]# pwd /run/gluster [root at qint-tor01-c7 gluster]# ls 5027ba011969a8b2eca99ca5c9fb77ae.socket shared_storage changelog-9fe3f3fdd745db918d7d5c39fbe94017.sock snaps changelog-a9bf0a82aba38610df80c75a9adc45ad.sock Yesterday, we tried to deploy Ganesha HA with Gluster FSAL on a different cloud. and when I run the same command there, (same version of glusterfs and ganesha, same cent OS 7) - the command returned successfully, but it did not auto create the shared_storage directory. There were no logs either in /var/log/glusterfs/etc-glusterfs-glusterd.vol.log or /var/log/ganesha.log related to the command. However, I do see these logs written to the etc-glusterfs-glusterd.vol.log [2015-11-05 14:43:00.692762] W [socket.c:588:__socket_rwv] 0-nfs: readv on /var/run/gluster/9d5e1ba5e44bd1aa3331d2ee752a806a.socket failed (Invalid argument) on both ganesha nodes independent of the commands I execute. regarding this error, I did a ss -x | grep /var/run/gluster/9d5e1ba5e44bd1aa3331d2ee752a806a.socket and it appears that no process was using these sockets, on either machines. My questions: 1. what is the expected behvaior? Is the cluster.enable-shared-storage command expected to create shared storage? It seems odd to return a success message without creating the shared volume. 2. Any suggestions on how to get past this problem? Regards, Surya Ghatty "This too shall pass" ________________________________________________________________________________________________________ Surya Ghatty | Software Engineer | IBM Cloud Infrastructure Services Development | tel: (507) 316-0559 | ghatty at us.ibm.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151105/1b6d5499/attachment.html>
Kaleb KEITHLEY
2015-Nov-05 15:42 UTC
[Gluster-users] Question on HA Active-Active Ganesha setup
On 11/05/2015 10:13 AM, Surya K Ghatty wrote:> All... I need your help! I am trying to setup Highly available > Active-Active Ganesha configuration on two glusterfs nodes based on > instructions here: > > https://gluster.readthedocs.org/en/latest/Administrator%20Guide/Configuring%20HA%20NFS%20Server/ > and > http://www.slideshare.net/SoumyaKoduri/high-49117846 and > https://www.youtube.com/watch?v=Z4mvTQC-efM. > > > *My questions:* > > 1. what is the expected behvaior? Is the cluster.enable-shared-storage > command expected to create shared storage? It seems odd to return a > success message without creating the shared volume. > 2. Any suggestions on how to get past this problem? > > *Details:* > I am using glusterfs 3.7.5 and Ganesha 2.2.0.6 installable packages. I'm > installing > > Also, I am using the following command > > gluster volume set all cluster.enable-shared-storage enable > > that would automatically setup the shared_storage directory under > /run/gluster/ and automounts the shared volume for HA. > > This command was working perfectly fine, and I was able to setup ganesha > HA successfully on cent OS 7.0 running on bare metals - until now. > > > > [root at qint-tor01-c7 gluster]# gluster vol set all > cluster.enable-shared-storage enable > volume set: success > > [root at qint-tor01-c7 gluster]# pwd > /run/gluster > > [root at qint-tor01-c7 gluster]# ls > 5027ba011969a8b2eca99ca5c9fb77ae.socket shared_storage > changelog-9fe3f3fdd745db918d7d5c39fbe94017.sock snaps > changelog-a9bf0a82aba38610df80c75a9adc45ad.sock > > > Yesterday, we tried to deploy Ganesha HA with Gluster FSAL on a > different cloud. and when I run the same command there, (same version of > glusterfs and ganesha, same cent OS 7) - the command returned > successfully, but it did not auto create the shared_storage directory. > There were no logs either in > /var/log/glusterfs/etc-glusterfs-glusterd.vol.log > > or /var/log/ganesha.log related to the command. > > However, I do see these logs written to the etc-glusterfs-glusterd.vol.log > > [2015-11-05 14:43:00.692762] W [socket.c:588:__socket_rwv] 0-nfs: readv > on /var/run/gluster/9d5e1ba5e44bd1aa3331d2ee752a806a.socket failed > (Invalid argument) > > on both ganesha nodes independent of the commands I execute. > > regarding this error, I did a ss -x | grep > /var/run/gluster/9d5e1ba5e44bd1aa3331d2ee752a806a.socket > > and it appears that no process was using these sockets, on either machines. > > My questions: > > 1. what is the expected behvaior? Is the cluster.enable-shared-storage > command expected to create shared storage? It seems odd to return a > success message without creating the shared volume. > 2. Any suggestions on how to get past this problem? > Regards,The answer everyone hates to hear: It works for me. I suspect it's not working in your case because it wants to create a "replica 3" volume and you only have two nodes. My blog at http://blog.gluster.org/2015/10/linux-scale-out-nfsv4-using-nfs-ganesha-and-glusterfs-one-step-at-a-time/ documents what I did recently to set up a four node HA ganesha cluster for testing at the NFS Bake-a-thon that Red Hat hosted recently.> > Surya Ghatty > > "This too shall pass" > ________________________________________________________________________________________________________ > Surya Ghatty | Software Engineer | IBM Cloud Infrastructure Services > Development | tel: (507) 316-0559 | ghatty at us.ibm.com > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >
Soumya Koduri
2015-Nov-06 17:18 UTC
[Gluster-users] Question on HA Active-Active Ganesha setup
On 11/05/2015 08:43 PM, Surya K Ghatty wrote:> All... I need your help! I am trying to setup Highly available > Active-Active Ganesha configuration on two glusterfs nodes based on > instructions here: > > https://gluster.readthedocs.org/en/latest/Administrator%20Guide/Configuring%20HA%20NFS%20Server/ > and > http://www.slideshare.net/SoumyaKoduri/high-49117846 and > https://www.youtube.com/watch?v=Z4mvTQC-efM. > > > *My questions:* > > 1. what is the expected behvaior? Is the cluster.enable-shared-storage > command expected to create shared storage? It seems odd to return a > success message without creating the shared volume. > 2. Any suggestions on how to get past this problem? > > *Details:* > I am using glusterfs 3.7.5 and Ganesha 2.2.0.6 installable packages. I'm > installing > > Also, I am using the following command > > gluster volume set all cluster.enable-shared-storage enable > > that would automatically setup the shared_storage directory under > /run/gluster/ and automounts the shared volume for HA. > > This command was working perfectly fine, and I was able to setup ganesha > HA successfully on cent OS 7.0 running on bare metals - until now. > > > > [root at qint-tor01-c7 gluster]# gluster vol set all > cluster.enable-shared-storage enable > volume set: success > > [root at qint-tor01-c7 gluster]# pwd > /run/gluster > > [root at qint-tor01-c7 gluster]# ls > 5027ba011969a8b2eca99ca5c9fb77ae.socket shared_storage > changelog-9fe3f3fdd745db918d7d5c39fbe94017.sock snaps > changelog-a9bf0a82aba38610df80c75a9adc45ad.sock > > > Yesterday, we tried to deploy Ganesha HA with Gluster FSAL on a > different cloud. and when I run the same command there, (same version of > glusterfs and ganesha, same cent OS 7) - the command returned > successfully, but it did not auto create the shared_storage directory. > There were no logs either in > /var/log/glusterfs/etc-glusterfs-glusterd.vol.log > > or /var/log/ganesha.log related to the command. > > However, I do see these logs written to the etc-glusterfs-glusterd.vol.log > > [2015-11-05 14:43:00.692762] W [socket.c:588:__socket_rwv] 0-nfs: readv > on /var/run/gluster/9d5e1ba5e44bd1aa3331d2ee752a806a.socket failed > (Invalid argument) > > on both ganesha nodes independent of the commands I execute. > > regarding this error, I did a ss -x | grep > /var/run/gluster/9d5e1ba5e44bd1aa3331d2ee752a806a.socket > > and it appears that no process was using these sockets, on either machines. > > My questions: > > 1. what is the expected behvaior? Is the cluster.enable-shared-storage > command expected to create shared storage? It seems odd to return a > success message without creating the shared volume.yes. This command creates the volume "gluster_shared_storage" and mounts it at '/run/gluster/shared_storage'.> 2. Any suggestions on how to get past this problem?Has the "gluster_shared_storage" volume creation itself failed? Verify in 'gluster volume info'. Also check if the gluster nodes are in healthy state 'gluster peer status'. Try restarting 'glusterd' service on both the nodes. CCin Avra who may provide more details. Thanks, Soumya> Regards, > > Surya Ghatty > > "This too shall pass" > ________________________________________________________________________________________________________ > Surya Ghatty | Software Engineer | IBM Cloud Infrastructure Services > Development | tel: (507) 316-0559 | ghatty at us.ibm.com > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >