Taylor Lewick
2015-Aug-12 15:52 UTC
[Gluster-users] After Centos 6 yum update client fails to mount glusterfs volume
I have an 8 node gluster cluster running 3.6.2. Yesterday one of our admin ran a yum update on his Centos 6.6 machine, and it updated the gluster client to 3.7.3. (Client update only on his machine, not the gluster cluster servers). After the update he was not able to mount the gluster volume. He then removed and reinstalled the 3.6 gluster client, and he was able to mount the volume again. So to test/confirm, I did the same thing. Prior to the running the upgrade I was running 3.6.4 glusterfs and glusterfs-fuse, and I had the mountpoint mounted on my test server. I ran the yum update glusterfs, it completed successfully, and I could access the mount. I then unmounted the gluster mount, and when I tried to mount it, and I received the following error: Mount failed. Please check the log file for more details. Since I could access the mountpoint, it appears something has happened in this update to change default behavior from a client mount request. Any ideas what changes I need to make to be able to mount the volume? With 3.7 version of client, do you have to specifically set auth.allow options for volumes? Below is the logfile from the client [2015-08-12 15:47:10.982346] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.3 (args: /usr/sbin/glusterfs --volfile-server=infarchive01 --volfile-id=/archives /media) [2015-08-12 15:47:11.029757] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2015-08-12 15:47:11.058367] I [MSGID: 114020] [client.c:2118:notify] 0-archives-client-0: parent translators are ready, attempting connect on transport [2015-08-12 15:47:11.058529] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2015-08-12 15:47:11.065361] I [MSGID: 114020] [client.c:2118:notify] 0-archives-client-1: parent translators are ready, attempting connect on transport [2015-08-12 15:47:11.066220] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-archives-client-0: changing port to 49152 (from 0) [2015-08-12 15:47:11.073949] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-archives-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-08-12 15:47:11.074260] I [MSGID: 114020] [client.c:2118:notify] 0-archives-client-2: parent translators are ready, attempting connect on transport [2015-08-12 15:47:11.075504] W [MSGID: 114043] [client-handshake.c:1114:client_setvolume_cbk] 0-archives-client-0: failed to set the volume [Permission denied] [2015-08-12 15:47:11.075671] W [MSGID: 114007] [client-handshake.c:1143:client_setvolume_cbk] 0-archives-client-0: failed to get 'process-uuid' from reply dict [Invalid argument] [2015-08-12 15:47:11.075761] E [MSGID: 114044] [client-handshake.c:1149:client_setvolume_cbk] 0-archives-client-0: SETVOLUME on remote-host failed [Permission denied] [2015-08-12 15:47:11.075801] I [MSGID: 114049] [client-handshake.c:1240:client_setvolume_cbk] 0-archives-client-0: sending AUTH_FAILED event [2015-08-12 15:47:11.075840] E [fuse-bridge.c:5130:notify] 0-fuse: Server authenication failed. Shutting down. [2015-08-12 15:47:11.075892] I [fuse-bridge.c:5595:fini] 0-fuse: Unmounting '/media'. [2015-08-12 15:47:11.082675] I [MSGID: 114020] [client.c:2118:notify] 0-archives-client-3: parent translators are ready, attempting connect on transport [2015-08-12 15:47:11.084889] W [glusterfsd.c:1219:cleanup_and_exit] (-->/lib64/libpthread.so.0() [0x3fe5a07a51] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d] -->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received signum (15), shutting down -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150812/5367ec90/attachment.html>
Atin Mukherjee
2015-Aug-12 16:34 UTC
[Gluster-users] After Centos 6 yum update client fails to mount glusterfs volume
Please try any of the below two workarounds: There are two possible workarounds. Before upgrading, 1. Set 'client.bind-insecure off' on all volumes. This forces 3.7.3 clients to use secure ports to connect to the servers. This does not affect older clients as this setting is the default for them. 2. Set 'server.allow-insecure on' on all volumes. This enables servers to accept connections from insecure ports as well and allows the new clients to successfully connect to the servers. -Atin Sent from one plus one On Aug 12, 2015 9:22 PM, "Taylor Lewick" <tlewick at adknowledge.com> wrote:> I have an 8 node gluster cluster running 3.6.2. Yesterday one of our > admin ran a yum update on his Centos 6.6 machine, and it updated the > gluster client to 3.7.3. (Client update only on his machine, not the > gluster cluster servers). > > After the update he was not able to mount the gluster volume. > > He then removed and reinstalled the 3.6 gluster client, and he was able to > mount the volume again. > > > > So to test/confirm, I did the same thing. Prior to the running the > upgrade I was running 3.6.4 glusterfs and glusterfs-fuse, and I had the > mountpoint mounted on my test server. > > I ran the yum update glusterfs, it completed successfully, and I could > access the mount. I then unmounted the gluster mount, and when I tried to > mount it, and I received the following error: > > Mount failed. Please check the log file for more details. > > Since I could access the mountpoint, it appears something has happened in > this update to change default behavior from a client mount request. Any > ideas what changes I need to make to be able to mount the volume? > > With 3.7 version of client, do you have to specifically set auth.allow > options for volumes? > > Below is the logfile from the client > > [2015-08-12 15:47:10.982346] I [MSGID: 100030] [glusterfsd.c:2301:main] > 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.3 > (args: /usr/sbin/glusterfs --volfile-server=infarchive01 > --volfile-id=/archives /media) > > [2015-08-12 15:47:11.029757] I [MSGID: 101190] > [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread > with index 1 > > [2015-08-12 15:47:11.058367] I [MSGID: 114020] [client.c:2118:notify] > 0-archives-client-0: parent translators are ready, attempting connect on > transport > > [2015-08-12 15:47:11.058529] I [MSGID: 101190] > [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread > with index 2 > > [2015-08-12 15:47:11.065361] I [MSGID: 114020] [client.c:2118:notify] > 0-archives-client-1: parent translators are ready, attempting connect on > transport > > [2015-08-12 15:47:11.066220] I [rpc-clnt.c:1819:rpc_clnt_reconfig] > 0-archives-client-0: changing port to 49152 (from 0) > > [2015-08-12 15:47:11.073949] I [MSGID: 114057] > [client-handshake.c:1437:select_server_supported_programs] > 0-archives-client-0: Using Program GlusterFS 3.3, Num (1298437), Version > (330) > > [2015-08-12 15:47:11.074260] I [MSGID: 114020] [client.c:2118:notify] > 0-archives-client-2: parent translators are ready, attempting connect on > transport > > [2015-08-12 15:47:11.075504] W [MSGID: 114043] > [client-handshake.c:1114:client_setvolume_cbk] 0-archives-client-0: failed > to set the volume [Permission denied] > > [2015-08-12 15:47:11.075671] W [MSGID: 114007] > [client-handshake.c:1143:client_setvolume_cbk] 0-archives-client-0: failed > to get 'process-uuid' from reply dict [Invalid argument] > > [2015-08-12 15:47:11.075761] E [MSGID: 114044] > [client-handshake.c:1149:client_setvolume_cbk] 0-archives-client-0: > SETVOLUME on remote-host failed [Permission denied] > > [2015-08-12 15:47:11.075801] I [MSGID: 114049] > [client-handshake.c:1240:client_setvolume_cbk] 0-archives-client-0: sending > AUTH_FAILED event > > [2015-08-12 15:47:11.075840] E [fuse-bridge.c:5130:notify] 0-fuse: Server > authenication failed. Shutting down. > > [2015-08-12 15:47:11.075892] I [fuse-bridge.c:5595:fini] 0-fuse: > Unmounting '/media'. > > [2015-08-12 15:47:11.082675] I [MSGID: 114020] [client.c:2118:notify] > 0-archives-client-3: parent translators are ready, attempting connect on > transport > > [2015-08-12 15:47:11.084889] W [glusterfsd.c:1219:cleanup_and_exit] > (-->/lib64/libpthread.so.0() [0x3fe5a07a51] > -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d] > -->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received > signum (15), shutting down > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150812/85c8086a/attachment.html>