Atin Mukherjee
2015-Sep-12 17:34 UTC
[Gluster-users] After upgrade from 3.5 to 3.7 gluster local NFS is not starting on one of the servers
By any chance is your Gluster NFS server is already running? Output of netstat -nap | grep 38465 might give some clue? -Atin Sent from one plus one On Sep 12, 2015 10:54 PM, "Yaroslav Molochko" <onorua at gmail.com> wrote:> Hello, > > I have a problem reported in logs: > =================> [2015-09-12 13:56:06.271644] I [MSGID: 100030] [glusterfsd.c:2301:main] > 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 > (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p > /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S > /var/run/gluster/cb186678589f28e74c67da70fd06e736.socket) > [2015-09-12 13:56:06.277921] I [MSGID: 101190] > [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread > with index 1 > [2015-09-12 13:56:07.284888] I > [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured > rpc.outstanding-rpc-limit with value 16 > [2015-09-12 13:56:07.292484] W [MSGID: 112153] > [mount3.c:3910:mnt3svc_init] 0-nfs-mount: Exports auth has been disabled! > [2015-09-12 13:56:07.294357] E > [rpcsvc.c:1370:rpcsvc_program_register_portmap] 0-rpc-service: Could not > register with portmap 100005 3 38465 > [2015-09-12 13:56:07.294398] E [MSGID: 112088] > [nfs.c:341:nfs_init_versions] 0-nfs: Required program MOUNT3 registration > failed > [2015-09-12 13:56:07.294413] E [MSGID: 112109] [nfs.c:1482:init] 0-nfs: > Failed to initialize protocols > [2015-09-12 13:56:07.294426] E [MSGID: 101019] [xlator.c:428:xlator_init] > 0-nfs-server: Initialization of volume 'nfs-server' failed, review your > volfile again > [2015-09-12 13:56:07.294438] E [graph.c:322:glusterfs_graph_init] > 0-nfs-server: initializing translator failed > [2015-09-12 13:56:07.294448] E [graph.c:661:glusterfs_graph_activate] > 0-graph: init failed > [2015-09-12 13:56:07.294781] W [glusterfsd.c:1219:cleanup_and_exit] > (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x11a) [0x7fbe9c754b7a] > -->/usr/sbin/glusterfs(glusterfs_process_volfp+0x123) [0x7fbe9c74fcb3] > -->/usr/sbin/glusterfs(cleanup_and_exit+0x59) [0x7fbe9c74f329] ) 0-: > received signum (0), shutting down > ==================> > I've checked the page: > > http://www.gluster.org/community/documentation/index.php/Gluster_3.1:_NFS_Frequently_Asked_Questions > > I've found report in RedHat that it's necessary to remove -w from rpcbind > becuse some times it causes problems. > I did all that but still no luck on one of the servers, what is > interesting, the other server (peered) is working fine without any > problems. > > root at PSC01SERV008:/var/lib/glusterd/nfs# systemctl status nfs > ? nfs.service > Loaded: not-found (Reason: No such file or directory) > Active: inactive (dead) > > root at PSC01SERV008:/var/lib/glusterd/nfs# systemctl status rpcbind > ? rpcbind.service - RPC bind portmap service > Loaded: loaded (/etc/systemd/system/rpcbind.service; enabled; vendor > preset: enabled) > Drop-In: /run/systemd/generator/rpcbind.service.d > ??50-rpcbind-$portmap.conf > Active: active (running) since Sat 2015-09-12 13:55:07 UTC; 6min ago > Main PID: 9796 (rpcbind) > Memory: 428.0K > CGroup: /system.slice/rpcbind.service > ??9796 /sbin/rpcbind > > Sep 12 13:55:07 PSC01SERV008 systemd[1]: Starting RPC bind portmap > service... > Sep 12 13:55:07 PSC01SERV008 systemd[1]: Started RPC bind portmap service. > root at PSC01SERV008:/var/lib/glusterd/nfs# rpcinfo -p > program vers proto port service > 100000 4 tcp 111 portmapper > 100000 3 tcp 111 portmapper > 100000 2 tcp 111 portmapper > 100000 4 udp 111 portmapper > 100000 3 udp 111 portmapper > 100000 2 udp 111 portmapper > > I've tried to reinstall it agaon and again - but there is no luck. > > What I have: > cat /etc/lsb-release > DISTRIB_ID=Ubuntu > DISTRIB_RELEASE=15.04 > DISTRIB_CODENAME=vivid > DISTRIB_DESCRIPTION="Ubuntu 15.04" > > ii glusterfs-client 3.7.4-ubuntu1~vivid1 > amd64 clustered file-system (client package) > ii glusterfs-common 3.7.4-ubuntu1~vivid1 > amd64 GlusterFS common libraries and translator modules > ii glusterfs-server 3.7.4-ubuntu1~vivid1 > amd64 clustered file-system (server package) > > What else can I check? How can I fix it, what is most important :) > Thanks in advance! > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150912/0a800900/attachment.html>
Yaroslav Molochko
2015-Sep-13 04:08 UTC
[Gluster-users] After upgrade from 3.5 to 3.7 gluster local NFS is not starting on one of the servers
I wish this could be that simple: root at PSC01SERV008:/var/lib# netstat -nap | grep 38465 root at PSC01SERV008:/var/lib# ss -n | grep 38465 root at PSC01SERV008:/var/lib# 2015-09-13 1:34 GMT+08:00 Atin Mukherjee <atin.mukherjee83 at gmail.com>:> By any chance is your Gluster NFS server is already running? Output of > netstat -nap | grep 38465 might give some clue? > > -Atin > Sent from one plus one > On Sep 12, 2015 10:54 PM, "Yaroslav Molochko" <onorua at gmail.com> wrote: > >> Hello, >> >> I have a problem reported in logs: >> =================>> [2015-09-12 13:56:06.271644] I [MSGID: 100030] [glusterfsd.c:2301:main] >> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 >> (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p >> /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S >> /var/run/gluster/cb186678589f28e74c67da70fd06e736.socket) >> [2015-09-12 13:56:06.277921] I [MSGID: 101190] >> [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread >> with index 1 >> [2015-09-12 13:56:07.284888] I >> [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured >> rpc.outstanding-rpc-limit with value 16 >> [2015-09-12 13:56:07.292484] W [MSGID: 112153] >> [mount3.c:3910:mnt3svc_init] 0-nfs-mount: Exports auth has been disabled! >> [2015-09-12 13:56:07.294357] E >> [rpcsvc.c:1370:rpcsvc_program_register_portmap] 0-rpc-service: Could not >> register with portmap 100005 3 38465 >> [2015-09-12 13:56:07.294398] E [MSGID: 112088] >> [nfs.c:341:nfs_init_versions] 0-nfs: Required program MOUNT3 registration >> failed >> [2015-09-12 13:56:07.294413] E [MSGID: 112109] [nfs.c:1482:init] 0-nfs: >> Failed to initialize protocols >> [2015-09-12 13:56:07.294426] E [MSGID: 101019] [xlator.c:428:xlator_init] >> 0-nfs-server: Initialization of volume 'nfs-server' failed, review your >> volfile again >> [2015-09-12 13:56:07.294438] E [graph.c:322:glusterfs_graph_init] >> 0-nfs-server: initializing translator failed >> [2015-09-12 13:56:07.294448] E [graph.c:661:glusterfs_graph_activate] >> 0-graph: init failed >> [2015-09-12 13:56:07.294781] W [glusterfsd.c:1219:cleanup_and_exit] >> (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x11a) [0x7fbe9c754b7a] >> -->/usr/sbin/glusterfs(glusterfs_process_volfp+0x123) [0x7fbe9c74fcb3] >> -->/usr/sbin/glusterfs(cleanup_and_exit+0x59) [0x7fbe9c74f329] ) 0-: >> received signum (0), shutting down >> ==================>> >> I've checked the page: >> >> http://www.gluster.org/community/documentation/index.php/Gluster_3.1:_NFS_Frequently_Asked_Questions >> >> I've found report in RedHat that it's necessary to remove -w from rpcbind >> becuse some times it causes problems. >> I did all that but still no luck on one of the servers, what is >> interesting, the other server (peered) is working fine without any >> problems. >> >> root at PSC01SERV008:/var/lib/glusterd/nfs# systemctl status nfs >> ? nfs.service >> Loaded: not-found (Reason: No such file or directory) >> Active: inactive (dead) >> >> root at PSC01SERV008:/var/lib/glusterd/nfs# systemctl status rpcbind >> ? rpcbind.service - RPC bind portmap service >> Loaded: loaded (/etc/systemd/system/rpcbind.service; enabled; vendor >> preset: enabled) >> Drop-In: /run/systemd/generator/rpcbind.service.d >> ??50-rpcbind-$portmap.conf >> Active: active (running) since Sat 2015-09-12 13:55:07 UTC; 6min ago >> Main PID: 9796 (rpcbind) >> Memory: 428.0K >> CGroup: /system.slice/rpcbind.service >> ??9796 /sbin/rpcbind >> >> Sep 12 13:55:07 PSC01SERV008 systemd[1]: Starting RPC bind portmap >> service... >> Sep 12 13:55:07 PSC01SERV008 systemd[1]: Started RPC bind portmap service. >> root at PSC01SERV008:/var/lib/glusterd/nfs# rpcinfo -p >> program vers proto port service >> 100000 4 tcp 111 portmapper >> 100000 3 tcp 111 portmapper >> 100000 2 tcp 111 portmapper >> 100000 4 udp 111 portmapper >> 100000 3 udp 111 portmapper >> 100000 2 udp 111 portmapper >> >> I've tried to reinstall it agaon and again - but there is no luck. >> >> What I have: >> cat /etc/lsb-release >> DISTRIB_ID=Ubuntu >> DISTRIB_RELEASE=15.04 >> DISTRIB_CODENAME=vivid >> DISTRIB_DESCRIPTION="Ubuntu 15.04" >> >> ii glusterfs-client 3.7.4-ubuntu1~vivid1 >> amd64 clustered file-system (client package) >> ii glusterfs-common 3.7.4-ubuntu1~vivid1 >> amd64 GlusterFS common libraries and translator modules >> ii glusterfs-server 3.7.4-ubuntu1~vivid1 >> amd64 clustered file-system (server package) >> >> What else can I check? How can I fix it, what is most important :) >> Thanks in advance! >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-users >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150913/b42db11b/attachment.html>