莊尚豪
2015-Jun-03 09:59 UTC
[Gluster-users] gluster-3.7 cannot start volume & ganesha feature cannot turn on problem
Sorry I post wrong command. I want to turn on ganesha feature like your command. And these are the output. [root at n1 ~]# gluster nfs-ganesha enable Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue? (y/n) y nfs-ganesha: failed: Commit failed on localhost. Please check the log file for more details. And these are the detail log about turn on ganesha feature [2015-06-03 09:56:19.951124] W [common-utils.c:1676:gf_string2boolean] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7f9937dc0b77] (--> /lib64/libglusterfs.so.0(gf_string2boolean+0x13f)[0x7f9937dd8ccf] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_set_ganesha+0xaa)[0x7f992cc0a32a] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_validate+0x9b)[0x7f992cbbf76b] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(gd_stage_op_phase+0x1bd)[0x7f992cc369ed] ))))) 0-management: argument invalid [2015-06-03 09:56:19.951312] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped [2015-06-03 09:56:19.952278] I [glusterd-ganesha.c:364:check_host_list] 0-management: couldn't open the file /etc/ganesha/ganesha-ha.conf [2015-06-03 09:56:19.957777] E [glusterd-ganesha.c:670:pre_setup] 0-THIS->name: mkdir() failed on path /var/run/gluster/shared_storage/nfs-ganesha,errno: ????????? [2015-06-03 09:56:19.957874] E [glusterd-ganesha.c:254:glusterd_op_set_ganesha] 0-management: Initial NFS-Ganesha set up failed [2015-06-03 09:56:19.957899] E [glusterd-syncop.c:1396:gd_commit_op_phase] 0-management: Commit of operation 'Volume (null)' failed on localhost By the way, does glusterfs HA for ganesha support pnfs? Many thanks, Ben -----Original Message----- From: Soumya Koduri [mailto:skoduri at redhat.com] Sent: Wednesday, June 03, 2015 3:35 PM To: ???; 'Anoop C S'; gluster-users at gluster.org Cc: 'Meghana Madhusudhan' Subject: Re: [Gluster-users] gluster-3.7 cannot start volume & ganesha feature cannot turn on problem On 06/03/2015 12:01 PM, ??? wrote:> Thank for your reply. > > I have already upgrade glusterfs 3.7.1 and I can start the volume! > Therefore I am going to turn on ganesha feature by glusterfs. > > [root at n1 yum.repos.d]# gluster volume set ganesha ganesha.enable on > volume set: failed: The option nfs-ganesha should be enabled before setting ganesha.enable.This error means that 'nfs-ganesha' feature is not enabled. It needs to be enabled prior to exporting any volumes via ganesha. cmd to be used - 'gluster nfs-ganesha enable'. In case if you already have it enabled, check if it was successful using below commands - 'showmount -e localhost' --> should display '/' 'pcs status' --> Should display all the nodes' (which you have configured for HA ) pcs status with out any errors reported. Thanks, Soumya> > There is the log in "/var/log/glusterfs/etc-glusterfs-glusterd.vol.log" > > [2015-06-03 06:12:29.976469] W [common-utils.c:1676:gf_string2boolean] > (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7f9937dc0b77] > (--> /lib64/libglusterfs.so.0(gf_string2boolean+0x13f)[0x7f9937dd8ccf] > (--> > /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_s > et_ganesha+0xaa)[0x7f992cc0a32a] (--> > /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_v > alidate+0x9b)[0x7f992cbbf76b] (--> > /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(gd_stage_op_phase+0 > x1bd)[0x7f992cc369ed] ))))) 0-management: argument invalid > [2015-06-03 06:12:29.976823] I [MSGID: 106132] > [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already > stopped > [2015-06-03 06:12:29.976938] I > [glusterd-ganesha.c:364:check_host_list] 0-management: couldn't open > the file /etc/ganesha/ganesha-ha.conf > [2015-06-03 06:12:29.984693] E [glusterd-ganesha.c:670:pre_setup] > 0-THIS->name: mkdir() failed on path > /var/run/gluster/shared_storage/nfs-ganesha,errno: ????????? > [2015-06-03 06:12:29.984851] E > [glusterd-ganesha.c:254:glusterd_op_set_ganesha] 0-management: Initial > NFS-Ganesha set up failed > [2015-06-03 06:12:29.984883] E > [glusterd-syncop.c:1396:gd_commit_op_phase] 0-management: Commit of > operation 'Volume (null)' failed on localhost > > The Chinese log means that the path is empty or no directory. > It seems that glusterfs does not build ganesha configuration in initialization. > How do I continue next step? > > Thanks, > Ben > > > -----Original Message----- > From: Anoop C S [mailto:achiraya at redhat.com] > Sent: Tuesday, June 02, 2015 11:18 PM > To: benchuang at iii.org.tw; gluster-users at gluster.org > Cc: Meghana Madhusudhan; Soumya Koduri > Subject: Re: [Gluster-users] gluster-3.7 cannot start volume & ganesha > feature cannot turn on problem > > > Can you please attach the glusterd logs here? You are having trouble to even start the volume here right? > And also HA configuration is mandatory to use NFS-Ganesha in this release. Once you have the volume started, I can help you with the remaining steps in detail. > > Thanks > Meghana > > ----- Original Message ----- > From: "Anoop C S" <achiraya at redhat.com> > To: gluster-users at gluster.org > Cc: "Meghana Madhusudhan" <mmadhusu at redhat.com>, "Soumya Koduri" > <skoduri at redhat.com> > Sent: Tuesday, June 2, 2015 4:38:39 PM > Subject: Re: [Gluster-users] gluster-3.7 cannot start volume & ganesha > feature cannot turn on problem manadatroy > > > On 06/02/2015 01:42 PM, ??? wrote: >> Hi all, >> >> I have two question for glusterfs-3.7 on fedora-22 >> >> I used to have a glusterfs cluster version 3.6.2. >> >> The following configuration can be work in version-3.6.2, but not in >> version-3.7 >> >> >> >> There is 2 node for glusterfs. >> >> OS: fedora 22 >> >> Gluster: 3.7 on >> https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.0/ >> >> >> >> #gluster peer probe n1 >> >> #gluster volume create ganesha n1:/data/brick1/gv0 >> n2:/data/brick1/gv0 >> >> >> >> Volume Name: ganesha >> >> Type: Distribute >> >> Volume ID: cbb8d360-0025-419c-a12b-b29e4b91d7f8 >> >> Status: Created >> >> Number of Bricks: 2 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: n1:/data/brick1/gv0 >> >> Brick2: n2:/data/brick1/gv0 >> >> Options Reconfigured: >> >> performance.readdir-ahead: on >> >> >> >> The problem to start the volume "ganesha" >> >> #gluster volume start ganesha >> >> >> >> volume start: ganesha: failed: Commit failed on localhost. Please >> check the log file for more details. >> >> >> >> LOG in /var/log/glusterfs/bricks/data-brick1-gv0.log >> >> >> >> [2015-06-02 08:02:55.232923] I [MSGID: 100030] >> [glusterfsd.c:2294:main] 0-/usr/sbin/glusterfsd: Started running >> /usr/sbin/glusterfsd version 3.7.0 (args: /usr/sbin/glusterfsd -s >> n2 --volfile-id ganesha.n2.data-brick1-gv0 -p >> /var/lib/glusterd/vols/ganesha/run/n2-data-brick1-gv0.pid -S >> /var/run/gluster/73ea8a39514304f5ebd440321d784386.socket >> --brick-name /data/brick1/gv0 -l >> /var/log/glusterfs/bricks/data-brick1-gv0.log --xlator-option >> *-posix.glusterd-uuid=35547067-d343-4fee-802a-0e911b5a07cd >> --brick-port 49157 --xlator-option >> ganesha-server.listen-port=49157) >> >> [2015-06-02 08:02:55.284206] I >> [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started >> thread with index 1 >> >> [2015-06-02 08:02:55.397923] W [xlator.c:192:xlator_dynload] >> 0-xlator: /usr/lib64/glusterfs/3.7.0/xlator/features/changelog.so: >> undefined symbol: changelog_select_event >> > > This particular error for "undefined symbol changelog_select_event" > was identified recently and corresponding fix [ > http://review.gluster.org/#/c/11004/ ] is already in master and > hopefully will be available with v3.7.1. > >> [2015-06-02 08:02:55.397963] E [graph.y:212:volume_type] 0-parser: >> Volume 'ganesha-changelog', line 30: type 'features/changelog' is not >> valid or not found on this machine >> >> [2015-06-02 08:02:55.397992] E [graph.y:321:volume_end] 0-parser: >> "type" not specified for volume ganesha-changelog >> >> [2015-06-02 08:02:55.398214] E [MSGID: 100026] >> [glusterfsd.c:2149:glusterfs_process_volfp] 0-: failed to construct >> the graph >> >> [2015-06-02 08:02:55.398423] W >> [glusterfsd.c:1219:cleanup_and_exit] (--> 0-: received signum (0), >> shutting down >> >> >> >> I cannot google method to resolve it. >> >> Does anyone have across this problem? >> >> >> >> Another question is the feature in nfs-ganesha(version 2.2) >> >> The volume command I cannot turn on this feature. >> >> I try to copy the demo glusterfs-ganesha video but cannot work. >> >> Demo link: >> https://plus.google.com/events/c9omal6366f2cfkcd0iuee5ta1o >> >> >> >> [root at n1 brick1]# gluster nfs-ganesha enable >> >> Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the >> trusted pool. Do you still want to continue? (y/n) y >> >> nfs-ganesha: failed: Commit failed on localhost. Please check the log >> file for more details. >> >> > > Adding ganesha folks to the thread. > >> >> Does anyone have the detail configuration? >> >> THANKS for giving advice. >> >> >> >> Regards, >> >> Ben >> >> >> >> >> >> >> _______________________________________________ Gluster-users mailing >> list Gluster-users at gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-users >> > >
Soumya Koduri
2015-Jun-03 13:05 UTC
[Gluster-users] gluster-3.7 cannot start volume & ganesha feature cannot turn on problem
Hi Ben, Current NFS-Ganesha version (>V2.2.0) support pNFS with Gluster but not with HA. That means you need to disable HA to use pNFS. More details are in the doc - https://github.com/gluster/glusterfs/blob/master/doc/features/mount_gluster_volume_using_pnfs.md For HA, as mentioned in the demo video, there are certain pre-requisites to be followed. Noting down them below - 1. Populate /etc/hosts (or ensure DNS) on all machines. 2. Disable and stop NetworkManager service, enable and start network service on all machines 3. Create a gluster shared volume 'gluster_shared_storage' and mount it on '/var/run/gluster/shared_storage' on all the cluster nodes using glusterfs native mount. 4. Touch /etc/ganesha/ganesha.conf (empty file) 5.. `yum -y install pacemaker cman pcs ccs resource-agents corosync` on all machines 6. set cluster auth password on all machines - `echo hacluster | passwd --stdin hacluster` - pcs cluster auth on all the nodes 7. Enable IPv6 on all the cluster nodes. 8. Enable and start pcsd on all machines - `chkconfig --add pcsd; chkconfig pcsd on; service pcsd start` 9. Populate /etc/ganesha/ganesha-ha.conf on all the cluster nodes. Please make sure if you have followed all of them. From the logs, looks like step3 and step4 are missed. Thanks, Soumya On 06/03/2015 03:29 PM, ??? wrote:> Sorry I post wrong command. > I want to turn on ganesha feature like your command. > And these are the output. > > [root at n1 ~]# gluster nfs-ganesha enable > Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue? (y/n) y > nfs-ganesha: failed: Commit failed on localhost. Please check the log file for more details. > > And these are the detail log about turn on ganesha feature > > [2015-06-03 09:56:19.951124] W [common-utils.c:1676:gf_string2boolean] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7f9937dc0b77] (--> /lib64/libglusterfs.so.0(gf_string2boolean+0x13f)[0x7f9937dd8ccf] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_set_ganesha+0xaa)[0x7f992cc0a32a] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_validate+0x9b)[0x7f992cbbf76b] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(gd_stage_op_phase+0x1bd)[0x7f992cc369ed] ))))) 0-management: argument invalid > [2015-06-03 09:56:19.951312] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped > [2015-06-03 09:56:19.952278] I [glusterd-ganesha.c:364:check_host_list] 0-management: couldn't open the file /etc/ganesha/ganesha-ha.conf > [2015-06-03 09:56:19.957777] E [glusterd-ganesha.c:670:pre_setup] 0-THIS->name: mkdir() failed on path /var/run/gluster/shared_storage/nfs-ganesha,errno: ????????? > [2015-06-03 09:56:19.957874] E [glusterd-ganesha.c:254:glusterd_op_set_ganesha] 0-management: Initial NFS-Ganesha set up failed > [2015-06-03 09:56:19.957899] E [glusterd-syncop.c:1396:gd_commit_op_phase] 0-management: Commit of operation 'Volume (null)' failed on localhost > > > > By the way, does glusterfs HA for ganesha support pnfs? > > Many thanks, > Ben > > -----Original Message----- > From: Soumya Koduri [mailto:skoduri at redhat.com] > Sent: Wednesday, June 03, 2015 3:35 PM > To: ???; 'Anoop C S'; gluster-users at gluster.org > Cc: 'Meghana Madhusudhan' > Subject: Re: [Gluster-users] gluster-3.7 cannot start volume & ganesha feature cannot turn on problem > > > > On 06/03/2015 12:01 PM, ??? wrote: >> Thank for your reply. >> >> I have already upgrade glusterfs 3.7.1 and I can start the volume! >> Therefore I am going to turn on ganesha feature by glusterfs. >> >> [root at n1 yum.repos.d]# gluster volume set ganesha ganesha.enable on >> volume set: failed: The option nfs-ganesha should be enabled before setting ganesha.enable. > > This error means that 'nfs-ganesha' feature is not enabled. It needs to be enabled prior to exporting any volumes via ganesha. > > cmd to be used - 'gluster nfs-ganesha enable'. > > In case if you already have it enabled, check if it was successful using below commands - > > 'showmount -e localhost' --> should display '/' > 'pcs status' --> Should display all the nodes' (which you have configured for HA ) pcs status with out any errors reported. > > Thanks, > Soumya > > >> >> There is the log in "/var/log/glusterfs/etc-glusterfs-glusterd.vol.log" >> >> [2015-06-03 06:12:29.976469] W [common-utils.c:1676:gf_string2boolean] >> (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7f9937dc0b77] >> (--> /lib64/libglusterfs.so.0(gf_string2boolean+0x13f)[0x7f9937dd8ccf] >> (--> >> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_s >> et_ganesha+0xaa)[0x7f992cc0a32a] (--> >> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_v >> alidate+0x9b)[0x7f992cbbf76b] (--> >> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(gd_stage_op_phase+0 >> x1bd)[0x7f992cc369ed] ))))) 0-management: argument invalid >> [2015-06-03 06:12:29.976823] I [MSGID: 106132] >> [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already >> stopped >> [2015-06-03 06:12:29.976938] I >> [glusterd-ganesha.c:364:check_host_list] 0-management: couldn't open >> the file /etc/ganesha/ganesha-ha.conf >> [2015-06-03 06:12:29.984693] E [glusterd-ganesha.c:670:pre_setup] >> 0-THIS->name: mkdir() failed on path >> /var/run/gluster/shared_storage/nfs-ganesha,errno: ????????? >> [2015-06-03 06:12:29.984851] E >> [glusterd-ganesha.c:254:glusterd_op_set_ganesha] 0-management: Initial >> NFS-Ganesha set up failed >> [2015-06-03 06:12:29.984883] E >> [glusterd-syncop.c:1396:gd_commit_op_phase] 0-management: Commit of >> operation 'Volume (null)' failed on localhost >> >> The Chinese log means that the path is empty or no directory. >> It seems that glusterfs does not build ganesha configuration in initialization. >> How do I continue next step? >> >> Thanks, >> Ben >> >> >> -----Original Message----- >> From: Anoop C S [mailto:achiraya at redhat.com] >> Sent: Tuesday, June 02, 2015 11:18 PM >> To: benchuang at iii.org.tw; gluster-users at gluster.org >> Cc: Meghana Madhusudhan; Soumya Koduri >> Subject: Re: [Gluster-users] gluster-3.7 cannot start volume & ganesha >> feature cannot turn on problem >> >> >> Can you please attach the glusterd logs here? You are having trouble to even start the volume here right? >> And also HA configuration is mandatory to use NFS-Ganesha in this release. Once you have the volume started, I can help you with the remaining steps in detail. >> >> Thanks >> Meghana >> >> ----- Original Message ----- >> From: "Anoop C S" <achiraya at redhat.com> >> To: gluster-users at gluster.org >> Cc: "Meghana Madhusudhan" <mmadhusu at redhat.com>, "Soumya Koduri" >> <skoduri at redhat.com> >> Sent: Tuesday, June 2, 2015 4:38:39 PM >> Subject: Re: [Gluster-users] gluster-3.7 cannot start volume & ganesha >> feature cannot turn on problem manadatroy >> >> >> On 06/02/2015 01:42 PM, ??? wrote: >>> Hi all, >>> >>> I have two question for glusterfs-3.7 on fedora-22 >>> >>> I used to have a glusterfs cluster version 3.6.2. >>> >>> The following configuration can be work in version-3.6.2, but not in >>> version-3.7 >>> >>> >>> >>> There is 2 node for glusterfs. >>> >>> OS: fedora 22 >>> >>> Gluster: 3.7 on >>> https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.0/ >>> >>> >>> >>> #gluster peer probe n1 >>> >>> #gluster volume create ganesha n1:/data/brick1/gv0 >>> n2:/data/brick1/gv0 >>> >>> >>> >>> Volume Name: ganesha >>> >>> Type: Distribute >>> >>> Volume ID: cbb8d360-0025-419c-a12b-b29e4b91d7f8 >>> >>> Status: Created >>> >>> Number of Bricks: 2 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: n1:/data/brick1/gv0 >>> >>> Brick2: n2:/data/brick1/gv0 >>> >>> Options Reconfigured: >>> >>> performance.readdir-ahead: on >>> >>> >>> >>> The problem to start the volume "ganesha" >>> >>> #gluster volume start ganesha >>> >>> >>> >>> volume start: ganesha: failed: Commit failed on localhost. Please >>> check the log file for more details. >>> >>> >>> >>> LOG in /var/log/glusterfs/bricks/data-brick1-gv0.log >>> >>> >>> >>> [2015-06-02 08:02:55.232923] I [MSGID: 100030] >>> [glusterfsd.c:2294:main] 0-/usr/sbin/glusterfsd: Started running >>> /usr/sbin/glusterfsd version 3.7.0 (args: /usr/sbin/glusterfsd -s >>> n2 --volfile-id ganesha.n2.data-brick1-gv0 -p >>> /var/lib/glusterd/vols/ganesha/run/n2-data-brick1-gv0.pid -S >>> /var/run/gluster/73ea8a39514304f5ebd440321d784386.socket >>> --brick-name /data/brick1/gv0 -l >>> /var/log/glusterfs/bricks/data-brick1-gv0.log --xlator-option >>> *-posix.glusterd-uuid=35547067-d343-4fee-802a-0e911b5a07cd >>> --brick-port 49157 --xlator-option >>> ganesha-server.listen-port=49157) >>> >>> [2015-06-02 08:02:55.284206] I >>> [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started >>> thread with index 1 >>> >>> [2015-06-02 08:02:55.397923] W [xlator.c:192:xlator_dynload] >>> 0-xlator: /usr/lib64/glusterfs/3.7.0/xlator/features/changelog.so: >>> undefined symbol: changelog_select_event >>> >> >> This particular error for "undefined symbol changelog_select_event" >> was identified recently and corresponding fix [ >> http://review.gluster.org/#/c/11004/ ] is already in master and >> hopefully will be available with v3.7.1. >> >>> [2015-06-02 08:02:55.397963] E [graph.y:212:volume_type] 0-parser: >>> Volume 'ganesha-changelog', line 30: type 'features/changelog' is not >>> valid or not found on this machine >>> >>> [2015-06-02 08:02:55.397992] E [graph.y:321:volume_end] 0-parser: >>> "type" not specified for volume ganesha-changelog >>> >>> [2015-06-02 08:02:55.398214] E [MSGID: 100026] >>> [glusterfsd.c:2149:glusterfs_process_volfp] 0-: failed to construct >>> the graph >>> >>> [2015-06-02 08:02:55.398423] W >>> [glusterfsd.c:1219:cleanup_and_exit] (--> 0-: received signum (0), >>> shutting down >>> >>> >>> >>> I cannot google method to resolve it. >>> >>> Does anyone have across this problem? >>> >>> >>> >>> Another question is the feature in nfs-ganesha(version 2.2) >>> >>> The volume command I cannot turn on this feature. >>> >>> I try to copy the demo glusterfs-ganesha video but cannot work. >>> >>> Demo link: >>> https://plus.google.com/events/c9omal6366f2cfkcd0iuee5ta1o >>> >>> >>> >>> [root at n1 brick1]# gluster nfs-ganesha enable >>> >>> Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the >>> trusted pool. Do you still want to continue? (y/n) y >>> >>> nfs-ganesha: failed: Commit failed on localhost. Please check the log >>> file for more details. >>> >>> >> >> Adding ganesha folks to the thread. >> >>> >>> Does anyone have the detail configuration? >>> >>> THANKS for giving advice. >>> >>> >>> >>> Regards, >>> >>> Ben >>> >>> >>> >>> >>> >>> >>> _______________________________________________ Gluster-users mailing >>> list Gluster-users at gluster.org >>> http://www.gluster.org/mailman/listinfo/gluster-users >>> >> >> >