ABHISHEK PALIWAL
2016-May-04 06:59 UTC
[Gluster-users] [Gluster-devel] Exporting Gluster Volume
i am changing the nfs.port using gluster volume set gv0 nfs.port 2049 but it automatically changing MOUNTD as well and nfs.disable is also off gluster volume get gv0 nfs.disable Option Value ------ ----- nfs.disable off On Wed, May 4, 2016 at 12:25 PM, Soumya Koduri <skoduri at redhat.com> wrote:> Not sure why mountd is registering on 2049 port number. Please try > resetting volume options (if its fine) using "gluster v reset <volname> > force" and enable only nfs and verify the behavior. > > "gluster v set <volname> nfs.disable off" > > Thanks, > Soumya > > On 05/04/2016 12:17 PM, ABHISHEK PALIWAL wrote: > >> still having the same problem not able to see NETFACL option in rpcinfo -p >> >> rpcinfo -p >> program vers proto port service >> 100000 4 tcp 111 portmapper >> 100000 3 tcp 111 portmapper >> 100000 2 tcp 111 portmapper >> 100000 4 udp 111 portmapper >> 100000 3 udp 111 portmapper >> 100000 2 udp 111 portmapper >> 100005 3 tcp 2049 mountd >> 100005 1 tcp 2049 mountd >> 100003 3 tcp 2049 nfs >> 100227 3 tcp 2049 >> >> >> >> >> On Wed, May 4, 2016 at 12:09 PM, Soumya Koduri <skoduri at redhat.com >> <mailto:skoduri at redhat.com>> wrote: >> >> >> >> On 05/04/2016 12:04 PM, ABHISHEK PALIWAL wrote: >> >> but gluster mentioned that Gluster NFS works on 38465-38469 that >> why I >> took these ports. >> >> >> yes. Gluster-NFS uses ports 38465-38469 to register the side-band >> protocols it needs like MOUNTD, NLM, NFSACL etc.. I am not sure, but >> maybe the wording in that documentation was misleading. >> >> Thanks, >> Soumya >> >> >> Regards, >> Abhishek >> >> On Wed, May 4, 2016 at 12:00 PM, Soumya Koduri >> <skoduri at redhat.com <mailto:skoduri at redhat.com> >> <mailto:skoduri at redhat.com <mailto:skoduri at redhat.com>>> wrote: >> >> > nfs.port >> > 38465 >> >> Is there any reason behind choosing this port for NFS >> protocol? This >> port number seems to be used for MOUNT (V3) protocol as >> well and >> that may have resulted in port registration failures. Could >> you >> please change nfs port to default (2049) or some other port >> (other >> than 38465-38469). Also please make sure kernel-nfs is >> stopeed. >> >> Thanks, >> Soumya >> >> >> On 05/04/2016 11:49 AM, ABHISHEK PALIWAL wrote: >> >> Hi Soumya, >> >> >> Please find attached nfs.log file. >> >> Regards, >> Abhishek >> >> On Wed, May 4, 2016 at 11:45 AM, ABHISHEK PALIWAL >> <abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>>>> wrote: >> >> HI Soumya, >> >> Thanks for reply. >> >> Yes, I am getting following error in >> /var/log/glusterfs/nfs.log file >> >> [2016-04-25 06:27:23.721851] E [MSGID: 112109] >> [nfs.c:1482:init] >> 0-nfs: Failed to initialize protocols >> >> Please suggest me how can I resolve it. >> >> Regards, >> Abhishek >> >> On Wed, May 4, 2016 at 11:33 AM, Soumya Koduri >> <skoduri at redhat.com <mailto:skoduri at redhat.com> >> <mailto:skoduri at redhat.com <mailto:skoduri at redhat.com>> >> <mailto:skoduri at redhat.com >> <mailto:skoduri at redhat.com> <mailto:skoduri at redhat.com >> <mailto:skoduri at redhat.com>>>> wrote: >> >> Hi Abhishek, >> >> Below 'rpcinfo' output doesn't list 'nfsacl' >> protocol. >> That must >> be the reason client is not able set ACLs. >> Could you please >> check the log file '/var/lib/glusterfs/nfs.log' >> if >> there are any >> errors logged with respect protocol >> registration failures. >> >> Thanks, >> Soumya >> >> On 05/04/2016 11:15 AM, ABHISHEK PALIWAL wrote: >> >> Hi Niels, >> >> Please reply it is really urgent. >> >> Regards, >> Abhishek >> >> On Tue, May 3, 2016 at 11:36 AM, ABHISHEK >> PALIWAL >> <abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>> >> <mailto:abhishpaliwal at gmail.com <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>>> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>>>>> wrote: >> >> Hi Niels, >> >> Do you require more logs... >> >> Regards, >> Abhishek >> >> On Mon, May 2, 2016 at 4:58 PM, >> ABHISHEK PALIWAL >> <abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>>> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>> >> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com> >> <mailto:abhishpaliwal at gmail.com >> <mailto:abhishpaliwal at gmail.com>>>>> wrote: >> >> Hi Niels, >> >> >> Here is the output of rpcinfo -p >> $NFS_SERVER >> >> root at 128:/# rpcinfo -p $NFS_SERVER >> program vers proto port >> service >> 100000 4 tcp 111 >> portmapper >> 100000 3 tcp 111 >> portmapper >> 100000 2 tcp 111 >> portmapper >> 100000 4 udp 111 >> portmapper >> 100000 3 udp 111 >> portmapper >> 100000 2 udp 111 >> portmapper >> 100005 3 tcp 38465 >> mountd >> 100005 1 tcp 38465 >> mountd >> 100003 3 tcp 38465 nfs >> 100227 3 tcp 38465 >> >> >> out of mount command >> >> #mount -vvv -t nfs -o acl,vers=3 >> 128.224.95.140:/gv0 /tmp/e >> mount: fstab path: "/etc/fstab" >> mount: mtab path: "/etc/mtab" >> mount: lock path: "/etc/mtab~" >> mount: temp path: "/etc/mtab.tmp" >> mount: UID: 0 >> mount: eUID: 0 >> mount: spec: "128.224.95.140: >> /gv0" >> mount: node: "/tmp/e" >> mount: types: "nfs" >> mount: opts: "acl,vers=3" >> mount: external mount: argv[0] >> "/sbin/mount.nfs" >> mount: external mount: argv[1] >> "128.224.95.140:/gv0" >> mount: external mount: argv[2] >> "/tmp/e" >> mount: external mount: argv[3] >> "-v" >> mount: external mount: argv[4] >> "-o" >> mount: external mount: argv[5] >> "rw,acl,vers=3" >> mount.nfs: timeout set for Mon >> May 2 >> 16:58:58 2016 >> mount.nfs: trying text-based >> options >> 'acl,vers=3,addr=128.224.95.140' >> mount.nfs: prog 100003, trying >> vers=3, prot=6 >> mount.nfs: trying 128.224.95.140 >> prog >> 100003 vers 3 >> prot TCP >> port 38465 >> mount.nfs: prog 100005, trying >> vers=3, prot=17 >> mount.nfs: portmap query >> retrying: RPC: >> Program not >> registered >> mount.nfs: prog 100005, trying >> vers=3, prot=6 >> mount.nfs: trying 128.224.95.140 >> prog >> 100005 vers 3 >> prot TCP >> port 38465 >> >> >> On Mon, May 2, 2016 at 4:36 PM, >> Niels de Vos >> <ndevos at redhat.com >> <mailto:ndevos at redhat.com> <mailto:ndevos at redhat.com >> <mailto:ndevos at redhat.com>> >> <mailto:ndevos at redhat.com <mailto:ndevos at redhat.com> >> <mailto:ndevos at redhat.com <mailto:ndevos at redhat.com>>> >> <mailto:ndevos at redhat.com >> <mailto:ndevos at redhat.com> >> <mailto:ndevos at redhat.com <mailto:ndevos at redhat.com>> >> >> <mailto:ndevos at redhat.com >> <mailto:ndevos at redhat.com> >> <mailto:ndevos at redhat.com >> <mailto:ndevos at redhat.com>>>>> wrote: >> >> On Mon, May 02, 2016 at >> 04:14:01PM +0530, >> ABHISHEK PALIWAL >> wrote: >> > HI Team, >> > >> > I am exporting gluster >> volume using >> GlusterNFS with ACL >> support but at NFS >> > client while running >> 'setfacl' command >> getting "setfacl: >> /tmp/e: Remote I/O >> > error" >> > >> > >> > Following is the NFS >> option status >> for Volume: >> > >> > nfs.enable-ino32 >> > no >> > nfs.mem-factor >> > 15 >> > nfs.export-dirs >> > on >> > nfs.export-volumes >> > on >> > nfs.addr-namelookup >> > off >> > nfs.dynamic-volumes >> > off >> > nfs.register-with-portmap >> > on >> > nfs.outstanding-rpc-limit >> > 16 >> > nfs.port >> > 38465 >> > nfs.rpc-auth-unix >> > on >> > nfs.rpc-auth-null >> > on >> > nfs.rpc-auth-allow >> > all >> > nfs.rpc-auth-reject >> > none >> > nfs.ports-insecure >> > off >> > nfs.trusted-sync >> > off >> > nfs.trusted-write >> > off >> > nfs.volume-access >> > read-write >> > nfs.export-dir >> > >> > nfs.disable >> > off >> > nfs.nlm >> > on >> > nfs.acl >> > on >> > nfs.mount-udp >> > off >> > nfs.mount-rmtab >> > /var/lib/glusterd/nfs/rmtab >> > nfs.rpc-statd >> > /sbin/rpc.statd >> > nfs.server-aux-gids >> > off >> > nfs.drc >> > off >> > nfs.drc-size >> > 0x20000 >> > nfs.read-size >> (1 * >> > 1048576ULL) >> > nfs.write-size >> (1 * >> > 1048576ULL) >> > nfs.readdir-size >> (1 * >> > 1048576ULL) >> > nfs.exports-auth-enable >> > (null) >> > >> nfs.auth-refresh-interval-sec >> > (null) >> > nfs.auth-cache-ttl-sec >> (null) >> > >> > Command to mount exported >> gluster >> volume on >> NFS client is >> > >> > mount -v -t nfs -o >> acl,vers=3 >> 128.224.95.140:/gv0 /tmp/e >> >> Could you post the output of >> mounting with >> 'mount -vvv ...'? >> In previous >> emails I've asked for the >> output of >> 'rpcinfo -p >> $NFS_SERVER', I do not >> think I've seen that yet. >> >> The port used for NFSv3 ACLs >> on the >> NFS-server >> should be >> listed in >> 'netstat -tulpen' and the PID >> of the >> process >> should be the >> one of the >> Gluster/NFS service. >> >> HTH, >> Niels >> >> >> > setfacl -m u:nobody:r /tmp/e >> > setfacl: /tmp/e: Remote I/O >> error >> > >> > -- >> > >> > >> > >> > >> > Regards >> > Abhishek Paliwal >> >> > >> _______________________________________________ >> > Gluster-devel mailing list >> > Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org> >> <mailto:Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org>> >> <mailto:Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org> >> <mailto:Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org>>> >> <mailto:Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org> >> <mailto:Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org>> >> <mailto:Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org> >> <mailto:Gluster-devel at gluster.org >> <mailto:Gluster-devel at gluster.org>>>> >> > >> http://www.gluster.org/mailman/listinfo/gluster-devel >> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >> <mailto:Gluster-users at gluster.org >> <mailto:Gluster-users at gluster.org>> >> <mailto:Gluster-users at gluster.org >> <mailto:Gluster-users at gluster.org> >> <mailto:Gluster-users at gluster.org >> <mailto:Gluster-users at gluster.org>>> >> http://www.gluster.org/mailman/listinfo/gluster-users >> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >-- Regards Abhishek Paliwal -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160504/8bb6f886/attachment.html>
Soumya Koduri
2016-May-04 07:58 UTC
[Gluster-users] [Gluster-devel] Exporting Gluster Volume
Even on my setup if I change nfs.port, all the other services also started registering on those ports. Can you please file a bug for it. That seems like a bug (or is it intentional..Niels?). 100005 3 tcp 2049 mountd 100005 1 tcp 2049 mountd 100003 3 tcp 2049 nfs 100021 4 tcp 2049 nlockmgr 100227 3 tcp 2049 nfs_acl However, as you can see NFSACL did come up. Try restarting volume " gluster v start <volname> force" and check your logs again. Thanks, Soumya On 05/04/2016 12:29 PM, ABHISHEK PALIWAL wrote:> i am changing the nfs.port using gluster volume set gv0 nfs.port 2049 > but it automatically changing MOUNTD as well > > and nfs.disable is also off > > gluster volume get gv0 nfs.disable > Option Value > ------ ----- > nfs.disable off > > >