Hi,
I got these errors 3 times since I'm testing gluster with nfs-ganesha. The
clients are php apps and when this happen, clients got strange php session
error. Below, the first error only happen once but other errors happen every
time a clients try to create a new session file. To make php apps work again, I
had to restart the client. Do you have an idea of what's happening here ?
31/08/2017 17:02:55 : epoch 59a450e6 : GLUSTER-NODE3 :
ganesha.nfsd-2067[work-161] nfs4_State_Del :STATE :CRIT :hashtable get latch
failed: 2
31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 :
ganesha.nfsd-2067[work-31] create_nfs4_owner :NFS4 LOCK :CRIT :Related
{STATE_OPEN_OWNER_NFSV4 0x7f0274475840: clientid={0x7f036836b0f0
ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED Client={0x7f0278001940
name=(24:Linux NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1
refcount=466} owner=(24:0x6f70656e2069643a00000027000000000000a95056901336)
confirmed=0 seqid=0 refcount=2} doesn't match for {STATE_LOCK_OWNER_NFSV4
0x7f0328479b10: clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6
Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2
devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466}
owner=(20:0x6c6f636b2069643a000000270000000000000000) confirmed=0 seqid=0
related_owner={STATE_OPEN_OWNER_NFSV4 0x7f03204cb2d0: clientid={0x7f036836b0f0
ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED Client={0x7f0278001940
name=(24:Linux NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1
refcount=466} owner=(24:0x6f70656e2069643a00000027000000000000a90f9b20feca)
confirmed=0 seqid=0 refcount=3} refcount=5}
31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 :
ganesha.nfsd-2067[work-31] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed to create
new lock owner Lock: obj=0x7f036c3a8268, fileid=11757051714723246668, type=READ
, start=0x0, end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4
0x7f0274475840: clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6
Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2
devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466}
owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) confirmed=0
seqid=0 refcount=2}
31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 :
ganesha.nfsd-2067[work-71] create_nfs4_owner :NFS4 LOCK :CRIT :Related
{STATE_OPEN_OWNER_NFSV4 0x7f0274475840: clientid={0x7f036836b0f0
ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED Client={0x7f0278001940
name=(24:Linux NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1
refcount=466} owner=(24:0x6f70656e2069643a00000027000000000000a95056901336)
confirmed=0 seqid=0 refcount=2} doesn't match for {STATE_LOCK_OWNER_NFSV4
0x7f0328479b10: clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6
Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2
devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466}
owner=(20:0x6c6f636b2069643a000000270000000000000000) confirmed=0 seqid=0
related_owner={STATE_OPEN_OWNER_NFSV4 0x7f03204cb2d0: clientid={0x7f036836b0f0
ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED Client={0x7f0278001940
name=(24:Linux NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1
refcount=466} owner=(24:0x6f70656e2069643a00000027000000000000a90f9b20feca)
confirmed=0 seqid=0 refcount=3} refcount=5}
31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 :
ganesha.nfsd-2067[work-71] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed to create
new lock owner Lock: obj=0x7f036c3a8268, fileid=11757051714723246668,
type=WRITE, start=0x0, end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4
0x7f0274475840: clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6
Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2
devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466}
owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) confirmed=0
seqid=0 refcount=2}
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170901/1aacfa6f/attachment.html>
On 09/02/2017 02:09 AM, Renaud Fortier wrote:> Hi, > > I got these errors 3 times since I?m testing gluster with nfs-ganesha. > The clients are php apps and when this happen, clients got strange php > session error. Below, the first error only happen once but other errors > happen every time a clients try to create a new session file. To make > php apps work again, I had to restart the client. Do you have an idea of > what?s happening here ? > > 31/08/2017 17:02:55 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-161] nfs4_State_Del :STATE :CRIT :hashtable get > latch failed: 2 > > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-31] create_nfs4_owner :NFS4 LOCK :CRIT :Related > {STATE_OPEN_OWNER_NFSV4 0x7f0274475840: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} doesn't match for > {STATE_LOCK_OWNER_NFSV4 0x7f0328479b10: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(20:0x6c6f636b2069643a000000270000000000000000) confirmed=0 > seqid=0 related_owner={STATE_OPEN_OWNER_NFSV4 0x7f03204cb2d0: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 Counter=0x00000008} > CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) > refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a90f9b20feca) > confirmed=0 seqid=0 refcount=3} refcount=5}The open owners doesn't seem to be matching. Pkt trace may help in further debugging. Also we haven't tested v4.1 and v4.2 protocols extensively with glusterfs. Could you please check the behavior with v4.0 protocol mount. Thanks, Soumya> > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-31] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed > to create new lock owner Lock: obj=0x7f036c3a8268, > fileid=11757051714723246668, type=READ , start=0x0, > end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4 0x7f0274475840: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 Counter=0x00000008} > CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) > refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} > > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-71] create_nfs4_owner :NFS4 LOCK :CRIT :Related > {STATE_OPEN_OWNER_NFSV4 0x7f0274475840: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} doesn't match for > {STATE_LOCK_OWNER_NFSV4 0x7f0328479b10: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(20:0x6c6f636b2069643a000000270000000000000000) confirmed=0 > seqid=0 related_owner={STATE_OPEN_OWNER_NFSV4 0x7f03204cb2d0: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 Counter=0x00000008} > CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) > refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a90f9b20feca) > confirmed=0 seqid=0 refcount=3} refcount=5} > > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-71] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed > to create new lock owner Lock: obj=0x7f036c3a8268, > fileid=11757051714723246668, type=WRITE, start=0x0, > end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4 0x7f0274475840: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 Counter=0x00000008} > CONFIRMED Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) > refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >
Hi, This was my fstab line : gluster-node3v:/dev /data nfs4 noatime,_netdev,auto 0 0 I tought it was the way to mount nfs v4.0 but now, I don't think so ! So I changed it to : gluster-node3v:/dev /data nfs nfsvers=4,noatime,_netdev,auto 0 0 If I look at the ganesha (niv_debug) logs, now I see it is mount in nfs v4.0 instead of v4.2. I will first try with this and ask for help if it fails again. Thank you Renaud -----Message d'origine----- De?: Soumya Koduri [mailto:skoduri at redhat.com] Envoy??: 2 septembre 2017 03:10 ??: Renaud Fortier <Renaud.Fortier at fsaa.ulaval.ca>; gluster-users at gluster.org Cc?: nfs-ganesha-support at lists.sourceforge.net Objet?: Re: [Gluster-users] ganesha error ? On 09/02/2017 02:09 AM, Renaud Fortier wrote:> Hi, > > I got these errors 3 times since I'm testing gluster with nfs-ganesha. > The clients are php apps and when this happen, clients got strange php > session error. Below, the first error only happen once but other > errors happen every time a clients try to create a new session file. > To make php apps work again, I had to restart the client. Do you have > an idea of what's happening here ? > > 31/08/2017 17:02:55 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-161] nfs4_State_Del :STATE :CRIT :hashtable get > latch failed: 2 > > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-31] create_nfs4_owner :NFS4 LOCK :CRIT :Related > {STATE_OPEN_OWNER_NFSV4 0x7f0274475840: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} doesn't match for > {STATE_LOCK_OWNER_NFSV4 0x7f0328479b10: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(20:0x6c6f636b2069643a000000270000000000000000) confirmed=0 > seqid=0 related_owner={STATE_OPEN_OWNER_NFSV4 0x7f03204cb2d0: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 > Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux > NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a90f9b20feca) > confirmed=0 seqid=0 refcount=3} refcount=5}The open owners doesn't seem to be matching. Pkt trace may help in further debugging. Also we haven't tested v4.1 and v4.2 protocols extensively with glusterfs. Could you please check the behavior with v4.0 protocol mount. Thanks, Soumya> > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-31] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed > to create new lock owner Lock: obj=0x7f036c3a8268, > fileid=11757051714723246668, type=READ , start=0x0, > end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4 0x7f0274475840: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 > Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux > NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} > > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-71] create_nfs4_owner :NFS4 LOCK :CRIT :Related > {STATE_OPEN_OWNER_NFSV4 0x7f0274475840: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} doesn't match for > {STATE_LOCK_OWNER_NFSV4 0x7f0328479b10: clientid={0x7f036836b0f0 > ClientID={Epoch=0x59a450e6 Counter=0x00000008} CONFIRMED > Client={0x7f0278001940 name=(24:Linux NFSv4.2 devel-web3) refcount=4} > t_delta=0 reservations=1 refcount=466} > owner=(20:0x6c6f636b2069643a000000270000000000000000) confirmed=0 > seqid=0 related_owner={STATE_OPEN_OWNER_NFSV4 0x7f03204cb2d0: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 > Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux > NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a90f9b20feca) > confirmed=0 seqid=0 refcount=3} refcount=5} > > 31/08/2017 17:04:00 : epoch 59a450e6 : GLUSTER-NODE3 : > ganesha.nfsd-2067[work-71] nfs4_op_lock :NFS4 LOCK :EVENT :LOCK failed > to create new lock owner Lock: obj=0x7f036c3a8268, > fileid=11757051714723246668, type=WRITE, start=0x0, > end=0xffffffffffffffff, owner={STATE_OPEN_OWNER_NFSV4 0x7f0274475840: > clientid={0x7f036836b0f0 ClientID={Epoch=0x59a450e6 > Counter=0x00000008} CONFIRMED Client={0x7f0278001940 name=(24:Linux > NFSv4.2 devel-web3) refcount=4} t_delta=0 reservations=1 refcount=466} > owner=(24:0x6f70656e2069643a00000027000000000000a95056901336) > confirmed=0 seqid=0 refcount=2} > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >