Dan Bretherton
2013-Feb-21 14:46 UTC
[Gluster-users] I/O error repaired only by owner or root access
Dear All- Several users are having a lot of trouble reading files belonging to other users. Here is an example. [sms05dab at jupiter ~]$ ls -l /users/gcs/WORK/ORCA1/ORCA1-R07-MEAN/Ctl ls: /users/gcs/WORK/ORCA1/ORCA1-R07-MEAN/Ctl: Input/output error The corresponding nfs.log messages are shown below. [2013-02-21 12:11:39.204659] W [nfs3.c:727:nfs3svc_getattr_stat_cbk] 0-nfs: fe2ba5b8: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN => -1 (Invalid argument) [2013-02-21 12:11:39.204778] W [nfs3-helpers.c:3389:nfs3_log_common_res] 0-nfs-nfsv3: XID: fe2ba5b8, GETATTR: NFS: 22(Invalid argument for operation), POSIX: 22(Invalid argument) [2013-02-21 12:11:39.215345] I [dht-common.c:954:dht_lookup_everywhere_cbk] 0-nemo2-dht: deleting stale linkfile /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN on nemo2-replicate-0 [2013-02-21 12:11:39.225674] W [client3_1-fops.c:592:client3_1_unlink_cbk] 0-nemo2-client-1: remote operation failed: Permission denied [2013-02-21 12:11:39.225786] W [client3_1-fops.c:592:client3_1_unlink_cbk] 0-nemo2-client-0: remote operation failed: Permission denied [2013-02-21 12:11:39.681029] W [client3_1-fops.c:258:client3_1_mknod_cbk] 0-nemo2-client-18: remote operation failed: Permission denied. Path: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN (1662aa0a-d43b-4c2e-9be9-407eb7a89e85) [2013-02-21 12:11:39.681400] W [client3_1-fops.c:258:client3_1_mknod_cbk] 0-nemo2-client-19: remote operation failed: Permission denied. Path: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN (1662aa0a-d43b-4c2e-9be9-407eb7a89e85) [2013-02-21 12:11:39.682268] W [nfs3.c:1627:nfs3svc_readlink_cbk] 0-nfs: 2ca5b8: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN => -1 (Invalid argument) [2013-02-21 12:11:39.682338] W [nfs3-helpers.c:3403:nfs3_log_readlink_res] 0-nfs-nfsv3: XID: 2ca5b8, READLINK: NFS: 22(Invalid argument for operation), POSIX: 22(Invalid argument), target: (null) I managed to access the same directory as the owner (or any user with write access including root) without any trouble, and after that access from my normal user account was fine as well. The permissions on the directory allowed read access by everyone, but the "Permission denied" messages in nfs.log indicate that some sort of operation is not being allowed when the directory is accessed by other users. I have seen this happen with files and directories, and with the GlusterFS native client and NFS. I presume this is a bug; I would be grateful if someone could confirm this. I would file a bug report, but the trouble is that I don't know how to reproduce the problem that causes the I/O error in the first place. It only happens with some files and directories, not all. Would a bug report without any way to reproduce the error be any use, and can anyone suggest a way to dig deeper (eg looking at xattrs) next time I come across an example? -Dan. -- Dan Bretherton ESSC Computer System Manager Department of Meteorology Harry Pitt Building, 3 Earley Gate University of Reading Reading, RG6 7BE (or RG6 6AL for postal service deliveries) UK Tel. +44 118 378 5205, Fax: +44 118 378 6413 -- ## Please sponsor me to run in VSO's 30km Race to the Eye ## ## http://www.justgiving.com/DanBretherton ##
Rajesh Amaravathi
2013-Feb-22 11:56 UTC
[Gluster-users] I/O error repaired only by owner or root access
Hi Dan, Could you please provide the following info> (1) the exact permissions of the file you are accessing and its parent directory, (2) the user from which 'ls -l' is issued, and (3) the owner of the file, and the parent directory. You could open a bug for it if it is seen several times. Regards, Rajesh Amaravathi, Software Engineer, GlusterFS RedHat Inc. ----- Original Message -----> From: "Dan Bretherton" <d.a.bretherton at reading.ac.uk> > To: "gluster-users" <gluster-users at gluster.org> > Sent: Thursday, February 21, 2013 8:16:40 PM > Subject: [Gluster-users] I/O error repaired only by owner or root access > > Dear All- > Several users are having a lot of trouble reading files belonging to > other users. Here is an example. > > [sms05dab at jupiter ~]$ ls -l /users/gcs/WORK/ORCA1/ORCA1-R07-MEAN/Ctl > ls: /users/gcs/WORK/ORCA1/ORCA1-R07-MEAN/Ctl: Input/output error > > The corresponding nfs.log messages are shown below. > > [2013-02-21 12:11:39.204659] W [nfs3.c:727:nfs3svc_getattr_stat_cbk] > 0-nfs: fe2ba5b8: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN => -1 > (Invalid argument) > [2013-02-21 12:11:39.204778] W > [nfs3-helpers.c:3389:nfs3_log_common_res] > 0-nfs-nfsv3: XID: fe2ba5b8, GETATTR: NFS: 22(Invalid argument for > operation), POSIX: 22(Invalid argument) > [2013-02-21 12:11:39.215345] I > [dht-common.c:954:dht_lookup_everywhere_cbk] 0-nemo2-dht: deleting > stale > linkfile /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN on > nemo2-replicate-0 > [2013-02-21 12:11:39.225674] W > [client3_1-fops.c:592:client3_1_unlink_cbk] 0-nemo2-client-1: remote > operation failed: Permission denied > [2013-02-21 12:11:39.225786] W > [client3_1-fops.c:592:client3_1_unlink_cbk] 0-nemo2-client-0: remote > operation failed: Permission denied > [2013-02-21 12:11:39.681029] W > [client3_1-fops.c:258:client3_1_mknod_cbk] 0-nemo2-client-18: remote > operation failed: Permission denied. Path: > /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN > (1662aa0a-d43b-4c2e-9be9-407eb7a89e85) > [2013-02-21 12:11:39.681400] W > [client3_1-fops.c:258:client3_1_mknod_cbk] 0-nemo2-client-19: remote > operation failed: Permission denied. Path: > /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN > (1662aa0a-d43b-4c2e-9be9-407eb7a89e85) > [2013-02-21 12:11:39.682268] W [nfs3.c:1627:nfs3svc_readlink_cbk] > 0-nfs: > 2ca5b8: /gorgon/users/gcs/WORK/ORCA1/ORCA1-R07-MEAN => -1 (Invalid > argument) > [2013-02-21 12:11:39.682338] W > [nfs3-helpers.c:3403:nfs3_log_readlink_res] 0-nfs-nfsv3: XID: 2ca5b8, > READLINK: NFS: 22(Invalid argument for operation), POSIX: 22(Invalid > argument), target: (null) > > I managed to access the same directory as the owner (or any user with > write access including root) without any trouble, and after that > access > from my normal user account was fine as well. The permissions on the > directory allowed read access by everyone, but the "Permission > denied" > messages in nfs.log indicate that some sort of operation is not being > allowed when the directory is accessed by other users. I have seen > this happen with files and directories, and with the GlusterFS native > client and NFS. > > I presume this is a bug; I would be grateful if someone could confirm > this. I would file a bug report, but the trouble is that I don't > know > how to reproduce the problem that causes the I/O error in the first > place. It only happens with some files and directories, not all. > Would > a bug report without any way to reproduce the error be any use, and > can > anyone suggest a way to dig deeper (eg looking at xattrs) next time I > come across an example? > > -Dan. > > -- > Dan Bretherton > ESSC Computer System Manager > Department of Meteorology > Harry Pitt Building, 3 Earley Gate > University of Reading > Reading, RG6 7BE (or RG6 6AL for postal service deliveries) > UK > Tel. +44 118 378 5205, Fax: +44 118 378 6413 > -- > ## Please sponsor me to run in VSO's 30km Race to the Eye ## > ## http://www.justgiving.com/DanBretherton ## > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users >