Xavier Hernandez
2015-Jan-07 14:22 UTC
[Gluster-users] Input/Output Error when deleting folder
Can you send me the result of the following command on all bricks ? getfattr -m. -e hex -d <brick path>/Rec218 On 01/07/2015 03:15 PM, RASTELLI Alessandro wrote:> Hi, > we just did a clean reboot of the client. > > The config is as follows (2 bricks on each node): > > [root at gluster01-mi ~]# gluster volume info > Volume Name: storage-recorder > Type: Disperse > Volume ID: 97587d68-3834-43cc-8b95-11996e013bf2 > Status: Started > Number of Bricks: 1 x (8 + 2) = 10 > Transport-type: tcp > Bricks: > Brick1: gluster01-mi:/brick1/recorder > Brick2: gluster01-mi:/brick2/recorder > Brick3: gluster02-mi:/brick1/recorder > Brick4: gluster02-mi:/brick2/recorder > Brick5: gluster03-mi:/brick1/recorder > Brick6: gluster03-mi:/brick2/recorder > Brick7: gluster04-mi:/brick1/recorder > Brick8: gluster04-mi:/brick2/recorder > Brick9: gluster05-mi:/brick1/recorder > Brick10: gluster05-mi:/brick2/recorder > > A. > > -----Original Message----- > From: Xavier Hernandez [mailto:xhernandez at datalab.es] > Sent: mercoled? 7 gennaio 2015 15:11 > To: RASTELLI Alessandro > Cc: gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI Gabriele; TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca > Subject: Re: [Gluster-users] Input/Output Error when deleting folder > > Did you stopped the client or unmounted the volume while the application was still writing to it ? > > And what is the volume configuration ? (gluster volume info) > > Xavi > > On 01/07/2015 02:21 PM, RASTELLI Alessandro wrote: >> Hi Xavier, >> >> nothing special happened to volumes and bricks, we just restarted the >> client that writes into the folder Rec218. >> >> Before rebooting, the folder was OK. >> >> Thank you >> >> Alessandro >> >> *From:*Xavier Hernandez [mailto:xhernandez at datalab.es] >> *Sent:* mercoled? 7 gennaio 2015 13:11 >> *To:* RASTELLI Alessandro >> *Cc:* gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI Gabriele; >> TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca >> *Subject:* Re: [Gluster-users] Input/Output Error when deleting folder >> >> Hi Alessandro, >> >> what is the volume configuration ? >> did happen something special to the volume, bricks or that directory ? >> did you directly removed or changed something in the bricks ? >> did you replaced some brick ? >> >> Xavi >> >> On 01/07/2015 12:09 PM, RASTELLI Alessandro wrote: >> >> Hi, >> >> we encounter this error when I try to delete a folder on a >> Gluster3.6.1 environment (dispersed volume): >> >> [2015-01-07 10:27:49.008999] W [fuse-bridge.c:483:fuse_entry_cbk] >> 0-glusterfs-fuse: 10890: LOOKUP() /Rec218 => -1 (Input/output >> error) >> >> [2015-01-07 10:27:49.009542] W [ec-common.c:164:ec_check_status] >> 0-storage-recorder-disperse-0: Operation failed on some subvolumes >> (up=3FF, mask=3FF, remaining=0, good=3CF, bad=30) >> >> [2015-01-07 10:27:50.326991] E >> [ec-heal.c:656:ec_heal_prepare_others] >> 0-storage-recorder-disperse-0: Don't know how to heal error 116 >> >> ls command gives the output: >> >> ?????????? ? ? ? ? ? Rec218 >> >> I can't do any operation on the folder (move, rename, delete) >> >> Can you please indicate how to fix? >> >> Thanks >> >> Alessandro >> >> >> >> >> _______________________________________________ >> >> Gluster-users mailing list >> >> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >> >> http://www.gluster.org/mailman/listinfo/gluster-users >>
RASTELLI Alessandro
2015-Jan-07 14:31 UTC
[Gluster-users] Input/Output Error when deleting folder
Hmmm... different results on different nodes! Below the output for each brick for each of the 5 nodes. [root at gluster01-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick1/recorder/Rec218 trusted.ec.version=0x000000000000693a trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff [root at gluster01-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick2/recorder/Rec218 trusted.ec.version=0x000000000000693a trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff [root at gluster02-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick1/recorder/Rec218 trusted.ec.version=0x000000000000693a trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff [root at gluster02-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick2/recorder/Rec218 trusted.ec.version=0x000000000000693a trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff [root at gluster03-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick1/recorder/Rec218 trusted.gfid=0xbda849fca556469ead84ed074f2c1bcd [root at gluster03-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick2/recorder/Rec218 trusted.gfid=0xbda849fca556469ead84ed074f2c1bcd [root at gluster04-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick1/recorder/Rec218 trusted.ec.version=0x0000000000006939 trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff [root at gluster04-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick2/recorder/Rec218 trusted.ec.version=0x0000000000006939 trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff [root at gluster05-mi ~]# getfattr -m. -e hex -d /brick1/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick1/recorder/Rec218 trusted.ec.version=0x000000000000693a trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff [root at gluster05-mi ~]# getfattr -m. -e hex -d /brick2/recorder/Rec218 getfattr: Removing leading '/' from absolute path names # file: brick2/recorder/Rec218 trusted.ec.version=0x000000000000693a trusted.gfid=0xa9d904af0d9e4018acb2881bd8b3c2e4 trusted.glusterfs.dht=0x000000010000000000000000ffffffff -----Original Message----- From: Xavier Hernandez [mailto:xhernandez at datalab.es] Sent: mercoled? 7 gennaio 2015 15:22 To: RASTELLI Alessandro Cc: gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI Gabriele; TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca Subject: Re: [Gluster-users] Input/Output Error when deleting folder Can you send me the result of the following command on all bricks ? getfattr -m. -e hex -d <brick path>/Rec218 On 01/07/2015 03:15 PM, RASTELLI Alessandro wrote:> Hi, > we just did a clean reboot of the client. > > The config is as follows (2 bricks on each node): > > [root at gluster01-mi ~]# gluster volume info Volume Name: > storage-recorder > Type: Disperse > Volume ID: 97587d68-3834-43cc-8b95-11996e013bf2 > Status: Started > Number of Bricks: 1 x (8 + 2) = 10 > Transport-type: tcp > Bricks: > Brick1: gluster01-mi:/brick1/recorder > Brick2: gluster01-mi:/brick2/recorder > Brick3: gluster02-mi:/brick1/recorder > Brick4: gluster02-mi:/brick2/recorder > Brick5: gluster03-mi:/brick1/recorder > Brick6: gluster03-mi:/brick2/recorder > Brick7: gluster04-mi:/brick1/recorder > Brick8: gluster04-mi:/brick2/recorder > Brick9: gluster05-mi:/brick1/recorder > Brick10: gluster05-mi:/brick2/recorder > > A. > > -----Original Message----- > From: Xavier Hernandez [mailto:xhernandez at datalab.es] > Sent: mercoled? 7 gennaio 2015 15:11 > To: RASTELLI Alessandro > Cc: gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI Gabriele; > TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca > Subject: Re: [Gluster-users] Input/Output Error when deleting folder > > Did you stopped the client or unmounted the volume while the application was still writing to it ? > > And what is the volume configuration ? (gluster volume info) > > Xavi > > On 01/07/2015 02:21 PM, RASTELLI Alessandro wrote: >> Hi Xavier, >> >> nothing special happened to volumes and bricks, we just restarted the >> client that writes into the folder Rec218. >> >> Before rebooting, the folder was OK. >> >> Thank you >> >> Alessandro >> >> *From:*Xavier Hernandez [mailto:xhernandez at datalab.es] >> *Sent:* mercoled? 7 gennaio 2015 13:11 >> *To:* RASTELLI Alessandro >> *Cc:* gluster-users at gluster.org; CAZZANIGA Stefano; UBERTINI >> Gabriele; TECHNOLOGY - Supporto Sistemi OTT e Cloud; ORLANDO Luca >> *Subject:* Re: [Gluster-users] Input/Output Error when deleting >> folder >> >> Hi Alessandro, >> >> what is the volume configuration ? >> did happen something special to the volume, bricks or that directory ? >> did you directly removed or changed something in the bricks ? >> did you replaced some brick ? >> >> Xavi >> >> On 01/07/2015 12:09 PM, RASTELLI Alessandro wrote: >> >> Hi, >> >> we encounter this error when I try to delete a folder on a >> Gluster3.6.1 environment (dispersed volume): >> >> [2015-01-07 10:27:49.008999] W [fuse-bridge.c:483:fuse_entry_cbk] >> 0-glusterfs-fuse: 10890: LOOKUP() /Rec218 => -1 (Input/output >> error) >> >> [2015-01-07 10:27:49.009542] W [ec-common.c:164:ec_check_status] >> 0-storage-recorder-disperse-0: Operation failed on some subvolumes >> (up=3FF, mask=3FF, remaining=0, good=3CF, bad=30) >> >> [2015-01-07 10:27:50.326991] E >> [ec-heal.c:656:ec_heal_prepare_others] >> 0-storage-recorder-disperse-0: Don't know how to heal error 116 >> >> ls command gives the output: >> >> ?????????? ? ? ? ? ? Rec218 >> >> I can't do any operation on the folder (move, rename, delete) >> >> Can you please indicate how to fix? >> >> Thanks >> >> Alessandro >> >> >> >> >> _______________________________________________ >> >> Gluster-users mailing list >> >> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >> >> http://www.gluster.org/mailman/listinfo/gluster-users >>