mabi
2017-Jul-29 11:06 UTC
[Gluster-users] Possible stale .glusterfs/indices/xattrop file?
Hi, Sorry for mailing again but as mentioned in my previous mail, I have added an arbiter node to my replica 2 volume and it seem to have gone fine except for the fact that there is one single file which needs healing and does not get healed as you can see here from the output of a "heal info": Brick node1.domain.tld:/data/myvolume/brick Status: Connected Number of entries: 0 Brick node2.domain.tld:/data/myvolume/brick <gfid:29e0d13e-1217-41cc-9bda-1fbbf781c397> Status: Connected Number of entries: 1 Brick arbiternode.domain.tld:/srv/glusterfs/myvolume/brick Status: Connected Number of entries: 0 On my node2 the respective .glusterfs/indices/xattrop directory contains two files as you can see below: ls -lai /data/myvolume/brick/.glusterfs/indices/xattrop total 76180 10 drw------- 2 root root 4 Jul 29 12:15 . 9 drw------- 5 root root 5 Apr 28 22:15 .. 2798404 ---------- 2 root root 0 Apr 28 22:51 29e0d13e-1217-41cc-9bda-1fbbf781c397 2798404 ---------- 2 root root 0 Apr 28 22:51 xattrop-6fa49ad5-71dd-4ec2-9246-7b302ab92d38 I tried to find the real file on my brick where this xattrop file points to using its inode number (command: find /data/myvolume/brick/data -inum 8394642) but it does not find any associated file. So my question here is, is it possible that this is a stale file which just forgot to get deleted from the indices/xattrop file by gluster for some unknown reason? If yes is it safe for me to delete these two files? or what would be the correct process in that case? Thank you for your input. Mabi -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170729/82334667/attachment.html>
Ravishankar N
2017-Jul-30 01:31 UTC
[Gluster-users] Possible stale .glusterfs/indices/xattrop file?
On 07/29/2017 04:36 PM, mabi wrote:> Hi, > > Sorry for mailing again but as mentioned in my previous mail, I have > added an arbiter node to my replica 2 volume and it seem to have gone > fine except for the fact that there is one single file which needs > healing and does not get healed as you can see here from the output of > a "heal info": > > Brick node1.domain.tld:/data/myvolume/brick > Status: Connected > Number of entries: 0 > > Brick node2.domain.tld:/data/myvolume/brick > <gfid:29e0d13e-1217-41cc-9bda-1fbbf781c397> > Status: Connected > Number of entries: 1 > > Brick arbiternode.domain.tld:/srv/glusterfs/myvolume/brick > Status: Connected > Number of entries: 0 > > On my node2 the respective .glusterfs/indices/xattrop directory > contains two files as you can see below: > > ls -lai /data/myvolume/brick/.glusterfs/indices/xattrop > total 76180 > 10 drw------- 2 root root 4 Jul 29 12:15 . > 9 drw------- 5 root root 5 Apr 28 22:15 .. > 2798404 ---------- 2 root root 0 Apr 28 22:51 > 29e0d13e-1217-41cc-9bda-1fbbf781c397 > 2798404 ---------- 2 root root 0 Apr 28 22:51 > xattrop-6fa49ad5-71dd-4ec2-9246-7b302ab92d38 > > > > I tried to find the real file on my brick where this xattrop file > points to using its inode number (command: find > /data/myvolume/brick/data -inum 8394642) but it does not find any > associated file. > > So my question here is, is it possible that this is a stale file which > just forgot to get deleted from the indices/xattrop file by gluster > for some unknown reason? If yes is it safe for me to delete these two > files? or what would be the correct process in that case?The 'xattrop-6fa...' is the base entry. gfids of files that need heal are hard linked to this entry, so nothing needs to be done for it. But you need to find out why '29e0d13...' is not healing. Launch the heal and observe the glustershd logs for errors. I suppose the inode number for .glusterfs/29/e0/29e0d13e-1217-41cc-9bda-1fbbf781c397 is what is 8394642. Is .glusterfs/29/e0/29e0d13e-1217-41cc-9bda-1fbbf781c397 a regular file or symlink? Does it exist in the other 2 bricks? What is the link count (as seen from stat <file>)? -Ravi> > Thank you for your input. > Mabi > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170730/25ed72f0/attachment.html>
mabi
2017-Jul-30 08:54 UTC
[Gluster-users] Possible stale .glusterfs/indices/xattrop file?
Hi Ravi, Thanks for your hints. Below you will find the answer to your questions. First I tried to start the healing process by running: gluster volume heal myvolume and then as you suggested watch the output of the glustershd.log file but nothing appeared in that log file after running the above command. I checked the files which need to be healing using the "heal <volume> info" command and it still shows that very same GFID on node2 to be healed. So nothing changed here. The file /data/myvolume/brick/.glusterfs/indices/xattrop/29e0d13e-1217-41cc-9bda-1fbbf781c397 is only on node2 and not on my nod1 nor on my arbiternode. This file seems to be a regular file and not a symlink. Here is the output of the stat command on it from my node2: File: ?/data/myvolume/brick/.glusterfs/indices/xattrop/29e0d13e-1217-41cc-9bda-1fbbf781c397? Size: 0 Blocks: 1 IO Block: 512 regular empty file Device: 25h/37d Inode: 2798404 Links: 2 Access: (0000/----------) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2017-04-28 22:51:15.215775269 +0200 Modify: 2017-04-28 22:51:15.215775269 +0200 Change: 2017-07-30 08:39:03.700872312 +0200 Birth: - I hope this is enough info for a starter, else let me know if you need any more info. I would be glad to resolve this weird file which needs to be healed but can not. Best regards, Mabi> -------- Original Message -------- > Subject: Re: [Gluster-users] Possible stale .glusterfs/indices/xattrop file? > Local Time: July 30, 2017 3:31 AM > UTC Time: July 30, 2017 1:31 AM > From: ravishankar at redhat.com > To: mabi <mabi at protonmail.ch>, Gluster Users <gluster-users at gluster.org> > > On 07/29/2017 04:36 PM, mabi wrote: > >> Hi, >> Sorry for mailing again but as mentioned in my previous mail, I have added an arbiter node to my replica 2 volume and it seem to have gone fine except for the fact that there is one single file which needs healing and does not get healed as you can see here from the output of a "heal info": >> Brick node1.domain.tld:/data/myvolume/brick >> Status: Connected >> Number of entries: 0 >> Brick node2.domain.tld:/data/myvolume/brick >> <gfid:29e0d13e-1217-41cc-9bda-1fbbf781c397> >> Status: Connected >> Number of entries: 1 >> Brick arbiternode.domain.tld:/srv/glusterfs/myvolume/brick >> Status: Connected >> Number of entries: 0 >> On my node2 the respective .glusterfs/indices/xattrop directory contains two files as you can see below: >> ls -lai /data/myvolume/brick/.glusterfs/indices/xattrop >> total 76180 >> 10 drw------- 2 root root 4 Jul 29 12:15 . >> 9 drw------- 5 root root 5 Apr 28 22:15 .. >> 2798404 ---------- 2 root root 0 Apr 28 22:51 29e0d13e-1217-41cc-9bda-1fbbf781c397 >> 2798404 ---------- 2 root root 0 Apr 28 22:51 xattrop-6fa49ad5-71dd-4ec2-9246-7b302ab92d38 >> I tried to find the real file on my brick where this xattrop file points to using its inode number (command: find /data/myvolume/brick/data -inum 8394642) but it does not find any associated file. >> So my question here is, is it possible that this is a stale file which just forgot to get deleted from the indices/xattrop file by gluster for some unknown reason? If yes is it safe for me to delete these two files? or what would be the correct process in that case? > > The 'xattrop-6fa...' is the base entry. gfids of files that need heal are hard linked to this entry, so nothing needs to be done for it. But you need to find out why '29e0d13...' is not healing. Launch the heal and observe the glustershd logs for errors. I suppose the inode number for .glusterfs/29/e0/29e0d13e-1217-41cc-9bda-1fbbf781c397 is what is 8394642. Is .glusterfs/29/e0/29e0d13e-1217-41cc-9bda-1fbbf781c397 a regular file or symlink? Does it exist in the other 2 bricks? What is the link count (as seen from stat <file>)? > -Ravi > >> Thank you for your input. >> Mabi >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> >> http://lists.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170730/77e3b626/attachment.html>