search for: gfid

...nt-glus1-081017:/exp/b4/gv0 Brick11: tpc-cent-glus2-081017:/exp/b4/gv0 Brick12: tpc-arbiter1-100617:/exp/b4/gv0 (arbiter) Options Reconfigured: nfs.disable: on transport.address-family: inet [root at tpc-cent-glus1-081017 ~]# gluster volume heal gv0 info Brick tpc-cent-glus1-081017:/exp/b1/gv0 <gfid:108694db-c039-4b7c-bd3d-ad6a15d811a2> <gfid:6d5ade20-8996-4de2-95d5-20ef98004742> <gfid:bc6cdc3d-5c46-4597-a7eb-282b21e9bdd5> <gfid:3c2ff4d1-3662-4214-8f21-f8f47dbdbf06> <gfid:053e2fb1-bc89-476e-a529-90dffa39963c> <removed to save scrolling> Status: Connected Numb...

...fo output from all the bricks of that replica pair. getfattr -d -e hex -m . <file path on brick> Thanks & Regards Karthik On 16-Oct-2017 8:16 PM, "Matt Waymack" <mwaymack at nsgdv.com> wrote: Hi all, I have a volume where the output of volume heal info shows several gfid entries to be healed, but they?ve been there for weeks and have not healed. Any normal file that shows up on the heal info does get healed as expected, but these gfid entries do not. Is there any way to remove these orphaned entries from the volume so they are no longer stuck in the heal process?...

...; Brick12: tpc-arbiter1-100617:/exp/b4/gv0 (arbiter) > > Options Reconfigured: > > nfs.disable: on > > transport.address-family: inet > > > > [root at tpc-cent-glus1-081017 ~]# gluster volume heal gv0 info > > Brick tpc-cent-glus1-081017:/exp/b1/gv0 > > <gfid:108694db-c039-4b7c-bd3d-ad6a15d811a2> > > <gfid:6d5ade20-8996-4de2-95d5-20ef98004742> > > <gfid:bc6cdc3d-5c46-4597-a7eb-282b21e9bdd5> > > <gfid:3c2ff4d1-3662-4214-8f21-f8f47dbdbf06> > > <gfid:053e2fb1-bc89-476e-a529-90dffa39963c> > > > &gt...

...#39;/' from absolute path names # file: exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2 security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.afr.dirty=0x000000000000000000000000 trusted.afr.gv0-client-2=0x000000000000000100000000 trusted.gfid=0x108694dbc0394b7cbd3dad6a15d811a2 trusted.gfid2path.9a2f5ada22eb9c45=0x38633262623330322d323466332d346463622d393630322d3839356136396461363131662f435f564f4c2d623030312d693637342d63642d63772e6d6435 [root at tpc-cent-glus2-081017 ~]# getfattr -d -e hex -m . /exp/b1/gv0/.glusterfs/10/86/108694db-c039...

I've been following this particular thread as I have a similar issue (RAID6 array failed out with 3 dead drives at once while a 12 TB load was being copied into one mounted space - what a mess) I have >700K GFID entries that have no path data:Example:getfattr -d -e hex -m . .glusterfs/00/00/0000a5ef-5af7-401b-84b5-ff2a51c10421# file: .glusterfs/00/00/0000a5ef-5af7-401b-84b5- ff2a51c10421security.selinux=0x73797374656d5f753a6f626a6563745f723a756e 6c6162656c65645f743a733000trusted.bit- rot.version=0x02000000...

...the mount, so it will create the entry on the arbiter brick and then run the volume heal to do the healing. Follow these steps to resolve the issue: (first try this on one file and check whether it gets healed. If it gets healed then do this for all the remaining files) 1. Get the file path for the gfids you got from heal info output. find <brickpath> -samefile <brickpath/.glusterfs/<first two bits of gfid>/<next 2 bits of gfid>/<full gfid> 2. Do ls/stat on the file from mount. 3. Run volume heal. 4. Check the heal info output to see whether the file got healed. If...

...of the nodes going read only, thats now resolved and I have ran the following and had errors. gluster volume heal repository info split-brain This is showing the following and has around 1041 lines. 2013-06-20 06:34:05 /shareddocuments/Product Media/images/big/w/7355.jpg 2013-06-20 06:34:05 <gfid:929454d9-d3f2-44cb-a96a-5ffdc9ed538a> 2013-06-20 06:34:05 <gfid:4de362d1-8b91-4be2-8900-5b727e612912> 2013-06-20 06:34:05 <gfid:eb57c8a4-8f22-470d-bfa8-f6780d57207a> 2013-06-20 06:34:05 <gfid:55722221-66c3-4950-88cb-b8eeeb6951b8> 2013-06-20 06:34:05 <gfid:3597bbd5-29ec-4414-...

...ve done the trick, thanks! From: Karthik Subrahmanya [mailto:ksubrahm at redhat.com] Sent: Monday, October 23, 2017 1:52 AM To: Jim Kinney <jim.kinney at gmail.com>; Matt Waymack <mwaymack at nsgdv.com> Cc: gluster-users <Gluster-users at gluster.org> Subject: Re: [Gluster-users] gfid entries in volume heal info that do not heal Hi Jim & Matt, Can you also check for the link count in the stat output of those hardlink entries in the .glusterfs folder on the bricks. If the link count is 1 on all the bricks for those entries, then they are orphaned entries and you can delete t...

Hi Jim, Can you check whether the same hardlinks are present on both the bricks & both of them have the link count 2? If the link count is 2 then "find <brickpath> -samefile <brickpath/.glusterfs/<first two bits of gfid>/<next 2 bits of gfid>/<full gfid>" should give you the file path. Regards, Karthik On Tue, Oct 24, 2017 at 3:28 AM, Jim Kinney <jim.kinney at gmail.com> wrote: > I'm not so lucky. ALL of mine show 2 links and none have the attr data > that supplies the path t...

...3:18 AM, Jim Kinney <jim.kinney at gmail.com> wrote: > I've been following this particular thread as I have a similar issue > (RAID6 array failed out with 3 dead drives at once while a 12 TB load was > being copied into one mounted space - what a mess) > > I have >700K GFID entries that have no path data: > Example: > getfattr -d -e hex -m . .glusterfs/00/00/0000a5ef- > 5af7-401b-84b5-ff2a51c10421 > # file: .glusterfs/00/00/0000a5ef-5af7-401b-84b5-ff2a51c10421 > security.selinux=0x73797374656d5f753a6f626a6563 > 745f723a756e6c6162656c65645f743a733000...

I have 14,734 GFIDS that are different. All the different ones are only on the brick that was live during the outage and concurrent file copy- in. The brick that was down at that time has no GFIDs that are not also on the up brick. As the bricks are 10TB, the find is going to be a long running process. I'm runnin...

...ailto:ksubrahm at redhat.com] > > > Sent: Monday, October 23, 2017 1:52 AM > > To: Jim Kinney <jim.kinney at gmail.com>; Matt Waymack <mwaymack at nsgdv.c > om> > > Cc: gluster-users <Gluster-users at gluster.org> > > Subject: Re: [Gluster-users] gfid entries in volume heal info that do > not heal > > > > > Hi Jim & Matt, > > Can you also check for the link count in the stat output of those > hardlink entries in the .glusterfs folder on the bricks. > > If the link count is 1 on all the bricks for those...

Hi Ravi, thanks for your answer, sure there you go: # gluster volume heal applicatif info Brick ipvr7.xxx:/mnt/gluster-applicatif/brick <gfid:e3b5ef36-a635-4e0e-bd97-d204a1f8e7ed> <gfid:f8030467-b7a3-4744-a945-ff0b532e9401> <gfid:def47b0b-b77e-4f0e-a402-b83c0f2d354b> <gfid:46f76502-b1d5-43af-8c42-3d833e86eb44> <gfid:d27a71d2-6d53-413d-b88c-33edea202cc2> <gfid:7e7f02b2-3f2d-41ff-9cad-cd3b5a1e506a> Status:...

I guess you're right aboug gfid, I got that: [2017-07-07 07:35:15.197003] W [MSGID: 108008] [afr-self-heal-name.c:354:afr_selfheal_name_gfid_mismatch_check] 0-applicatif-replicate-0: GFID mismatch for <gfid:3fa785b5-4242-4816-a452-97da1a5e45c6>/snooper b9222041-72dd-43a3-b0ab-4169dbd9a87f on applicatif-client-1 and 60056f9...

What does the mount log say when you get the EIO error on snooper? Check if there is a gfid mismatch on snooper directory or the files under it for all 3 bricks. In any case the mount log or the glustershd.log of the 3 nodes for the gfids you listed below should give you some idea on why the files aren't healed. Thanks. On 07/07/2017 03:10 PM, Florian Leleu wrote: > > Hi Rav...

On 07/07/2017 03:39 PM, Florian Leleu wrote: > > I guess you're right aboug gfid, I got that: > > [2017-07-07 07:35:15.197003] W [MSGID: 108008] > [afr-self-heal-name.c:354:afr_selfheal_name_gfid_mismatch_check] > 0-applicatif-replicate-0: GFID mismatch for > <gfid:3fa785b5-4242-4816-a452-97da1a5e45c6>/snooper > b9222041-72dd-43a3-b0ab-4169dbd9a87f o...

...entries: 0 Brick ngluster-1.network.hoggins.fr:/export/brick/mailer Status: Connected Number of entries: 0 Brick ngluster-2.network.hoggins.fr:/export/brick/mailer Status: Connected Number of entries: 0 Brick ngluster-3.network.hoggins.fr:/export/brick/mailer <gfid:98642fd6-f8a4-4966-9c30-32fedbecfc1a> Status: Connected Number of entries: 1 Brick ngluster-1.network.hoggins.fr:/export/brick/rom Status: Connected Number of entries: 0 Brick ngluster-2.network.hoggins.fr:/export/brick/rom Status: Connected Number of entries: 0...

...rom a backup source, I'm thinking of rerunning rsync to overwrite files with same create/modify times on the mount to realign things (maybe?) I ran a giant ls/stat on the mount but nothing changed. Ran it again with no changes. gluster-health-report Loaded reports: glusterd-op-version, georep, gfid-mismatch-dht-report, glusterd-peer-disconnect, disk_usage, errors_in_logs, coredump, glusterd, glusterd_volume_version_cksum_errors, kernel_issues, errors_in_logs, ifconfig, nic-health, process_status [ OK] Disk used percentage path=/ percentage=7[ OK] Disk used percentage path=/var per...

...bc7204cd.meta found on cached subvol ovirt-350-zone1-replicate-5 [2018-04-03 02:07:57.967489] I [MSGID: 109070] [dht-common.c:2796:dht_lookup_linkfile_cbk] 0-ovirt-350-zone1-dht: Lookup of /.shard/927c6620-848b-4064-8c88-68a332b645c2.7 on ovirt-350-zone1-replicate-3 (following linkfile) failed ,gfid = 00000000-0000-0000-0000-000000000000 [No such file or directory] [2018-04-03 02:07:57.974815] I [MSGID: 109069] [dht-common.c:2095:dht_lookup_unlink_stale_linkto_cbk] 0-ovirt-350-zone1-dht: Returned with op_ret 0 and op_errno 0 for /.shard/927c6620-848b-4064-8c88-68a332b645c2.3 [2018-04-03 02...

...und on cached subvol ovirt-350-zone1-replicate-5 > [2018-04-03 02:07:57.967489] I [MSGID: 109070] > [dht-common.c:2796:dht_lookup_linkfile_cbk] 0-ovirt-350-zone1-dht: Lookup > of /.shard/927c6620-848b-4064-8c88-68a332b645c2.7 on > ovirt-350-zone1-replicate-3 (following linkfile) failed ,gfid = > 00000000-0000-0000-0000-000000000000 [No such file or directory] > [2018-04-03 02:07:57.974815] I [MSGID: 109069] > [dht-common.c:2095:dht_lookup_unlink_stale_linkto_cbk] > 0-ovirt-350-zone1-dht: Returned with op_ret 0 and op_errno 0 for > /.shard/927c6620-848b-4064-8c88-68a332b6...