On 22/05/19 12:39 PM, Hu Bert wrote:> Hi @ll,
>
> today i updated and rebooted the 3 servers of my replicate 3 setup;
> after the 3rd one came up again i noticed this error:
>
> [2019-05-22 06:41:26.781165] E [MSGID: 108008]
> [afr-self-heal-common.c:392:afr_gfid_split_brain_source]
> 0-workdata-replicate-0: Gfid mismatch detected for
> <gfid:751233b0-7789-4550-bd95-4dd9c8f57c19>/120710351>,
> 82025ab3-8034-4257-9628-d8ebde909629 on workdata-client-2 and
> eaf2f31e-b4a7-4fa8-b710-d6ff9cd4eace on workdata-client-1.
120710351 seems to be the entry that is in split-brain. Is
/staticmap/120/710/120710351 the complete path to that entry? (check if
gfid:751233b0-7789-4550-bd95-4dd9c8f57c19 corresponds to the gfid of 710).
You can then try "gluster volume heal workdata split-brain source-brick
gluster1:/gluster/md4/workdata /staticmap/120/710/120710351"
-Ravi
> [2019-05-22 06:41:27.069969] W [MSGID: 108027]
> [afr-common.c:2270:afr_attempt_readsubvol_set] 0-workdata-replicate-0:
> no read subvols for /staticmap/120/710/120710351
> [2019-05-22 06:41:27.808532] W [fuse-bridge.c:582:fuse_entry_cbk]
> 0-glusterfs-fuse: 1834335: LOOKUP() /staticmap/120/710/120710351 => -1
> (Transport endpoint is not connected)
>
> A simple 'gluster volume heal workdata' didn't help;
'gluster volume
> heal workdata info' says:
>
> Brick gluster1:/gluster/md4/workdata
> /staticmap/120/710
> /staticmap/120/710/120710351
> <gfid:fe7fdbe8-9a39-4793-8d38-6dfdd3d5089b>
> Status: Connected
> Number of entries: 3
>
> Brick gluster2:/gluster/md4/workdata
> /staticmap/120/710
> /staticmap/120/710/120710351
> <gfid:fe7fdbe8-9a39-4793-8d38-6dfdd3d5089b>
> Status: Connected
> Number of entries: 3
>
> Brick gluster3:/gluster/md4/workdata
> /staticmap/120/710/120710351
> Status: Connected
> Number of entries: 1
>
> There's a mismatch in one directory; I tried to follow these
instructions:
> https://docs.gluster.org/en/latest/Troubleshooting/resolving-splitbrain/
>
> gluster volume heal workdata split-brain source-brick
> gluster1:/gluster/md4/workdata
> gfid:fe7fdbe8-9a39-4793-8d38-6dfdd3d5089b
> Healing gfid:fe7fdbe8-9a39-4793-8d38-6dfdd3d5089b failed: File not in
> split-brain.
> Volume heal failed.
>
> Is there any other documentation for gfid mismatch and how to resolve this?
>
>
> Thx,
> Hubert
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users