Ashish Pandey
2016-Jul-21 05:27 UTC
[Gluster-users] bitrot error correction in gluster 3.8 distributed dispersed setup
Hi, "gluster v heal " command does not heal corrupt files detected by bitrot feature. You have modified a file from back end and corrupted it. I think to correct your corrupted files (in fact the part of file in case of disperse volume) you have to delete that part and run gluster v heal <volname> full to correct the file. Keeping Kotresh in cc for better input of bitrot. Ashish ----- Original Message ----- From: "itlinux_team" <itlinux at imppc.org> To: gluster-users at gluster.org Sent: Wednesday, July 20, 2016 7:43:56 PM Subject: [Gluster-users] bitrot error correction in gluster 3.8 distributed dispersed setup Hi all, I'm testing the bitrot feature on a distributed dispersed setup: 3 x (8 + 4). I simulated a bit-flip on the part of a couple of files in a brick, having bitrot feature enabled before. The output from ''gluster volume bitrot vol_1 scrub status'' shows that two files are corrupted as shown below: --- Volume name : vol_1 State of scrub: Active Scrub impact: normal Scrub frequency: daily Bitrot error log location: /var/log/glusterfs/bitd.log Scrubber error log location: /var/log/glusterfs/scrub.log ========================================================= Node: localhost Number of Scrubbed files: 564 Number of Skipped files: 0 Last completed scrub time: 2016-07-20 13:37:16 Duration of last scrub (D:M:H:M:S): 0:0:0:50 Error count: 0 ========================================================= Node: glusterserver1 Number of Scrubbed files: 562 Number of Skipped files: 0 Last completed scrub time: 2016-07-20 13:37:16 Duration of last scrub (D:M:H:M:S): 0:0:0:51 Error count: 2 Corrupted object's [GFID]: e3cbffbe-12bd-4e1c-8dcf-2ef87ac99990 2550475e-cf9e-4781-9a05-9e11c1dd6b7a ========================================================= Node: glusterserver3 Number of Scrubbed files: 564 Number of Skipped files: 0 Last completed scrub time: 2016-07-20 13:37:16 Duration of last scrub (D:M:H:M:S): 0:0:0:50 --- I have executed ''gluster volume heal vol_1 full'' but it does not seem to fix the files, however the file that the client reads is the correct one. How should I handle this kind of corruption within gluster? Thanks! I?aki. _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://www.gluster.org/mailman/listinfo/gluster-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160721/6b6f0a5f/attachment.html>
Kotresh Hiremath Ravishankar
2016-Jul-21 05:45 UTC
[Gluster-users] bitrot error correction in gluster 3.8 distributed dispersed setup
Hi, Please follow below steps to restore: 1. Determine the path of each corrupted object from gfid shown from bitrot status command # find /bricks/*/.glusterfs -name GFID /bricks/brick1/.glusterfs/path/to/GFID # find /bricks -samefile /bricks/brick1/.glusterfs/path/to/GFID /bricks/brick1/.glusterfs/path/to/GFID /bricks/brick1/path/to/corrupted_file 2. Delete the corrupted files including .glusterfs GFID link file Delete the corrupted files from the path output by the find command. 3. Heal the file If you have client self-heal enabled, the file is healed the next time that you access it. If you do not have client self-heal enabled, you must manually heal the volume with the following command. # gluster volume heal VOLNAME The next time that the bitrot scrubber runs, this GFID is no longer listed (unless it has become corrupted again). Thanks and Regards, Kotresh H R ----- Original Message -----> From: "Ashish Pandey" <aspandey at redhat.com> > To: "itlinux_team" <itlinux at imppc.org> > Cc: gluster-users at gluster.org, "Kotresh Hiremath Ravishankar" <khiremat at redhat.com> > Sent: Thursday, July 21, 2016 10:57:51 AM > Subject: Re: [Gluster-users] bitrot error correction in gluster 3.8 distributed dispersed setup > > > Hi, > > "gluster v heal " command does not heal corrupt files detected by bitrot > feature. > You have modified a file from back end and corrupted it. > > I think to correct your corrupted files (in fact the part of file in case of > disperse volume) you have to delete that part and run gluster v heal > <volname> full > to correct the file. > > Keeping Kotresh in cc for better input of bitrot. > > Ashish > > > ----- Original Message ----- > > From: "itlinux_team" <itlinux at imppc.org> > To: gluster-users at gluster.org > Sent: Wednesday, July 20, 2016 7:43:56 PM > Subject: [Gluster-users] bitrot error correction in gluster 3.8 distributed > dispersed setup > > Hi all, > > I'm testing the bitrot feature on a distributed dispersed setup: 3 x (8 > + 4). > I simulated a bit-flip on the part of a couple of files in a brick, > having bitrot feature enabled before. The output from ''gluster volume > bitrot vol_1 scrub status'' shows that two files are corrupted as shown > below: > > > --- > Volume name : vol_1 > > State of scrub: Active > > Scrub impact: normal > > Scrub frequency: daily > > Bitrot error log location: /var/log/glusterfs/bitd.log > > Scrubber error log location: /var/log/glusterfs/scrub.log > > > ========================================================> > Node: localhost > > Number of Scrubbed files: 564 > > Number of Skipped files: 0 > > Last completed scrub time: 2016-07-20 13:37:16 > > Duration of last scrub (D:M:H:M:S): 0:0:0:50 > > Error count: 0 > > > ========================================================> > Node: glusterserver1 > > Number of Scrubbed files: 562 > > Number of Skipped files: 0 > > Last completed scrub time: 2016-07-20 13:37:16 > > Duration of last scrub (D:M:H:M:S): 0:0:0:51 > > Error count: 2 > > Corrupted object's [GFID]: > > e3cbffbe-12bd-4e1c-8dcf-2ef87ac99990 > > 2550475e-cf9e-4781-9a05-9e11c1dd6b7a > > > ========================================================> > Node: glusterserver3 > > Number of Scrubbed files: 564 > > Number of Skipped files: 0 > > Last completed scrub time: 2016-07-20 13:37:16 > > Duration of last scrub (D:M:H:M:S): 0:0:0:50 > --- > > > I have executed ''gluster volume heal vol_1 full'' but it does not seem > to fix the files, however the file that the client reads is the correct > one. > > How should I handle this kind of corruption within gluster? > > Thanks! > I?aki. > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users > >