Hi Kotresh, i have used below command to verify any open fd for file. "ls -l /proc/*/fd | grep filename". as soon as write completes there no open fd's, if there is any alternate option. please let me know will also try that. Also, below is my scrub status in my test setup. number of skipped files slow reducing day by day. I think files are skipped due to bitrot signature process is not completed yet. where can i see scrub skipped files? Volume name : glsvol1 State of scrub: Active (Idle) Scrub impact: normal Scrub frequency: daily Bitrot error log location: /var/log/glusterfs/bitd.log Scrubber error log location: /var/log/glusterfs/scrub.log ======================================================== Node: localhost Number of Scrubbed files: 1644 Number of Skipped files: 1001 Last completed scrub time: 2016-09-20 11:59:58 Duration of last scrub (D:M:H:M:S): 0:0:39:26 Error count: 0 ======================================================== Node: 10.1.2.3 Number of Scrubbed files: 1644 Number of Skipped files: 1001 Last completed scrub time: 2016-09-20 10:50:00 Duration of last scrub (D:M:H:M:S): 0:0:38:17 Error count: 0 ======================================================== Node: 10.1.2.4 Number of Scrubbed files: 981 Number of Skipped files: 1664 Last completed scrub time: 2016-09-20 12:38:01 Duration of last scrub (D:M:H:M:S): 0:0:35:19 Error count: 0 ======================================================== Node: 10.1.2.1 Number of Scrubbed files: 1263 Number of Skipped files: 1382 Last completed scrub time: 2016-09-20 11:57:21 Duration of last scrub (D:M:H:M:S): 0:0:37:17 Error count: 0 ======================================================== Node: 10.1.2.2 Number of Scrubbed files: 1644 Number of Skipped files: 1001 Last completed scrub time: 2016-09-20 11:59:25 Duration of last scrub (D:M:H:M:S): 0:0:39:18 Error count: 0 ======================================================== Thanks Amudhan On Wed, Sep 21, 2016 at 11:45 AM, Kotresh Hiremath Ravishankar < khiremat at redhat.com> wrote:> Hi Amudhan, > > I don't think it's the limitation with read data from the brick. > To limit the usage of CPU, throttling is done using token bucket > algorithm. The log message showed is related to it. But even then > I think it should not take 12 minutes for check-sum calculation unless > there is an fd open (might be internal). Could you please cross verify > if there are any fd opened on that file by looking into /proc? I will > also test it out in the mean time and get back to you. > > Thanks and Regards, > Kotresh H R > > ----- Original Message ----- > > From: "Amudhan P" <amudhan83 at gmail.com> > > To: "Kotresh Hiremath Ravishankar" <khiremat at redhat.com> > > Cc: "Gluster Users" <gluster-users at gluster.org> > > Sent: Tuesday, September 20, 2016 3:19:28 PM > > Subject: Re: [Gluster-users] 3.8.3 Bitrot signature process > > > > Hi Kotresh, > > > > Please correct me if i am wrong, Once a file write completes and as soon > as > > closes fds, bitrot waits for 120 seconds and starts hashing and update > > signature for the file in brick. > > > > But, what i am feeling that bitrot takes too much of time to complete > > hashing. > > > > below is test result i would like to share. > > > > > > writing data in below path using dd : > > > > /mnt/gluster/data/G (mount point) > > -rw-r--r-- 1 root root 10M Sep 20 12:19 test53-bs10M-c1.nul > > -rw-r--r-- 1 root root 100M Sep 20 12:19 test54-bs10M-c10.nul > > > > No any other write or read process is going on. > > > > > > Checking file data in one of the brick. > > > > -rw-r--r-- 2 root root 2.5M Sep 20 12:23 test53-bs10M-c1.nul > > -rw-r--r-- 2 root root 25M Sep 20 12:23 test54-bs10M-c10.nul > > > > file's stat and getfattr info from brick, after write process completed. > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test53-bs10M-c1.nul > > File: ?test53-bs10M-c1.nul? > > Size: 2621440 Blocks: 5120 IO Block: 4096 regular file > > Device: 821h/2081d Inode: 536874168 Links: 2 > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > Access: 2016-09-20 12:23:28.798886647 +0530 > > Modify: 2016-09-20 12:23:28.994886646 +0530 > > Change: 2016-09-20 12:23:28.998886646 +0530 > > Birth: - > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test54-bs10M-c10.nul > > File: ?test54-bs10M-c10.nul? > > Size: 26214400 Blocks: 51200 IO Block: 4096 regular file > > Device: 821h/2081d Inode: 536874169 Links: 2 > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > Access: 2016-09-20 12:23:42.902886624 +0530 > > Modify: 2016-09-20 12:23:44.378886622 +0530 > > Change: 2016-09-20 12:23:44.378886622 +0530 > > Birth: - > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > test53-bs10M-c1.nul > > # file: test53-bs10M-c1.nul > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > trusted.ec.config=0x0000080501000200 > > trusted.ec.size=0x0000000000a00000 > > trusted.ec.version=0x00000000000000500000000000000050 > > trusted.gfid=0xe2416bd1aae4403c88f44286273bbe99 > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > test54-bs10M-c10.nul > > # file: test54-bs10M-c10.nul > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > trusted.ec.config=0x0000080501000200 > > trusted.ec.size=0x0000000006400000 > > trusted.ec.version=0x00000000000003200000000000000320 > > trusted.gfid=0x54e018dd8c5a4bd79e0317729d8a57c5 > > > > > > > > file's stat and getfattr info from brick, after bitrot signature updated. > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test53-bs10M-c1.nul > > File: ?test53-bs10M-c1.nul? > > Size: 2621440 Blocks: 5120 IO Block: 4096 regular file > > Device: 821h/2081d Inode: 536874168 Links: 2 > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > Access: 2016-09-20 12:25:31.494886450 +0530 > > Modify: 2016-09-20 12:23:28.994886646 +0530 > > Change: 2016-09-20 12:27:00.994886307 +0530 > > Birth: - > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > test53-bs10M-c1.nul > > # file: test53-bs10M-c1.nul > > trusted.bit-rot.signature=0x0102000000000000006de7493c5c > 90f643357c268fbaaf461c1567e0334e4948023ce17268403aa37a > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > trusted.ec.config=0x0000080501000200 > > trusted.ec.size=0x0000000000a00000 > > trusted.ec.version=0x00000000000000500000000000000050 > > trusted.gfid=0xe2416bd1aae4403c88f44286273bbe99 > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test54-bs10M-c10.nul > > File: ?test54-bs10M-c10.nul? > > Size: 26214400 Blocks: 51200 IO Block: 4096 regular file > > Device: 821h/2081d Inode: 536874169 Links: 2 > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > Access: 2016-09-20 12:25:47.510886425 +0530 > > Modify: 2016-09-20 12:23:44.378886622 +0530 > > Change: 2016-09-20 12:38:05.954885243 +0530 > > Birth: - > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > test54-bs10M-c10.nul > > # file: test54-bs10M-c10.nul > > trusted.bit-rot.signature=0x010200000000000000394c345f0b > 0c63ee652627a62eed069244d35c4d5134e4f07d4eabb51afda47e > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > trusted.ec.config=0x0000080501000200 > > trusted.ec.size=0x0000000006400000 > > trusted.ec.version=0x00000000000003200000000000000320 > > trusted.gfid=0x54e018dd8c5a4bd79e0317729d8a57c5 > > > > > > (Actual time taken for reading file from brick for md5sum) > > > > gfstst-node5:/media/disk2/brick2/data/G$ time md5sum test53-bs10M-c1.nul > > 8354dcaa18a1ecb52d0895bf00888c44 test53-bs10M-c1.nul > > > > real 0m0.045s > > user 0m0.007s > > sys 0m0.003s > > > > gfstst-node5:/media/disk2/brick2/data/G$ time md5sum > test54-bs10M-c10.nul > > bed3c0a4a1407f584989b4009e9ce33f test54-bs10M-c10.nul > > > > real 0m0.166s > > user 0m0.062s > > sys 0m0.011s > > > > As you can see that 'test54-bs10M-c10.nul' file took around 12 minutes to > > update bitort signature (pls refer stat output for the file). > > > > what would be the cause for such a slow read?. Any limitation in read > data > > from brick? > > > > Also, i am seeing this line bitd.log, what does this mean? > > [bit-rot.c:1784:br_rate_limit_signer] 0-glsvol1-bit-rot-0: [Rate Limit > > Info] "tokens/sec (rate): 131072, maxlimit: 524288 > > > > > > Thanks > > Amudhan P > > > > > > > > On Mon, Sep 19, 2016 at 1:00 PM, Kotresh Hiremath Ravishankar < > > khiremat at redhat.com> wrote: > > > > > Hi Amudhan, > > > > > > Thanks for testing out the bitrot feature and sorry for the delayed > > > response. > > > Please find the answers inline. > > > > > > Thanks and Regards, > > > Kotresh H R > > > > > > ----- Original Message ----- > > > > From: "Amudhan P" <amudhan83 at gmail.com> > > > > To: "Gluster Users" <gluster-users at gluster.org> > > > > Sent: Friday, September 16, 2016 4:14:10 PM > > > > Subject: Re: [Gluster-users] 3.8.3 Bitrot signature process > > > > > > > > Hi, > > > > > > > > Can anyone reply to this mail. > > > > > > > > On Tue, Sep 13, 2016 at 12:49 PM, Amudhan P < amudhan83 at gmail.com > > > > wrote: > > > > > > > > > > > > > > > > Hi, > > > > > > > > I am testing bitrot feature in Gluster 3.8.3 with disperse EC volume > 4+1. > > > > > > > > When i write single small file (< 10MB) after 2 seconds i can see > bitrot > > > > signature in bricks for the file, but when i write multiple files > with > > > > different size ( > 10MB) it takes long time (> 24hrs) to see bitrot > > > > signature in all the files. > > > > > > The default timeout for signing to happen is 120 seconds. So the > > > signing will happen > > > 120 secs after the last fd gets closed on that file. So if the file > is > > > being written > > > continuously, it will not be signed until 120 secs after it's last > fd is > > > closed. > > > > > > > > My questions are. > > > > 1. I have enabled scrub schedule as hourly and throttle as normal, > does > > > this > > > > make any impact in delaying bitrot signature? > > > No. > > > > 2. other than "bitd.log" where else i can watch current status of > bitrot, > > > > like number of files added for signature and file status? > > > Signature will happen after 120 sec of last fd closure, as said > above. > > > There is not status command which tracks the signature of the > files. > > > But there is bitrot status command which tracks the number of > files > > > scrubbed. > > > > > > #gluster vol bitrot <volname> scrub status > > > > > > > > > > 3. where i can confirm that all the files in the brick are bitrot > signed? > > > > > > As said, signing information of all the files is not tracked. > > > > > > > 4. is there any file read size limit in bitrot? > > > > > > I didn't get. Could you please elaborate this ? > > > > > > > 5. options for tuning bitrot for faster signing of files? > > > > > > Bitrot feature is mainly to detect silent corruption (bitflips) of > > > files due to long > > > term storage. Hence the default is 120 sec of last fd closure, the > > > signing happens. > > > But there is a tune able which can change the default 120 sec but > > > that's only for > > > testing purposes and we don't recommend it. > > > > > > gluster vol get master features.expiry-time > > > > > > For testing purposes, you can change this default and test. > > > > > > > > Thanks > > > > Amudhan > > > > > > > > > > > > _______________________________________________ > > > > Gluster-users mailing list > > > > Gluster-users at gluster.org > > > > http://www.gluster.org/mailman/listinfo/gluster-users > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160921/a81a2b85/attachment.html>
Kotresh Hiremath Ravishankar
2016-Sep-21 09:58 UTC
[Gluster-users] 3.8.3 Bitrot signature process
Hi Amudhan, Don't grep for the filename, glusterfs maintains hardlink in .glusterfs directory for each file. Just check 'ls -l /proc/<respective brick pid>/fd' for any fds opened for a file in .glusterfs and check if it's the same file. Thanks and Regards, Kotresh H R ----- Original Message -----> From: "Amudhan P" <amudhan83 at gmail.com> > To: "Kotresh Hiremath Ravishankar" <khiremat at redhat.com> > Cc: "Gluster Users" <gluster-users at gluster.org> > Sent: Wednesday, September 21, 2016 1:33:10 PM > Subject: Re: [Gluster-users] 3.8.3 Bitrot signature process > > Hi Kotresh, > > i have used below command to verify any open fd for file. > > "ls -l /proc/*/fd | grep filename". > > as soon as write completes there no open fd's, if there is any alternate > option. please let me know will also try that. > > > > > Also, below is my scrub status in my test setup. number of skipped files > slow reducing day by day. I think files are skipped due to bitrot signature > process is not completed yet. > > where can i see scrub skipped files? > > > Volume name : glsvol1 > > State of scrub: Active (Idle) > > Scrub impact: normal > > Scrub frequency: daily > > Bitrot error log location: /var/log/glusterfs/bitd.log > > Scrubber error log location: /var/log/glusterfs/scrub.log > > > ========================================================> > Node: localhost > > Number of Scrubbed files: 1644 > > Number of Skipped files: 1001 > > Last completed scrub time: 2016-09-20 11:59:58 > > Duration of last scrub (D:M:H:M:S): 0:0:39:26 > > Error count: 0 > > > ========================================================> > Node: 10.1.2.3 > > Number of Scrubbed files: 1644 > > Number of Skipped files: 1001 > > Last completed scrub time: 2016-09-20 10:50:00 > > Duration of last scrub (D:M:H:M:S): 0:0:38:17 > > Error count: 0 > > > ========================================================> > Node: 10.1.2.4 > > Number of Scrubbed files: 981 > > Number of Skipped files: 1664 > > Last completed scrub time: 2016-09-20 12:38:01 > > Duration of last scrub (D:M:H:M:S): 0:0:35:19 > > Error count: 0 > > > ========================================================> > Node: 10.1.2.1 > > Number of Scrubbed files: 1263 > > Number of Skipped files: 1382 > > Last completed scrub time: 2016-09-20 11:57:21 > > Duration of last scrub (D:M:H:M:S): 0:0:37:17 > > Error count: 0 > > > ========================================================> > Node: 10.1.2.2 > > Number of Scrubbed files: 1644 > > Number of Skipped files: 1001 > > Last completed scrub time: 2016-09-20 11:59:25 > > Duration of last scrub (D:M:H:M:S): 0:0:39:18 > > Error count: 0 > > ========================================================> > > > > Thanks > Amudhan > > > On Wed, Sep 21, 2016 at 11:45 AM, Kotresh Hiremath Ravishankar < > khiremat at redhat.com> wrote: > > > Hi Amudhan, > > > > I don't think it's the limitation with read data from the brick. > > To limit the usage of CPU, throttling is done using token bucket > > algorithm. The log message showed is related to it. But even then > > I think it should not take 12 minutes for check-sum calculation unless > > there is an fd open (might be internal). Could you please cross verify > > if there are any fd opened on that file by looking into /proc? I will > > also test it out in the mean time and get back to you. > > > > Thanks and Regards, > > Kotresh H R > > > > ----- Original Message ----- > > > From: "Amudhan P" <amudhan83 at gmail.com> > > > To: "Kotresh Hiremath Ravishankar" <khiremat at redhat.com> > > > Cc: "Gluster Users" <gluster-users at gluster.org> > > > Sent: Tuesday, September 20, 2016 3:19:28 PM > > > Subject: Re: [Gluster-users] 3.8.3 Bitrot signature process > > > > > > Hi Kotresh, > > > > > > Please correct me if i am wrong, Once a file write completes and as soon > > as > > > closes fds, bitrot waits for 120 seconds and starts hashing and update > > > signature for the file in brick. > > > > > > But, what i am feeling that bitrot takes too much of time to complete > > > hashing. > > > > > > below is test result i would like to share. > > > > > > > > > writing data in below path using dd : > > > > > > /mnt/gluster/data/G (mount point) > > > -rw-r--r-- 1 root root 10M Sep 20 12:19 test53-bs10M-c1.nul > > > -rw-r--r-- 1 root root 100M Sep 20 12:19 test54-bs10M-c10.nul > > > > > > No any other write or read process is going on. > > > > > > > > > Checking file data in one of the brick. > > > > > > -rw-r--r-- 2 root root 2.5M Sep 20 12:23 test53-bs10M-c1.nul > > > -rw-r--r-- 2 root root 25M Sep 20 12:23 test54-bs10M-c10.nul > > > > > > file's stat and getfattr info from brick, after write process completed. > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test53-bs10M-c1.nul > > > File: ?test53-bs10M-c1.nul? > > > Size: 2621440 Blocks: 5120 IO Block: 4096 regular file > > > Device: 821h/2081d Inode: 536874168 Links: 2 > > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > > Access: 2016-09-20 12:23:28.798886647 +0530 > > > Modify: 2016-09-20 12:23:28.994886646 +0530 > > > Change: 2016-09-20 12:23:28.998886646 +0530 > > > Birth: - > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test54-bs10M-c10.nul > > > File: ?test54-bs10M-c10.nul? > > > Size: 26214400 Blocks: 51200 IO Block: 4096 regular file > > > Device: 821h/2081d Inode: 536874169 Links: 2 > > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > > Access: 2016-09-20 12:23:42.902886624 +0530 > > > Modify: 2016-09-20 12:23:44.378886622 +0530 > > > Change: 2016-09-20 12:23:44.378886622 +0530 > > > Birth: - > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > > test53-bs10M-c1.nul > > > # file: test53-bs10M-c1.nul > > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > > trusted.ec.config=0x0000080501000200 > > > trusted.ec.size=0x0000000000a00000 > > > trusted.ec.version=0x00000000000000500000000000000050 > > > trusted.gfid=0xe2416bd1aae4403c88f44286273bbe99 > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > > test54-bs10M-c10.nul > > > # file: test54-bs10M-c10.nul > > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > > trusted.ec.config=0x0000080501000200 > > > trusted.ec.size=0x0000000006400000 > > > trusted.ec.version=0x00000000000003200000000000000320 > > > trusted.gfid=0x54e018dd8c5a4bd79e0317729d8a57c5 > > > > > > > > > > > > file's stat and getfattr info from brick, after bitrot signature updated. > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test53-bs10M-c1.nul > > > File: ?test53-bs10M-c1.nul? > > > Size: 2621440 Blocks: 5120 IO Block: 4096 regular file > > > Device: 821h/2081d Inode: 536874168 Links: 2 > > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > > Access: 2016-09-20 12:25:31.494886450 +0530 > > > Modify: 2016-09-20 12:23:28.994886646 +0530 > > > Change: 2016-09-20 12:27:00.994886307 +0530 > > > Birth: - > > > > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > > test53-bs10M-c1.nul > > > # file: test53-bs10M-c1.nul > > > trusted.bit-rot.signature=0x0102000000000000006de7493c5c > > 90f643357c268fbaaf461c1567e0334e4948023ce17268403aa37a > > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > > trusted.ec.config=0x0000080501000200 > > > trusted.ec.size=0x0000000000a00000 > > > trusted.ec.version=0x00000000000000500000000000000050 > > > trusted.gfid=0xe2416bd1aae4403c88f44286273bbe99 > > > > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ stat test54-bs10M-c10.nul > > > File: ?test54-bs10M-c10.nul? > > > Size: 26214400 Blocks: 51200 IO Block: 4096 regular file > > > Device: 821h/2081d Inode: 536874169 Links: 2 > > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > > Access: 2016-09-20 12:25:47.510886425 +0530 > > > Modify: 2016-09-20 12:23:44.378886622 +0530 > > > Change: 2016-09-20 12:38:05.954885243 +0530 > > > Birth: - > > > > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ sudo getfattr -m. -e hex -d > > > test54-bs10M-c10.nul > > > # file: test54-bs10M-c10.nul > > > trusted.bit-rot.signature=0x010200000000000000394c345f0b > > 0c63ee652627a62eed069244d35c4d5134e4f07d4eabb51afda47e > > > trusted.bit-rot.version=0x020000000000000057daa7b50002e5b4 > > > trusted.ec.config=0x0000080501000200 > > > trusted.ec.size=0x0000000006400000 > > > trusted.ec.version=0x00000000000003200000000000000320 > > > trusted.gfid=0x54e018dd8c5a4bd79e0317729d8a57c5 > > > > > > > > > (Actual time taken for reading file from brick for md5sum) > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ time md5sum test53-bs10M-c1.nul > > > 8354dcaa18a1ecb52d0895bf00888c44 test53-bs10M-c1.nul > > > > > > real 0m0.045s > > > user 0m0.007s > > > sys 0m0.003s > > > > > > gfstst-node5:/media/disk2/brick2/data/G$ time md5sum > > test54-bs10M-c10.nul > > > bed3c0a4a1407f584989b4009e9ce33f test54-bs10M-c10.nul > > > > > > real 0m0.166s > > > user 0m0.062s > > > sys 0m0.011s > > > > > > As you can see that 'test54-bs10M-c10.nul' file took around 12 minutes to > > > update bitort signature (pls refer stat output for the file). > > > > > > what would be the cause for such a slow read?. Any limitation in read > > data > > > from brick? > > > > > > Also, i am seeing this line bitd.log, what does this mean? > > > [bit-rot.c:1784:br_rate_limit_signer] 0-glsvol1-bit-rot-0: [Rate Limit > > > Info] "tokens/sec (rate): 131072, maxlimit: 524288 > > > > > > > > > Thanks > > > Amudhan P > > > > > > > > > > > > On Mon, Sep 19, 2016 at 1:00 PM, Kotresh Hiremath Ravishankar < > > > khiremat at redhat.com> wrote: > > > > > > > Hi Amudhan, > > > > > > > > Thanks for testing out the bitrot feature and sorry for the delayed > > > > response. > > > > Please find the answers inline. > > > > > > > > Thanks and Regards, > > > > Kotresh H R > > > > > > > > ----- Original Message ----- > > > > > From: "Amudhan P" <amudhan83 at gmail.com> > > > > > To: "Gluster Users" <gluster-users at gluster.org> > > > > > Sent: Friday, September 16, 2016 4:14:10 PM > > > > > Subject: Re: [Gluster-users] 3.8.3 Bitrot signature process > > > > > > > > > > Hi, > > > > > > > > > > Can anyone reply to this mail. > > > > > > > > > > On Tue, Sep 13, 2016 at 12:49 PM, Amudhan P < amudhan83 at gmail.com > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > I am testing bitrot feature in Gluster 3.8.3 with disperse EC volume > > 4+1. > > > > > > > > > > When i write single small file (< 10MB) after 2 seconds i can see > > bitrot > > > > > signature in bricks for the file, but when i write multiple files > > with > > > > > different size ( > 10MB) it takes long time (> 24hrs) to see bitrot > > > > > signature in all the files. > > > > > > > > The default timeout for signing to happen is 120 seconds. So the > > > > signing will happen > > > > 120 secs after the last fd gets closed on that file. So if the file > > is > > > > being written > > > > continuously, it will not be signed until 120 secs after it's last > > fd is > > > > closed. > > > > > > > > > > My questions are. > > > > > 1. I have enabled scrub schedule as hourly and throttle as normal, > > does > > > > this > > > > > make any impact in delaying bitrot signature? > > > > No. > > > > > 2. other than "bitd.log" where else i can watch current status of > > bitrot, > > > > > like number of files added for signature and file status? > > > > Signature will happen after 120 sec of last fd closure, as said > > above. > > > > There is not status command which tracks the signature of the > > files. > > > > But there is bitrot status command which tracks the number of > > files > > > > scrubbed. > > > > > > > > #gluster vol bitrot <volname> scrub status > > > > > > > > > > > > > 3. where i can confirm that all the files in the brick are bitrot > > signed? > > > > > > > > As said, signing information of all the files is not tracked. > > > > > > > > > 4. is there any file read size limit in bitrot? > > > > > > > > I didn't get. Could you please elaborate this ? > > > > > > > > > 5. options for tuning bitrot for faster signing of files? > > > > > > > > Bitrot feature is mainly to detect silent corruption (bitflips) of > > > > files due to long > > > > term storage. Hence the default is 120 sec of last fd closure, the > > > > signing happens. > > > > But there is a tune able which can change the default 120 sec but > > > > that's only for > > > > testing purposes and we don't recommend it. > > > > > > > > gluster vol get master features.expiry-time > > > > > > > > For testing purposes, you can change this default and test. > > > > > > > > > > Thanks > > > > > Amudhan > > > > > > > > > > > > > > > _______________________________________________ > > > > > Gluster-users mailing list > > > > > Gluster-users at gluster.org > > > > > http://www.gluster.org/mailman/listinfo/gluster-users > > > > > > > > > >