jungeun kim
2018-Jan-12 09:17 UTC
[Gluster-users] Reading over than the file size on dispersed volume
Hi All,
I'm using gluster as dispersed volume and I send to ask for very serious
thing.
I have 3 servers and there are 9 bricks.
My volume is like below.
------------------------------------------------------
Volume Name: TEST_VOL
Type: Disperse
Volume ID: be52b68d-ae83-46e3-9527-0e536b867bcc
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (6 + 3) = 9
Transport-type: tcp
Bricks:
Brick1: server1:/data/brick1
Brick2: server2:/data/brick1
Brick3: server3:/data/brick1
Brick4: server1:/data/brick2
Brick5: server2:/data/brick2
Brick6: server3:/data/brick2
Brick7: server1:/data/brick3
Brick8: server2:/data/brick3
Brick9: server3:/data/brick3
Options Reconfigured:
network.ping-timeout: 10
performance.write-behind: on
features.quota-deem-statfs: on
features.inode-quota: on
features.quota: on
transport.address-family: inet
nfs.disable: on
------------------------------------------------------
I generated the temporary files by various size to test.
But I have found out some files MD5 mismatch after uploading to glusterfs.
e.g
fallocate -l 130352 test1
fallocate -l 2281569280 test2
fallocate -l 667417616 test3
I have to calculate to make MD5 after uploading.
But the total size is different when reading the files to EOF. and MD5 is
different, too.
( I write 42byte header in each files when create the file )
char buf[128*1024] = {0,};
offset += 42;
while ((retr = glfs_pread(fd, (void*)&buf, sizeof(buf), offset, 0)) >
0) {
DEBUG("retr : %ld, offset : %ld(%d)", retr,offset,errno) ;
offset += retr;
MD5_Update(&context, buf, retr);
}
DEBUG("total : %d", offset);
[DEBUG] retr : 130394, offset : 42(0) (star_glusterfs.c:96)
[DEBUG] retr : 1024, offset : 130436(0) (star_glusterfs.c:96)
[DEBUG] retr : 636, offset : 131460(0) (star_glusterfs.c:96)
[DEBUG] total : 132096 (star_glusterfs.c:109)
Why the glfs_pread(or glfs_read) function reads until over than the file
size ?
This is fine on dispersed (4+2) and (8+1) volume but it is happend on (6+3)
or (6+4) or (7+2) disperse volume.
I want to know exact cause.
Thank you.
Regards,
Jenny
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180112/ff5effdf/attachment.html>
Jim Kinney
2018-Jan-12 18:13 UTC
[Gluster-users] Reading over than the file size on dispersed volume
On a disperse setup shouldn't the various nodes have different parts of the files and thus the md5sums would be different? On Fri, 2018-01-12 at 18:17 +0900, jungeun kim wrote:> Hi All, > > I'm using gluster as dispersed volume and I send to ask for very > serious thing. > I have 3 servers and there are 9 bricks. > > My volume is like below. > ------------------------------------------------------ > Volume Name: TEST_VOL > Type: Disperse > Volume ID: be52b68d-ae83-46e3-9527-0e536b867bcc > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 x (6 + 3) = 9 > Transport-type: tcp > Bricks: > Brick1: server1:/data/brick1 > Brick2: server2:/data/brick1 > Brick3: server3:/data/brick1 > Brick4: server1:/data/brick2 > Brick5: server2:/data/brick2 > Brick6: server3:/data/brick2 > Brick7: server1:/data/brick3 > Brick8: server2:/data/brick3 > Brick9: server3:/data/brick3 > Options Reconfigured: > network.ping-timeout: 10 > performance.write-behind: on > features.quota-deem-statfs: on > features.inode-quota: on > features.quota: on > transport.address-family: inet > nfs.disable: on > ------------------------------------------------------ > > I generated the temporary files by various size to test. > But I have found out some files MD5 mismatch after uploading to > glusterfs. > > e.g > fallocate -l 130352 test1 > fallocate -l 2281569280 test2 > fallocate -l 667417616 test3 > > I have to calculate to make MD5 after uploading. > But the total size is different when reading the files to EOF. and > MD5 is different, too. > ( I write 42byte header in each files when create the file ) > > > char buf[128*1024] = {0,}; > offset += 42; > while ((retr = glfs_pread(fd, (void*)&buf, sizeof(buf), offset, > 0)) > 0) { > DEBUG("retr : %ld, offset : %ld(%d)", retr,offset,errno) ; > offset += retr; > MD5_Update(&context, buf, retr); > } > > DEBUG("total : %d", offset); > > [DEBUG] retr : 130394, offset : 42(0) (star_glusterfs.c:96) > [DEBUG] retr : 1024, offset : 130436(0) (star_glusterfs.c:96) > [DEBUG] retr : 636, offset : 131460(0) (star_glusterfs.c:96) > [DEBUG] total : 132096 (star_glusterfs.c:109) > > > Why the glfs_pread(or glfs_read) function reads until over than the > file size ? > > This is fine on dispersed (4+2) and (8+1) volume but it is happend on > (6+3) or (6+4) or (7+2) disperse volume. > I want to know exact cause. > Thank you. > > Regards, > > Jenny > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users-- James P. Kinney III Every time you stop a school, you will have to build a jail. What you gain at one end you lose at the other. It's like feeding a dog on his own tail. It won't fatten the dog. - Speech 11/23/1900 Mark Twain http://heretothereideas.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180112/8da71fd2/attachment.html>