eagleeyes
2009-Jul-14 09:19 UTC
[Gluster-users] So tangly of Glusterfs 2.0.4 distributing after dht expansion !!!!
HI:
Today i tested glusterfs2.0.4 with fuse-2.8.0-pre3 , for dht expansion . I
found a problem.
I use
[root at nio1 data6]# ls /data/
data1 data2 data3 data4 data5 data6 as data dir ,
first i used data1+data2+data3 ,mount dot /mnt and touch 11 22 33 44 55
66 , created "ddd eee jjj pppppp uuu yy " with some word in
them .
then those files in data was :
ll -h /data/*
/data/data1:
total 0
-rw-r--r-- 1 root root 0 Jul 14 17:14 11
-rw-r--r-- 1 root root 0 Jul 14 17:14 33
/data/data2:
total 12K
-rw-r--r-- 1 root root 0 Jul 14 17:14 22
-rw-r--r-- 1 root root 0 Jul 14 17:14 44
-rw-r--r-- 1 root root 116 Jul 14 17:15 ddd
-rw-r--r-- 1 root root 116 Jul 14 17:15 jjj
-rw-r--r-- 1 root root 116 Jul 14 17:15 yy
/data/data3:
total 12K
-rw-r--r-- 1 root root 0 Jul 14 17:14 55
-rw-r--r-- 1 root root 0 Jul 14 17:14 66
-rw-r--r-- 1 root root 116 Jul 14 17:15 eee
-rw-r--r-- 1 root root 116 Jul 14 17:15 pppppp
-rw-r--r-- 1 root root 116 Jul 14 17:15 uuu
and then i expansion dht client, used data1+data2+data3+data4+data5+data6
,mount dot /mnt
this time i find
ll -h /data/*
/data/data1:
total 0
-rw-r--r-- 1 root root 0 Jul 14 16:57 11
-rw-r--r-- 1 root root 0 Jul 14 16:57 33
/data/data2:
total 16K
---------T 1 root root 0 Jul 14 16:58 11
-rw-r--r-- 1 root root 0 Jul 14 16:57 22
-rw-r--r-- 1 root root 0 Jul 14 16:57 44
-rw-r--r-- 1 root root 126 Jul 14 16:56 ddd
-rw-r--r-- 1 root root 126 Jul 14 16:56 jjj
-rw-r--r-- 1 root root 126 Jul 14 16:56 yy
/data/data3:
total 24K
---------T 1 root root 0 Jul 14 16:58 44
-rw-r--r-- 1 root root 0 Jul 14 16:57 55
-rw-r--r-- 1 root root 0 Jul 14 16:57 66
---------T 1 root root 0 Jul 14 16:58 ddd
-rw-r--r-- 1 root root 126 Jul 14 16:56 eee
---------T 1 root root 0 Jul 14 16:58 jjj
-rw-r--r-- 1 root root 126 Jul 14 16:56 pppppp
-rw-r--r-- 1 root root 126 Jul 14 16:56 uuu
/data/data4:
total 8.0K
---------T 1 root root 0 Jul 14 16:58 22
---------T 1 root root 0 Jul 14 16:58 yy
/data/data5:
total 4.0K
---------T 1 root root 0 Jul 14 16:58 55
/data/data6:
total 16K
---------T 1 root root 0 Jul 14 16:58 66
---------T 1 root root 0 Jul 14 16:58 eee
---------T 1 root root 0 Jul 14 16:58 pppppp
---------T 1 root root 0 Jul 14 16:58 uuu
files' names appeared in data4 data5 data6 ,but no byte . Why was it ?
If a directory i can comprehend ,but those was files and appeared files was
not all files in data1+data2+data3 .
Why the files distributing was so tangly .
2009-07-14
eagleeyes
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090714/0b6ce582/attachment.html>
NovA
2009-Jul-14 13:41 UTC
[Gluster-users] So tangly of Glusterfs 2.0.4 distributing after dht expansion !!!!
Hi! Empty files with sticky bit (access mode ---------T) are link-files, which extended attributes contains the location of the real file. It's OK, that these link-files have been created after changing the data layout. It seems you have not got the idea how the DHT works. It's rather clear actually, as I understand it. When one accesses a file in a GlusterFS directory the following actions are taken. 1) A special number (hash) is generated from the file name. 2) Based on this number a cluster subvolume is chosen where to look the file's data. 3) The data is transfered form/to the selected subvolume. 3a) Instead of the real data, the selected subvolume can contain a link to the other subvolume where the data really resides, then the data transfered from/to there. The algorithm of choosing a subvolume by hash (in the 2nd item) depends on the total number of available subvolumes. So it will select different subvolumes if the number changes. To ensure availability of files created in previous layout GlusterFS creates link-files, which you see. Best wishes, Andrey 2009/7/14 eagleeyes <eagleeyes at 126.com>:> HI: > Today i tested glusterfs2.0.4 with fuse-2.8.0-pre3 , for dht expansion > . I found a problem. > I use > [root at nio1 data6]# ls /data/ > data1 data2 data3 data4 data5 data6 as data dir , > > first i used data1+data2+data3 ,mount dot /mnt and touch > 11 22 33 44 55 66 , created "ddd eee jjj pppppp uuu yy " with > some word in them . > then those files in data was : > > ll -h /data/* > /data/data1: > total 0 > -rw-r--r-- 1 root root 0 Jul 14 17:14 11 > -rw-r--r-- 1 root root 0 Jul 14 17:14 33 > /data/data2: > total 12K > -rw-r--r-- 1 root root 0 Jul 14 17:14 22 > -rw-r--r-- 1 root root 0 Jul 14 17:14 44 > -rw-r--r-- 1 root root 116 Jul 14 17:15 ddd > -rw-r--r-- 1 root root 116 Jul 14 17:15 jjj > -rw-r--r-- 1 root root 116 Jul 14 17:15 yy > /data/data3: > total 12K > -rw-r--r-- 1 root root 0 Jul 14 17:14 55 > -rw-r--r-- 1 root root 0 Jul 14 17:14 66 > -rw-r--r-- 1 root root 116 Jul 14 17:15 eee > -rw-r--r-- 1 root root 116 Jul 14 17:15 pppppp > -rw-r--r-- 1 root root 116 Jul 14 17:15 uuu > > and then i expansion dht client, used data1+data2+data3+data4+data5+data6 > ,mount dot /mnt > this time i find > ll -h /data/* > /data/data1: > total 0 > -rw-r--r-- 1 root root 0 Jul 14 16:57 11 > -rw-r--r-- 1 root root 0 Jul 14 16:57 33 > /data/data2: > total 16K > ---------T 1 root root 0 Jul 14 16:58 11 > -rw-r--r-- 1 root root 0 Jul 14 16:57 22 > -rw-r--r-- 1 root root 0 Jul 14 16:57 44 > -rw-r--r-- 1 root root 126 Jul 14 16:56 ddd > -rw-r--r-- 1 root root 126 Jul 14 16:56 jjj > -rw-r--r-- 1 root root 126 Jul 14 16:56 yy > /data/data3: > total 24K > ---------T 1 root root 0 Jul 14 16:58 44 > -rw-r--r-- 1 root root 0 Jul 14 16:57 55 > -rw-r--r-- 1 root root 0 Jul 14 16:57 66 > ---------T 1 root root 0 Jul 14 16:58 ddd > -rw-r--r-- 1 root root 126 Jul 14 16:56 eee > ---------T 1 root root 0 Jul 14 16:58 jjj > -rw-r--r-- 1 root root 126 Jul 14 16:56 pppppp > -rw-r--r-- 1 root root 126 Jul 14 16:56 uuu > /data/data4: > total 8.0K > ---------T 1 root root 0 Jul 14 16:58 22 > ---------T 1 root root 0 Jul 14 16:58 yy > /data/data5: > total 4.0K > ---------T 1 root root 0 Jul 14 16:58 55 > /data/data6: > total 16K > ---------T 1 root root 0 Jul 14 16:58 66 > ---------T 1 root root 0 Jul 14 16:58 eee > ---------T 1 root root 0 Jul 14 16:58 pppppp > ---------T 1 root root 0 Jul 14 16:58 uuu > > files' names appeared in data4 data5 data6 ,but no byte . Why was it ? > If a directory i can comprehend ,but those was files and appeared files > was not all files in data1+data2+data3 . > Why the files distributing was so tangly . > > 2009-07-14 > ________________________________ > eagleeyes > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users > >