Gluster users - Jul 2009 - Adding new volumes to DHT

I have a question about this paragraph from the "Understanding DHT  
Translator":

"Currently hash works based on directory level distribution. i.e, a  
given file's parent directory will have information of how the hash  
numbers are mapped to subvolumes. So, adding new node doesn't disturb  
any current setup as the files/dirs present already have its  
information preserved. Whatever new directory gets created, will start  
considering new volume for scheduling files."

The last sentence suggests that if I have a single directory on a DHT  
volume and it is getting full, adding additional subvolumes to the DHT  
volume will not help because all the files in a directory will only  
ever live in subvolumes that existed at the time the directory is  
created.  Is that true?

Also, it sounds like an entry mapping each filename (hash number) to a  
subvolume is stored in the extended attributes of the parent directory  
for that file.  What is the practical limit for the number of files  
that can be stored in a single directory under this system?  It seems  
like eventually doing a lookup in the directory's attributes would  
itself become a very expensive operation.

Thanks,
Barry

Barry,

i just did a test about that.
if you add another subvolume to the DHT volume and add new files into  
an existing directory they will be distributed over all DHT-subvolumes.

I assume that the distribution on old subvolumes will stop when the  
free-space threshold is reached, but i did not test that.

Moritz


Am 21.07.2009 um 18:57 schrieb Barry Jaspan:
> I have a question about this paragraph from the "Understanding DHT  
> Translator":
>
> "Currently hash works based on directory level distribution. i.e, a  
> given file's parent directory will have information of how the hash  
> numbers are mapped to subvolumes. So, adding new node doesn't  
> disturb any current setup as the files/dirs present already have its  
> information preserved. Whatever new directory gets created, will  
> start considering new volume for scheduling files."
>
> The last sentence suggests that if I have a single directory on a  
> DHT volume and it is getting full, adding additional subvolumes to  
> the DHT volume will not help because all the files in a directory  
> will only ever live in subvolumes that existed at the time the  
> directory is created.  Is that true?
>
> Also, it sounds like an entry mapping each filename (hash number) to  
> a subvolume is stored in the extended attributes of the parent  
> directory for that file.  What is the practical limit for the number  
> of files that can be stored in a single directory under this  
> system?  It seems like eventually doing a lookup in the directory's  
> attributes would itself become a very expensive operation.
>
> Thanks,
> Barry
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users