Gluster users - Jan 2012 - Gluster limits and best practices

Hi Everyone,

We are evaluating Gluster as the filesystem for a large scale web application
where users upload many images. I wanted to see what are the limitations of
Gluster as far as number of files/directories and more importantly what are some
best practices?

We are thinking of using distributed + replicated mode with 3 copies of each
file. Is that the best approach?

In addition, how should we store the files, can we have one huge flat namespace
(i.e. UUIDs for names) or should we have a hierarchical directory structure just
like a regular filesystem to distribute the files into many sub-directories?

Thanks,

Drew

>
> We are evaluating Gluster as the filesystem for a large scale web
> application where users upload many images. I wanted to see what are the
> limitations of Gluster as far as number of files/directories and more
> importantly what are some best practices?
>
> We are thinking of using distributed + replicated mode with 3 copies of
> each file. Is that the best approach?
>
> Best option for number of copies is 2 (considering the performance), but
if your use case requires 3 copies, then that should work fine too. But
modification/creation time would be more with replica count 3.

> In addition, how should we store the files, can we have one huge flat
> namespace (i.e. UUIDs for names) or should we have a hierarchical directory
> structure just like a regular filesystem to distribute the files into many
> sub-directories?
>
> We recommend using directory structure as regular filesystem, because
having millions of files in a same directory would make the directory
listing/crawling operation to perform poorly.


Regards,
Amar
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120112/9a5d5fb7/attachment.html>