Gluster users - Feb 2010 - my own unify - raid5 setup, is it possible?

Hi all,

I've been toying around with glusterfs 2.0.9 and I'm really liking it. I
very much enjoy it's simple modular design.

My current test setup is a unify cluster, using a afr/loadbalanced
namespace.

Could I duplicate files to multiple data bricks in the cluster to
provide a raid5-like setup? I very much want to be able to shutdown a
single machine in the cluster and still have a fully functional
filesystem. I'm very happy to write the application that does the
copying over the data bricks myself.

Another advantage could be that I can decide on a per-file basis how
many copies of a file exist in the filesystem. (Two would be a minimum
for me) The real-world scenario: This would be the data filesystem for a
webserver cluster setup. You can imagine images used on a homepage are
requested more frequent than others.

I imagine the order in which I specify the subvolumes in the unify
volume needs to be different over my clients to make good use of the
multiple copies over the different bricks.

What problems can I expect with this setup?
Have others tried a similar setup?
Am I missing a GlusterFS feature that would implement what I want, in a
much easier way?

Thank you for your time!

Some other minor remarks:

* Is there a reason the AFR translator is missing in the wiki on:
http://gluster.com/community/documentation/index.php/Translators ?

* The wiki search does not work for small words (AFR for example) This
is because MySQL full text search indexes are used, and mysql has a
minimum char length. Add 'ft_min_word_len=2' to the mysql my.cnf file
(in the mysqld section) To create indexes of 2 and 3 letter words too.
This results in larger indexes, but I think it's worth it.

Hi Casper,

On Fri, Feb 12, 2010 at 1:32 PM, Casper Langemeijer <casper at bcx.nl>
wrote:
> Hi all,
>
> I've been toying around with glusterfs 2.0.9 and I'm really liking
it. I
> very much enjoy it's simple modular design.
>
> My current test setup is a unify cluster, using a afr/loadbalanced
> namespace.
>
> Could I duplicate files to multiple data bricks in the cluster to
> provide a raid5-like setup? I very much want to be able to shutdown a
> single machine in the cluster and still have a fully functional
> filesystem. I'm very happy to write the application that does the
> copying over the data bricks myself.
>
We recommend using distribute translator instead of unify. But with
distribute (even with unify) data is not striped. Both translators (unify
and distribute) are used to aggregate multiple storage nodes into a single
filesystem. If you want to increase read performance using stripe, you can
use stripe translator.

> Another advantage could be that I can decide on a per-file basis how
> many copies of a file exist in the filesystem. (Two would be a minimum
> for me) The real-world scenario: This would be the data filesystem for a
> webserver cluster setup. You can imagine images used on a homepage are
> requested more frequent than others.
>
replicate (formerly known as afr) does not  support maintaining different
number of replicas for different files.

> I imagine the order in which I specify the subvolumes in the unify
> volume needs to be different over my clients to make good use of the
> multiple copies over the different bricks.
>
> What problems can I expect with this setup?
> Have others tried a similar setup?
> Am I missing a GlusterFS feature that would implement what I want, in a
> much easier way?
>
> Thank you for your time!
>
> Some other minor remarks:
>
> * Is there a reason the AFR translator is missing in the wiki on:
> http://gluster.com/community/documentation/index.php/Translators ?
>
AFR is renamed as replicate.

>
> * The wiki search does not work for small words (AFR for example) This
> is because MySQL full text search indexes are used, and mysql has a
> minimum char length. Add 'ft_min_word_len=2' to the mysql my.cnf
file
> (in the mysqld section) To create indexes of 2 and 3 letter words too.
> This results in larger indexes, but I think it's worth it.
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>


-- 
Raghavendra G