Gluster users - Mar 2011 - How to use gluster for WAN/Data Center replication

How to setup gluster for WAN/Data Center replication? Are there others
using it this way?

Also, how to make the writes asynchronuous for data center replication?

We have a requirement to replicate data to other data center as well.

Hi, 
According to my understanding, there is no data replication mechanism for WAN.
I know that Sector file system have this function you want and it uses enhanced
UDT protocol.
The way you can use in Gluster is to replicate data by using the original TCP
protocol.
But it is influenced by network a lot.
I tried this way before but the performance was not good.
I asked gluster.com about the question and they said they are going to provide
the function this year, for your reference.

Best Regards, 
Sylar Shen

-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at
gluster.org] On Behalf Of Mohit Anchlia
Sent: Thursday, March 10, 2011 8:44 AM
To: gluster-users at gluster.org
Subject: [Gluster-users] How to use gluster for WAN/Data Center replication

How to setup gluster for WAN/Data Center replication? Are there others
using it this way?

Also, how to make the writes asynchronuous for data center replication?

We have a requirement to replicate data to other data center as well.
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Hi,
I have done a setup(see my setup below) on multi site datacenter with gluster
and currently it doesn't work properly but there is some workaround.
The main problem is that replication is synchronous and there is currently no
way to turn it in async mod. I've done some test
(iozone,tar,bonnie++,script...) and performance
is poor with small files especially. We are using an url to access servers :
glusterfs.cluster.inetcompany.com
This url is in DNS GSLB(geo DNS)+RR (Round Robin)
It means that client from datacenter 1 will always be binded randomly on storage
node from his Datacenter.
They use this command for mounting the filesystem : 
mount -t glusterfs glusterfs.cluster.inetcompany.com:/venus /users/glusterfs_mnt

If one node fails , it is remove from de list of the DNS, client do a new DNS
query and he is binded on active node of his Datacenter.
You could use Wan accelerator also.

We currently are in intra site mode and we are waiting for Async replication
feature expected in version 3.2. It should come soon.


Volume Name: venus
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 4 = 8
Transport-type: tcp
Bricks:
Brick1: serv1:/users/exp1  \
Brick2: serv2:/users/exp2   > R?plica pool 1 \
Brick3: serv3:/users/exp3  /                  \
Brick4: serv4:/users/exp4                       =Envoyer>Distribution
Brick5: serv5:/users/exp5  \                  /
Brick6: serv6:/users/exp6   > R?plica pool 2 /
Brick7: serv7:/users/exp7  /
Brick8: serv8:/users/exp8

Datacenter 1 : Brick 1,2,5,6
Datacenter 2 : Brick 3,4,7,8
Distance between Datacenters : 500km
Latency between Datacenters : 11ms
Datarate between Datacenters : ~100Mb/s



Regards,
Anthony


>Message: 3
>Date: Wed, 9 Mar 2011 16:44:27 -0800
>From: Mohit Anchlia <mohitanchlia at gmail.com>
>Subject: [Gluster-users] How to use gluster for WAN/Data Center
>	replication
>To: gluster-users at gluster.org
>Message-ID:
>	<AANLkTi=dkK=zX0QdCfnKeLJ5nkF1dF3+g1hxDzFZNvwx at mail.gmail.com>
>Content-Type: text/plain; charset=ISO-8859-1
> 
>How to setup gluster for WAN/Data Center replication? Are there others
>using it this way?
> 
>Also, how to make the writes asynchronuous for data center replication?
> 
>We have a requirement to replicate data to other data center as well.

2011/3/10 Mohit Anchlia <mohitanchlia at
gmail.com>:
> How to setup gluster for WAN/Data Center replication? Are there others
> using it this way?
AFAIK, GlusterFS performance depends mostly on network latency ->
Ethernet vs. Infiniband
The high latency of a WAN connection will make every (uncached) file
or metadata access very slow, compared to a local network.
> Also, how to make the writes asynchronuous for data center replication?
>
> We have a requirement to replicate data to other data center as well.
If you want to DIY:

http://fak3r.com/geek/howto-build-your-own-open-source-dropbox-clone/
http://code.google.com/p/lsyncd/

might be an option...

Disclaimer: I have not tried a setup like that yet, I don't know if it
would work with two GlusterFS-mountpoints.


regards,

Harald St?rzebecher