Gluster users - Oct 2010 - Possible to use gluster w/ email services + Tuning for fast replication

Gluster ver: glusterfs-core/rdma/fuse-3.1.0-1 
OS: Centos 5.5 x86_64 (Two VMWare Nodes) 

Hello All, 

I am relatively new to gluster, I've started diving into Gluster for a
couple of weeks now. I was wondering if anyone is using Gluster with an email
server, e.g. postfix or sendmail. I am curious to know on how fast gluster
replicates files since MTA's do a lot of IO. I'd imagine some gluster
tuning involved to handle a lot files needing to be replicated.

Right now, I am testing out a 2 node setup, with one server replicating data to
another node. One thing I noticed was when I created a file or directory on the
server, the new data does not replicate to the other node. The only time data is
synced from server to the other node is when I run "gluster volume
rebalance test start". Is this normal? I had envisioned gluster would
constantly replicate changes from the server to the other nodes, am I off base?

The commands I used to create the volume from the server (hostnameA) 

1) gluster volume create test replica 2 transport tcp hostnameA:/opt
hostnameB:/opt

2) gluster volume start test 

3) mount -t glusterfs hostnameA:/test /mnt 


My current volume info: 

Volume Name: test 
Type: Replicate 
Status: Started 
Number of Bricks: 2 
Transport-type: tcp 
Bricks: 
Brick1: hostnameA:/opt 
Brick2: hostnameB:/opt 
Options Reconfigured: 
performance.cache-size: 6GB 

Greatly appreciate anyone has any pointers on how to improve replication. 

My best to you, 

~~Rick King 











DISCLAIMER: This e-mail and any files transmitted with it ('Message') is
intended only for the use of the recepient (s) named and may contain
confidential information. Opinions, conclusion and other information in this
message that do not relate to the official business of King7.

> Right now, I am testing out a 2 node setup, with one server replicating
data to another node. One thing I noticed was when I created a file or directory
on the server, the new data does not replicate to the other node. The only time
data is synced from server to the other node is when I run "gluster volume
rebalance test start". Is this normal? I had envisioned gluster would
constantly replicate changes from the server to the other nodes, am I off base?
Are you examining the second node directly, ie not by mounting it?  I 
think the point is that replication only happens when you "observe"
the
second node?

Glusterfs is targeted for HTPC applications where typically the nodes 
are all connected over high performance interlinks.  It appears that 
performance degrades very quickly as the latency between nodes increases 
and so whether the solution works for you is largely going to be 
determined by the latency between nodes on your network connection.

I'm not actually sure what some representative numbers should be?  I 
have two machines hooked up using bonded-rr intel gigabit cards 
(crossover to each other) and these ping at around 0.3ms.  However, I 
have one other machine on a gigabit connection, hooked up to a switch 
and that sometimes drops to around 0.15ms...  I believe infiniband will 
drop that latency to some few tens of microseconds?

So basically every file access on my system would suffer a 0.3ms access 
latency.  This is better than a spining disk with no cache which comes 
in more like 3-10ms, but obviously it's still not brilliant

Please let us know how you get on?

Good luck

Ed W