Gluster users - Sep 2012 - Recommended settings for geo-replication for folders with lots of files

Hi,

I'm currently using GlusterFS in an installation and it works quite well.
Only problem I'm currently facing is synching files to the slave on folders
with lots of files (e.g. images). Is there some recommended way to speed
this up? Synchronisation will take quite some time if there are changes in
one of these folders. I tried to search the Gluster.org website or via
google for a solution but couldn't find anything to solve the issue. Any
pointers would be appreciated. Thanks in advance.

Regards,

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120924/7f92552f/attachment.html>

You don't mention geo-synchronization specifically but you mentioned 
slave. The only time there is a slave is when you're using 
geo-synchronization, otherwise you're making an all too common mistake 
and writing to a brick. Bricks are back end storage for GlusterFS, not 
user storage. Volumes need to be accessed via a client mount.

On 09/23/2012 10:17 PM, Alexander Sopicki wrote:
> Hi,
>
> I'm currently using GlusterFS in an installation and it works quite 
> well. Only problem I'm currently facing is synching files to the slave 
> on folders with lots of files (e.g. images). Is there some recommended 
> way to speed this up? Synchronisation will take quite some time if 
> there are changes in one of these folders. I tried to search the 
> Gluster.org website or via google for a solution but couldn't find 
> anything to solve the issue. Any pointers would be appreciated. Thanks 
> in advance.
>
> Regards,
>
> Alex
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120923/05933e1e/attachment.html>

Well, you can use geo-replication to sync files from master (a GlusterFS volume)
to a directory tree on a remote slave. Due to the overhead of syncing xtime
(geo-rep's way of versioning ), large number of small files are sync'd
in an non optimal way.

There is a tunable that speeds this up but to use that the slave needs to be
GlusterFS volume. In that case doing this would speed up things:

# gluster volume geo-replication <master> <slave> config
use-rsync-xattrs On

Thanks, 
-Venky 

----- Original Message -----
> From: "Alexander Sopicki" <asopicki at googlemail.com>
> To: Gluster-users at gluster.org
> Sent: Monday, September 24, 2012 10:47:03 AM
> Subject: [Gluster-users] Recommended settings for geo-replication for
> folders with lots of files
> Hi,
> I'm currently using GlusterFS in an installation and it works quite
> well. Only problem I'm currently facing is synching files to the
> slave on folders with lots of files (e.g. images). Is there some
> recommended way to speed this up? Synchronisation will take quite
> some time if there are changes in one of these folders. I tried to
> search the Gluster.org website or via google for a solution but
> couldn't find anything to solve the issue. Any pointers would be
> appreciated. Thanks in advance.
> Regards,
> Alex
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120924/769b0b9d/attachment.html>