rsync - Feb 2008 - File locking for simultaneous rsync sessions

Hello all,

I have two web servers both mounting the same file system (with GFS). I 
have a third web server which is independent from the first two, using 
only its own local file systems. I need to mirror a directory within the 
shared file system on the first two web servers to the local file system 
on the third web server. To maintain redundancy, both of the first two 
web servers will be pushing the files to the third web server via rsync 
started up by cron as well as on-demand by the web developers.

My concern is that if rsync is running on the first two web servers at 
the same time, that there may be a chance for the two rsync sessions to 
write to the same file(s) on the third web server at the same time, 
possibly corrupting the files or interrupting the transfers.

Is there any sort of locking mechanism built in into rsync to prevent 
this from happening? Would running an rsync daemon on the third web 
server versus running it under an ssh shell make any difference to this? 
Has anyone else worked around this type of problem before and what 
solution did you use? I was thinking of having the scripts on the first 
two web servers refer to a lock file before firing up rsync, but was 
hoping there was something built in to rsync to avoid this added complexity.

Thanks in advance for any advice,
Andrew
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3225 bytes
Desc: S/MIME Cryptographic Signature
Url :
http://lists.samba.org/archive/rsync/attachments/20080214/f887a2e5/smime.bin

On Thu, 2008-02-14 at 13:51 -0500, Andrew Culver wrote:
> I have two web servers both mounting the same file system (with GFS). I 
> have a third web server which is independent from the first two, using 
> only its own local file systems. I need to mirror a directory within the 
> shared file system on the first two web servers to the local file system 
> on the third web server. To maintain redundancy, both of the first two 
> web servers will be pushing the files to the third web server via rsync 
> started up by cron as well as on-demand by the web developers.
> 
> My concern is that if rsync is running on the first two web servers at 
> the same time, that there may be a chance for the two rsync sessions to 
> write to the same file(s) on the third web server at the same time, 
> possibly corrupting the files or interrupting the transfers.
> 
> Is there any sort of locking mechanism built in into rsync to prevent 
> this from happening?
Rsync does no file locking.  Multiple rsync processes writing 
concurrently to the destination generally won't make a terrible mess
since each process updates a file by writing the new data to a temporary
file and moving it over the original, but the processes may waste
resources by duplicating each other's work.  If you're using --delete,
the processes might delete each other's temporary files, which could
cause a problem; to avoid that, either protect the temporary files with
--filter=P_.*.?????? or have each process use its own --temp-dir outside
the destination.
> Would running an rsync daemon on the third web 
> server versus running it under an ssh shell make any difference to this? 
The rsync daemon has a "max connections" setting that you could use to
prevent the first and second hosts from pushing at the same time.

Matt