rsync - Dec 2008 - Run rsync through intermediary server with SSH

I'm using rsync, ssh, and cron glued together with Python as a
push-based synchronization system. From a single location, I push
content out to various offices.  I log stdout/stderr on the master
server to make sure everything is running smoothly.

I would now like for some of our "regional hubs" to take on some of
the
load (bandwidth-wise), while still retaining my centralized master for
command execution and logging.  So far, I simply send the desired rsync
command as a remote ssh command.  It looks like this:

ssh gsync@cppdf "rsync -ritzOK --delete-after --timeout=600 --progress
--stats --verbose --bwlimit='350'
--exclude-from='/home/gsync/.gsync/config/office-filters/sr'
--filter='.
/home/gsync/.gsync/config/general-filters/junk' --filter=':-
gsync-filter' '/s/cad/Revit/'
gsync@sr-pdf:'\"/s/cad/Revit\"'"

The idea is that I can push to our regional offices, and then have them
push to their near-by peers using their own bandwidth, yet still collect
output at the master.

The only problem I'm running into is this:  if I kill the ssh session on
the master, the remote rsync process continues to run on the
intermediary server to the final destination.  This can lead to multiple
rsync jobs running from the intermediary to the same destination server,
flooding it.  I'm having difficulty figuring out what the appropriate
mojo is to ensure that the rsync jobs die with the ssh connections.
Hope this is clear.  Any advice?

Thanks,

Jimmie

You could setup SSH port forwarding to forward the rsync commands to your
remote servers, then you'll be sure rsync fails when ssh is killed. The
commands should be something like this:

ssh gsync@cppdf -L 10000:sr-pdf:873 &

rsync gsync@localhost --port 10000


See e.g.
http://www.ssh.com/support/documentation/online/ssh/adminguide/32/Port_Forwa
rding.html for more info.


Regards,

Jeroen van der Vegt	 
System designer 
________________________________

Technolution B.V.	 
Telephone:         +31(0)182 59 40 00	 
Fax:              +31(0)182 53 97 36	 
E-mail:           Jeroen.van.der.Vegt@technolution.eu	 
Visit us at:      www.technolution.eu	 
Mailing address:  P.O. Box 2013 - 2800 BD Gouda - The Netherlands	 
Address:          Zuidelijk Halfrond 1 - 2801 DD Gouda - The Netherlands

________________________________

This e-mail is intended exclusively for the addressee(s), and may not be
passed on to, or made available for use by any person other than the
addressee(s). Technolution B.V. rules out any and every liability resulting
from any electronic transmission. 
> -----Original Message-----
> From: rsync-bounces+jeroen.van.der.vegt=technolution.nl@lists.samba.org
> [mailto:rsync-bounces+jeroen.van.der.vegt=technolution.nl@lists.samba.org]
> On Behalf Of Jimmie Fulton
> Sent: dinsdag 9 december 2008 3:15
> To: rsync@lists.samba.org
> Subject: Run rsync through intermediary server with SSH
> 
> I'm using rsync, ssh, and cron glued together with Python as a
> push-based synchronization system. From a single location, I push
> content out to various offices.  I log stdout/stderr on the master
> server to make sure everything is running smoothly.
> 
> I would now like for some of our "regional hubs" to take on some
of the
> load (bandwidth-wise), while still retaining my centralized master for
> command execution and logging.  So far, I simply send the desired rsync
> command as a remote ssh command.  It looks like this:
> 
> ssh gsync@cppdf "rsync -ritzOK --delete-after --timeout=600 --progress
> --stats --verbose --bwlimit='350'
> --exclude-from='/home/gsync/.gsync/config/office-filters/sr'
--filter='.
> /home/gsync/.gsync/config/general-filters/junk' --filter=':-
> gsync-filter' '/s/cad/Revit/'
gsync@sr-pdf:'\"/s/cad/Revit\"'"
> 
> The idea is that I can push to our regional offices, and then have them
> push to their near-by peers using their own bandwidth, yet still collect
> output at the master.
> 
> The only problem I'm running into is this:  if I kill the ssh session
on
> the master, the remote rsync process continues to run on the
> intermediary server to the final destination.  This can lead to multiple
> rsync jobs running from the intermediary to the same destination server,
> flooding it.  I'm having difficulty figuring out what the appropriate
> mojo is to ensure that the rsync jobs die with the ssh connections.
> Hope this is clear.  Any advice?
> 
> Thanks,
> 
> Jimmie
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html