has someone come up with a trick to let disconnected ssh connections be recovered without terminating and having to restart rsync (perhaps by wrapping ssh or something)? i have a very large (but pretty stable/unchanging) tree of files that i need to sync daily, at night, between two geographically different servers. i'm using rsync -avz --force -e ssh SOURCE DEST. the connection is pretty slow and quite unreliable and many times the rsync process is terminated in the middle because of disconnected tcp connection. restarting rsync would begin the process of building the filelist which is pretty slow. -- dave
On Tue, Jan 28, 2003 at 06:41:20AM -0700, David Garamond wrote:> has someone come up with a trick to let disconnected ssh connections be > recovered without terminating and having to restart rsync (perhaps by > wrapping ssh or something)? > > i have a very large (but pretty stable/unchanging) tree of files that i > need to sync daily, at night, between two geographically different > servers. i'm using rsync -avz --force -e ssh SOURCE DEST. the connection > is pretty slow and quite unreliable and many times the rsync process is > terminated in the middle because of disconnected tcp connection. > restarting rsync would begin the process of building the filelist which > is pretty slow.Not really doable. When the connection is lost the remote process dies. Even rsyncd couldn't sensibly support a reconnect. There is just too much context to restart the remote end without restarting the local as well. What i'd look into doing in your case would be to break the tree up into smaller jobs that would run quicker and be less likely to be fail. Wrap each sub-job in a loop that repeats on failure. The --from-files option (patch form only for now) could have some possibilities here but you'd still need to identify deleted files. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
has someone come up with a trick to let disconnected ssh connections be recovered without terminating and having to restart rsync (perhaps by wrapping ssh or something)? i have a very large (but pretty stable/unchanging) tree of files that i need to sync daily, at night, between two geographically different servers. i'm using rsync -avz --force -e ssh SOURCE DEST. the connection is pretty slow and quite unreliable and many times the rsync process is terminated in the middle because of disconnected tcp connection. restarting rsync would begin the process of building the filelist which is pretty slow. -- dave
On Tue, 28 Jan 2003, David Garamond wrote [twice!]:> has someone come up with a trick to let disconnected ssh connections be > recovered without terminating and having to restart rsync (perhaps by > wrapping ssh or something)?Maybe my google skills are lacking, but I couldn't find anything. Both Xvnc and screen support reconnecting interactive sessions, so there's no reason we couldn't do the same sort of thing for ssh. It's a bit more complicated because we need to save the state at both ends. We also need some different options, how often to try to reconnect, when to just give up, what ways to try to connect, etc... Performance isn't much of an issue, since we'd only need it for unreliable networks, and those are usually the slow ones. Would rsync do anything odd if a network write took 4 hours? The wrapper around ssh would prevent rsync from seeing any errors, but it wouldn't hide the timing. Are there any timeouts in rsync? Assuming it hasn't already been done, my thought would be to see how screen works and model the remote end on it. The local side would be a mirror of the remote end with the addition of some code to (re)start and end sessions. If I wrote it, the wrapper would be GPL'd, so I can freely borrow code from screen. -- Paulh
On 30 Jan 2003, David Garamond <davegaramond@icqmail.com> wrote:> has someone come up with a trick to let disconnected ssh connections be > recovered without terminating and having to restart rsync (perhaps by > wrapping ssh or something)?Ooh, interesting idea... You might do it with some kind of wrapper at both ends... Alternatively, by changing ssh options perhaps you can get the process to stay open even if the link goes away, by increasing timeouts and so on... -- Martin