rsync - Dec 2006 - How to improve speed of a single file transfer over an unstable link?

Long time fan and user of rsync, with a question. First some background...

What I'm trying to do is copy a 21GB backup file from my parent's house
to
my home to help them with their new computer. But the link is sporadic,
thus a continous copy will not succeed. The session dies after 25mins - 2
hours.

Thus, I've scripted the following script 'get_me.sh'

   #!/bin/sh
   mv .file.bkf.* ./file.bkf
   rsync --timeout 90 user@remote:/dir/file.bkf ./file.bkf
   ./getme.sh

So, the script moves the temp file created by rsync onto the file itself,
then calls rsync to continue sync'ing, and then after rsync losses it's
connection, the script calls itself and the cycle starts again.

Not very elegant but it's working. Sort of. I'm now starting to decrease
the overall throughput of the transfer since I keep checking to make sure
that the data is still the same on both side, so here's the question...

Would the "append" flag work well for this situation? I'd normally
try a
few tests myself, but according to my data, it'll be at least another 4
days until the file is finished, and my parent's leave in 5 days. So,
I'm
a little hesitant to "experiment" on the transfer in progress.

Thanks,

Scott
PS> I am joining the mailing list but, please cc me on replies for the
short term.

On Fri 29 Dec 2006, Scott C. Kennedy wrote:
> 
> Thus, I've scripted the following script 'get_me.sh'
> 
>    #!/bin/sh
>    mv .file.bkf.* ./file.bkf
>    rsync --timeout 90 user@remote:/dir/file.bkf ./file.bkf
>    ./getme.sh
> 
> So, the script moves the temp file created by rsync onto the file itself,
> then calls rsync to continue sync'ing, and then after rsync losses
it's
> connection, the script calls itself and the cycle starts again.
You do know about the --partial option? That basically takes care of
this...  although I'm wondering why your rsync doesn't delete the
tmpfile after the transfer is interrupted.
Using --inplace may also be useful.
> Not very elegant but it's working. Sort of. I'm now starting to
decrease
> the overall throughput of the transfer since I keep checking to make sure
> that the data is still the same on both side, so here's the question...
rsync will check the existing data upon the start of each transfer,
unless...
> Would the "append" flag work well for this situation? I'd
normally try a
> few tests myself, but according to my data, it'll be at least another 4
> days until the file is finished, and my parent's leave in 5 days. So,
I'm
> a little hesitant to "experiment" on the transfer in progress.
the --append option will assume that the partial data there is correct,
only too short. This is a good idea, as it saves reading the already
transported data, saving time. And --append implies --inplace.


Paul Slootman