rsync - Nov 2006 - Is it okay to rsync the same source to the same dest multiple times?

I'm doing a huge rsync from one net to another (about a terabyte) over
a dedicated DS-3.  One rsync doesn't even come close to saturating the
line, especially with compression enabled.  Is it okay to run multiple
rsyncs copying the same source to the same destination or do I have to
take care that they work on disjoint sections of the source directory
tree?

Thanks,

Skip Montanaro

On Wed, Nov 08, 2006 at 10:17:18PM +0000, Skip Montanaro
wrote:
> Is it okay to run multiple rsyncs copying the same source to the same
> destination or do I have to take care that they work on disjoint
> sections of the source directory tree?
If you run the same command multiple times, you risk having multiple
rsync working on transferring the same file at the same time.  This
wastes bandwidth, but shouldn't cause any long-term problems:  since
rsync will notice if the file didn't reconstruct properly, having
multiple people update the file might at most cause the file to get
resent needlessly.

If you can segment the list, or at least use some disjoint excludes so
that the processes get out of lockstep with each other, you'll be better
off.

If you are limited by CPU when running multiple rsyncs, you may need to
turn off compression.  If you are limited by disk I/O on the receiving
side, you can turn off the checksumming-rsync algorithm (--whole-file).
Experiment and see what works best for you.

..wayne..