rsync - Mar 2003 - Local copying with minimal disk activity.

How do I minimize the amount of disk activity?

I'm copying a large number of files:

	rsync -av -W /from/here/. /to/here/.

I'm using rsync because I need the incremental updates and other features. 
Otherwise, I'd use tar piped to tar or some similar technique.

Note: Both the source and destination are "local".  Well, not really. 
The
source is a network-mounted partition (SAMBA), and the destination is a
real disk.

I'd like to minimize the amount of network activity since that seems to be
my bottleneck.  For example, without the "-W" option, every file is
read
twice (once to checksum, once to copy).

Are there other options that I could enable to speed up the copy?

Is using -W dangerous?  If I'm doing an incremental update (a file or two
changed on the source, so I need to recopy the data) will -W catch as much
as checksuming if the file timestamps change?

Thanks in advance!
--Tom

On Mon, Mar 03, 2003 at 10:56:15AM -0500, Tom Limoncelli
wrote:
> How do I minimize the amount of disk activity?
> 
> I'm copying a large number of files:
> 
> 	rsync -av -W /from/here/. /to/here/.
> 
> I'm using rsync because I need the incremental updates and other
features.
> Otherwise, I'd use tar piped to tar or some similar technique.
> 
> Note: Both the source and destination are "local".  Well, not
really.  The
> source is a network-mounted partition (SAMBA), and the destination is a
> real disk.
I rsync thinks it is running local->local it will use
whole-file anyway unless you specify --no-whole-file.
> I'd like to minimize the amount of network activity since that seems to
be
> my bottleneck.  For example, without the "-W" option, every file
is read
> twice (once to checksum, once to copy).
The double-read only applies to the destination, not the source.
> Are there other options that I could enable to speed up the copy?
Install rsync on the source and eliminate smb.
> Is using -W dangerous?  If I'm doing an incremental update (a file or
two
> changed on the source, so I need to recopy the data) will -W catch as much
> as checksuming if the file timestamps change?
-W is not dangerous.  Since rsync does a checksum as part of
the transfer it is less dangerous than other means.

With --whole-file the sender disk load is about as small as
it could be.  (Speaking only for the sender) We walk the tree
and stat(2) each file, then we read each file that the
stat(2) info indicates as changed.  This compares favorably
with tar and other methods which must do at least that much.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt