rsync - Jan 2004 - Performance problem

Hi, I am running rsync version 2.5.7 (stock distro on redhat linux, ES
and 9) to rsync a directory with ~300k files in it from a machine on a
10Mbit internet tap to a machine with a 100Mbit internet tap. The
problem is that I am only getting about 500Kbps during the transfer. I
have tested link speed by scping a file between the two systems,
performance was approximately 7.5Mbit/sec.  Ping time between the two
locations is about 2.5 ms (only 20 miles apart)

This is obviously a problem. I have also tried rsyncing just a few files
(say 400) to rule out problems related to huge file lists, however I had
the same problem. 

Is this a known problem? Am I missing something really obvious?

I have tried testing drive performance in each machine and have found it
to be acceptable. 

The line that I am using to do the rsync is the following:

rsync -avvu --exclude-from=/root/sync_list.exclude --delete -e ssh
--stats /data/ dest:/data/


Any idea?

 -Andrew

-- 
Andrew E. Boyce-Lewis
Conduit-IT
Phone: 	800 493-5045 x 210
Fax:	(814) 867-8248
Cell:   (814) 360-6868
GPG key available at http://dev.conduit-it.com/~aboyce/aboyce.gpg

Please AVOID sending me WORD, EXCEL or POWERPOINT attachments.
See http://www.fsf.org/philosophy/no-word-attachments.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.samba.org/archive/rsync/attachments/20040112/308405ed/attachment.bin

On Mon, Jan 12, 2004 at 06:22:04PM -0500, Andrew Boyce-Lewis
wrote:
> Hi, I am running rsync version 2.5.7 (stock distro on redhat linux, ES
> and 9) to rsync a directory with ~300k files in it from a machine on a
> 10Mbit internet tap to a machine with a 100Mbit internet tap. The
> problem is that I am only getting about 500Kbps during the transfer. I
> have tested link speed by scping a file between the two systems,
> performance was approximately 7.5Mbit/sec.  Ping time between the two
> locations is about 2.5 ms (only 20 miles apart)
> 
> This is obviously a problem. I have also tried rsyncing just a few files
It is?  Rsync and scp do two completely different things and
their performance is not comparable.
> (say 400) to rule out problems related to huge file lists, however I had
> the same problem. 
> 
> Is this a known problem? Am I missing something really obvious?
> 
> I have tried testing drive performance in each machine and have found it
> to be acceptable. 
But how is the filesystem performance at doing what rsync
does to it?  Raw sequential throughput is irrelevant.
> The line that I am using to do the rsync is the following:
> 
> rsync -avvu --exclude-from=/root/sync_list.exclude --delete -e ssh
> --stats /data/ dest:/data/
Try --whole-file;  if that improves things you are disk bound.

Drop to just one -v; -vv is for debugging and adversely
affects network buffering.

It probably isn't your exclude file but a large exclude file
would add overhead.

2.5.7 is a security release with minimal changes from 2.5.6
which is almost a year old.  There have been a lot of
changes since then most of which are in 2.6.0.  Since 2.6.0
there have been a number of changes related to performance
including one that dramatically improves the network
performance.  All of this you would have known if you had
read the list archives or checked NEWS.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt