rsync - Nov 2001 - Rsync: Re: patch to enable faster mirroring of large filesyst ems

I was at first, but then removed it. The results were still insufficiently
fast.
> Were you using the -c option of rsync?  It sounds like you 
> were and it's
> extremely slow.  I knew somebody who once went to 
> extraordinary lengths to
> avoid the overhead of -c, making a big patch to rsync to 
> cache checksums,
> when all he had to do was not use -c.

On Thu, Nov 29, 2001 at 12:59:00PM -0600, Keating, Tim
wrote:
> I was at first, but then removed it. The results were still insufficiently
> fast.
> 
> > Were you using the -c option of rsync?  It sounds like you 
> > were and it's
> > extremely slow.  I knew somebody who once went to 
> > extraordinary lengths to
> > avoid the overhead of -c, making a big patch to rsync to 
> > cache checksums,
> > when all he had to do was not use -c.
23 minutes to check 3200 files is definitely unexpected.  What options did
you end up using?  Normally rsync will only check the modification
timestamps and the sizes of the files on both sides (that is, only a
stat()) and if they match it will not do anything else.

- Dave Dykstra

It seems to me the new options --read-batch and --write-batch should go 
a long way towards reducing any time spent in creation of checksums and
file lists, so you should definitely give 2.4.7pre4 a try.  This is just
a guess since I haven't actually used those options myself, but seems
worth looking into.

BTW, could we please have some real documentation about these options? 
What's
in the man page doesn't come nearly close to telling what is cached and
how to make use of it.  Some examples of how people are using this option
may be illuminating for those of us who don't have the time or inclination 
to figure it out from the code.


-- Alberto


In message
<2292DBED5A978A498EABCCE95524499E0281D89C@osi-postal.osi.ad.ea.com>,
 "Keating, Tim" writes:
> I was at first, but then removed it. The results were still insufficiently
> fast.
> 
> > Were you using the -c option of rsync?  It sounds like you 
> > were and it's
> > extremely slow.  I knew somebody who once went to 
> > extraordinary lengths to
> > avoid the overhead of -c, making a big patch to rsync to 
> > cache checksums,
> > when all he had to do was not use -c.
> 


****************************************************************************
Alberto Accomazzi                          mailto:aaccomazzi@cfa.harvard.edu
NASA Astrophysics Data System                      http://adsabs.harvard.edu
Harvard-Smithsonian Center for Astrophysics        http://cfawww.harvard.edu
60 Garden Street, MS 83, Cambridge, MA 02138 USA   
****************************************************************************