rsync - Jul 2006 - Max number of files to transfer

Hey,

Is it possible to make rsync only transfer a maximum amount of files at 
one time. I want to do this to make the initial set up time of the 
filelist before transfer to be reasonably predictable.

Rsync is basically going to transfer from one mailbox to another mailbox 
but I want to be able to only transfer say 50 files at a time. I know 
this can be solved by controlling the number of files in the source 
directory but was wondering if there is anyway to achieve this using 
rsync itself.

Thanks in advance,
Adrian

On Thu, Jul 06, 2006 at 10:29:10AM +0200, Adrian Cooke
wrote:
> Is it possible to make rsync only transfer a maximum amount of files
> at one time?
No, that is not something that rsync supports.  It would be fairly easy
to add such a check into the generator.c code, but I'm not presently
inclined to include this in the regular release, so this would need to
be something that you patch into your own version of rsync.

..wayne..

Is there a version of Rsync for Openvms?

Wayne Davison wrote:
> On Thu, Jul 06, 2006 at 01:59:54PM -0400, Bob Bagwill wrote:
>> You probably already considered this, but you could also do something
>> like this: [...]
> 
> That divides the entire file-list up into 50-file chunks, whereas I
> believe that the original poster wanted to allow 50 differing files to
> be updated before quitting (out of 50+N files checked for being
> up-to-date).
Yes this is correct. The link that is being used is the limiting factor 
and the time that the link is available is also a limiting factor. So I 
want to be able to quickly startup rsync(by limiting the file scanning) 
and then transfer those in one go, and not create and terminate rsync 
sessions as I go.
> If the file-system scan is not the limiting factor that is slowing
> things down, then running rsync with --dry-run could be used to output a
> list of the files that need to be updated, and then that list could be
> divided into groups of 50 and iterated over.
> 
> ..wayne..
I dont think this wont really work because it will take to long to run 
because the link is to slow. This is because with --dry-run rsync still 
has to talk to the remote host if I understand it correctly and I do not 
want this.

For the remote to local transfer the problem should be solved by using a 
command to be passed to the remote shell as described in 
http://rsync.samba.org/examples.html "Fancy footwork with remote file 
lists". For the local to remote transfer I will take a look at the src 
or will just keep two directories and transfer the correct amount of 
files into the directory that I am going to rsync before sending.

Once the files are placed in the directory they will not change. In a 
previous post Matt McCutchen said that I can use --append to improve 
this start up time.

In the man page it says that this assumes that if the destination file 
is shorter than the source file it will then assume that the rest of the 
file must simply be appended to the file in the destination. Will this 
always be the case here because the source file never changes? It says 
that it implies --inplace but I need to used --partial-dir, will this 
cause a conflict?

Cheers,

Adrian