rsync - Aug 2012 - Cache file list in daemon mode?

Hello.  I suspect that what I want to do is not possible with rsync, but 
this is the best place to double-check.

We are pushing files to a remote target that stores them on a very slow 
network file system.  There are also over a million files on the 
target.  Consequently, running rsync to push an update takes hours while 
the remote side enumerates and stats all those files.

I thought, that wouldn't be necessary if the remote side was running 
rsync in daemon mode, and that it only built its internal map of the 
files there once after startup, thereafter updating that in-memory list 
with every push it receives.  For that to work, there would have to be 
some flag I could set to promise rsyncd that no files in the target 
would be updated through any means other than rsyncd.  I looked for such 
an option, did some experimenting with write-only targets, straced the 
daemon and saw I wasn't getting anywhere.

Our alternatives are options like building a list of what has changed 
and copying only those things across.  There's some housekeeping and 
race condition avoidance in there that makes it more than a couple of 
simple commands and replicates some of what rsync knows how to do 
already.  By any chance is what I want to do possible with rsyncd, or 
some other tool?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

is it possible to talk directly to the NFS server via rsyncd or rsync
over ssh?  Eliminating the extra hop through a network mount should
make a big difference.

On 08/02/12 18:30, Peter Scott wrote:
> Hello.  I suspect that what I want to do is not possible with
> rsync, but this is the best place to double-check.
> 
> We are pushing files to a remote target that stores them on a very
> slow network file system.  There are also over a million files on
> the target.  Consequently, running rsync to push an update takes
> hours while the remote side enumerates and stats all those files.
> 
> I thought, that wouldn't be necessary if the remote side was
> running rsync in daemon mode, and that it only built its internal
> map of the files there once after startup, thereafter updating that
> in-memory list with every push it receives.  For that to work,
> there would have to be some flag I could set to promise rsyncd that
> no files in the target would be updated through any means other
> than rsyncd.  I looked for such an option, did some experimenting
> with write-only targets, straced the daemon and saw I wasn't
> getting anywhere.
> 
> Our alternatives are options like building a list of what has
> changed and copying only those things across.  There's some
> housekeeping and race condition avoidance in there that makes it
> more than a couple of simple commands and replicates some of what
> rsync knows how to do already.  By any chance is what I want to do
> possible with rsyncd, or some other tool?
- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlAbApcACgkQVKC1jlbQAQcYHgCg6HYJATQn4kf/duAC+owxPq5w
TNAAoNFXN8MAcKbBEbXk2cKgF/4MNgE5
=D9DM
-----END PGP SIGNATURE-----