Jason Usher
2011-Sep-29 23:57 UTC
rsync (or rsync-like tool) provides local metadata cache ?
Let's say I have a slow, or high latency (or both) link between two hosts, and transferring data is expensive. Is it possible with rsync (or any other tool like rsync) to maintain a local metadata cache, such that I can rsync without transferring data to and from the hosts about what files are already in place ? That is, I already "know" what is out there and don't need to check anything - I can just use the state information on the sending end to send only the data required, and don't need to traverse the remote filesystem for sizes/datestamps/hashes/etc. Does this exist ?
Kevin Korb
2011-Sep-29 23:58 UTC
rsync (or rsync-like tool) provides local metadata cache ?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You want something real time like lsyncd. On 09/29/11 19:57, Jason Usher wrote:> Let's say I have a slow, or high latency (or both) link between two hosts, and transferring data is expensive. > > Is it possible with rsync (or any other tool like rsync) to maintain a local metadata cache, such that I can rsync without transferring data to and from the hosts about what files are already in place ? > > That is, I already "know" what is out there and don't need to check anything - I can just use the state information on the sending end to send only the data required, and don't need to traverse the remote filesystem for sizes/datestamps/hashes/etc. > > Does this exist ?- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6FBhgACgkQVKC1jlbQAQe4cgCg91qePrvo0j5XvEgOxMOBEoHo P70AnRHby3Rd/wihPZXY3rLOXtCVZBl9 =GiFl -----END PGP SIGNATURE-----
Johannes Totz
2011-Sep-30 14:16 UTC
rsync (or rsync-like tool) provides local metadata cache ?
On 30/09/2011 00:57, Jason Usher wrote:> Let's say I have a slow, or high latency (or both) link between two > hosts, and transferring data is expensive. > > Is it possible with rsync (or any other tool like rsync) to maintain > a local metadata cache, such that I can rsync without transferring > data to and from the hosts about what files are already in place ? > > That is, I already "know" what is out there and don't need to check > anything - I can just use the state information on the sending end to > send only the data required, and don't need to traverse the remote > filesystem for sizes/datestamps/hashes/etc. > > Does this exist ?Check out the checksum database patches. Maybe these could help in your case...