Hello, This is my first posting to the rsync list. I mirror a database containing directories which contain a very large number of files (say 30,000), and sending the file list can often take longer than transferring the new files. (Rsync ends up sending nearly the same file list on every transfer, with only the addition of a few new files.) Has the rsync team considered an rsync option which would remember the last file list on both ends, and only send changes to the list? In my case this would reduce the number of files in the file list from 300,000 per day to under 1,000 per day. Might be a good addition to rsync-3.0. Thanks in advance. Peter Salameh psalameh@ucsd.edu
On 10/1/07, Peter Salameh <psalameh@ucsd.edu> wrote:> Has the rsync team considered an rsync option which would remember the > last file list on both ends, and only send changes to the list?Perhaps a more natural approach is to use the delta-transfer algorithm to send the file list. Jamie Lokier suggested this approach here: http://lists.samba.org/archive/rsync/2007-August/018345.html Matt
I think the next release is supposed to speed this up. _____________________________ Stephen Zemlicka Integrated Computer Technologies PH. 608-558-5926 E-Mail stevezemlicka@gmail.com -----Original Message----- From: rsync-bounces+stevezemlicka=gmail.com@lists.samba.org [mailto:rsync-bounces+stevezemlicka=gmail.com@lists.samba.org] On Behalf Of Peter Salameh Sent: Monday, October 01, 2007 5:42 PM To: rsync@lists.samba.org Subject: reducing file list bytes transferred Hello, This is my first posting to the rsync list. I mirror a database containing directories which contain a very large number of files (say 30,000), and sending the file list can often take longer than transferring the new files. (Rsync ends up sending nearly the same file list on every transfer, with only the addition of a few new files.) Has the rsync team considered an rsync option which would remember the last file list on both ends, and only send changes to the list? In my case this would reduce the number of files in the file list from 300,000 per day to under 1,000 per day. Might be a good addition to rsync-3.0. Thanks in advance. Peter Salameh psalameh@ucsd.edu -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
On 10/1/07, Stephen Zemlicka <stevezemlicka@gmail.com> wrote:> I think the next release is supposed to speed this up.Are you thinking of incremental recursion? It interleaves file list and file data transmission but does not decrease the total amount of file-list data sent. Matt
I thought it was supposed to do it more effeciently _____________________________ Stephen Zemlicka Integrated Computer Technologies PH. 608-558-5926 E-Mail stevezemlicka@gmail.com -----Original Message----- From: hashproduct@gmail.com [mailto:hashproduct@gmail.com] On Behalf Of Matt McCutchen Sent: Monday, October 01, 2007 8:10 PM To: Stephen Zemlicka Cc: Peter Salameh; rsync@lists.samba.org Subject: Re: reducing file list bytes transferred On 10/1/07, Stephen Zemlicka <stevezemlicka@gmail.com> wrote:> I think the next release is supposed to speed this up.Are you thinking of incremental recursion? It interleaves file list and file data transmission but does not decrease the total amount of file-list data sent. Matt
On 10/1/07, Stephen Zemlicka <stevezemlicka@gmail.com> wrote:> I thought it was supposed to do it more effecientlyNot that I know of. (Wayne, please correct me if necessary.) Matt