Hello, I use rsync v3.0.7 with "--delete-during", but it appears to act more like "--delete-before". I've checked the docs and list archives, but I am still discovering some of the subtleties of rsync, and I can't see where I'm going wrong. There's an example of my issue below... can anybody set me straight? =-) o We archive a small number of large files, each in its own directory. Both storage and bandwidth are limited. o We keep only one edition of each file at any given time. The filenames change with every revision. o We want to minimize the number of incomplete files at any point during the lengthy transfer. o Given more space, I would simply use "--delete-delay", or keep multiple file revisions. It looks like "--delete-during" should operate in-fix, i.e. step into a directory, delete the old file, pull the new file, and then move on to the next directory. For my transfers, all deletes seem to take place up front. The destination directory is emptied at the beginning of the session, which is what I want to avoid in case the link is severed, and the transfer is interrupted. Otherwise, rsync works as expected. Both ends use v3.0.7 packages from sunfreeware.com. The server config is pretty basic, and I see no errors on the source host. Am I looking at this the right way, or am I missing something simple? Let me know if you have any questions, and thanks in advance for any advice. -cheers, CSB P.S. My workaround is to run a separate rsync session for each subdirectory, followed by a final run from the top level. That seems to work alright, but it's rather a kludge. -c -------------------------------- desthost $ ls -l */* -rw-r--r-- 1 user group 1048576 Sep 1 16:32 0/file-0 -rw-r--r-- 1 user group 1048576 Sep 1 16:32 1/file-1 -rw-r--r-- 1 user group 1048576 Sep 1 16:32 2/file-2 -rw-r--r-- 1 user group 1048576 Sep 1 16:32 3/file-3 desthost $ rsync -a --delete-during -i srchost::del-test . *deleting 0/file-0 *deleting 1/file-1 *deleting 2/file-2 *deleting 3/file-3 .d..t...... 0/>f+++++++++ 0/file-00.d..t...... 1/>f+++++++++ 1/file-11.d..t...... 2/>f+++++++++ 2/file-22.d..t...... 3/>f+++++++++ 3/file-33desthost $ ls -l */* -rw-r--r-- 1 user group 1048576 Sep 1 16:32 0/file-00 -rw-r--r-- 1 user group 1048576 Sep 1 16:32 1/file-11 -rw-r--r-- 1 user group 1048576 Sep 1 16:32 2/file-22 -rw-r--r-- 1 user group 1048576 Sep 1 16:32 3/file-33 -------------------------------- srchost $ cat /etc/rsyncd.conf pid file = /var/run/rsyncd.pid address = srchost hosts allow = desthost log file = /var/log/rsync.log [del-test] comment = Try to delete files infix path = /var/tmp/del-test --------------------------------
On 10-09-09 01:07, Craig Bell wrote:> > Hello, I use rsync v3.0.7 with "--delete-during", but it appears to act more like "--delete-before". I've checked the docs and list archives, but I am still discovering some of the subtleties of rsync, and I can't see where I'm going wrong. There's an example of my issue below... can anybody set me straight? =-)Functionally yes. In more details, if you for example do rsync A -> B with --delete-before, rsync will do: 1) 1st delete pass, honoring global (on B or sent from A, if rsync was executed on A) and per directory (on B) rules of what to protect/delete (see --filter's options R & P) 2) 2nd transfer pass If you used --delete-after - the order of the above would be reversed - first transfer (which would have potentially useful effect of first transferring [updated] per directory rules from A -> B, and then using them for deletion during 2nd pass (delete). If you used --delete-during - rsync would combine 1st and 2nd pass of --delete-before into single pass, where deletion and transfer would be done on directory per directory basis, recursively. --delete-delay combined with --delete during would alter that combined pass in the following way - on a dir-by-dir basis: 1) find what to delete 2) transfer files 3) delete This is almost like --delete-after, but it won't work if you expect updated per directory rules to be used for delete pass.> > It looks like "--delete-during" should operate in-fix, i.e. step into a directory, delete the old file, pull the new file, and then move on to the next directory. For my transfers, all deletes seem to take place up front. The destination directory is emptied at the beginning of the session, which is what I want to avoid in case the link is severed, and the transfer is interrupted.--delete-delay is probably what you're after. Also check options: --ignore-existing, --ignore-non-exisitng, --partial, --delay-updates
On 08.09.2010 23:07, Craig Bell wrote:> > Hello, I use rsync v3.0.7 with "--delete-during", but it appears to > act more like "--delete-before". I've checked the docs and list > archives, but I am still discovering some of the subtleties of > rsync, and I can't see where I'm going wrong. There's an example of > my issue below... can anybody set me straight? =-)My observation of current rsync in "Incremental Recursion"-Mode (default), with default "--delete", is that the files it deletes are about 1000 files further down the list than the file it currently transfers. So question is: Are there more than 1000 files in total? Bis denn -- Real Programmers consider "what you see is what you get" to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a "you asked for it, you got it" text editor -- complicated, cryptic, powerful, unforgiving, dangerous.
Carlos Carvalho wrote:> Did you try with --no-i-r? In your somewhat particular case it might do what you want.Thanks Carlos, I tried --no-i-r with my test data, but it still effectively deletes before. -cheers, CSB
Michal Soltys wrote:> Try for example something analogous to:Thanks Michal, I have a workaround that provides the behavior that I want to see: for dir in 0 1 2 3 ; do rsync -a --del srchost::del-test/${dir} . done rsync -a --del srchost::del-test .> On a related note, if you call rsync locally, you will see strict boundariesThat's interesting! I tried this, --del does indeed behave strictly "infix" when both paths are local. Output below -- this is exactly what I would like to see for the remote transfer. Thx... -cheers, CSB $ rsync -i -a --del ./rsync-test ./rsync-test-2 .d..t...... rsync-test/0/ *deleting rsync-test/0/file-0>f+++++++++ rsync-test/0/file-00.d..t...... rsync-test/1/ *deleting rsync-test/1/file-1>f+++++++++ rsync-test/1/file-11.d..t...... rsync-test/2/ *deleting rsync-test/2/file-2>f+++++++++ rsync-test/2/file-22.d..t...... rsync-test/3/ *deleting rsync-test/3/file-3>f+++++++++ rsync-test/3/file-33
Hi, Not sure if post below made it to the list, so re-sending. Apologies if you already got this. On Sep 10, 2:29 pm, Craig.B...<http://groups.google.com/groups/unlock?_done=/group/mailing.unix.rsync/browse_thread/thread/1c489b0c136e09d7&msg=a40df0e3b5329994> @standard.com (Craig Bell) wrote:> Michal Soltys wrote:<snip>> > On a related note, if you call rsync locally, you will see strictboundaries> That's interesting! I tried this, --del does indeed behave strictly"infix" when both paths are local.> Output below -- this is exactly what I would like to see for the remotetransfer. Thx... -cheers, CSB Hi, Is NFS mount of destination a possibility? Would NFS mount give the same behavior as "rsync locally"? Any shortcomings in an NFS mount/rsync local? Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20100914/1de82ade/attachment.html>