rsync - Dec 2005 - [Bug 3304] New: on --delay-updates remove .~tmp~ recursively

https://bugzilla.samba.org/show_bug.cgi?id=3304

           Summary: on --delay-updates remove .~tmp~ recursively
           Product: rsync
           Version: 2.6.6
          Platform: x64
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P3
         Component: core
        AssignedTo: wayned@samba.org
        ReportedBy: olleg_s@mail.ru
         QAContact: rsync-qa@samba.org


On --delay-updates delayed files saved in .~tmp~. On error (disconnect, server
or client killed) Delayed files keeped in .~tmp~. (May be better remove it on
disconnect, but this is another bug and don't concerned client kill or
poweroff
client station.) On next rsync they are rewriten and .~tmp~ removed. But if one
of delayed files removed on server side before next rsync, it (in .~tmp~) and
.~tmp~ don't removed after next rsync.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=3304


robe@amd.co.at changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |robe@amd.co.at




------- Comment #1 from robe@amd.co.at  2006-01-16 07:47 MST -------
and here an easy reproducible testcase:

mirror:/tmp/rsynctest# mkdir src dest
mirror:/tmp/rsynctest# echo foo > src/abc
mirror:/tmp/rsynctest# rsync -av --delay-updates --delete src/ dest/
building file list ... done
./
abc

sent 113 bytes  received 48 bytes  322.00 bytes/sec
total size is 4  speedup is 0.02
mirror:/tmp/rsynctest# mkdir dest/.~tmp~
mirror:/tmp/rsynctest# rsync -av --delay-updates --delete src/ dest/
building file list ... done
./

sent 63 bytes  received 26 bytes  178.00 bytes/sec
total size is 4  speedup is 0.04
mirror:/tmp/rsynctest# ls -la dest/
total 16
drwxr-xr-x  3 root root 4096 Jan 16 15:45 .
drwxr-xr-x  4 root root 4096 Jan 16 15:45 ..
drwxr-xr-x  2 root root 4096 Jan 16 15:46 .~tmp~
-rw-r--r--  1 root root    4 Jan 16 15:45 abc
mirror:/tmp/rsynctest#


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=3304


wayned@samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED




------- Comment #2 from wayned@samba.org  2006-01-17 12:38 MST -------
It might not be safe for rsync to delete a non-empty .~tmp~ dir if multiple
rsync commands are running at the same time (hopefully updating different
files).  One could argue that two simultaneously running rsyncs should really
be using different partial-dir settings to avoid other potential failures (such
as an empty directory being removed by one rsync right before the other can
open a temp file in it).  I'll consider allowing this, but it might be
better
to make this an option, or to only enable it if a custom --partial-dir name is
chosen.

As for your example, rsync does not remove the empty .~tmp~ dir because it
didn't use it during that run.  If rsync had updated a file in the
destination
directory, it would have removed the dir (as long as it was empty).  This
behavior is certainly debatable, and might change in the future.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=3304





------- Comment #3 from robe@amd.co.at  2006-01-17 14:01 MST -------
Perfectly understandable. As for now a hint in the in the manpage (or at
runtime, when 'stale' .~tmp~ directories are encountered) should
suffice, since
the current behaviour can lead to unexpected (and unexplainable) disk-space
"loss".

At least my (and probably others too) assumption of rsync was, that a
"rsync
--delete src/ dest/" ensured that the two trees are identical, and
--delay-updates silently breaks this assumption.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=3304





------- Comment #4 from hashproduct@verizon.net  2006-01-17 14:56 MST -------
Perhaps the special treatment of the .~tmp~ directory could be implemented by
placing a receiving-end protect filter on any files inside it.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=3304





------- Comment #5 from hashproduct@verizon.net  2006-01-19 13:15 MST -------
Surprise: according to the man page of 2.6.6, a receiving-end protect filter is
indeed placed on the partial dir, which is the same as the delay-updates dir. 
Never mind comment #4.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=3304


wayned@samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED




------- Comment #6 from wayned@samba.org  2006-02-03 16:51 MST -------
I have checked-in some doc changes that talk about what the user can do to
cleanup old partial-dirs.  For instance, the user can specify a combination of
--delete-after with a "risk" filter rule:

rsync -ai --delay-updates -f 'R .~tmp~/' --delete-after src/ dest/

And rsync will no longer exclude the .~tmp~ dirs from being deleted at the end
of the run.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.