rsync - Mar 2010 - DO NOT REPLY [Bug 7194] New: Getting --inplace and --sparse to work together

https://bugzilla.samba.org/show_bug.cgi?id=7194

           Summary: Getting --inplace and --sparse to work together
           Product: rsync
           Version: 3.0.7
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P3
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: jansen at webgods.de
         QAContact: rsync-qa at samba.org


I created a small patch to get --inplace and --sparse to work together. The
main problem with this is, as far as I can see, that the write code just skips
blocks of zeroes instead of writing them, as is requested by the --sparse
switch. If the target already exists and has data in places where the source
has zeroes, this data will never get erased.
My patch simply reads these regions first and makes sure they really are zero.
If they are not, it clears them.
It would be good if someone with a deeper knowledge of the implications could
have a look at the patch to see if I missed something.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7194





------- Comment #1 from jansen at webgods.de  2010-03-02 01:44 CST -------
Created an attachment (id=5440)
 --> (https://bugzilla.samba.org/attachment.cgi?id=5440&action=view)
Patch as described


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7194


ildar at altlinux.ru changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ildar at altlinux.ru




------- Comment #2 from ildar at altlinux.ru  2010-11-05 15:40 CST -------
https://bugzilla.samba.org/show_bug.cgi?id=7778 might be related


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7194


jansen at webgods.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
Attachment #5440 is|0                           |1
           obsolete|                            |




------- Comment #3 from jansen at webgods.de  2010-11-12 01:46 CST -------
Created an attachment (id=6065)
 --> (https://bugzilla.samba.org/attachment.cgi?id=6065&action=view)
updated patch

This updated patch contains a small bugfix. With this version we have
successfully copied many TB of data snapshot-by-snapshot and verified the
result with gdiff.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7194


wayned at samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED




------- Comment #4 from wayned at samba.org  2011-01-13 09:53 CST -------
An improvement like this seems good for 3.1.0.  Your patch looks nice, but I do
want to make sure that it is leveraging knowledge from the generator -- if
we're doing a normal (not whole-file) transfer, then a seek over identical
bytes doesn't need to read the bytes again.  Does you patch take that into
consideration?

Also, there's already a change in the 3.1.0dev code in sparse_end() that
replaces that write-a-0-at-the-end-of-the-file code with an ftruncate() call
(if ftruncate() is available), which will avoid allocating an extra block at
the end of the file to hold the 0.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7194





------- Comment #5 from jansen at webgods.de  2011-01-14 08:39 CST -------
I assume that when write_file is called, the receiver already has determined
that the buffer has to be written to disk. In case of inplace, we cannot assume
that the file contains zeros in this location, nor it's sparse. So we need
to
make sure it really contains what we require.
I don't see how any information from the generator can help to avoid this
step.
Can you please elaborate on your question?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7194

--- Comment #6 from Arne Jansen <jansen at webgods.de> 2011-05-30 08:51:05
UTC ---
Is it ready to be pulled into 3.1, or do you need anything else?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7194

--- Comment #7 from Ildar Muyukov <ildar at altlinux.ru> 2011-11-07
07:23:06 UTC ---
Wayne?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7194

--- Comment #8 from David Taylor <davidt-samba-bugzilla at yadt.co.uk>
2013-09-08 09:50:35 UTC ---
Is there any update available on what's stopping this patch being merged?

It would be very useful...

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7194

David Taylor <davidt-samba-bugzilla at yadt.co.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |davidt-samba-bugzilla at yadt.
                   |                            |co.uk

--- Comment #9 from David Taylor <davidt-samba-bugzilla at yadt.co.uk>
2013-09-08 09:53:13 UTC ---
Is there any update available on what's stopping this patch being merged?

It would be very useful...

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.