rsync - Dec 2013 - Making --inplace and --sparse compatible

It seems to me that it should be possible to make --inplace and
--sparse compatible.

The manpage says

    -S, --sparse
           ... Conflicts with --inplace because it's not
           possible to overwrite data in a sparse fashion.

but I think it _is_ possible to do that, provided the receiving
side opens the dest file read-write instead of write-only.

When writing within the existing length of the dest, it should be
possible -- when processing a src block which is all zero and thus
would be seeked over instead of written when writing sparsely to a
file being newly created -- to read that block from the dest and
refrain from writing it back if it is already zero in the dest file.
(Of course, if it is _not_ all zero in the dest, the non-zero bytes
would have to be overwritten with zeros.)

Granted this approach will not introduce sparseness into an existing
non-sparse dest, but it at least will preserve any sparseness which
already exists in the dest.

I'm currently using 3.0.7, but I think the same issue still exists
in 3.1.0 since there's no mention of such a change in the release
notes and the 3.1.0 manpage still documents the same limitation.

Thoughts?

On 10.12.2013 21:05, Perry Hutchison wrote:
> It seems to me that it should be possible to make --inplace and
> --sparse compatible.
> 
> The manpage says
> 
>     -S, --sparse
>            ... Conflicts with --inplace because it's not
>            possible to overwrite data in a sparse fashion.
> 
> but I think it _is_ possible to do that, provided the receiving
> side opens the dest file read-write instead of write-only.
> 
> When writing within the existing length of the dest, it should be
> possible -- when processing a src block which is all zero and thus
> would be seeked over instead of written when writing sparsely to a
> file being newly created -- to read that block from the dest and
> refrain from writing it back if it is already zero in the dest file.
> (Of course, if it is _not_ all zero in the dest, the non-zero bytes
> would have to be overwritten with zeros.)
> 
> Granted this approach will not introduce sparseness into an existing
> non-sparse dest, but it at least will preserve any sparseness which
> already exists in the dest.
> 
> I'm currently using 3.0.7, but I think the same issue still exists
> in 3.1.0 since there's no mention of such a change in the release
> notes and the 3.1.0 manpage still documents the same limitation.
And there is hole-punching nowadays (at least for linux and it should 
work on the major filesystems. Not sure about others). So a file can be 
made sparse after the fact.

And nowadays (linux again) there is fiemap and the older(?) fibmap and 
the newest addition seek_hole/seek_data to determine sparseness.
Altough i'm unsure which work for which filesystems and also if support 
is mainlined or still in development for the major filesytems.

So nowadays or in the near future (at least for linux with the major 
filesystems) determining sparseness and creating sparseness "after the 
fact" is possible.



-- 

Matthias