rsync - Sep 2023 - rsync --delete with empty source folder for fast snapshot deletion: Permissions of hardlinked files are changed to 644. Workaround?

Context
-------

I am one of the active developers of the open source application "Back in
Time"
which uses "rsync" as backend and I want to fix an open issue:

    "Back in Time"-Bug:
    https://github.com/bit-team/backintime/issues/994#issuecomment-1724211507

"Back in Time" uses "--link-dest" to reduce traffic and
storage by hardlinking
unchanged files in the backups/snapshots and tries to keep the permissions
by using "--perms --group --owner" by default.

Older snapshots are then deleted according to a schedule.



Issue
-----

When deleting a complete snapshot folder with rsync using an empty source folder
(which is a best practice for faster deletion than "rm -f") the
permissions of hardlinked
files are changed from eg. 444/-r--r--r-- to 644/-rw-r--r-- for all deleted
files
with existing hardlinks:

    mkdir empty
    rsync -a --delete -s empty/ snapshot1/

This distorts the backup history.

There is a corrsponding 6-year-old issue in Bugzilla incl. a patch but the issue
is still unfixed:

    https://bugzilla.samba.org/show_bug.cgi?id=12806



Questions
---------

1. Is there a reliable workaround ("--super" is proposed in the issue
but may probably not always work)?
2. If a rsync developer is reading this:  Is there any chance to fix this?



Steps to reproduce
------------------

I have attached a bash script "setup.sh" as txt file which makes the
issue fully reproducible (on Ubuntu 22.04).
-------------- next part --------------
#!/bin/bash
clear
echo
echo "Minimal reproducible example (MRE)"
echo "----------------------------------"
echo
echo "Tested with"
echo "- rsync  version 3.2.7  protocol version 31"
echo "- ext4 file system"
echo
rm -rf MRE
mkdir MRE
cd MRE
mkdir source
touch source/read_only.txt
chmod 444 source/read_only.txt

mkdir snapshot1
# rsync --recursive --times --devices --specials --hard-links --human-readable
-s --links --perms --executability --group --owner source/* snapshot1/
rsync --perms --executability --group --owner source/* snapshot1/

mkdir snapshot2
# rsync --recursive --times --devices --specials --hard-links --human-readable
-s --links --perms --executability --group --owner --link-dest=../snapshot1
source/* snapshot2/
rsync --perms --executability --group --owner --link-dest=../snapshot1 source/*
snapshot2/
# stat snapshot1/read_only.txt 
echo "File stats BEFORE rsync --delete":
echo
stat snapshot2/read_only.txt

mkdir empty
# delete snapshot1 using an empty source directory (shall be the fastest way to
delete a subdir)
# --debug=ALL
rsync -a --delete -s empty/ snapshot1/
# rm -f snapshot1/  # works as expected (unlink does neither change
'Changed' time nor file permissions)
# strace -c rm -rf snapshot1
echo
stat snapshot2/read_only.txt

echo
echo "Results (after deleting snapshot1 via rsync --delete):"
echo "1. The 'Change' time of the hardlinked file was updated"
echo "2. The hardlinked file has now different access rights (644 instead
of 444 so it is writable again!)"
echo "This does NOT happen if 'rm -f snapshot1/' is used!"
echo

echo "Files in snapshot2:"
ls -l snapshot2

I had intended to come back to this but because I didn't really think I 
had anything to add to the discussion I put it at a low enough priority 
that I forgot about it.  But I saw your bug report and was surprised to 
see that I was already unhelpful on this topic but because that original 
poster didn't have access to do an rm while you are trying to get a 
performance boost.

I have heard in the past that rsyncing an empty dir over a tree to 
delete the tree is faster than an rm -rf but I can't say I have ever 
benchmarked it to get any actual numbers.  But now that I am hearing 
that rsync actually adds a bunch of pointless chmods to the process.  Is 
it still faster given this problem?  If so maybe we should be trying to 
investigate why rm is so slow.  Otherwise, I would agree that an 
optimization to not do a bunch of pointless chmods is a good idea.  I 
would suspect that they are only there to prevent permission denied 
errors perhaps on some obscure filesystem (I tested on vfat since it is 
so non-compatible but read only files unlink fine there).

On 9/18/23 19:42, rsync--- via rsync wrote:
> Context
> -------
> 
> I am one of the active developers of the open source application "Back
in Time"
> which uses "rsync" as backend and I want to fix an open issue:
> 
>      "Back in Time"-Bug:
>     
https://github.com/bit-team/backintime/issues/994#issuecomment-1724211507
> 
> "Back in Time" uses "--link-dest" to reduce traffic and
storage by hardlinking
> unchanged files in the backups/snapshots and tries to keep the permissions
> by using "--perms --group --owner" by default.
> 
> Older snapshots are then deleted according to a schedule.
> 
> 
> 
> Issue
> -----
> 
> When deleting a complete snapshot folder with rsync using an empty source
folder
> (which is a best practice for faster deletion than "rm -f") the
permissions of hardlinked
> files are changed from eg. 444/-r--r--r-- to 644/-rw-r--r-- for all deleted
files
> with existing hardlinks:
> 
>      mkdir empty
>      rsync -a --delete -s empty/ snapshot1/
> 
> This distorts the backup history.
> 
> There is a corrsponding 6-year-old issue in Bugzilla incl. a patch but the
issue is still unfixed:
> 
>      https://bugzilla.samba.org/show_bug.cgi?id=12806
> 
> 
> 
> Questions
> ---------
> 
> 1. Is there a reliable workaround ("--super" is proposed in the
issue but may probably not always work)?
> 2. If a rsync developer is reading this:  Is there any chance to fix this?
> 
> 
> 
> Steps to reproduce
> ------------------
> 
> I have attached a bash script "setup.sh" as txt file which makes
the issue fully reproducible (on Ubuntu 22.04).
> 
> 
-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			https://sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,