rsync - Sep 2023 - rsync --delete with empty source folder for fast snapshot deletion: Permissions of hardlinked files are changed to 644. Workaround?

On Thu, 2023-09-21 at 20:08 -0400, Kevin Korb via rsync wrote:
> I have heard in the past that rsyncing an empty dir over a tree to 
> delete the tree is faster than an rm -rf but I can't say I have ever 
> benchmarked it to get any actual numbers.
This **may** indeed be a myth (for a long time now) re-cited again and again and
- could no longer be valid today
- could apply only when deleting explicitly named files but not deleting the
complete folder
  (as we need to do in "Back in Time")

At least I could not find a holistic benchmark with many files and different
scenarios
(file systems, rsync'ing locally vs. over network, snapshot sizes, number of
files, file sizes, rsync and rm versions...)

Q: Does `rsync` provide a test case that I could use as basis to prepare such a
holistic benchmark?
> But now that I am hearing 
> that rsync actually adds a bunch of pointless chmods to the process.? Is 
> it still faster given this problem?? If so maybe we should be trying to 
> investigate why rm is so slow.
Just by strace'ing I saw `rm` mainly calls unlink, `rsync` does not.

So I decided to do a quick test using the Linux kernel source tree since 
it has lots of files.  I duplicated a tree, used 'find . -type f -exec 
chmod 444 {} +' to make read only files for rsync to want to chmod, then 
used cp -al to make several duplicate trees using hard linked files.  An 
rm -rf on one such tree took .97 seconds while an rsync deletion took 
1.25 seconds.  Clearly I need a bigger test to play with as that margin 
could easily change just by different outputs if either had a -v.  But 
also, I did not experience the problem you are describing.  My surviving 
hard links in the duplicate trees were still 444.

BTW, it seems that rsync uses unlink() while rm uses unlinkat().  No 
idea if/why there is a performance diff there but I didn't look into it 
as it is now time for me to go to work.

On 9/22/23 04:14, rsync at altfeld-im.de wrote:
> On Thu, 2023-09-21 at 20:08 -0400, Kevin Korb via rsync wrote:
> 
>> I have heard in the past that rsyncing an empty dir over a tree to
>> delete the tree is faster than an rm -rf but I can't say I have
ever
>> benchmarked it to get any actual numbers.
> 
> This **may** indeed be a myth (for a long time now) re-cited again and
again and
> - could no longer be valid today
> - could apply only when deleting explicitly named files but not deleting
the complete folder
>    (as we need to do in "Back in Time")
> 
> At least I could not find a holistic benchmark with many files and
different scenarios
> (file systems, rsync'ing locally vs. over network, snapshot sizes,
number of files, file sizes, rsync and rm versions...)
> 
> Q: Does `rsync` provide a test case that I could use as basis to prepare
such a holistic benchmark?
> 
>> But now that I am hearing
>> that rsync actually adds a bunch of pointless chmods to the process.?
Is
>> it still faster given this problem?? If so maybe we should be trying to
>> investigate why rm is so slow.
> 
> Just by strace'ing I saw `rm` mainly calls unlink, `rsync` does not.
> 
> 
-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			https://sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,