rsync - Nov 2010 - DO NOT REPLY [Bug 7809] New: I/O errors other than IOERR_GENERAL should not suppress deletion

https://bugzilla.samba.org/show_bug.cgi?id=7809

           Summary: I/O errors other than IOERR_GENERAL should not suppress
                    deletion
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P3
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: matt at mattmccutchen.net
         QAContact: rsync-qa at samba.org


Jordan Russell pointed out (bug 3653 comment #20) that in the current
implementation, rsync skips deletion if there is any io_error, not just
IOERR_GENERAL.  My understanding was that IOERR_GENERAL is supposed to trigger
that behavior because it indicates that existing source files may be missing
from the file list, but other "I/O errors" should not trigger it.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7809

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #1 from Wayne Davison <wayned at samba.org> 2011-06-04
20:05:35 UTC ---
I've committed a fix to the 3.1.0dev git.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7809

--- Comment #2 from Jeffrey Simon <jas at rephunter.net> ---
This is an old ticket, but I am getting the same or similar problem in 2023
with rsync on macOS Ventura 13.4.1.

Here is the first attempt and partial results:

 rsync -av --delete  /Volumes/Backup1/ /Volumes/Backup2/ 

building file list ... rsync: opendir
"/Volumes/Backup1/.DocumentRevisions-V100" failed: Permission denied
(13)
rsync: opendir "/Volumes/Backup1/.TemporaryItems" failed: Permission
denied
(13)
rsync: opendir "/Volumes/Backup1/.Trashes" failed: Permission denied
(13)
done
IO error encountered -- skipping file deletion
...


With the following command that adds the exclusions the same results occur:

rsync -av --delete --exclude=/Volumes/Backup1/.DocumentRevisions-V100
--exclude=/Volumes/Backup1/.TemporaryItems --exclude=/Volumes/Backup1/.Trashes
/Volumes/Backup1/ /Volumes/Backup2/
building file list ... rsync: opendir
"/Volumes/Backup1/.DocumentRevisions-V100" failed: Permission denied
(13)
rsync: opendir "/Volumes/Backup1/.TemporaryItems" failed: Permission
denied
(13)
rsync: opendir "/Volumes/Backup1/.Trashes" failed: Permission denied
(13)
done
IO error encountered -- skipping file deletion
VM Archive/.DS_Store

sent 1570115 bytes  received 42 bytes  1046771.33 bytes/sec
total size is 568948981975  speedup is 362351.65
rsync error: some files could not be transferred (code 23) at
/AppleInternal/Library/BuildRoots/c2cb9645-dafc-11ed-aa26-6ec1e3b3f7b3/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(996)
[sender=2.6.9]


The only resolution to that issue is to use add the use of sudo:

sudo rsync -av --delete --exclude=/Volumes/Backup1/.DocumentRevisions-V100
--exclude=/Volumes/Backup1/.TemporaryItems --exclude=/Volumes/Backup1/.Trashes
/Volumes/Backup1/ /Volumes/Backup2/
building file list ... done
.DocumentRevisions-V100/
.DocumentRevisions-V100/.cs/
.DocumentRevisions-V100/AllUIDs/
.DocumentRevisions-V100/AllUIDs/1/
.DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/
.DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/3C2D59C9-44A2-471D-866B-282495F0718D.txt
.DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/9876415A-EA39-4A70-843F-7F9C8FFE75BF.txt
.DocumentRevisions-V100/db-V1/
.DocumentRevisions-V100/db-V1/db.sqlite-wal
.DocumentRevisions-V100/purgatory/
.DocumentRevisions-V100/staging/
.DocumentRevisions-V100/staging/501-60852-3Yny2b0q/
.TemporaryItems/
.TemporaryItems/folders.0/
.TemporaryItems/folders.0/TemporaryItems/
.TemporaryItems/folders.501/
.TemporaryItems/folders.501/TemporaryItems/
1P-Vault/
Download Archive/Windows/ZTree/
RH-archive/20230507/
VM Archive/W10-upgr-from-W7.pvm/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/MacOS/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/Resources/
VM
Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/Resources/English.lproj/
VM Archive/W10-upgr-from-W7.pvm/Windows 7-0.hdd/
VM Archive/W10-upgr-from-W7.pvm/Windows Disks/

sent 1666683 bytes  received 740 bytes  666969.20 bytes/sec
total size is 568949268052  speedup is 341214.72


The following are my conclusions:

1. The bug in this ticket still occurs in my environment.
2. Using the --exclude parameter would seem to be a workaround, but it is not.
3. Adding sudo along with the --exclude parameters and the command works as
expected from the command line.

However, the intended use case is to have this command run from launchd. For
some reason running the launchd command as root does not solve this, and even
adding sudo as part of the command does not solve it. This final aspect might
have nothing to do with rsync, but could rather be in my configuration or
usage. I am including this information just to be complete.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7809

--- Comment #3 from Jeffrey Simon <jas at rephunter.net> ---
One other point to follow up my first post of 2023-07-21:

None of these issues occurred on macOS Monterey 12.4.x

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7809

--- Comment #4 from Kevin Korb <rsync at sanitarium.net> ---
Your excludes aren't working because excludes are relative to the source dir
not /.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7809

--- Comment #5 from Jeffrey Simon <jas at rephunter.net> ---
"Excludes are relative to the source dir". Are you saying that the
excludes
should be --exclude=.DocumentRevisions-V100 --exclude=.TemporaryItems
--exclude=.Trashes?

That is a rhetorical question, because I no longer get an issue with with
excludes. After I added sudo, those errors no longer occur.

However, I am still having a problem, as follows:

1. The plain rsync command (with sudo and excludes) works from the command
line.

2. The same rsync command (with sudo and excludes) works as part of a script
from the command line.

3. The script does not work from launchd running as root.

Because rsync appears to be working properly, my issues are most likely with
the environmental differences between command line and launchd. So except for
the oddities that these issues only showed up on macOS Ventura, but did not
happen on macOS Monterey, I believe the issue can be considered somewhat
resolved. I say "somewhat" because it is not clear to me why sudo
should be
necessary from the command line, but it is.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7809

--- Comment #6 from Jeffrey Simon <jas at rephunter.net> ---
3. The script does not work from launchd running as root.

I should have given the failure mode, which is the following:

rsync: opendir "/Volumes/Backup1/." failed: Operation not permitted
(1)
rsync error: some files could not be transferred (code 23) at
/AppleInternal/Library/BuildRoots/c2cb9645-dafc-11ed-aa26-6ec1e3b3f7b3/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(996)
[sender=2.6.9]

-- 
You are receiving this mail because:
You are the QA Contact for the bug.