That sort of snafu is why find(1) has the -depth directive. Does rsync have anything similar? Kevin Korb via rsync <rsync at lists.samba.org> wrote:> There maybe a proper solution but an obvious workaround would be to run > rsync twice. The first time without the --fileflags option. > > --no-perms wouldn't help. That is only the standard unix permissions. > > On 10/30/21 8:04 PM, Fred Fugate via rsync wrote: > > Hi, > > > > I have some subdirectories within a home directory that are chflagged > > schg, aka system immutable. > > > > When I rsync the home directory to another machine, I get lots of errors > > for these subdirectories and they fail to copy to the target. NOTE: The > > target is empty; rsync is copying to an empty remote machine, not trying > > to overwrite anything. > > > > I understand why this is happening; it's the schg flag. But is there a > > workaround using some combination of rsync options or multiple passes? > > > > Here is an example. > > > > The source directory is /Users/redacted. > > > > Within /Users/redacted is a subdirectory foo that has flag schg set: > > /Users/redacted/Documents/artwork/foo > > > > # ls -laO /Users/redacted/Documents/artwork > > total 80 > > drwxr-xr-x ?? 20 redacted ??staff ??- ?? ?? ?? ??680 Sep 21 21:06 . > > drwx------@ 609 redacted ??staff ??- ?? ?? ??20706 Oct 29 16:07 .. > > -rw-r--r--@ ?? 1 redacted ??staff ??- ?? ?? ??18436 Sep 21 20:55 somefile > > drwxrwxrwx@ ??18 redacted ??staff ??schg ?? ?? 612 Apr 12 ??2006 foo > > > > > > My rsync args are: > > > > --verbose --archive --one-file-system --acls --hard-links --xattrs > > --protect-args --delete-after --numeric-ids --itemize-changes --crtimes > > --fileflags --force-change > > --rsync-path=/opt/rsync323/bin/rsync > > > > My rsync version is 3.2.3 > > > > My execution and output looks like this, run as root: > > > > # /opt/rsync323/bin/rsync [ARGS ABOVE] /Users/redacted > > remotemachine.domain.com:/Users > > . > > . > > rsync: [receiver] mkstemp > > "/Users/redacted/Documents/artwork/foo/bar/.background.tiff.vYOAS2" > > failed: Operation not permitted (1) > > rsync: [receiver] mkstemp > > "/Users/redacted/Documents/artwork/foo/bar/.dc4.orange.CMYK.tiff.rYCmAP" > > failed: Operation not permitted (1) > > rsync: [receiver] mkstemp > > "/Users/redacted/Documents/artwork/foo/bar/.barcode.tiff.2E4mec" failed: > > Operation not permitted (1) > > . > > . > > > > In a nutshell: > > > > Subdirectory foo gets created on the receiver, but it gets created with > > the schg flag set. > > > > No further copying into foo can happen after that, because the schg flag > > prevents that. > > > > Subdirectory bar cannot be created under foo, and > > .background.tiff.vYOAS2 and other files cannot be created under bar, etc. > > > > How can I force the schg flags to be set on the receiver AFTER > > everything has been copied from the source? > > > > I really don't want to remove all of my schg settings on the source > > before rsyncing to the target. > > > > Would --no-perms allow this? But what would it break?
No, rsync normally handles such problems well. Unfortunately, --fileflags is an OS vendor added feature rather than an official rsync feature so it is less well thought out. On 10/31/21 3:51 AM, Perry Hutchison wrote:> That sort of snafu is why find(1) has the -depth directive. > Does rsync have anything similar? > > Kevin Korb via rsync <rsync at lists.samba.org> wrote: > >> There maybe a proper solution but an obvious workaround would be to run >> rsync twice. The first time without the --fileflags option. >> >> --no-perms wouldn't help. That is only the standard unix permissions. >> >> On 10/30/21 8:04 PM, Fred Fugate via rsync wrote: >>> Hi, >>> >>> I have some subdirectories within a home directory that are chflagged >>> schg, aka system immutable. >>> >>> When I rsync the home directory to another machine, I get lots of errors >>> for these subdirectories and they fail to copy to the target. NOTE: The >>> target is empty; rsync is copying to an empty remote machine, not trying >>> to overwrite anything. >>> >>> I understand why this is happening; it's the schg flag. But is there a >>> workaround using some combination of rsync options or multiple passes? >>> >>> Here is an example. >>> >>> The source directory is /Users/redacted. >>> >>> Within /Users/redacted is a subdirectory foo that has flag schg set: >>> /Users/redacted/Documents/artwork/foo >>> >>> # ls -laO /Users/redacted/Documents/artwork >>> total 80 >>> drwxr-xr-x ?? 20 redacted ??staff ??- ?? ?? ?? ??680 Sep 21 21:06 . >>> drwx------@ 609 redacted ??staff ??- ?? ?? ??20706 Oct 29 16:07 .. >>> -rw-r--r--@ ?? 1 redacted ??staff ??- ?? ?? ??18436 Sep 21 20:55 somefile >>> drwxrwxrwx@ ??18 redacted ??staff ??schg ?? ?? 612 Apr 12 ??2006 foo >>> >>> >>> My rsync args are: >>> >>> --verbose --archive --one-file-system --acls --hard-links --xattrs >>> --protect-args --delete-after --numeric-ids --itemize-changes --crtimes >>> --fileflags --force-change >>> --rsync-path=/opt/rsync323/bin/rsync >>> >>> My rsync version is 3.2.3 >>> >>> My execution and output looks like this, run as root: >>> >>> # /opt/rsync323/bin/rsync [ARGS ABOVE] /Users/redacted >>> remotemachine.domain.com:/Users >>> . >>> . >>> rsync: [receiver] mkstemp >>> "/Users/redacted/Documents/artwork/foo/bar/.background.tiff.vYOAS2" >>> failed: Operation not permitted (1) >>> rsync: [receiver] mkstemp >>> "/Users/redacted/Documents/artwork/foo/bar/.dc4.orange.CMYK.tiff.rYCmAP" >>> failed: Operation not permitted (1) >>> rsync: [receiver] mkstemp >>> "/Users/redacted/Documents/artwork/foo/bar/.barcode.tiff.2E4mec" failed: >>> Operation not permitted (1) >>> . >>> . >>> >>> In a nutshell: >>> >>> Subdirectory foo gets created on the receiver, but it gets created with >>> the schg flag set. >>> >>> No further copying into foo can happen after that, because the schg flag >>> prevents that. >>> >>> Subdirectory bar cannot be created under foo, and >>> .background.tiff.vYOAS2 and other files cannot be created under bar, etc. >>> >>> How can I force the schg flags to be set on the receiver AFTER >>> everything has been copied from the source? >>> >>> I really don't want to remove all of my schg settings on the source >>> before rsyncing to the target. >>> >>> Would --no-perms allow this? But what would it break?-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Thanks Kevin. I tried leaving off --fileflags and got entirely different error output: rsync: [generator] failed to set file flags on "/Users/redacted/Library/Accounts": Operation not permitted (1) In this example, the Library subdir on the source has the "hidden" flag set; my understanding of this error is that it is not able to set that flag on the destination. # ls -laO@ /Users/redacted/Library total 96 drwx------@ 72 502 staff hidden 2448 Oct 8 18:27 . When I checked the Library subdir on the destination after copying, it was indeed missing the hidden flag. I'm not sure how I could work around both the original error caused by using --fileflags, and this error caused by omitting --fileflags. Also not sure why rsync is trying to honor the hidden flag when I'm NOT using --fileflags, but that is another post for the future. I do have a solution currently: rsync 3.0.9, the only other version I keep installed on my systems. It does not have a problem with --fileflags. So for this task I am downgrading to version 3.0.9. I guess using 3.0.9 will be fine. But if anyone has any insight or cautionary advice on 3.0.9 vs 3.2.3, I want to hear it. The systems are Macs, some still with HFS+ file systems and others with APFS. Thanks, Fred On Sun, Oct 31, 2021 at 6:51 AM Kevin Korb via rsync <rsync at lists.samba.org> wrote:> No, rsync normally handles such problems well. Unfortunately, > --fileflags is an OS vendor added feature rather than an official rsync > feature so it is less well thought out. > > On 10/31/21 3:51 AM, Perry Hutchison wrote: > > That sort of snafu is why find(1) has the -depth directive. > > Does rsync have anything similar? > > > > Kevin Korb via rsync <rsync at lists.samba.org> wrote: > > > >> There maybe a proper solution but an obvious workaround would be to run > >> rsync twice. The first time without the --fileflags option. > >> > >> --no-perms wouldn't help. That is only the standard unix permissions. > >> > >> On 10/30/21 8:04 PM, Fred Fugate via rsync wrote: > >>> Hi, > >>> > >>> I have some subdirectories within a home directory that are chflagged > >>> schg, aka system immutable. > >>> > >>> When I rsync the home directory to another machine, I get lots of > errors > >>> for these subdirectories and they fail to copy to the target. NOTE: The > >>> target is empty; rsync is copying to an empty remote machine, not > trying > >>> to overwrite anything. > >>> > >>> I understand why this is happening; it's the schg flag. But is there a > >>> workaround using some combination of rsync options or multiple passes? > >>> > >>> Here is an example. > >>> > >>> The source directory is /Users/redacted. > >>> > >>> Within /Users/redacted is a subdirectory foo that has flag schg set: > >>> /Users/redacted/Documents/artwork/foo > >>> > >>> # ls -laO /Users/redacted/Documents/artwork > >>> total 80 > >>> drwxr-xr-x ?? 20 redacted ??staff ??- ?? ?? ?? ??680 Sep 21 21:06 . > >>> drwx------@ 609 redacted ??staff ??- ?? ?? ??20706 Oct 29 16:07 .. > >>> -rw-r--r--@ ?? 1 redacted ??staff ??- ?? ?? ??18436 Sep 21 20:55 > somefile > >>> drwxrwxrwx@ ??18 redacted ??staff ??schg ?? ?? 612 Apr 12 ??2006 foo > >>> > >>> > >>> My rsync args are: > >>> > >>> --verbose --archive --one-file-system --acls --hard-links --xattrs > >>> --protect-args --delete-after --numeric-ids --itemize-changes --crtimes > >>> --fileflags --force-change > >>> --rsync-path=/opt/rsync323/bin/rsync > >>> > >>> My rsync version is 3.2.3 > >>> > >>> My execution and output looks like this, run as root: > >>> > >>> # /opt/rsync323/bin/rsync [ARGS ABOVE] /Users/redacted > >>> remotemachine.domain.com:/Users > >>> . > >>> . > >>> rsync: [receiver] mkstemp > >>> "/Users/redacted/Documents/artwork/foo/bar/.background.tiff.vYOAS2" > >>> failed: Operation not permitted (1) > >>> rsync: [receiver] mkstemp > >>> > "/Users/redacted/Documents/artwork/foo/bar/.dc4.orange.CMYK.tiff.rYCmAP" > >>> failed: Operation not permitted (1) > >>> rsync: [receiver] mkstemp > >>> "/Users/redacted/Documents/artwork/foo/bar/.barcode.tiff.2E4mec" > failed: > >>> Operation not permitted (1) > >>> . > >>> . > >>> > >>> In a nutshell: > >>> > >>> Subdirectory foo gets created on the receiver, but it gets created with > >>> the schg flag set. > >>> > >>> No further copying into foo can happen after that, because the schg > flag > >>> prevents that. > >>> > >>> Subdirectory bar cannot be created under foo, and > >>> .background.tiff.vYOAS2 and other files cannot be created under bar, > etc. > >>> > >>> How can I force the schg flags to be set on the receiver AFTER > >>> everything has been copied from the source? > >>> > >>> I really don't want to remove all of my schg settings on the source > >>> before rsyncing to the target. > >>> > >>> Would --no-perms allow this? But what would it break? > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: https://sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20211101/fcf9d1db/attachment.htm>