rsync 3.1.1: --ignore-missing-args / --delete-missing args problem

Axel Kittenberger
Fri, 07 Apr 2017 14:01:37 -0000

With this two options on a very live system you may need to take into
account this bug as well I reported a while ago:

https://bugzilla.samba.org/show_bug.cgi?id=12569

Due to this I'm currently not using  --ignore-missing-args /
--delete-missing but rather -exclude=* -include=- and a autocreated
hierachical filter rule list on stdin.

PS: Yes to simply ignore code 24 would be correct. However, it creates exit
code 2 in these cases.

On Fri, Apr 7, 2017 at 2:58 PM, Georgy Fedorov via rsync <
rsync at lists.samba.org> wrote:

> Dear All,
>
> We sometimes have to replicate large "live" filesystems with many
(
> sometimes millions, up to few hundred millions ) files on them. ( Copying
> actively used files is of course a bad idea, but it really helps to keep
> the delta small, so one final transfer can later save the day. )
>
> The problem, as one may guess, is that some files may disappear during the
> process, so rsync finishes with an error code 23.
> The good news is that since version 3.1.0 rsync has two options to address
> this problem: --ignore-missing-args and --delete-missing-args .
>
> The bad news, however, is that even with any of these, when the file
> disappears, rsync handles the file transfer properly, but then tries to set
> times or attributes on these files nevertheless. That fails with errno 22 (
> EINVAL ) and still leads to exit code 23, which is a bit of an annoyance.
>
> I am currently trying to come up with a patch to address this issue, since
> I'd like to have exit code 0 for either of --ignore-missing-args or
> --delete-missing-args, when the files are, well, missing.
>
> Probably it is not the right way to address this problem, but in the same
> vein as --ignore-missing-args are implemented, the patch can go as follows:
> https://gist.github.com/anonymous/96ba8bf10f864a93fd9203f75c43ffe9 .
>
> Basically we need to do two changes:
>
> (1) in options.c, make sure that the "missing_args" value is
transferred
> when set ; and
> (2) in rsync.c, in the "missing_args" case, replace FERROR_XFER
with
> something else, since apparently any log message with logcode FERROR_XFER
> sets the flag 'got_xfer_error' in log.c, and that finally leads to
exit
> code RERR_PARTIAL (23), which is what we are trying to avoid.
>
> I am currently testing this on fairly big datasets to see if there's
> something missing, and will write more when I see how it goes ( as I said,
> the data sets are fairly large ).
>
> There shall be a better way to address that, but this is all I can do with
> a very shallow acquaintance with rsync source code. Nevertheless, it would
> be great if that could be fixed in the trunk one way or another )
>
> Kind regards, George
>
> --
> George Fedorov
> Senior Systems Specialist
> Melbourne School of Engineering
> The University of Melbourne, Victoria 3010, Australia
> phone: *****
> email: *****http://www.eng.unimelb.edu.au/
>
>
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/
> mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20170407/848a8e16/attachment.html>