rsync - Aug 2003 - rsync returns 23 (RERR_PARTIAL) when a file has been deleted after the list has been created.

All,

I am using rsync to backup a list of files that are in use by an
application. Files can be deleted, created, or modified by the application
whilst rsync is running. I am getting the following output line, in amongst
lots of succesful transfers, when running rsync, :

send_files failed to open //ca/spool/smdiq/005.973416/9734161182.1: No such
file or directory

This then results in an RC of 23 (RERR_PARTIAL),

I believe this is happening because between the time rsync compiles the
file list and the time rsync attempts to transfer the file the application
has deleted it.

My problem here is that rsync is run as part of a script that performs the
backup and the script must be able to detect whether the rsync was
succesful. The script currently treats a non-zero RC as a failure
conidtion. I think that this particular transfer failure should not be
flagged as RC 23,  as other RC 23 conditions are actual failures as opposed
to the file list becoming outdated during the copy process.

Any thoughts?? Am I barking up the wrong tree?

PS: I am running rsync-2.5.6

Ev.

On Wed, Aug 06, 2003 at 01:48:06PM +0100, Evan Skinner1
wrote:
> I believe this is happening because between the time rsync compiles
> the file list and the time rsync attempts to transfer the file the
> application has deleted it.
Yes, that can cause this error code to be returned.
> I think that this particular transfer failure should not be flagged as
> RC 23, as other RC 23 conditions are actual failures as opposed to the
> file list becoming outdated during the copy process.
Yeah, it's something that seems desirable to me (to differentiate this
specific partial-transfer condition).

One potential solution would be to not return an error code if the open
failures were all ENOENT errors on the sending side.

Another solution is to create a new file-vanished-during-transfer error
(for when the open errors consisted only of ENOENT errors), but we'd
give it error code 23 (for backward-compatibility reasons).  All other
partial-transfer errors would get a new error code with the same error
text as before.  (I wouldn't want to see a new option added to make this
new separation selectable, though.)

Thoughts?

..wayne..

> I like the idea of splitting file vanished during transfer
> (VANISHED) a lot.
> I'd rather 23 stay partial transfer and file vanished be a
> new error (24 or 40).  When pulling you depend on the remote
> end to set this error making this protocol dependent.
> Your wrapper won't know it is running down-graded.
A new error code for only ENOENT errors sounds great - whilst
keeping the old 23 code for all other partial transfers.
> I'd rather have a VANISHED (warning) mistagged as a possibly
> more serious PARTIAL (error) than have a more serious
> PARTIAL tagged as a VANISHED.  After all, PARTIAL usually
> means you want to re-run but VANISHED can be ignored.
Absolutely.

I've made changes to the code to this effect and tested that
it gives the desired results. The RERR_VANISHED RC is only
returned if the only errors that occured are ENOENT during
send_files. I'm going to use this modified rsync for my project.

Is there a place to submit these code changes or should I leave
you guys to make the change in due course?

Ev.

On Wed, Aug 06, 2003 at 09:54:47AM -0700, Wayne Davison
wrote:
> On Wed, Aug 06, 2003 at 01:48:06PM +0100, Evan Skinner1 wrote:
> > I believe this is happening because between the time rsync compiles
> > the file list and the time rsync attempts to transfer the file the
> > application has deleted it.
>
> Yes, that can cause this error code to be returned.
>
> > I think that this particular transfer failure should not be flagged as
> > RC 23, as other RC 23 conditions are actual failures as opposed to the
> > file list becoming outdated during the copy process.
>
> Yeah, it's something that seems desirable to me (to differentiate this
> specific partial-transfer condition).
>
> One potential solution would be to not return an error code if the open
> failures were all ENOENT errors on the sending side.
>
> Another solution is to create a new file-vanished-during-transfer error
> (for when the open errors consisted only of ENOENT errors), but we'd
> give it error code 23 (for backward-compatibility reasons).  All other
> partial-transfer errors would get a new error code with the same error
> text as before.  (I wouldn't want to see a new option added to make
this
> new separation selectable, though.)
>
> Thoughts?



--
________________________________________________________________
             J.W. Schultz            Pegasystems Technologies
             email address:                      jw@pegasys.ws