rsync - Mar 2023 - Trying to diagnose incomplete file transfer

I second Francis here. You don't need to diagnose incomplete file transfers
as long as you have racing conditions as you described. This leads to strange
result inevitably.
NEVER start several rsync jobs manipulating the same data - especially if there
are modifications to BOTH sides source and destination.

You do not necessarily define a service like Francis suggests. A simple
semaphore approach suffices. Perhaps even something like

# ps fax | grep -v grep | grep $0 && exit

to prevent this exact command "$0" to start concurrently.

Hardy

Am 04.03.23 um 08:38 schrieb Francis.Montagnac--- via
rsync:
> 
> Hi.
> 
> On Sat, 04 Mar 2023 00:39:52 -0600 Albert Croft via rsync wrote:
> 
>> The rsync commands may be launched from command-line or cron, but use
>> the same format and options in either case. As a result, there may be
>> multiple rsync processes pulling files from the same remote path to the
>> same local path.
> 
> I think you should first prevent this to happen.
> 
> If your receiving machine is using systemd:
> 
>    - define a X.service for doing the rsync
>    - define a X.timer unit to replace using cron
>    - launch from the command line with: systemctl start X
>      - this will not start a new rsync if one runs already (if X.service is
>        running)
>

Oh, yeah, I missed that part.  Yeah, don't do that; it's easy to add
a lock file to a shell script.

On Sun, Mar 05, 2023 at 12:30:16PM +0100, Hardy via rsync
wrote:
> I second Francis here. You don't need to diagnose incomplete file
transfers as long as you have racing conditions as you described. This leads to
strange result inevitably.
> NEVER start several rsync jobs manipulating the same data - especially if
there are modifications to BOTH sides source and destination.
> 
> You do not necessarily define a service like Francis suggests. A simple
semaphore approach suffices. Perhaps even something like
> 
> # ps fax | grep -v grep | grep $0 && exit
> 
> to prevent this exact command "$0" to start concurrently.
> 
> Hardy
> 
> Am 04.03.23 um 08:38 schrieb Francis.Montagnac--- via rsync:
> > 
> > Hi.
> > 
> > On Sat, 04 Mar 2023 00:39:52 -0600 Albert Croft via rsync wrote:
> > 
> > > The rsync commands may be launched from command-line or cron, but
use
> > > the same format and options in either case. As a result, there
may be
> > > multiple rsync processes pulling files from the same remote path
to the
> > > same local path.
> > 
> > I think you should first prevent this to happen.
> > 
> > If your receiving machine is using systemd:
> > 
> >    - define a X.service for doing the rsync
> >    - define a X.timer unit to replace using cron
> >    - launch from the command line with: systemctl start X
> >      - this will not start a new rsync if one runs already (if
X.service is
> >        running)
> > 
> 
> -- 
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html