rsync - Jun 2016 - Can rsync assume that the destination directory is empty ?

Hello,

I'm currently using rsync to backup up to 1 TB of small files of 
relatively small files (hundreds of Ko mostly)

My backup strategy is to use a full backup and then backup the diff 
every day using hardlink with the previous backup. This means that each 
time I use rsync, the destination directory is empty.

Using strace, I can see that rsync call a 'lstat' command to try to see 
if the file already exists in my destination directory. Is there an 
option to tell rsync that the destination directory is empty ?

Do you think that avoiding this call can improve rsync performances in 
this specific case ?

I tried reading the source code, but I'm not exactly sure where this 
lstat call happens.

Thanks a lot,

Arnaud Aujon Chevallier

There isn't an option for that and it isn't actually required that the
target directory be empty (just a good idea).  Plus it has to do the
stat calls on the other end anyway so I doubt there would be much
performance benefit.

Maybe --ignore-times would cause it to not look but I kinda doubt it and
I am too tired to do an strace right now ;)

On 06/09/2016 05:32 AM, Arnaud Aujon Chevallier wrote:
> Hello,
> 
> I'm currently using rsync to backup up to 1 TB of small files of
> relatively small files (hundreds of Ko mostly)
> 
> My backup strategy is to use a full backup and then backup the diff
> every day using hardlink with the previous backup. This means that each
> time I use rsync, the destination directory is empty.
> 
> Using strace, I can see that rsync call a 'lstat' command to try to
see
> if the file already exists in my destination directory. Is there an
> option to tell rsync that the destination directory is empty ?
> 
> Do you think that avoiding this call can improve rsync performances in
> this specific case ?
> 
> I tried reading the source code, but I'm not exactly sure where this
> lstat call happens.
> 
> Thanks a lot,
> 
> Arnaud Aujon Chevallier
> 
> 
-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20160609/143fbbf7/signature.sig>

Actually, don't do --ignore-times.  Even if it did prevent the stat
calls it would also tell rsync to not care about matching files in the
--link-dest dir which would be very bad.

On 06/09/2016 06:27 AM, Kevin Korb wrote:
> There isn't an option for that and it isn't actually required that
the
> target directory be empty (just a good idea).  Plus it has to do the
> stat calls on the other end anyway so I doubt there would be much
> performance benefit.
> 
> Maybe --ignore-times would cause it to not look but I kinda doubt it and
> I am too tired to do an strace right now ;)
> 
> On 06/09/2016 05:32 AM, Arnaud Aujon Chevallier wrote:
>> Hello,
>>
>> I'm currently using rsync to backup up to 1 TB of small files of
>> relatively small files (hundreds of Ko mostly)
>>
>> My backup strategy is to use a full backup and then backup the diff
>> every day using hardlink with the previous backup. This means that each
>> time I use rsync, the destination directory is empty.
>>
>> Using strace, I can see that rsync call a 'lstat' command to
try to see
>> if the file already exists in my destination directory. Is there an
>> option to tell rsync that the destination directory is empty ?
>>
>> Do you think that avoiding this call can improve rsync performances in
>> this specific case ?
>>
>> I tried reading the source code, but I'm not exactly sure where
this
>> lstat call happens.
>>
>> Thanks a lot,
>>
>> Arnaud Aujon Chevallier
>>
>>
> 
> 
> 
-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20160609/22773362/signature.sig>