rsync - Jun 2014 - --dry-run reliability ?

Hi,

I'm using rsync to backup my files. I have also added a pre-backup step
where the script checks if there is enough space on the destination for rsync to
run successfully. The goal is to make room (delete oldest backups) if that's
not the case.

I'm running rsync with -ahAXS --numeric-ids --link-dest options.

For my pre-backup step, I'm currently using the same options + --dry-run
--no-h --stats. I then parse the "Total transferred file size" line,
add a 20% (!) padding and compare the result with the result of a "df"
call on destination.

It seemd to work properly until I decided to add some ACL on the source device.
I didn't really know how rsync would handle the situation, but understood
that it may have to backup all the files (instead of hardlinking them) because
of this change.

The dry run reported that it would need about 1400 MB and I have around 30 000
MB of free space, so it should have ran properly. Sadly, I got the "no
space left on device" error.

So, is it safe to rely on the stats provided by a dry run to achieve my goal ?
Does the --dry-run option take care of ACL/permissions changes ?


Thanks a lot for your work and help, rsync is really a wonderful tool.

Cheers,

-- 
Fran?ois

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The problem here isn't transfer size it is that rsync is storing the
affected files with and without the ACL even though the file data is
the same.

The solution is to run a single manual rsync using the old method that
predates --link-dest...

1, find your most recent non-ACL backup
2, cp -al thatbackup newbackup
3, rsync -ahAXS --numeric-ids your data over top of the newbackup dir

When you do this rsync will set the ACLs on the existing files (and in
every old backup containing the same inode) rather than duplicating
the file to store it with and without the ACL.

On 06/13/2014 05:14 PM, daitheflu at free.fr wrote:
> Hi,
> 
> I'm using rsync to backup my files. I have also added a pre-backup
> step where the script checks if there is enough space on the
> destination for rsync to run successfully. The goal is to make room
> (delete oldest backups) if that's not the case.
> 
> I'm running rsync with -ahAXS --numeric-ids --link-dest options.
> 
> For my pre-backup step, I'm currently using the same options +
> --dry-run --no-h --stats. I then parse the "Total transferred file
> size" line, add a 20% (!) padding and compare the result with the
> result of a "df" call on destination.
> 
> It seemd to work properly until I decided to add some ACL on the
> source device. I didn't really know how rsync would handle the
> situation, but understood that it may have to backup all the files
> (instead of hardlinking them) because of this change.
> 
> The dry run reported that it would need about 1400 MB and I have
> around 30 000 MB of free space, so it should have ran properly.
> Sadly, I got the "no space left on device" error.
> 
> So, is it safe to rely on the stats provided by a dry run to
> achieve my goal ? Does the --dry-run option take care of
> ACL/permissions changes ?
> 
> 
> Thanks a lot for your work and help, rsync is really a wonderful
> tool.
> 
> Cheers,
> 
- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlObawsACgkQVKC1jlbQAQeaNACfeUKGW5bC1TPu4LpBTMpGHatD
NkgAoN8xD/82tg2NbrRu2xoS4yL8pN0/
=l28J
-----END PGP SIGNATURE-----