rsync - Dec 2014 - Re[2]: --link-dest --inplace updates files without unlinking. What to do?

>BTW, if you want it to always have that behavior (it can save a lot of
>backup space) you can use the old cp -al method instead of --link-dest
>so that the target dir starts out completely populated.
You mean making "cp -al" on the remote and then start rsync to newly
created dir with --partial and without --link-dest, --inplace, --append-verify?
What is benefits? Even metadata will not leak to old files?
=Askar Safin
http://vk.com/safinaskar
Kazan, Russia

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I mean instead of having --link-dest=previous_backup and the target
being empty (or starting that way in your case) you cp -al the
prevous_backup to the new "incomplete" one.  Now you have a tree full
of all hard links.  Now you can rsync to the target without any
- --link-dest reference.  You are right about the other options except
that you would now need --delete as well.  Rsync will replace updated
files as you described earlier, change the metadata on the existing
links as I described earlier, add the new files, and delete the hard
links cp made which are no longer needed.

This was how we did backups with rsync before rsync had --link-dest.
On large trees (lots of files don't care about size) it can take
significantly longer because of the extra pass of stat() calls and the
making of links that will then get deleted but otoh you can do the cp
- -al run between backup windows and potentially reduce the amount of
time of the actual run.

I backed up my stuff this way for years.  It worked well.  When
- --link-dest first came out I hated it because it insisted on listing
every directory since it was creating them and I liked my backup
output to only be real changes.  Then when I first switched to
- --link-dest after the output was fixed I ran into this metadata
problem.  I decided to add a group named music and I did a chgrp -R
(and a chmod -R g+r) on my entire music collection.  In the old cp -al
method that would be no big deal as it would just be an (I am talking
- --itemize-changes output here) cf---p-g--- but in the new --link-dest
method it became a >f---p-g--- which meant my entire music collection
was being duplicated on the backup.  I went back to cp -al for a while.

At present, I work for a shared server web hosting provider.  We used
rsync --link-dest (we have actually switched to rsyncing to ZFS
subvolumes with snapshots which is a longer story).  It became
critical to have the exact metadata in every backup.  When a client
says "hey, my webapp worked Tuesday and now it doesn't but I didn't
change anything what did you guys do?!?!?" I need to be able to look
in the backups and see if any permissions have changed.  So, maybe I
have extra copies of some php, html, and jpg files but I know how they
were at the time they were backed up.

On 12/26/2014 09:24 PM, Askar Safin wrote:
>> BTW, if you want it to always have that behavior (it can save a
>> lot of backup space) you can use the old cp -al method instead of
>> --link-dest so that the target dir starts out completely
>> populated.
> You mean making "cp -al" on the remote and then start rsync to
> newly created dir with --partial and without --link-dest,
> --inplace, --append-verify? What is benefits? Even metadata will
> not leak to old files? == Askar Safin http://vk.com/safinaskar 
> Kazan, Russia
> 
- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlSeHHwACgkQVKC1jlbQAQfXggCfXoRuT2tF/CwYBJm2PW+17Zq4
sGUAnRwilEg6eoDC81I6tZidoidDxpOQ
=Sp3G
-----END PGP SIGNATURE-----

>I mean instead of having --link-dest=previous_backup and the target
>being empty (or starting that way in your case) you cp -al the
>prevous_backup to the new "incomplete" one.  Now you have a tree
full
>of all hard links.  Now you can rsync to the target without any
>- --link-dest reference.
I meant the same.
>You are right about the other options except
>that you would now need --delete as well.
I have --delete-after.
>Rsync will replace updated
>files as you described earlier, change the metadata on the existing
>links as I described earlier, add the new files, and delete the hard
>links cp made which are no longer needed.
Okey, I will stick with --link-dest, thanks a lot.
>So, maybe I
>have extra copies of some php, html, and jpg files but I know how they
>were at the time they were backed up.
You can still do "cp -al" and backup "ls -lR" or "find
-ls" output. :)

=Askar Safin
http://vk.com/safinaskar
Kazan, Russia