I'm using rsync to do an incremental backup of my desktop here, to a remote server as follows: #/usr/bin/bash old=$(date -d 'now - 1 week' +%Y-%m-%d) new=$(date +%Y-%m-%d) rsync -avP --delete --link-dest=../$dir /home/bakers bakers at perturb.org:/home/bakers/backup/$new/ This is actually working GREAT! The only problem is that sometimes the cronjob won't complete (internet is down, something like that). When it tries to run the next week it does --link-dest against a dir that doesn't exist. It happily complies and transfers EVERY file because there is no source to hardlink from. I'd really like rsync to exit and throw an error if the --link-dest isn't present. I can't find anything in the man page about any fancy --list-dest options. Am I missing it? - Scott
On 06/23/2011 03:40 PM, Lancashire, Pete wrote:> pretty easy to put a wrapper around the script or add it ..If it were local that would be easy. This is remote (via SSH), so I'd have to login before I run rsync to verify that directory is that. I was hoping I could save that step since rsync is already doing all that already. A --link-dest-required flag would be great. That way rsync doesn't copy a bunch of files since there is no hard link source. - Scott
Scott Baker (scott at perturb.org) wrote on 23 June 2011 15:30: >I'm using rsync to do an incremental backup of my desktop here, to a >remote server as follows: > >#/usr/bin/bash > >old=$(date -d 'now - 1 week' +%Y-%m-%d) >new=$(date +%Y-%m-%d) > >rsync -avP --delete --link-dest=../$dir /home/bakers >bakers at perturb.org:/home/bakers/backup/$new/ > >This is actually working GREAT! The only problem is that sometimes the >cronjob won't complete (internet is down, something like that). When it >tries to run the next week it does --link-dest against a dir that >doesn't exist. It happily complies and transfers EVERY file because >there is no source to hardlink from. > >I'd really like rsync to exit and throw an error if the --link-dest >isn't present. I can't find anything in the man page about any fancy >--list-dest options. Am I missing it? You should handle it in your script. You can pass several directories with link-dest and rsync will search for the file in all of them, in the order you gave. If it doesn't find one for hardlinking it pulls the file through the net. We do it here in our backup script to avoid useless copying, both when the backup machine starts the process and when the source of the files starts it.
> I'm using rsync to do an incremental backup of my desktop here, to a > remote server as follows: > > #/usr/bin/bash > > old=$(date -d 'now - 1 week' +%Y-%m-%d) > new=$(date +%Y-%m-%d) > > rsync -avP --delete --link-dest=../$dir /home/bakers > bakers at perturb.org:/home/bakers/backup/$new/ > > This is actually working GREAT! The only problem is that sometimes the > cronjob won't complete (internet is down, something like that). When it > tries to run the next week it does --link-dest against a dir that > doesn't exist. It happily complies and transfers EVERY file because > there is no source to hardlink from. > > I'd really like rsync to exit and throw an error if the --link-dest > isn't present. I can't find anything in the man page about any fancy > --list-dest options. Am I missing it?Hopefully, Scott's suggestion will solve this issue for you. You may also be interested in the way LBackup works for hard-linked incremental backups. It uses a numbering system rather than a date system. If you still want to have the date named entries within the file system, it is just a matter of enabling a post-action which keeps a date-named symbolic links directory to these backup snapshots up to date. Details on the LBackup post and pre actions are available from the following URLs : - http://www.lbackup.org/pre_and_post_actions (basic intro on LBackup pre/post actions) - http://bit.ly/lIMfAo (source code for the example post action script mentioned) This is all just another way of approaching this issue which you may find of interest. The one issue is that the script (source code link above) will need to be run on a system which has access to the file system which stores the snapshots. It would require modification if you wanted to run it on a remote system. Hope this helps :^) --------------------------------------------------------------------- This email is protected by LBackup, an open source backup solution. http://www.lbackup.org
Apparently Analagous Threads
- rsync incremental backup
- Incremental backup with only delta into a separate file.
- rsync --link-dest won't link even if existing file is out of date
- Help creating incremental backups using --backup-dir.
- has this been asked about before ? multilple exclude files