As part of my backup system, I use Rsync to keep a copy of each server on one central backup server. This backup server then uses StoreBackup to keep multiple iterations of each clone directory. So that the StoreBackup archives don't keep adding "redundant" and misleading backups, I update a flag file with the current date/time before doing the Rsync update, and test to see if this file is newer than the one in the latest StoreBackup backup. If it isn't, then I skip the StoreBackup for that server. The end result is that if a system is down or out of communication (one or two are at sites that can be offline for days), then the list of backups in StoreBackup will reflect that. Eg, if the system did a sync on the 1st, but not on 2nd - 5th, then there will be no backups for 2nd-5th, and when looking later I won't be "fooled" into thinking that I have a backup from (say) the 4th. Where this breaks down is if the sync fails part way through. The flag file has already been synced, so I have multiple backups which aren't actually complete. I actually have this at the moment. Just put a small system on a customer site, it has a database that creates 1GB journal files (not that it handles anything like that volume of data), and at the moment their connectivity is a bit flakey. My first thought was "do the flag file last", but a quick search confirms what I thought - that there isn't an option for this. So, does anyone have any suggestions how I might reasonably easily get the ability for my script to see if the previous sync completed ?
rsync -av /src/ /dst/ && touch /dst/done That should do it as the touch only happens if rsync exits with a code of 0. If you need to consider other non zero exit code, it is still doable, just a bit more shell code. There are surely other options as well, but this is probably the most simple. On Apr 28, 2015 3:47 AM, "Simon Hobson" <linux at thehobsons.co.uk> wrote:> As part of my backup system, I use Rsync to keep a copy of each server on > one central backup server. This backup server then uses StoreBackup to keep > multiple iterations of each clone directory. > So that the StoreBackup archives don't keep adding "redundant" and > misleading backups, I update a flag file with the current date/time before > doing the Rsync update, and test to see if this file is newer than the one > in the latest StoreBackup backup. If it isn't, then I skip the StoreBackup > for that server. > > The end result is that if a system is down or out of communication (one or > two are at sites that can be offline for days), then the list of backups in > StoreBackup will reflect that. Eg, if the system did a sync on the 1st, but > not on 2nd - 5th, then there will be no backups for 2nd-5th, and when > looking later I won't be "fooled" into thinking that I have a backup from > (say) the 4th. > > Where this breaks down is if the sync fails part way through. The flag > file has already been synced, so I have multiple backups which aren't > actually complete. > I actually have this at the moment. Just put a small system on a customer > site, it has a database that creates 1GB journal files (not that it handles > anything like that volume of data), and at the moment their connectivity is > a bit flakey. > > My first thought was "do the flag file last", but a quick search confirms > what I thought - that there isn't an option for this. > > So, does anyone have any suggestions how I might reasonably easily get the > ability for my script to see if the previous sync completed ? > > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20150428/923cbda0/attachment.html>
Michael Johnson - MJ <mj at revmj.com> wrote:> rsync -av /src/ /dst/ && touch /dst/doneAaaaahhhh, knew I'd miss some detail. All the syncs are pushed to the backup server. But that does give me an idea. I guess I could do that on the source, then sync the flag file over. rsync -avH ${other_gubbins} / user at remote.machine:/dest/ && touch /etc/donefile && rsync -av ${some_other_gubbins} /etc/donefile user at remote.machine:/dest/ That (or some variation of it) could work.
For a push job. Run the rsync for the files, if the exit code is 0, create the flag file and then rsync just that file on its own. On 28/04/2015 10:38 p.m., Simon Hobson wrote:> As part of my backup system, I use Rsync to keep a copy of each server on one central backup server. This backup server then uses StoreBackup to keep multiple iterations of each clone directory. > So that the StoreBackup archives don't keep adding "redundant" and misleading backups, I update a flag file with the current date/time before doing the Rsync update, and test to see if this file is newer than the one in the latest StoreBackup backup. If it isn't, then I skip the StoreBackup for that server. > > The end result is that if a system is down or out of communication (one or two are at sites that can be offline for days), then the list of backups in StoreBackup will reflect that. Eg, if the system did a sync on the 1st, but not on 2nd - 5th, then there will be no backups for 2nd-5th, and when looking later I won't be "fooled" into thinking that I have a backup from (say) the 4th. > > Where this breaks down is if the sync fails part way through. The flag file has already been synced, so I have multiple backups which aren't actually complete. > I actually have this at the moment. Just put a small system on a customer site, it has a database that creates 1GB journal files (not that it handles anything like that volume of data), and at the moment their connectivity is a bit flakey. > > My first thought was "do the flag file last", but a quick search confirms what I thought - that there isn't an option for this. > > So, does anyone have any suggestions how I might reasonably easily get the ability for my script to see if the previous sync completed ? > >