similar to: Backup scripts - recycling old backup directories (Kevin Korb)

Folks, Kevin Korb wrote: > Have you considered more advanced methods such as subvolume snapshots > provided by ZFS and BTRFS? At work we were forced to abandon rsync > - --link-dest because of the amount of time it takes to delete old > backups when the data is primarily many millions of small files > (shared web hosting company). We don't have more advanced methods like

rsync folks, Henri Shustak <henri.shustak at gmail.com> wrote: > LBackup always starts a new backup snapshot with an empty directory. I > have been looking at extending --link-dest options to scan beyond just > the previous successful backup to (failed backups / older backups). > However, there are all kinds of edge cases which are worth considering > with such a changes. At

Folks, We use rsync extensively for protecting data by making backups. Thank you to the authors and maintainers. Like many others, we use the --link-dest option to cut down on the space occupied by the backups. Unlike many others, we re-cycle old backup directories. Since most file systems change only slowly (ours average about 0.5% of files and about 1.5% of data being churned each day), a

Folks, Am 20.06.2016 um 22:01 schrieb Larry Irwin (gmail): > The scripts I use analyze the rsync log after it completes and then sftp's a summary to the root of the just completed rsync. > If no summary is found or the summary is that it failed, the folder rotation for that set is skipped and that folder is re-used on the subsequent rsync. > The key here is that the folder

Thanks to Kevin and Paul for responses. We use a modified Tower of Hanoi scheme (on top of rsync and --link-dest and recycling) for deciding which backups to keep. Here is a sample of our holdings for one area: home.20111124.seq.0 set 0 home.20130512.seq.512 set 10 home.20140203.seq.768 set 9 home.20140414.seq.832 set 7 home.20140708.seq.896 set 8 home.20140815.seq.928 set 6

rsync Folks, The following explanatory text is by me and the patches are by Rowan McKenzie for use by the Advanced Scientific Computing group at CSIRO. This patch builds upon the --link-dest patch by Bryant Hansen (Thanks heaps!). 1. The original patch provided an alternate behaviour for rsync when using the --link-dest option. When there are identical files in the source and link-dest areas,

Folks, Thinking again about the issue of changed attributes for a file in source that is identical in content to a file in destination that is hard linked get propagated to the --link-dest directories I would like to suggest a new flag, (perhaps called something like --preserve-linked-attributes), to allow callers to get the behaviour that is needed to allow the use of recycled

Kevin, Thanks for your response. Some observations are inter-lined below. Rob. Regards Rob. Bell e-mail: Robert.Bell at csiro.au -- Dr Robert C. Bell, BSc (Hons) PhD Technical Services Manager Advanced Scientific Computing CSIRO IM&T Phone: +61 3 9669 8102 | Mobile: +61 428 108 333 | CSIRO 93 3810 Robert.Bell at csiro.au | http://www.csiro.au/ | http://www.hpsc.csiro.au/

> Our backup procudures have provision for looking back at previous directories, but there is not much to be gained with recycled directories. Without recycling, and after a failure, the latest available backup may not have much in it. Just wanted to point out that LBackup has a number of checks in place to detect failures during a backup. If this happens, then that backup is not labeled as a

How do you handle snapshotting? or do you leave that to the block/fs virtualization layer? /kc On Fri, Apr 17, 2015 at 01:35:27PM +1200, Henri Shustak said: >> Our backup procudures have provision for looking back at previous directories, but there is not much to be gained with recycled directories. Without recycling, and after a failure, the latest available backup may not have much

Peter, I was about to post the below to the rsync mailing list, but wondered whether your code which captures rsync output may have allowed rsync to continue past what I think would normally be a fatal error? Thanks Rob. P.S. Should I put the final backups of the old pearcey /home back into their correct slots - 5 backups were recycled, but nothing was added to them? The .Snapshots area is

Folks, We faced a similar situation to that which Ken described - we recycle backup directories, for good reason. There is a patch to solve the problem. Our systems administrator provided the following description of the patches we use: ============================================================================ 1. rsync_link_dest improvement by Bryant Hansen Normally, existing files in

Folks, bugs at stupidkitties.com wrote: " I was wondering if it'd be feasible to add an option (or multiple options, depending on the desired granularity) to cause rsync to treat changes in file attributes the same way it treats changes to the file's contents when it encounters hard links at the destination. Maybe something like --relink-p or something?" One of colleagues has

rsync watchers, I've been using rsync version 3.1.1+, and do like the improved --stats output: e.g. Number of files: 2480403 (reg: 2034290, dir: 233487, link: 212174, special: 452) Number of created files: 356 (reg: 6, dir: 3, link: 347) Number of regular files transferred: 16 ... - thanks to the developers. However, I want to be able to look at the --stats output and make a decision

Hi everyone, I'm thinking about breaking up a large package I have to avoid loading some of the more specialised parts when it isn't necessary. The package is dependent on shared libraries. The base one will keep all of the core stuff like constructors etc and the specialised parts will need to access these. It is obviously possible to set the "local" flag to false, however

Way back: > > ---------- Forwarded message ---------- Date: Mon, 2 Jun 2003 > > 09:56:50 -0700 (PDT) From: Michael Rubel <mrubel at > > galcit.caltech.edu> To: rsync at lists.samba.org Subject: > > --link-dest when target and compare_dir both have file > > > > Hi J.W. et al, > > > > Kevin Everets was kind enough to inform me about some

Paul Wayne, Kevin, Teodor and others, Thanks for your contributions in response to my postings. Paul: I was very imprecise if not plain wrong in my description. :-( Thanks for explaining what really happens. > "Rsync will not update an existing file in-place unless you use the > --inplace option. So --whole-file is irrelevant for this. > Rsync (without --inplace) will always

Thanks-- as Duncan Murdoch also noted, R_HISTFILE can be used to solve my problem, as below. BTW it's not easy to find out about R_HISTFILE in the help system or manuals, though, unless you already know it exists-- it only seems to be described under startup options, not things to do with history. However, Duncan's reply did hint at an inconsistency, in that savehistory() defaults to

Sorry if this has been covered here somewhere in the past, but ... Does anyone know why logical vectors are *silently* recycled, even when they are incommensurate lengths, when doing logical indexing? This is as documented: For ?[?-indexing only: ?i?, ?j?, ?...? can be logical vectors, indicating elements/slices to select. Such vectors are recycled if necessary to match

Hi, The following function use to work, but now it doesn't giving the error "> CallSnow(, 100) Using snow package, asking for 2 nodes 2 slaves are spawned successfully. 0 failed. Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: no applicable method for 'lapply' applied to an object of class "list" ". Where this is the