similar to: rsync - using a --files-from list to cut out scanning. How to handle deletions?

Kevin, Thanks for your response. Some observations are inter-lined below. Rob. Regards Rob. Bell e-mail: Robert.Bell at csiro.au -- Dr Robert C. Bell, BSc (Hons) PhD Technical Services Manager Advanced Scientific Computing CSIRO IM&T Phone: +61 3 9669 8102 | Mobile: +61 428 108 333 | CSIRO 93 3810 Robert.Bell at csiro.au | http://www.csiro.au/ | http://www.hpsc.csiro.au/

Paul Wayne, Kevin, Teodor and others, Thanks for your contributions in response to my postings. Paul: I was very imprecise if not plain wrong in my description. :-( Thanks for explaining what really happens. > "Rsync will not update an existing file in-place unless you use the > --inplace option. So --whole-file is irrelevant for this. > Rsync (without --inplace) will always

Folks, Thinking again about the issue of changed attributes for a file in source that is identical in content to a file in destination that is hard linked get propagated to the --link-dest directories I would like to suggest a new flag, (perhaps called something like --preserve-linked-attributes), to allow callers to get the behaviour that is needed to allow the use of recycled

rsync Folks, The following explanatory text is by me and the patches are by Rowan McKenzie for use by the Advanced Scientific Computing group at CSIRO. This patch builds upon the --link-dest patch by Bryant Hansen (Thanks heaps!). 1. The original patch provided an alternate behaviour for rsync when using the --link-dest option. When there are identical files in the source and link-dest areas,

Folks, Kevin Korb wrote: > Have you considered more advanced methods such as subvolume snapshots > provided by ZFS and BTRFS? At work we were forced to abandon rsync > - --link-dest because of the amount of time it takes to delete old > backups when the data is primarily many millions of small files > (shared web hosting company). We don't have more advanced methods like

Folks, Am 20.06.2016 um 22:01 schrieb Larry Irwin (gmail): > The scripts I use analyze the rsync log after it completes and then sftp's a summary to the root of the just completed rsync. > If no summary is found or the summary is that it failed, the folder rotation for that set is skipped and that folder is re-used on the subsequent rsync. > The key here is that the folder

Kevin, Thanks for the reply and interest in this topic. Comments below. Regards Rob. > I did consider that but rejected it for 2 reasons... > > 1. Backup run time. We have a 4 hour window to run backups at night. > Using recycled directories significantly extended the backup run > time. The deletion time is eliminated but frankly, we have the other > 20 hours of the day

rsync watchers, I've been using rsync version 3.1.1+, and do like the improved --stats output: e.g. Number of files: 2480403 (reg: 2034290, dir: 233487, link: 212174, special: 452) Number of created files: 356 (reg: 6, dir: 3, link: 347) Number of regular files transferred: 16 ... - thanks to the developers. However, I want to be able to look at the --stats output and make a decision

Way back: > > ---------- Forwarded message ---------- Date: Mon, 2 Jun 2003 > > 09:56:50 -0700 (PDT) From: Michael Rubel <mrubel at > > galcit.caltech.edu> To: rsync at lists.samba.org Subject: > > --link-dest when target and compare_dir both have file > > > > Hi J.W. et al, > > > > Kevin Everets was kind enough to inform me about some

Wayne, Thanks for the information - that looks useful for the future, and would replace our option. Rob. > Date: Fri, 24 Aug 2012 23:10:43 -0700 > From: Wayne Davison <wayned at samba.org> > To: Robert Bell <Robert.Bell at csiro.au> > Cc: rsync at lists.samba.org > Subject: Re: [patch] link-dest messages and max-size warnings (fwd) > > On Thu, Aug 23, 2012 at

Folks, bugs at stupidkitties.com wrote: " I was wondering if it'd be feasible to add an option (or multiple options, depending on the desired granularity) to cause rsync to treat changes in file attributes the same way it treats changes to the file's contents when it encounters hard links at the destination. Maybe something like --relink-p or something?" One of colleagues has

Dear rsync folks, Re the recent discussion of: Subject: RE: Does files-from work with --delete? We have come across the same issue, in doing partial backups of a file system and using the --files-from and --link-dest feature to a re-cycled area. Files that should be deleted tend to build up. I mentioned this issue to a colleague. He suggested that after the backup is done, a reverse rsync -n

rsync folks, Henri Shustak <henri.shustak at gmail.com> wrote: > LBackup always starts a new backup snapshot with an empty directory. I > have been looking at extending --link-dest options to scan beyond just > the previous successful backup to (failed backups / older backups). > However, there are all kinds of edge cases which are worth considering > with such a changes. At

Folks, We faced a similar situation to that which Ken described - we recycle backup directories, for good reason. There is a patch to solve the problem. Our systems administrator provided the following description of the patches we use: ============================================================================ 1. rsync_link_dest improvement by Bryant Hansen Normally, existing files in

Thanks to Kevin and Paul for responses. We use a modified Tower of Hanoi scheme (on top of rsync and --link-dest and recycling) for deciding which backups to keep. Here is a sample of our holdings for one area: home.20111124.seq.0 set 0 home.20130512.seq.512 set 10 home.20140203.seq.768 set 9 home.20140414.seq.832 set 7 home.20140708.seq.896 set 8 home.20140815.seq.928 set 6

Looks like there is a buglet in 'installed.packages', around line 17: for (lib in lib.loc) { dest <- file.path(tempdir(), paste("libloc_", URLencode(lib,=20 TRUE), paste(fields, collapse =3D ","), ".rds", sep =3D "")) if (!noCache && file.exists(dest) && file.info(dest)$mtime >=20

Dear r-help, I have a radar data file in the following format: nrays for (1 to n_rays) { some_data_info for (1 to n_gates) { some_data_info} } N_rays=360 and n_gates=950 The data looks something like: Header line1 Ray1 header gate1 var1, var2,.....,varN ... gate950 var1, var2,.....,varN Ray2 header gate1 var1, var2,.....,varN ...

On 15/04/07, Silvia Pfeiffer <silviapfeiffer1@gmail.com> wrote: > Hi Ian, > > hmm, there was a whole bunch up on media.aidabrowser.com, but it seems > that CSIRO have taken that site offline. > > I thought there was a version of the media in cmmlwiki that produces > annodex files, but there may be a bug on the server that prevents > that. Conrad - is this a known

Peter, I was about to post the below to the rsync mailing list, but wondered whether your code which captures rsync output may have allowed rsync to continue past what I think would normally be a fatal error? Thanks Rob. P.S. Should I put the final backups of the old pearcey /home back into their correct slots - 5 backups were recycled, but nothing was added to them? The .Snapshots area is

> If your goal is to reduce storage, and scanning inodes doesnt matter, > use --link-dest for targets. However, that'll keep a backup for every > time that you run it, by link-desting yesterday's copy. The goal was not to reduce storage, it was to reduce work. A full rsync takes more than the whole night, and the destination server is almost unusable for anything else when it