similar to: Recycling directories and backup performance. Was: Re: rsync --link-dest won't link even if existing file is out of date (fwd)

How do you handle snapshotting? or do you leave that to the block/fs virtualization layer? /kc On Fri, Apr 17, 2015 at 01:35:27PM +1200, Henri Shustak said: >> Our backup procudures have provision for looking back at previous directories, but there is not much to be gained with recycled directories. Without recycling, and after a failure, the latest available backup may not have much

> Our backup procudures have provision for looking back at previous directories, but there is not much to be gained with recycled directories. Without recycling, and after a failure, the latest available backup may not have much in it. Just wanted to point out that LBackup has a number of checks in place to detect failures during a backup. If this happens, then that backup is not labeled as a

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/14/2015 11:35 PM, Henri Shustak wrote: >> Ill take a look but I imagine I cant backup the 80 Million files >> I need to in under the 5 hours i have for nightly >> maintenance/backups. Currently it's possible by recycling >> directories... I would expect that recycling directories actually makes this worse. With an

I am a developer on the LBackup project. An LBackup user recently posted a question to the mailing list asking about the following error. > "rsync: unpack_smb_acl: sys_acl_get_info(): Unknown error: 0 (0)" Link to thread : <http://www.mail-archive.com/lbackup-discussion at lists.connect.homeunix.com/msg00040.html> My understanding of this error is that when copying files via

I THINK YOUR A VERY SAD BUNCH OF LIFES WAISTERS On Wed, Dec 8, 2010 at 7:00 PM, <rsync-request at lists.samba.org> wrote: > Send rsync mailing list submissions to > ? ? ? ?rsync at lists.samba.org > > To subscribe or unsubscribe via the World Wide Web, visit > ? ? ? ?https://lists.samba.org/mailman/listinfo/rsync > or, via email, send a message with subject or body

Feature request: allow --link-dest dir to be linked to even if file exists in target. This statement from the man page is adhered to too strongly IMHO: "This option works best when copying into an empty destination hierarchy, as rsync treats existing files as definitive (so it never looks in the link-dest dirs when a destination file already exists)". I was suprised by this behaviour

As Ben mentioned, ZFS snapshots is one possible approach. Another approach is to have a faster storage system. I have seen considerable speed improvements with rsync on similar data sets by say upgrading the storage sub system. -------------------------------------------------------------------- This email is protected by LBackup, an open source backup solution http://www.lbackup.org

I am involved with the development of lbackup. This message to the rsync mailing list is related to the following thread on the lbackup-disccussion mailing list : http://tinyurl.com/lbackup-discussion-diskfull Essentially, I am curious to if any one using rsync 3.0.7 on Mac OS (10.6) Server has experienced an out of disk space error and not had a message similar to the following reported : >

Hi All, I am writing a backup program for my computer. brief outline is as follows. Running ubuntu 10.04 2 main partitions, / and /home, both ext3. 1 external usb hdd, ext3, mounted to /backups/main. once every couple of days, rsync backs up, using following command, everything worth backing up in / and /home partitions to a folder /backups/main/Full. command: "rsync -vrhRupElog

I am involved with the development of LBackup <http://www.lbackup.org> Recently there was a post to the lbackup-dicussion mailing list. <http://tinyurl.com/lbackup-discussion-device-not> Within the post they reported the following error: > rsync: read > "/Volumes/Drobo_HD/lbackups/theserver.Homes/Section.0/Faculty/andrewmac/.DS_Store": > Device not configured

Folks, We faced a similar situation to that which Ken described - we recycle backup directories, for good reason. There is a patch to solve the problem. Our systems administrator provided the following description of the patches we use: ============================================================================ 1. rsync_link_dest improvement by Bryant Hansen Normally, existing files in

Just chiming in slightly off topic. As a first step if you are going to be backing up files to some media with a computer it would be a really good idea to ensure, that the hardware being used is not faulty. I am not saying that your hardware is faulty. However, it would be worth checking this somehow. Check the drive media for bad blocks, check that all the cables are working well. Ensure the

I have a volume at 192.168.0.2 on my local network. I'd like to rsync the entire volume to a backup volume, skip all files already present on the backup volume (same name and the same or earlier timestamp), and include all flags, dates, etc. as applicable to Mac OS X 10.6.x. Can someone tell me the proper command to do this.

Kevin, Thanks for the reply and interest in this topic. Comments below. Regards Rob. > I did consider that but rejected it for 2 reasons... > > 1. Backup run time. We have a 4 hour window to run backups at night. > Using recycled directories significantly extended the backup run > time. The deletion time is eliminated but frankly, we have the other > 20 hours of the day

Folks, Kevin Korb wrote: > Have you considered more advanced methods such as subvolume snapshots > provided by ZFS and BTRFS? At work we were forced to abandon rsync > - --link-dest because of the amount of time it takes to delete old > backups when the data is primarily many millions of small files > (shared web hosting company). We don't have more advanced methods like

Folks, Thinking again about the issue of changed attributes for a file in source that is identical in content to a file in destination that is hard linked get propagated to the --link-dest directories I would like to suggest a new flag, (perhaps called something like --preserve-linked-attributes), to allow callers to get the behaviour that is needed to allow the use of recycled

rsync Folks, The following explanatory text is by me and the patches are by Rowan McKenzie for use by the Advanced Scientific Computing group at CSIRO. This patch builds upon the --link-dest patch by Bryant Hansen (Thanks heaps!). 1. The original patch provided an alternate behaviour for rsync when using the --link-dest option. When there are identical files in the source and link-dest areas,

Thanks to Kevin and Paul for responses. We use a modified Tower of Hanoi scheme (on top of rsync and --link-dest and recycling) for deciding which backups to keep. Here is a sample of our holdings for one area: home.20111124.seq.0 set 0 home.20130512.seq.512 set 10 home.20140203.seq.768 set 9 home.20140414.seq.832 set 7 home.20140708.seq.896 set 8 home.20140815.seq.928 set 6

I am wondering if there is an option which is similar to --progress that will not display the file names. A percentage indicator would be fine. There is some related discussion at the following URL : http://lists.samba.org/archive/rsync/2004-March/008998.html Any replies regarding ways to achieve this kind of output are welcomed. In addition, a reply like "sorry not supported yet"

I'm using rsync 3.0.7 on Mac OS X 10.6, compiled according to Mike Bombich's instructions at http://www.bombich.com/rsync.html. Rsync repeatedly exits with a protocol data stream error when trying to copy some com.apple.FinderInfo extended attributes. While testing this issue, I found that rsync is not actually copying all extended attributes even when there is no error message. I'm