similar to: glob exclude vs include behaviour

Considering you cant INCLUDE a directory that is a symlink... which would be really handy right now for me to resolve a mapping of 103 -> meaningful_name for backups, instead im resorting to temporary bind mounts of 103 onto meaningful_name, and when the bind mount isnt there, the --del is emptying meaningful_name accidentally at times. I think both situations could benefit from a

On Wed, Jul 01, 2015 at 02:05:50PM +0100, Simon Hobson said: >As I read this, the default is to look at the file size/timestamp and if they match then do nothing as they are assumed to be identical. So unless you have specified this, then files which have already been copied should be ignored - the check should be quite low in CPU, at least compared to the "cost" of

And what's performance like? I've heard lots of COW systems performance drops through the floor when there's many snapshots. /kc On Tue, Jul 14, 2015 at 08:59:25AM +0200, Paul Slootman said: >On Mon 13 Jul 2015, Andrew Gideon wrote: >> >> On the other hand, I do confess that I am sometimes miffed at the waste >> involved in a small change to a very

Folks, We faced a similar situation to that which Ken described - we recycle backup directories, for good reason. There is a patch to solve the problem. Our systems administrator provided the following description of the patches we use: ============================================================================ 1. rsync_link_dest improvement by Bryant Hansen Normally, existing files in

Sounds to me like maintaining the metadata cache is important - and tuning the filesystem to do so would be more beneficial than caching writes, especially with a backup target where a write already written will likely never be read again (and isnt a big deal if it is since so few files are changed compared to the total # of inodes to scan). Your report of the minutes for the re-sync shows the

Wow, it took me a few seconds to figure out what you were trying to do. What's wrong with rm? Also I think trying to leverage the side of disqualifying all source files just to get the delete effect (very clever but somewhat obtuse!) risks creating a temporary file of some kind in the target at the start of the operation, and if you cant even mkdir then that exceeds disk quota immediately

What is taking time, scanning inodes on the destination, or recopying the entire backup because of either source read speed, target write speed or a slow interconnect between them? Do you keep a full new backup every day, or are you just overwriting the target directory? /kc On Wed, Jul 01, 2015 at 10:06:57AM +0200, Dirk van Deun said: >> If your goal is to reduce storage, and scanning

How do you handle snapshotting? or do you leave that to the block/fs virtualization layer? /kc On Fri, Apr 17, 2015 at 01:35:27PM +1200, Henri Shustak said: >> Our backup procudures have provision for looking back at previous directories, but there is not much to be gained with recycled directories. Without recycling, and after a failure, the latest available backup may not have much

If your goal is to reduce storage, and scanning inodes doesnt matter, use --link-dest for targets. However, that'll keep a backup for every time that you run it, by link-desting yesterday's copy. Y end up with a backup tree dir per day, with files hardlinked against all other backup dirs. My (and many others) here's solution is to mv $ancientbackup $today; rsync --del

Feature request: allow --link-dest dir to be linked to even if file exists in target. This statement from the man page is adhered to too strongly IMHO: "This option works best when copying into an empty destination hierarchy, as rsync treats existing files as definitive (so it never looks in the link-dest dirs when a destination file already exists)". I was suprised by this behaviour

This has been a consideration. But it pains me that a tiny change/addition to the rsync option set would save much time and space for other legit use cases. We know rsync very well, we dont know ZFS very well (licensing kept the tech out of our linux-centric operations). We've been using it but we're not experts yet. Thanks for the suggestion. /kc On Mon, Apr 06, 2015 at 12:07:05PM

Not to mention the fact that ZFS requires considerable hardware resources (CPU & memory) to perform well. It also requires you to learn a whole new terminology to wrap your head around it. It's certainly not a trivial swap to say the least... Thanks, -Clint On Mon, Apr 6, 2015 at 9:12 AM, Ken Chase <rsync-list-m829 at sizone.org> wrote: > This has been a consideration. But it

Hi all, Here''s what I''ve come up with so far for a pure Ruby Dir.glob for MS Windows. It almost works. The problem right now is the [] notation, which I''m not translating properly into a regex. I haven''t started on the ''**'' notation yet either, but I figure that''s more of a control flow issue. Feel free to disagree with me and/or

Rsync version: rsync-2.5.6-3mdk (Mandrake 9.2) I see from the CVS log that some of the following awkwardness may be fixed (or at least different) in the next public release. I'm looking forward to that. In the interim, here are some problems: --------- Problem 1 - unexpected consequence of replacing / with ** --------- The following exclude works because the explicit slash causes a match

I investigated the rsync code and found the reason why. For every file in the source, it searches the entire filter-list looking to see if that filename is on the exclude/include list. Most aren't, so it compares (350K - 72K) * 72K names (the non-listed files) plus (72K * 72K/2) names (the ones that are listed), for a total of about 22,608,000,000 strcmp's. That's 22 BILLION

Version 3.0.0cvs protocol version 30.PR4 Capabilities: 64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints, socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, append, no ACLs, xattrs, iconv The exclude test fails on OS X Darwin Kernel Version 8.10.1 RELEASE_I386. It just hangs there until CTL-C. The same test passes on the G4 PPC Darwin 8.10.0, and on i686 Xeon

https://bugzilla.samba.org/show_bug.cgi?id=14371 Bug ID: 14371 Summary: Combined Exclude & Protect Filter Type Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 Component: core Assignee: wayne at opencoder.net

https://bugzilla.samba.org/show_bug.cgi?id=3099 --- Comment #8 from Chip Schweiss <chip at innovates.com> --- I would argue that optionally all directory scanning should be made parallel. Modern file systems perform best when request queues are kept full. The current mode of rsync scanning directories does nothing to take advantage of this. I currently use scripts to split a couple

> If your goal is to reduce storage, and scanning inodes doesnt matter, > use --link-dest for targets. However, that'll keep a backup for every > time that you run it, by link-desting yesterday's copy. The goal was not to reduce storage, it was to reduce work. A full rsync takes more than the whole night, and the destination server is almost unusable for anything else when it

https://bugzilla.mindrot.org/show_bug.cgi?id=1634 Summary: [PATCH] openbsd-compat/glob.h conflicts with system glob.h Product: Portable OpenSSH Version: 5.2p1 Platform: All OS/Version: FreeBSD Status: NEW Severity: normal Priority: P2 Component: Miscellaneous AssignedTo: