Hello,
Since the very first day I've been using rsync - some 15 years ago -
the filtering rules caused great grieve. Their behaviour is just not
the way I'd expect it be be and as I read the manpage. Usually I end
up with some hand-written recipes, carefully documented,y including all
the gotchas.
This time however I failed and I see no other way than to ask for
advice.
Given the following structure
project/
.rsync-filter
project.git/
.git/
Now, the following command (rsync 3.1.1)
rsync -av --del --deleted-excluded -F \
/path/to/project/ /path/to/backup/project/
should transfer
- sync: project/project.git/.git/
- skip: Everything else in project/project.git/
- sync: Everything else in project/
In other words: Don't transfer the git repo except for .git/ itself.
Yes, this means files not checked in are lost, that's by intention.
Now, how is .rsync-filter supposed to look? The first of many failing
attempts was
+ *.git/.git/*
- *.git/
This however kills the entire project.git/ directory, in violation of
"the first matching pattern is acted on". Given previous bad
experiences I've tried (using a script) all 128 combination of
- line order
- '+' or '-' at the start of any line
- '*' appended to any line, or not, with without trailing slash
but no avail.
Any clue?
Christoph
Try: + *.git/ + *.git/.git/*** - *.git/* (you probably want to add --prune-empty-dirs) Also, when debugging filters/excludes use -vv as it will tell you which pattern is causing it to do what. Also, there is no point in using -v without --itemize-changes. On 05/03/2016 04:07 PM, Christoph Biedl wrote:> Hello, > > Since the very first day I've been using rsync - some 15 years ago - > the filtering rules caused great grieve. Their behaviour is just not > the way I'd expect it be be and as I read the manpage. Usually I end > up with some hand-written recipes, carefully documented,y including all > the gotchas. > > This time however I failed and I see no other way than to ask for > advice. > > Given the following structure > > project/ > .rsync-filter > project.git/ > .git/ > > Now, the following command (rsync 3.1.1) > > rsync -av --del --deleted-excluded -F \ > /path/to/project/ /path/to/backup/project/ > > should transfer > > - sync: project/project.git/.git/ > - skip: Everything else in project/project.git/ > - sync: Everything else in project/ > > In other words: Don't transfer the git repo except for .git/ itself. > Yes, this means files not checked in are lost, that's by intention. > > Now, how is .rsync-filter supposed to look? The first of many failing > attempts was > > + *.git/.git/* > - *.git/ > > This however kills the entire project.git/ directory, in violation of > "the first matching pattern is acted on". Given previous bad > experiences I've tried (using a script) all 128 combination of > > - line order > - '+' or '-' at the start of any line > - '*' appended to any line, or not, with without trailing slash > > but no avail. > > Any clue? > > Christoph >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20160503/bb95e30a/signature.sig>
On Tue, May 3, 2016 at 1:07 PM, Christoph Biedl <cbiedl at gmx.de> wrote:> + *.git/.git/* > - *.git/ >>From the man page near the start of the "INCLUDE/EXCLUDE PATTERN RULES"section: *Note that, when using the --recursive (-r) option (which is implied by> -a), every subcomponent of every path is visited from the top down, so > include/exclude patterns get applied recursively to each subcomponent’s > full name (e.g. to include "/foo/bar/baz" the subcomponents "/foo" and > "/foo/bar" must not be excluded). The exclude patterns actually > short-circuit the directory traversal stage when rsync finds the files to > send.*Thus, your latter exclude prevents the first include from ever seeing any data that it could match. You'd need to use something more like this: + *.git/.git/ - *.git/* ..wayne.. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20160504/01c86246/attachment.html>
I hate to say anything remotely negative to Wayne but... That wording from the man page makes almost no sense without the examples directly after it (and I have read it many times and know what it is saying). When I go all RTFM on this topic I usually tell them to 'man rsync', search for file-will-not-be-found and start reading from that line. Once you understand the broken and correct examples you can decipher what the explanation above means. On 05/04/2016 09:03 PM, Wayne Davison wrote:> > On Tue, May 3, 2016 at 1:07 PM, Christoph Biedl <cbiedl at gmx.de > <mailto:cbiedl at gmx.de>> wrote: > > + *.git/.git/* > - *.git/ > > > From the man page near the start of the "INCLUDE/EXCLUDE PATTERN RULES" > section: > > /Note that, when using the --recursive (-r) option (which is implied > by -a), every subcomponent of every path is visited from the top > down, so include/exclude patterns get applied recursively to each > subcomponent’s full name (e.g. to include "/foo/bar/baz" the > subcomponents "/foo" and "/foo/bar" must not be excluded). The > exclude patterns actually short-circuit the directory traversal > stage when rsync finds the files to send./ > > > Thus, your latter exclude prevents the first include from ever seeing > any data that it could match. You'd need to use something more like this: > > + *.git/.git/ > - *.git/* > > > ..wayne.. > >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20160504/ccd38bc6/signature.sig>
Wayne Davison wrote...> On Tue, May 3, 2016 at 1:07 PM, Christoph Biedl <cbiedl at gmx.de> wrote: > > > + *.git/.git/* > > - *.git/ > > > > >From the man page near the start of the "INCLUDE/EXCLUDE PATTERN RULES" > section: > > *Note that, when using the --recursive (-r) option (which is implied by > > -a), every subcomponent of every path is visited from the top down, so > > include/exclude patterns get applied recursively to each subcomponent’s > > full name (e.g. to include "/foo/bar/baz" the subcomponents "/foo" and > > "/foo/bar" must not be excluded). The exclude patterns actually > > short-circuit the directory traversal stage when rsync finds the files to > > send.*Yeah, things like this are the reason why I got the impression rsync's filter behaviour is acting strange. It certainly makes perfect sense once you've understood in great detail how it works - until then you're just confused.> Thus, your latter exclude prevents the first include from ever seeing any > data that it could match. You'd need to use something more like this: > > + *.git/.git/ > - *.git/*No, that one kills files inside *.git/.git/, too. Kevin's suggestion however does the trick. Thanks! Another working solution: + /*.git/.git/ - /*.git/* Christoph