Hi all, I thought I understand how include/exclude and their precedences work (I read the user guide about this), but I seem to miss something. I have 2 questions about this, separated by ===== Question 1: -------------- I want to exclude the "hidden" directories except some sub-dirs or files: .mybackup and its files) and the mozilla firefox bookmarks folder and its contents (.mozilla/firefox/plp6wzjm.default/bookmarkbackups/*) So my exclude-files contains: .* .mybackup/backup.log.* .mozilla/extensions *.*~ Desktop Downloads Movies Music Picturestmp Videos the include file contains: .kde4/share/apps/kabc/distlists .kde4/share/apps/kabc/std.vcf .mozilla/ .mozilla/firefox/ .mozilla/firefox/plp6wzjm.default/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* .mybackup Result: 1. Success: with both the .mybackup and the 2 .kde4/share/apps/kabc-files 2. Success: indeed I have my bookmarks in the ../bookmarkbackups/ folder 3. Failure: I have all other files, sub-dirs and their content as well in .mozilla/firefox/plp6wzjm.default. 4. when adding .mozilla/firefox/plp6wzjm.default/Cache .mozilla/firefox/plp6wzjm.default/chrome .mozilla/firefox/plp6wzjm.default/extensions .mozilla/firefox/plp6wzjm.default/firebug .mozilla/firefox/plp6wzjm.default/OfflineCache to the exclude file this folders are not in, but the "normal" .mozilla/firefox/plp6wzjm.default/ files are. Question: it seems that I miss the point how to do this. can you please give a hint about what I ? doing wrong? rsync command )some date-time variable for files, but that is working): rsync -axhhX --no-perms --omit-dir-times\ --backup --backup-dir=$NiceDate/updated --suffix=.$Now\ --delete --delete-excluded\ --backup-dir-dels=$NiceDate/deleted --suffix-dels=.`date +%H%M`\ --log-file=/home/hans/.mybackup/backup.log.$RawDate-$Now\ --include-from=/home/hans/.mybackup/include.txt\ --exclude-from=/home/hans/.mybackup/exclude.txt\ /home/hans /mnt/backup ======================================2nd, similar question: to play around with this stuff I wanted to test with the hidden files only (.*), so I tried to add all non-hidden directories and normal files to this exclude-test.txt, but did not get it working (it worked too much): exclude-test.txt .* .mybackup/backup.log.* [a-z]* [A-Z]* while my include-test is: .kde4/share/apps/kabc/distlists .kde4/share/apps/kabc/std.vcf .mozilla/ .mozilla/firefox/ .mozilla/firefox/plp6wzjm.default/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* .mybackup In this case I don't get any files backed up (dry-run, so checked in log-file only). same rsync command (other log-file and destination) ==============Thanks for any help, regards, Hans Troost
On 10-09-10 21:13, Hans Troost wrote:> > So my exclude-files contains: > .* > > the include file contains: > .kde4/share/apps/kabc/distlists > .kde4/share/apps/kabc/std.vcf > > > Result: > 1. Success: 2 .kde4/share/apps/kabc-filesAre you sure about that part ? '.*' ('H, .*' as seen by sending rsync) should short-circuit any .kde/ directory subtree and exclude it from file list/transfer. You would need something more elaborate to get just those two fles from .kde4 and nothing else. Anyway, putting that detail aside and if I'm reading things right. Your rules (combined) look like: +, .mozilla/ +, .mozilla/firefox/ +, .mozilla/firefox/plp6wzjm.default/ +, .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* +, .mybackup H, .* H, .mybackup/backup.log.* H, .mozilla/extensions H, *.*~ H, Desktop H, Downloads H, Movies H, Music H, Picturestmp H, Videos +, * (implicit final rule) H - because you added --delete-excluded, so receiving side will not protect them from deletion. For receiving rsync that means just: R, * (as "consider everything not part of the transfer a candidate to delete") For sending rsync: S, .mozilla/ S, .mozilla/firefox/ S, .mozilla/firefox/plp6wzjm.default/ S, .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* S, .mybackup H, .* <- this makes following 2 rules redundant H, .mybackup/backup.log.* H, .mozilla/extensions H, *.*~ H, Desktop H, Downloads H, Movies H, Music H, Picturestmp H, Videos S, * (implicit final rule) Which does exactly what you experienced:> 3. Failure: I have all other files, sub-dirs and their content as well in > .mozilla/firefox/plp6wzjm.default.They were all caught by the final implicit include, and none of them matched your H (exclude) rules.> 4. when adding > .mozilla/firefox/plp6wzjm.default/Cache > .mozilla/firefox/plp6wzjm.default/chrome > .mozilla/firefox/plp6wzjm.default/extensions > .mozilla/firefox/plp6wzjm.default/firebug > .mozilla/firefox/plp6wzjm.default/OfflineCache > to the exclude file this folders are not in, but the "normal" > .mozilla/firefox/plp6wzjm.default/ files are.Being part of exclude rules, they will be catched and excluded from transfer, but the rest will not. Note, that nothing from .mybackup that matches H (exclude) rules will be backed up.> > Question: it seems that I miss the point how to do this. can you please > give a hint about what I ? doing wrong? > > rsync command )some date-time variable for files, but that is working): > > rsync -axhhX --no-perms --omit-dir-times\ > --backup --backup-dir=$NiceDate/updated --suffix=.$Now\ > --delete --delete-excluded\ > --backup-dir-dels=$NiceDate/deleted --suffix-dels=.`date +%H%M`\ > --log-file=/home/hans/.mybackup/backup.log.$RawDate-$Now\ > --include-from=/home/hans/.mybackup/include.txt\ > --exclude-from=/home/hans/.mybackup/exclude.txt\ > /home/hans /mnt/backup > > ======================================> 2nd, similar question: > > to play around with this stuff I wanted to test with the hidden files only > (.*), so I tried to add all non-hidden directories and normal files to this > exclude-test.txt, but did not get it working (it worked too much): > > exclude-test.txt > .* > .mybackup/backup.log.* > [a-z]* > [A-Z]* > > while my include-test is: > .kde4/share/apps/kabc/distlists > .kde4/share/apps/kabc/std.vcf > .mozilla/ > .mozilla/firefox/ > .mozilla/firefox/plp6wzjm.default/ > .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* > .mybackup > > In this case I don't get any files backed up (dry-run, so checked in log-file > only).Hmm, you should get .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* .mybackup/ backed up. Otherwise the rest will be hidden from transfer (to be precise, any file/directory starting with a letter or .).
Hi Michal, (hope this mail is short enough for the list, previous one bounced: too big) Read, re-read, read again both your mail and the rsync-documentation (again) and think I get it now. First, regarding your comments (not all): 1. you are right, the .kabc\....-files were missing (did not check it properly). Now I have them (see my solution below) rest of your comments let me "understand what I read in the manual". One remark: the Exclude ".mybackup/backup*.log.* is not redundant, due to the Include ".mybackup". The ".mozilla/extensions" was redundant. 2 Excluding the [A-Z] and [a-z] works when specifying this: hans/[A-Z]* etc. Now I have the following which works perfect for me (test -version): rsync command: rsync -n -vvvvv -axhhX --no-perms --omit-dir-times --stats\ --backup --backup-dir=`date +%Y-%m-%d`/updated --suffix=.`date +%H%M`\ --delete --delete-excluded\ --backup-dir-dels=`date +%Y-%m-%d`/deleted --suffix-dels=.`date +%H%M`\ --log-file=/home/hans/.mybackup/backup-test.log.`date +%Y%m%d`-`date +%H%M`\ --include-from=/home/hans/.mybackup/include.txt\ --exclude-from=/home/hans/.mybackup/exclude.txt\ /home/hans /mnt/backup/test/ I herewith attach both the include.txt and exclude .txt and the listing of backed-up files. Best regards, Hans Troost -------------- next part -------------- # # to get KAdressbook entries + distribution lists # .kde4/ .kde4/share/ .kde4/share/apps/ .kde4/share/apps/kabc/ .kde4/share/apps/kabc/distlists .kde4/share/apps/kabc/std.vcf # # to get firefox bookmarks # .mozilla/ .mozilla/firefox/ .mozilla/firefox/plp6wzjm.default/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* # # to get the backup-scripts and associated files like this # .mybackup -------------- next part -------------- .* .mybackup/backup*.log.* .mozilla/* .mozilla/firefox/* .mozilla/firefox/plp6wzjm.default/* .kde4/* .kde4/share/* .kde4/share/apps/* .kde4/share/apps/kabc/* *.*~ Desktop Downloads Movies Music Pictures tmp Videos # # for testing purposes only will be commented out on real use # hans/[a-z]* # hans/[A-Z]* # hans/2* -------------- next part -------------- /mnt/backup/test: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 6 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 hans/ /mnt/backup/test/hans: total 0 drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 .kde4/ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 .mozilla/ drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 .mybackup/ /mnt/backup/test/hans/.kde4: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 share/ /mnt/backup/test/hans/.kde4/share: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 apps/ /mnt/backup/test/hans/.kde4/share/apps: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 kabc/ /mnt/backup/test/hans/.kde4/share/apps/kabc: total 28 drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ -rwxr-xr-x 1 hans hans 202 2010-09-11 11:55 distlists* -rwxr-xr-x 1 hans hans 21968 2010-09-11 11:55 std.vcf* /mnt/backup/test/hans/.mozilla: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 firefox/ /mnt/backup/test/hans/.mozilla/firefox: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 plp6wzjm.default/ /mnt/backup/test/hans/.mozilla/firefox/plp6wzjm.default: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 bookmarkbackups/ /mnt/backup/test/hans/.mozilla/firefox/plp6wzjm.default/bookmarkbackups: total 216 drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ -rwxr-xr-x 1 hans hans 33921 2010-09-05 19:16 bookmarks-2010-09-05.json* -rwxr-xr-x 1 hans hans 33921 2010-09-06 20:08 bookmarks-2010-09-06.json* -rwxr-xr-x 1 hans hans 34164 2010-09-07 21:25 bookmarks-2010-09-07.json* -rwxr-xr-x 1 hans hans 34164 2010-09-08 23:14 bookmarks-2010-09-08.json* -rwxr-xr-x 1 hans hans 34164 2010-09-10 14:30 bookmarks-2010-09-10.json* -rwxr-xr-x 1 hans hans 34164 2010-09-11 20:42 bookmarks-2010-09-11.json* /mnt/backup/test/hans/.mybackup: total 64 drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 ../ -rwxr-xr-x 1 hans hans 471 2010-09-10 20:44 backup-dry-run.sh* -rwxr-xr-x 1 hans hans 1305 2010-09-08 17:06 backup-old.sh* -rwxr-xr-x 1 hans hans 749 2010-09-10 19:43 backup.sh* -rwxr-xr-x 1 hans hans 457 2010-09-11 20:54 backup-test.sh* -rwxr-xr-x 1 hans hans 101 2010-09-10 20:48 exclude-old .txt* -rwxr-xr-x 1 hans hans 39 2010-09-10 21:01 exclude-test.txt* -rwxr-xr-x 1 hans hans 309 2010-09-11 21:08 exclude.txt* -rwxr-xr-x 1 hans hans 187 2010-09-10 19:24 include-test.txt* -rwxr-xr-x 1 hans hans 448 2010-09-11 21:08 include.txt* -rwxr-xr-x 1 hans hans 4480 2010-08-30 20:26 log-file legend.txt* -rwxr-xr-x 1 hans hans 1656 2010-08-31 20:37 mailing-list-question-01.txt* -rwxr-xr-x 1 hans hans 9132 2010-08-20 17:38 rsync.txt* -rwxr-xr-x 1 hans hans 3422 2010-09-11 21:04 test-results.txt*