rsync - Jun 2008 - OS/X Leopard Server and rsync backups

I'm dithering over this too much, I need some advice. I apologize for  
the length of this and hope I haven't left out any essential detail.

I want to do nightly snapshots of my Leopard server using an rsync  
script and a filter file to a locally connected firewire drive.

The rync command I'm considering is:
> rsync -vq -a -x --delete-excluded -A -X --filter=". $FLTR"
--link-
> dest=$OLD --log-file=$LOG // $NEW
and the filter file I'm considering is:
> - /Network/*
> - /Previous Systems.localized
> - /Volumes/*
> - /afs/*
> - /automount/*
> - /cores/*
> - /dev/*
> - /private/tmp/*
> - /private/var/imap/socket/*
> - /private/var/run/*
> - /private/var/spool/postfix/*
> - /private/var/spool/postfix/private/*
> - /private/var/spool/postfix/public/*
> - /private/var/vm/*
> - /proc
> - /tmp/*
> : .rsync-filter
> - */.Trash
> - .Spotlight-*/
The version of rsync is 3.0.2 from macports.

Here's what I think this all does (in other words, what I want it to  
do):

-vq	: produce error messages
-a	: archive mode
-x	: only one file system
--delete-excluded	: delete files no longer present & any excluded files
-A	: copy acls
-X	: copy extra attribute
--filter=". $FLTR"	: load filter from file $FLTR
--link-dest=$OLD	: copy only files that are different from those in  
directory $OLD
--log-file=$LOG		: write log into file $LOG
//	: the base of the source tree is the filesystem root directory
$NEW	: the copy is placee immediately under diectory $NEW

In the filter file, a line of the form "- /SOME-PATH/*" means copy the
directory /SOME-PATH but nothing under it.

A line of the form ": .rsync-filter" means incorporate filter rules  
from any .rsync-filter file found and apply them to everything in and  
beneath its directory.

The line "- */.Trash" means don't copy any .Trash file, even the
one
in the root directory.

The line "- .Spotlight-*/" means copy all directories whose names  
match ".Spotlight-*" but nothing beneath them.

The filter file is sort of the union of several examples I found on  
the Internet, I haven't been able to locate a "definitive" filter
file
for OS/X server. Do I have too much or too little in there?

So I'm hoping that someone who has some experience with this could  
look this over and let me know of any looming problems I haven't seen.

I have one other question: there are several different kinds of  
databases in /private/var, including openldap, squirrelmail, and so  
on. It seems to me that one way to handle the problem of getting valid  
backups of a variety of different database types would be to (somehow)  
bring the system down long enough to rsync /private/var only, then  
bring it back up and do the rest of the snapshot. This would probably  
be less than five minutes of downtime. Alternatively, the whole  
snapshot could be done with the system in single-user mode. I don't  
yet have a feel for how long the incremental snapshot will take. I  
could probably live with an hour or less per night of downtime,  
especially if it substantially increased the value of the snapshot.

Thanks in advance for any suggestions or comments.

Cheers,
Greg Shenaut

Few remarks:

Greg Shenaut wrote:
> 
> 
> --delete-excluded    : delete files no longer present & any excluded
files
Yes.

You can achieve the same with 'H' or 'R' instead of '-',
without having
to specify --delete-excluded. 'H' is sender-only exclude, 'R' is
receiver-only include.

In your case:

H somefile

will cause the extra file on the receiving side to disappear, unless 
it's matched by some other rule later, that would protect it otherwise.

Similary:

R somefile
- some*

would work as expected as well. Extra file on the receiver would be 
deleted, if it wasn't transferred - sender will not send it, and 'R'
will match before '-' on the receiving side. But e.g.
'someotherfile'
would match '-' on both sides, so it wouldn't be deleted, if it
wasn't
transferred.
> --link-dest=$OLD    : copy only files that are different from those in 
> directory $OLD
and hardlink the unchanged ones.
> //    : the base of the source tree is the filesystem root directory
One slash is enough.
> 
> A line of the form ": .rsync-filter" means incorporate filter
rules from
> any .rsync-filter file found and apply them to everything in and beneath 
> its directory.
> 
Yes. Careful about delete modes though - you will need --delete-after, 
if the per-dir rules are being transferred, and they differ from the 
ones already present on the receiving side (and you expect the new ones 
to influence deletions).
> The line "- */.Trash" means don't copy any .Trash file, even
the one in
> the root directory.
> 
* alone will match any non-empty component, so .Trash (dir or file) in 
the root will not be excluded.

- .Trash

would do the thing - the rule is inherited in every subdirectory (unless 
you inhibit that with .n $FLTR ), so it will match anything .Trash it 
encounters.

> The line "- .Spotlight-*/" means copy all directories whose names
match
> ".Spotlight-*" but nothing beneath them.
This one will exclude any directory that matches .Spotlight-* . So 
neither dir, nor its contents will be copied (as rsync won't visit the 
directory in the first place). If you have regular file matching 
.Spotlight-*, it will be copied though.


Btw - try running some test case rsync with -vv / -vvv / -i and/or with 
custom logging options - you will see when and how rsync does things.