rsync - Jun 2005 - How to prevent deletion of files on the client side?

I've a problem understanding the --delete option (well, I think I understand
why it works that way but I miss how to make it work the way I want).

I want to use rsync to "mirror" two live Apache servers (I've
tried to simplify my problem to a small example, but it's part of a bigger
project). Regularly, the apache directory tree is rsynced from a master server
to its slave mirror. It's not an exact mirror, as both side have to keep
some private files: SSL certificates, logs, configuration...
The rsync daemon is running on the master and the client pulls the files from
the master.

On the master server, I defined the module [apache] in /etc/rsyncd.conf:
[apache]
        comment = Apache tree
        path = /usr/apache
        read only = true
        list = false
        uid = nobody
        gid = nobody
        auth users = webclone
        secrets file = /etc/rsyncd.passwd
        hosts allow = 192.168.241.0/24
        timeout = 600
        exclude from = /etc/rsyncd.exclude

I setup the /etc/rsyncd.exclude file to prevent transfering logs files or system
specific files (SSL certificates or passphrase):

- /logs/
- /conf/ssl*
- /conf/passphrase


Now, the mirror server pulls the apache module with the command:
rsync -avz --delete --delete-during rsync://websecure@sunray/apache/
/usr/apache/

The daemon runs as nobody on the server, and as the /conf/passphrase file is
excluded from transfer, it does not try to access it. If it tries, I would have
a permissions error as it is readable by root only.
But on the client side, I have to run as root to correctly set the permissions
of the files rsynced. I suspect that the passphrase file is not transmitted in
the files list to the client, as it is excluded on the server. But the --delete
on the client side removes it. The client thinks it's an extra older file,
because it does not appear in the transfer list and so removes it...

To keep it on the mirror, I have to exclude it again on the client side. So, I
have to maintain two lists of exclusions: one on the daemon side, one on the
clients.

Is it possible to keep only one list of exclusion on the server side or share it
from both sides, and benefit from the --delete option to remove files only in
the directories transfered?

I've tried various combinations, running the daemon as root and/or
protecting the file in the exclude list, but no one seems to work. To sum up, I
want:
- To exclude the /conf/passphrase from the transfer on the server side.
- The daemon to notify the client not to delete this file.
- The client to delete all other files in directories transfered.
- Manage the exclusions only in one place.
The important point which seems not to be possible is the second point.

Or perhaps have a set uid on the client side to restrict the power of the
--delete option?

Thanks if you can help me to keep a simple rsync configuration.

Pierre M?tras

On Wed, Jun 15, 2005 at 10:18:31AM -0400, M?tras Pierre
wrote:
> I suspect that the passphrase file is not transmitted in the files
> list to the client, as it is excluded on the server. But the --delete
> on the client side removes it. [...] To keep it on the mirror, I have
> to exclude it again on the client side.
See the rsyncd.conf manpage in the section on the "exclude" config
option for an explanation of why this is necessary:

    http://rsync.samba.org/ftp/rsync/rsyncd.conf.html
> Is it possible to keep only one list of exclusion on the server side
> or share it from both sides, and benefit from the --delete option to
> remove files only in the directories transfered?
You could just maintain the list on the receiving side only -- if you
only have one client pulling information, that is the easiest place to
maintain the list of exclusions.

Another possibility is to install 2.6.5 on both hosts and then switch
over to using --delete-after and the -F option to have rsync use per-
directory filter files (each named .rsync-filter) to limit the transfer
and the deletions.  Either exclude everything in one file in the root of
the transfer, or sprinkle .rsync-filter files around in the dirs where
you want to exclude something (possibly anchoring the exclude rules so
that they apply only to the current directory).  See the rsync manpage
in the FILTER RULES section for the rule format for the .rsync-filter
files (since filter rules aren't the same as includes/excludes, though
there is some similarity in the include and exclude rules):

    http://rsync.samba.org/ftp/rsync/rsync.html

..wayne..

Hi,
> > I suspect that the passphrase file is not transmitted in the files 
> > list to the client, as it is excluded on the server. But 
> the --delete 
> > on the client side removes it. [...] To keep it on the 
> mirror, I have 
> > to exclude it again on the client side.
> 
> See the rsyncd.conf manpage in the section on the "exclude" 
> config option for an explanation of why this is necessary:

The sentence I was missing is "Because this exclude list is not passed to
the client it only applies on the daemon: that is, it excludes files received by
a client when receiving from a daemon and files deleted on a daemon when sending
to a daemon, but it doesn't exclude files from being deleted on a client
when receiving from a daemon."

> > Is it possible to keep only one list of exclusion on the 
> server side 
> > or share it from both sides, and benefit from the --delete 
> option to 
> > remove files only in the directories transfered?
> 
> You could just maintain the list on the receiving side only 
> -- if you only have one client pulling information, that is 
> the easiest place to maintain the list of exclusions.
> 
> Another possibility is to install 2.6.5 on both hosts and 
> then switch over to using --delete-after and the -F option to 
> have rsync use per- directory filter files (each named 
> .rsync-filter) to limit the transfer and the deletions.  
> Either exclude everything in one file in the root of the 
> transfer, or sprinkle .rsync-filter files around in the dirs 
> where you want to exclude something (possibly anchoring the 
> exclude rules so that they apply only to the current 
> directory).  See the rsync manpage in the FILTER RULES 
> section for the rule format for the .rsync-filter files 
> (since filter rules aren't the same as includes/excludes, 
> though there is some similarity in the include and exclude rules):
> 
>     http://rsync.samba.org/ftp/rsync/rsync.html
> 
> ..wayne..
> 
All this is very powerful but complex to maintain and manage. The solution with
the excludes file on the client side is not possible as I have many clients and
for security I don't want to show some files from the server.

I've decided to go with a two steps process. As I have to create
"mirror" servers, the exclude list for both servers are the same. So I
rsync the exclude file first, and then rsync the actual directories tree using
the same exclude file both on the server and on the client. That way, I can
manage only the excludes list on the server and it is updated on all the
clients.

Thanks for your help.

Pierre M?tras