I've a problem understanding the --delete option (well, I think I understand why it works that way but I miss how to make it work the way I want). I want to use rsync to "mirror" two live Apache servers (I've tried to simplify my problem to a small example, but it's part of a bigger project). Regularly, the apache directory tree is rsynced from a master server to its slave mirror. It's not an exact mirror, as both side have to keep some private files: SSL certificates, logs, configuration... The rsync daemon is running on the master and the client pulls the files from the master. On the master server, I defined the module [apache] in /etc/rsyncd.conf: [apache] comment = Apache tree path = /usr/apache read only = true list = false uid = nobody gid = nobody auth users = webclone secrets file = /etc/rsyncd.passwd hosts allow = 192.168.241.0/24 timeout = 600 exclude from = /etc/rsyncd.exclude I setup the /etc/rsyncd.exclude file to prevent transfering logs files or system specific files (SSL certificates or passphrase): - /logs/ - /conf/ssl* - /conf/passphrase Now, the mirror server pulls the apache module with the command: rsync -avz --delete --delete-during rsync://websecure@sunray/apache/ /usr/apache/ The daemon runs as nobody on the server, and as the /conf/passphrase file is excluded from transfer, it does not try to access it. If it tries, I would have a permissions error as it is readable by root only. But on the client side, I have to run as root to correctly set the permissions of the files rsynced. I suspect that the passphrase file is not transmitted in the files list to the client, as it is excluded on the server. But the --delete on the client side removes it. The client thinks it's an extra older file, because it does not appear in the transfer list and so removes it... To keep it on the mirror, I have to exclude it again on the client side. So, I have to maintain two lists of exclusions: one on the daemon side, one on the clients. Is it possible to keep only one list of exclusion on the server side or share it from both sides, and benefit from the --delete option to remove files only in the directories transfered? I've tried various combinations, running the daemon as root and/or protecting the file in the exclude list, but no one seems to work. To sum up, I want: - To exclude the /conf/passphrase from the transfer on the server side. - The daemon to notify the client not to delete this file. - The client to delete all other files in directories transfered. - Manage the exclusions only in one place. The important point which seems not to be possible is the second point. Or perhaps have a set uid on the client side to restrict the power of the --delete option? Thanks if you can help me to keep a simple rsync configuration. Pierre M?tras
On Wed, Jun 15, 2005 at 10:18:31AM -0400, M?tras Pierre wrote:> I suspect that the passphrase file is not transmitted in the files > list to the client, as it is excluded on the server. But the --delete > on the client side removes it. [...] To keep it on the mirror, I have > to exclude it again on the client side.See the rsyncd.conf manpage in the section on the "exclude" config option for an explanation of why this is necessary: http://rsync.samba.org/ftp/rsync/rsyncd.conf.html> Is it possible to keep only one list of exclusion on the server side > or share it from both sides, and benefit from the --delete option to > remove files only in the directories transfered?You could just maintain the list on the receiving side only -- if you only have one client pulling information, that is the easiest place to maintain the list of exclusions. Another possibility is to install 2.6.5 on both hosts and then switch over to using --delete-after and the -F option to have rsync use per- directory filter files (each named .rsync-filter) to limit the transfer and the deletions. Either exclude everything in one file in the root of the transfer, or sprinkle .rsync-filter files around in the dirs where you want to exclude something (possibly anchoring the exclude rules so that they apply only to the current directory). See the rsync manpage in the FILTER RULES section for the rule format for the .rsync-filter files (since filter rules aren't the same as includes/excludes, though there is some similarity in the include and exclude rules): http://rsync.samba.org/ftp/rsync/rsync.html ..wayne..
Hi,> > I suspect that the passphrase file is not transmitted in the files > > list to the client, as it is excluded on the server. But > the --delete > > on the client side removes it. [...] To keep it on the > mirror, I have > > to exclude it again on the client side. > > See the rsyncd.conf manpage in the section on the "exclude" > config option for an explanation of why this is necessary:The sentence I was missing is "Because this exclude list is not passed to the client it only applies on the daemon: that is, it excludes files received by a client when receiving from a daemon and files deleted on a daemon when sending to a daemon, but it doesn't exclude files from being deleted on a client when receiving from a daemon."> > Is it possible to keep only one list of exclusion on the > server side > > or share it from both sides, and benefit from the --delete > option to > > remove files only in the directories transfered? > > You could just maintain the list on the receiving side only > -- if you only have one client pulling information, that is > the easiest place to maintain the list of exclusions. > > Another possibility is to install 2.6.5 on both hosts and > then switch over to using --delete-after and the -F option to > have rsync use per- directory filter files (each named > .rsync-filter) to limit the transfer and the deletions. > Either exclude everything in one file in the root of the > transfer, or sprinkle .rsync-filter files around in the dirs > where you want to exclude something (possibly anchoring the > exclude rules so that they apply only to the current > directory). See the rsync manpage in the FILTER RULES > section for the rule format for the .rsync-filter files > (since filter rules aren't the same as includes/excludes, > though there is some similarity in the include and exclude rules): > > http://rsync.samba.org/ftp/rsync/rsync.html > > ..wayne.. >All this is very powerful but complex to maintain and manage. The solution with the excludes file on the client side is not possible as I have many clients and for security I don't want to show some files from the server. I've decided to go with a two steps process. As I have to create "mirror" servers, the exclude list for both servers are the same. So I rsync the exclude file first, and then rsync the actual directories tree using the same exclude file both on the server and on the client. That way, I can manage only the excludes list on the server and it is updated on all the clients. Thanks for your help. Pierre M?tras
Seemingly Similar Threads
- --exclude-from works but "exclude from" in rsyncd.conf doesn't ?
- Excludes in /etc/rsyncd.conf
- Prevent dereferenced path of symlinked directories from printing on client side log output
- Bug report: deletion of files only on remote system is not logged.
- tables of frequencies