Sam Sexton
2003-May-09 20:56 UTC
Bug report: deletion of files only on remote system is not logged.
I was told that my earlier submission of this report contained a
user-hostile attachment (courtesy of Outlook), so here it is again, with
the log included below.
The problem: Files that exist only on the remote system are deleted
silently (i.e. not logged), rather than noisily as implied by the
recommendation of --dry-run in the rsync man page definition of --delete
to see what would be deleted. I am intending to use rsync to replace
rdist in an disaster recovery suite for several major banks and need to
be able to provide the same details of what files were deleted as at
present.
[128](samx@Edelfelt)~samx: uname -a
SunOS Edelfelt 5.8 Generic_108528-18 sun4u sparc
SUNW,UltraSPARC-IIi-cEngine
rsync version 2.5.6 protocol version 26
Copyright (C) 1996-2002 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks,
batchfiles,
no IPv6, 64-bit system inums, 64-bit internal inums
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
Script started on Tue May 06 14:23:28 2003
14:23
[436](H:root@Edelfelt)~MBhome/site: cat /etc/rsyncd.conf
motd file = /etc/rsyncd.motd
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
[freq_copy]
comment = Rsync definition for frequent copy
# Distribute following as MBHD/MBSTEW and run do_skel on it.
path = /export/home/mbstew
max connections = 1
lock file = /var/run/rsync_freq_copy
uid = root
gid = root
read only = no
list = yes
# The following file defines all inclusions and exclusions.
# exclude from = /usr/mb/recovery/conf/rsync_freq.conf
auth users = root
secrets file = /etc/rsyncd.secrets
hosts allow = 10.5.203.101,10.5.203.103
# This is probably for testing only, so I can see what it's like:
transfer logging
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.Z[daily_copy]
[daily_copy]
comment = Rsync definition for daily copy
# Distribute following as MBHD and run do_skel on it.
path = /export/home
max connections = 1
lock file = /var/run/rsync_daily_copy
uid = root
gid = root
read only = no
list = yes
# The following file defines all inclusions and exclusions.
# exclude from = /usr/mb/recovery/conf/rsync_daily.conf
auth users = root
secrets file = /etc/rsyncd.secrets
hosts allow = 10.5.203.101,10.5.203.103
# This is probably for testing only, so I can see what it's like:
transfer logging
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.Z[daily_copy]
[daily_etc]
comment = Rsync definition for daily /etc copy
path = /etc
max connections = 1
lock file = /var/run/rsync_daily_etc
uid = root
gid = root
read only = no
list = yes
# The following file defines all inclusions and exclusions.
# exclude from = /usr/mb/recovery/conf/rsync_daily_etc.conf
auth users = root
secrets file = /etc/rsyncd.secrets
hosts allow = 10.5.203.101,10.5.203.103
# This is probably for testing only, so I can see what it's like:
transfer logging
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.Z
# Ensuring that the relevant files are the same is on the target system,
which is the rsync server:
[438](H:root@Edelfelt)~MBhome/site: nonce_copy /etc/rsyncd.conf
updating host Tazdevil
updating: /etc/rsyncd.conf
[439](H:root@Edelfelt)~MBhome/site: nonce_copy rsynctest
updating host Tazdevil
updating: /export/home/MBhome/site/rsynctest
# Evidence that tazonly exists only on the remote system:
[442](H:root@Edelfelt)~MBhome/site: rsh taz ls -l /usr/mb/site
total 14
-rw-r----- 1 mbase mbadmin 205 Jan 27 09:24 MBvars.pl
-rwxr-x--- 1 mbase mbadmin 36 Feb 28 09:59 con
-rw-r----- 1 root other 0 May 1 14:37 newfile
-rwxr-x--- 1 mbase mbadmin 1110 Apr 7 13:45 newstewpass
-rwxr-x--- 1 mbase mbadmin 1747 May 6 14:23 rsynctest
-rw-r----- 1 root other 4 May 6 13:33 taznewer
-rw-r--r-- 1 root other 0 May 6 14:21 tazonly
# and not on the local system:
[443](H:root@Edelfelt)~MBhome/site: ls -l
total 14
-rw-r----- 1 mbase mbadmin 205 Jan 27 09:24 MBvars.pl
-rwxr-x--- 1 mbase mbadmin 36 Feb 28 09:59 con
-rw-r----- 1 root other 0 May 1 14:37 newfile
-rwxr-x--- 1 mbase mbadmin 1110 Apr 7 13:45 newstewpass
-rwxr-x--- 1 mbase mbadmin 1747 May 6 14:23 rsynctest
-rw-r----- 1 root other 4 May 6 13:31 taznewer
# Here's the test script:
[444](H:root@Edelfelt)~MBhome/site: cat rsynctest
#!/bin/ksh
log=/usr/mb/log/daily_copy.log
print "$(date) Starting daily copy" | tee $log
#rsync --dry-run \
rsync \
--archive \
--update \
--recursive \
--stats \
--devices \
--one-file-system \
--password-file=/etc/rsyncd.passwd \
--rsync-path=/usr/local/bin/rsync \
--verbose \
--verbose \
--delete \
--exclude-from=/usr/mb/recovery/conf/rsync_daily.conf \
/export/home/ root@Tazdevil::daily_copy >>$log
}
print "$(date) All copies completed" | tee -a $log
# Now run it:
[445](H:root@Edelfelt)~MBhome/site: rsynctest
Tue May 6 14:27:55 GMT 2003 Starting daily copy
Tue May 6 14:27:57 GMT 2003 All copies completed
# The file has gone:
[446](H:root@Edelfelt)~MBhome/site: rsh taz ls -l /usr/mb/site
total 14
-rw-r----- 1 mbase mbadmin 205 Jan 27 09:24 MBvars.pl
-rwxr-x--- 1 mbase mbadmin 36 Feb 28 09:59 con
-rw-r----- 1 root other 0 May 1 14:37 newfile
-rwxr-x--- 1 mbase mbadmin 1110 Apr 7 13:45 newstewpass
-rwxr-x--- 1 mbase mbadmin 1747 May 6 14:23 rsynctest
-rw-r----- 1 root other 4 May 6 13:33 taznewer
# But there's nothing in the log:
[447](H:root@Edelfelt)~MBhome/site: m log
[448](H:root@Edelfelt)~MBhome/log: cat daily_copy.log
Tue May 6 14:27:55 GMT 2003 Starting daily copy
opening tcp connection to Tazdevil port 873
# Connected to Tazdevil rsync server
building file list ...
expand file_list to 4000 bytes, did move
excluding directory lost+found because of pattern /lost+found
excluding directory MBhome/doc/geninfo because of pattern
/MBhome/doc/geninfo/
excluding directory MBhome/tmp because of pattern /MBhome/tmp/
excluding directory MBhome/backup because of pattern /MBhome/backup/
excluding directory MBhome/presift because of pattern /MBhome/presift/
excluding directory MBhome/conf/dbase/log because of pattern
/MBhome/conf/dbase/log
excluding file MBhome/conf/mbw_interlock because of pattern
/MBhome/conf/mbw_interlock
excluding file MBhome/conf/swapl_at_boot because of pattern
/MBhome/conf/*at_boot
excluding file MBhome/conf/swaps_at_boot because of pattern
/MBhome/conf/*at_boot
excluding file MBhome/conf/vmstat_at_boot because of pattern
/MBhome/conf/*at_boot
excluding file MBhome/conf/CRlive because of pattern /MBhome/conf/CRlive
excluding file MBhome/conf/run_daily.lastrun because of pattern
/MBhome/conf/*.lastrun
excluding file MBhome/conf/run_frequently.lastrun because of pattern
/MBhome/conf/*.lastrun
excluding file MBhome/conf/run_monthly.lastrun because of pattern
/MBhome/conf/*.lastrun
excluding directory MBhome/log because of pattern /MBhome/log/
excluding file MBhome/recovery/conf/run_freq because of pattern
/MBhome/recovery/conf/run_*
excluding file MBhome/recovery/conf/last_freq because of pattern
/MBhome/recovery/conf/last*
excluding file mbstew/NFcrashws.sw because of pattern *crash*.sw
expand file_list to 8000 bytes, did move
excluding file mbstew/nfbtcrash.sw because of pattern *crash*.sw
excluding file mbstew/continue.sw because of pattern continue.sw
excluding file sax/rel/lib/btask/CRbserv.sw because of pattern CR*.sw
excluding file sax/vam/ssm/CRssmstart.sw because of pattern CR*.sw
excluding file sax/vam/ssm/sockservC.sw because of pattern *C.sw
excluding file sax/vam/ssm/sockserverC.sw because of pattern *C.sw
excluding file sax/vam/ssm/continue.sw because of pattern continue.sw
excluding directory samx/.dt because of pattern /samx/.dt
excluding file samx/continue.sw because of pattern continue.sw
excluding file samx/core because of pattern core
excluding file samx/SWAP01 because of pattern SWAP*
excluding file samx/idshp0101 because of pattern idshp*
excluding file samx/SWAP because of pattern SWAP*
excluding directory projects because of pattern /projects
excluding directory kellym because of pattern /kellym
excluding directory TT_DB because of pattern /TT_DB
done
mbstew/MBmsg8.sf
total: matches=4232 tag_hits=4341 false_alarms=0 data=2100
Number of files: 1543
Number of files transferred: 1
Total file size: 262168186 bytes
Total transferred file size: 2964224 bytes
Literal data: 2100 bytes
Matched data: 2962124 bytes
File list size: 30800
Total bytes written: 50626
Total bytes read: 25545
wrote 50626 bytes read 25545 bytes 30468.40 bytes/sec
total size is 262168186 speedup is 3441.84
Tue May 6 14:27:57 GMT 2003 All copies completed
script done on Tue May 06 14:29:19 2003
/Sam
Sam Sexton <mailto:sam.sexton@reuters.com>
Reuters Coventry
Automated Dealing Technologies
Phone: +44 24 7625 6562
Fax: +44 24 7655 5203
-------------------------------------------------------------- --
Visit our Internet site at http://www.reuters.com
Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging
Any views expressed in this message are those of the individual
sender, except where the sender specifically states them to be
the views of Reuters Ltd.
jw schultz
2003-May-09 21:46 UTC
Bug report: deletion of files only on remote system is not logged.
On Fri, May 09, 2003 at 11:56:18AM +0100, Sam Sexton wrote:> I was told that my earlier submission of this report contained a > user-hostile attachment (courtesy of Outlook), so here it is again, with > the log included below. > > The problem: Files that exist only on the remote system are deleted > silently (i.e. not logged), rather than noisily as implied by the > recommendation of --dry-run in the rsync man page definition of --delete > to see what would be deleted. I am intending to use rsync to replace > rdist in an disaster recovery suite for several major banks and need to > be able to provide the same details of what files were deleted as at > present.Works for me. Reduce your testcase to the bone, run from the command-line. With nothing in the log that isn't needed. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt