Edward Farrar
2002-Jul-22 04:23 UTC
rsync: --delete fails with multiple source directories
Hello,
Rsync 2.5.5 is producing this error message and a core file when executing the
command "/usr/local/bin/rsync -av --delete --force
/net/OSCM/OS_ATLAS2/CONFIG/.
/net/OSCM/OS_TITAN1/2.6/CONFIG/. /OS/2.6/CONFIG"
building file list ... done
rsync: connection unexpectedly closed (8 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(150)
The /net/OSCM directory is an NFS mounted file system. The /OS directory is
local. From reading the man pacge for rsync, one of the syntax options is
"rsync [OPTION]... SRC [SRC]... DEST ". The two source directories
contain
several files with the same name. I am unsure how rsync selects which file to
transfer, but it appears to be the file with the most recent date. Is that
correct?
My basic goal is to merge the two source directories, choosing the most recent
file in either directory if a duplicate name exists, and then deleting any
"extra" files in the destination directory.
Any insight and help is greatly appreciated. I have searched the web and as
much documentation I could find, but have been unsuccessful in finding a
solution. I have also tried versions 2.5.4 and 2.3.1 with the same results.
Thanks,
Ed Farrar
-----------
system names removed for security reasons......
# uname -a
SunOS XXXXXXXXXXXXXXXXXX 5.8 Generic_108528-11 sun4u sparc
SUNW,Ultra-Enterprise
# /usr/local/bin/rsync --version
rsync version 2.5.5 protocol version 26
Copyright (C) 1996-2002 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
no IPv6, 64-bit system inums, 64-bit internal inums
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
# df -k /net/OSCM /OS
Filesystem kbytes used avail capacity Mounted on
XXXXXXXXX:/sol/oscm 41943040 29256720 12650144 70% /net/OSCM
/dev/md/dsk/d0 17402262 14459697 2768543 84% /OS
# du -sk /net/OSCM/OS_ATLAS2/CONFIG/. /net/OSCM/OS_TITAN1/2.6/CONFIG/.
320 /net/OSCM/OS_ATLAS2/CONFIG/.
368 /net/OSCM/OS_TITAN1/2.6/CONFIG/.
That's not enough information for us to be able to do anything with. Can you at least find a stack backtrace in the core file? If you could boil it down into something that someone else could reproduce that would help a lot. - Dave Dykstra On Mon, Jul 22, 2002 at 07:19:26AM -0400, Edward Farrar wrote:> Hello, > > Rsync 2.5.5 is producing this error message and a core file when executing the > command "/usr/local/bin/rsync -av --delete --force /net/OSCM/OS_ATLAS2/CONFIG/. > /net/OSCM/OS_TITAN1/2.6/CONFIG/. /OS/2.6/CONFIG" > > building file list ... done > rsync: connection unexpectedly closed (8 bytes read so far) > rsync error: error in rsync protocol data stream (code 12) at io.c(150) > > The /net/OSCM directory is an NFS mounted file system. The /OS directory is > local. From reading the man pacge for rsync, one of the syntax options is > "rsync [OPTION]... SRC [SRC]... DEST ". The two source directories contain > several files with the same name. I am unsure how rsync selects which file to > transfer, but it appears to be the file with the most recent date. Is that > correct? > > My basic goal is to merge the two source directories, choosing the most recent > file in either directory if a duplicate name exists, and then deleting any > "extra" files in the destination directory. > > Any insight and help is greatly appreciated. I have searched the web and as > much documentation I could find, but have been unsuccessful in finding a > solution. I have also tried versions 2.5.4 and 2.3.1 with the same results. > > Thanks, > Ed Farrar > > ----------- > system names removed for security reasons...... > > # uname -a > SunOS XXXXXXXXXXXXXXXXXX 5.8 Generic_108528-11 sun4u sparc > SUNW,Ultra-Enterprise > > # /usr/local/bin/rsync --version > rsync version 2.5.5 protocol version 26 > Copyright (C) 1996-2002 by Andrew Tridgell and others > <http://rsync.samba.org/> > Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, > no IPv6, 64-bit system inums, 64-bit internal inums > > rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you > are welcome to redistribute it under certain conditions. See the GNU > General Public Licence for details. > > # df -k /net/OSCM /OS > Filesystem kbytes used avail capacity Mounted on > XXXXXXXXX:/sol/oscm 41943040 29256720 12650144 70% /net/OSCM > /dev/md/dsk/d0 17402262 14459697 2768543 84% /OS > > # du -sk /net/OSCM/OS_ATLAS2/CONFIG/. /net/OSCM/OS_TITAN1/2.6/CONFIG/. > 320 /net/OSCM/OS_ATLAS2/CONFIG/. > 368 /net/OSCM/OS_TITAN1/2.6/CONFIG/. > > -- > To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Wayne Davison
2002-Jul-27 10:54 UTC
rsync: --delete fails with multiple source directories
On Mon, 22 Jul 2002, Edward Farrar wrote:> Rsync 2.5.5 is producing this error message and a core file when executing the > command "/usr/local/bin/rsync -av --delete --force /net/OSCM/OS_ATLAS2/CONFIG/. > /net/OSCM/OS_TITAN1/2.6/CONFIG/. /OS/2.6/CONFIG" > > building file list ... done > rsync: connection unexpectedly closed (8 bytes read so far) > rsync error: error in rsync protocol data stream (code 12) at io.c(150)I looked at the code in flist_find(), and I had the theory that the code would fail if it found a duplicate name as the last item in the flist. Sure enough, creating two directories with one duplicate between them would crash in the same way if that duplicated item is the last one alphabetically but would succeed otherwise. The problems stems from the flist_up() function marching right off the top of the list if the last item has its basename zeroed out (indicating it is a duplicate). The easiest fix appears to be to simply trim the high value to ignore removed items. Like so: Index: flist.c --- flist.c 11 Apr 2002 02:21:41 -0000 1.124 +++ flist.c 27 Jul 2002 17:40:10 -0000 @@ -1151,7 +1151,9 @@ { int low = 0, high = flist->count - 1; - if (flist->count <= 0) + while (high >= 0 && !flist->files[high]->basename) high--; + + if (high < 0) return -1; while (low != high) { I've tested it and it fixes my crashing testcase, so I'll commit this to CVS. ..wayne..