Phil Howard
2002-Jun-22 03:01 UTC
-z and -B65536 causing file corruption in 2.5.5 w/zlib 1.1.4
When using the -z and -B65536 options together, sometimes there is file
corruption. Client and server are both compiled against zlib 1.1.4, so
the gzip corruption shouldn't be there, right?
With -z and -B32768 there is an error, but it is detected in a different
way.
With -B65536 and without -z all seems to be going OK.
With -B16384 and with -z all seems to be going OK.
I'm guessing this is not a protocol limitation of rsync since it works
OK without -z, but maybe it is problem in chunk sizing with -z?
Here is what I encountered in a specific case where the file needed to be
updated, but failed to. I've been appending an "x" to the file to
force
it to be updated. The output of more verbose execution can be obtained
at http://phil.ipal.org/rsync-bug.log if you want to see that (doesn't
look helpful to me, but maybe it is to you).
Anyone know what's really happening here?
[text below contains many lines way longer than 80 characters]
============================================================================phil@polaris:/home/phil
20> echo -n x >>
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
phil@polaris:/home/phil 21> rsync -acPvz -B65536
hadar.ipal.net::slackware-8.1/PACKAGES.TXT
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
Welcome to the Internet Power and Light public file archive!
Please limit repeated updates to not more than once per hour.
receiving file list ...
1 file to consider
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
228031 100% 74.23kB/s 0:00:03
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
228031 100% 71.35kB/s 0:00:03
ERROR: file corruption in
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT.
File changed during transfer?
wrote 252 bytes read 18090 bytes 1930.74 bytes/sec
total size is 228031 speedup is 12.43
phil@polaris:/home/phil 22> rsync -acPv -B65536
hadar.ipal.net::slackware-8.1/PACKAGES.TXT
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
Welcome to the Internet Power and Light public file archive!
Please limit repeated updates to not more than once per hour.
receiving file list ...
1 file to consider
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
228031 100% 27.29kB/s 0:00:08
wrote 155 bytes read 31699 bytes 2769.91 bytes/sec
total size is 228031 speedup is 7.16
phil@polaris:/home/phil 23> echo -n x >>
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
phil@polaris:/home/phil 24> rsync -acPvz -B65536
hadar.ipal.net::slackware-8.1/PACKAGES.TXT
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
Welcome to the Internet Power and Light public file archive!
Please limit repeated updates to not more than once per hour.
receiving file list ...
1 file to consider
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
228031 100% 72.73kB/s 0:00:03
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
228031 100% 70.67kB/s 0:00:03
ERROR: file corruption in
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT.
File changed during transfer?
wrote 252 bytes read 18090 bytes 1746.86 bytes/sec
total size is 228031 speedup is 12.43
phil@polaris:/home/phil 25> rsync -acPvz -B32768
hadar.ipal.net::slackware-8.1/PACKAGES.TXT
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
Welcome to the Internet Power and Light public file archive!
Please limit repeated updates to not more than once per hour.
receiving file list ...
1 file to consider
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
inflate (token) returned -5
rsync error: error in rsync protocol data stream (code 12) at token.c(472)
rsync: connection unexpectedly closed (216 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(150)
phil@polaris:/home/phil 26> rsync -acPvz -B16384
hadar.ipal.net::slackware-8.1/PACKAGES.TXT
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
Welcome to the Internet Power and Light public file archive!
Please limit repeated updates to not more than once per hour.
receiving file list ...
1 file to consider
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
228031 100% 76.76kB/s 0:00:02
wrote 216 bytes read 9157 bytes 1442.00 bytes/sec
total size is 228031 speedup is 24.33
phil@polaris:/home/phil 27> rsync -acPvz -B65536
hadar.ipal.net::slackware-8.1/PACKAGES.TXT
/home2/rsync/hadar/mirrors/slackware/rsync/ftp.slackware.com/slackware/slackware-8.1/PACKAGES.TXT
Welcome to the Internet Power and Light public file archive!
Please limit repeated updates to not more than once per hour.
receiving file list ...
1 file to consider
wrote 116 bytes read 224 bytes 136.00 bytes/sec
total size is 228031 speedup is 670.68
phil@polaris:/home/phil 28> rsync --version
rsync version 2.5.5 protocol version 26
Copyright (C) 1996-2002 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
IPv6, 64-bit system inums, 64-bit internal inums
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
phil@polaris:/home/phil 29> which rsync
/usr/bin/rsync
phil@polaris:/home/phil 30> ls -l /usr/bin/rsync
-rwxr-xr-x 1 root root 548639 Jun 22 03:47 /usr/bin/rsync
phil@polaris:/home/phil 31> strings </usr/bin/rsync | fgrep 1.1.4
1.1.4
deflate 1.1.4 Copyright 1995-2002 Jean-loup Gailly
1.1.4
inflate 1.1.4 Copyright 1995-2002 Mark Adler
1.1.4
phil@polaris:/home/phil 32>
============================================================================
Checking out the server:
============================================================================root@hadar:/root
20> which rsync
/usr/bin/rsync
root@hadar:/root 21> which rsyncd
/usr/sbin/rsyncd
root@hadar:/root 22> ls -l /usr/bin/rsync
-rwxr-xr-x 1 root root 548639 Jun 22 03:47 /usr/bin/rsync
root@hadar:/root 23> ls -l /usr/sbin/rsyncd
-rwxr-xr-x 1 root root 548639 Jun 22 03:47 /usr/sbin/rsyncd
root@hadar:/root 24> cmp /usr/bin/rsync /usr/sbin/rsyncd
root@hadar:/root 25> strings </usr/bin/rsync | fgrep 1.1.4
1.1.4
deflate 1.1.4 Copyright 1995-2002 Jean-loup Gailly
1.1.4
inflate 1.1.4 Copyright 1995-2002 Mark Adler
1.1.4
root@hadar:/root 26> strings </usr/sbin/rsyncd | fgrep 1.1.4
1.1.4
deflate 1.1.4 Copyright 1995-2002 Jean-loup Gailly
1.1.4
inflate 1.1.4 Copyright 1995-2002 Mark Adler
1.1.4
root@hadar:/root 27>
============================================================================
--
-----------------------------------------------------------------
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| phil-nospam@ipal.net | Texas, USA | http://phil.ipal.org/ |
-----------------------------------------------------------------
Reasonably Related Threads
Wisdom of the Ancients
