Martin Pool
2002-Apr-20 19:38 UTC
rsync breaks on FreeBSD without -O2?(fwd from grog@FreeBSD.org) PR 36998
Apparently rsync breaks on FreeBSD if you turn off -O2 (which is the default.) The breakage is apparently inside zlib. You'd have to think it was a compiler bug, but perhaps not. I haven't tried to reproduce it yet. -- Martin ----- Forwarded message from grog@FreeBSD.org ----- Date: Fri, 19 Apr 2002 23:14:46 -0700 (PDT) From: <grog@FreeBSD.org> To: mbp@samba.org Subject: PR 36998>From nobody@FreeBSD.org Thu Apr 11 10:49:33 2002Return-Path: <nobody@FreeBSD.org> Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id B56BF37B416 for <freebsd-gnats-submit@FreeBSD.org>; Thu, 11 Apr 2002 10:49:32 -0700 (PDT) Received: (from nobody@localhost) by freefall.freebsd.org (8.11.6/8.11.6) id g3BHnWF98869; Thu, 11 Apr 2002 10:49:32 -0700 (PDT) (envelope-from nobody) Message-Id: <200204111749.g3BHnWF98869@freefall.freebsd.org> Date: Thu, 11 Apr 2002 10:49:32 -0700 (PDT) From: Donal Diamond <ddiamond@indigo.ie> To: freebsd-gnats-submit@FreeBSD.org Subject: rsync: -z option core dumps on large files X-Send-Pr-Version: www-1.0>Number: 36998 >Category: ports >Synopsis: rsync: -z option core dumps on large files >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-ports >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Apr 11 10:50:01 PDT 2002 >Closed-Date: >Last-Modified: Mon Apr 15 15:10:01 PDT 2002 >Originator: Donal Diamond >Release: 4.5-STABLE >Organization: >Environment:FreeBSD donal.eng.eircom.net 4.5-STABLE FreeBSD 4.5-STABLE #0: Thu Jan 31 19:55: 40 GMT 2002 diamondd@donal.eng.eircom.net:/usr/obj/usr/cvsup/stable-releng4/ src/sys/DD_NFSD-SMBFS i386>Description:rsync port core dumps tansferring large files if you use -z/--compress option.(Problem does not occur if you install manually from source.) I made a basic rysnc port Makefile (no patches/options etc) and it still coredumps. Here's a gdb from a normal port install that was tweaked to enable debug symbols: Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/libc.so.4...done. Reading symbols from /usr/libexec/ld-elf.so.1...done. #0 0x8061823 in inflate_codes (s=0x807b040, z=0x80777c0, r=0) at zlib/infcodes.c:200 200 while (f < s->window) /* modulo window size-"while" instead */ (gdb) bt #0 0x8061823 in inflate_codes (s=0x807b040, z=0x80777c0, r=0) at zlib/infcodes.c:200 #1 0x806104c in inflate_blocks (s=0x807b040, z=0x80777c0, r=-5) at zlib/infblock.c:340 #2 0x806243b in inflate (z=0x80777c0, f=0) at zlib/inflate.c:221 #3 0x805852f in recv_deflated_token (f=0, data=0xbfbfe8dc) at token.c:412 #4 0x805872e in recv_token (f=0, data=0xbfbfe8dc) at token.c:506 #5 0x804bd38 in receive_data (f_in=0, buf=0x0, fd=1, fname=0x8074a80 "master-bin.095", total_size=203874289) at receiver.c:224 #6 0x804c254 in recv_files (f_in=0, flist=0x807a090, local_name=0x0, f_gen=4) at receiver.c:456 #7 0x804fe23 in do_recv (f_in=0, f_out=1, flist=0x807a090, local_name=0x0) at main.c:398 #8 0x80500da in do_server_recv (f_in=0, f_out=1, argc=2, argv=0xbfbff718) at main.c:491 #9 0x8050188 in start_server (f_in=0, f_out=1, argc=2, argv=0xbfbff718) at main.c:519 #10 0x804e048 in local_child (argc=2, argv=0xbfbff718, f_in=0xbfbff900, f_out=0xbfbff904) at util.c:212 #11 0x804f969 in do_cmd (cmd=0x0, machine=0x0, user=0x0, path=0x807a080 ".", f_in=0xbfbff900, f_out=0xbfbff904) at main.c:234 #12 0x8050752 in start_client (argc=2, argv=0x8078060) at main.c:772 #13 0x80509dc in main (argc=2, argv=0x8078060) at main.c:921 #14 0x804a01d in _start () (gdb) p s $1 = (inflate_blocks_statef *) 0x807b040 (gdb) p *s $2 = {dummy = 6} (gdb) p z $3 = 0x80777c0 (gdb) p *z $4 = {next_in = 0x8091ff6 "!.?Y?????", avail_in = 9, total_in = 16374, next_out = 0x809a000 <Address 0x809a000 out of bounds>, avail_out = 0, total_out = 98304, msg = 0x0, state = 0x8078100, zalloc = 0x8064f18 <zcalloc>, zfree = 0x8064f30 <zcfree>, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}>How-To-Repeat:transfer a large (binary?) file with compression on. local to local transfers also fail. rsync -azv / rsync://sunsite.dk/ftp//mirrors/mysql/Downloads/Contrib/MyAccess.mda . rsync: connection unexpectedly closed (930 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(150)>Fix:>Release-Note: >Audit-Trail:From: Donal Diamond <donal.diamond@eircom.net> To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-ports@FreeBSD.org Cc: Subject: Re: ports/36998: rsync: -z option core dumps on large files Date: Thu, 11 Apr 2002 19:00:06 +0100 Apologies, I have a typo for the command. Command should be: rsync -azv rsync://sunsite.dk/ftp//mirrors/mysql/Downloads/Contrib/MyAccess.mda . Also, forgot to state rsync version: $rsync --version rsync version 2.5.5 protocol version 26 Copyright (C) 1996-2002 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, IPv6, 32-bit system inums, 64-bit internal inums Donal From: Oliver Braun <obraun@informatik.unibw-muenchen.de> To: Donal Diamond <ddiamond@indigo.ie> Cc: freebsd-gnats-submit@FreeBSD.ORG Subject: Re: ports/36998: rsync: -z option core dumps on large files Date: Thu, 11 Apr 2002 21:15:27 +0200 Thanks for your report. I am going to check this next week, since I am on a official journey from tomorrow morning until next week. Regards, Olli From: Oliver Braun <obraun@informatik.unibw-muenchen.de> To: Donal Diamond <donal.diamond@eircom.net> Cc: FreeBSD-gnats-submit@FreeBSD.ORG Subject: Re: ports/36998: [MAINTAINER UPDATE] solution for rsync: -z option core dumps on large files Date: Mon, 15 Apr 2002 23:42:25 +0200 [MAINTAINER UPDATE] rsync -z works if compiled with CFLAGS including at least -O2 optimization level. The following patch checks whether -O[2-6] is part of CFLAGS. Otherwise -O2 will be appended. [ From the gcc manual: If you use multiple `-O' options, with or without level numbers, the last such option is the one that is effective. ] Bumped PORTREVISION Regards, Olli -------------------------------------------------------------------------- diff -ruN rsync.old/Makefile rsync/Makefile --- rsync.old/Makefile Mon Apr 15 23:13:44 2002 +++ rsync/Makefile Mon Apr 15 23:25:50 2002 @@ -8,7 +8,7 @@ PORTNAME= rsync PORTVERSION= 2.5.5 -PORTREVISION= 0 +PORTREVISION= 1 CATEGORIES= net ipv6 MASTER_SITES= ftp://samba.anu.edu.au/pub/rsync/ \ ftp://sunsite.auc.dk/pub/unix/rsync/ \ @@ -18,6 +18,11 @@ MAINTAINER= obraun@informatik.unibw-muenchen.de GNU_CONFIGURE= yes + +OPTIMIZATION!= ${ECHO} "${CFLAGS}" | grep '\-O[2-6]' || true +.if empty(OPTIMIZATION) +CFLAGS+= -O2 +.endif .include <bsd.port.pre.mk>>Unformatted:----- End forwarded message ----- -- Martin
Jos Backus
2002-Apr-20 20:26 UTC
rsync breaks on FreeBSD without -O2?(fwd from grog@FreeBSD.org) PR 36998
On Sat, Apr 20, 2002 at 07:37:12PM -0700, Martin Pool wrote:> Apparently rsync breaks on FreeBSD if you turn off -O2 (which is the > default.) The breakage is apparently inside zlib. > > You'd have to think it was a compiler bug, but perhaps not. > > I haven't tried to reproduce it yet.Here's some mail from freebsd-cvs-all (<20020420171059.A593@xor.obsecurity.org>). Bruce Evans submitted this workaround which has been incorporated into the FreeBSD tree. I think we should do the same. I can do the honors :-) Jos <quote> On Sat, Apr 20, 2002 at 04:01:29PM -0700, Jos Backus wrote:> On Sat, Apr 20, 2002 at 01:22:16PM -0700, Kris Kennaway wrote: > > On Sat, Apr 20, 2002 at 01:19:47PM -0700, Anders Nordby wrote: > > > anders 2002/04/20 13:19:47 PDT > > > > > > Modified files: > > > net/rsync Makefile > > > Log: > > > Make port set -O2 in CFLAGS. Necessary for -z (compress) option to work > > > properly. > > > > Err, this sounds like the fault of the broken zlib code we had in the > > source tree, which was already fixed. > > rsync uses it's own slightly incompatible version of zlib. If you can point me > to a fix I can discuss the fix with the rsync folks and commit it. > > (If you don't hear from me it's because XO cut my DSL line - I'm in the > process of switching providers.)Here's the fix from bde which I applied to the libz in the tree: http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libz/infcodes.c.diff?r1=1.3&r2=1.4 Kris </quote> -- Jos Backus _/ _/_/_/ Santa Clara, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/ _/ jos@catnook.com _/_/ _/_/_/ use Std::Disclaimer;