I have a freebsd 6.3 server and a freebsd 7.0 server, I have a binary
I run of the freebsd 7 box but has recently been crashing, the binary
in question is ezbounce.
It was previously running for weeks no problems at all and then during
the past 2 weeks or so its had problems.
Like many shell programs it has a configure script and you then run
make or gmake to compile the binary.
On freebsd 6 it picks up /usr/local ssl libaries no problem and in
fact uses them without even haveing to specify the directory it auto
detects them over the base ssl. On freebsd 7 it uses the base
libraries even when telling it to search in /usr/local.
So I then decided to move the binary I compiled on freebsd 6 over to
the freebsd 7 box and when I ran ldd on the binary to my surprise it
is using the base libraries on freebsd 7.
ldd on binary on freebsd 6
libssl.so.5 => /usr/local/lib/libssl.so.5 (0x48102000)
libcrypto.so.5 => /usr/local/lib/libcrypto.so.5 (0x48143000)
libcrypt.so.3 => /lib/libcrypt.so.3 (0x4829f000)
libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so
(0x482b8000)
libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x482c0000)
libm.so.4 => /lib/libm.so.4 (0x48396000)
libpthread.so.2 => /lib/libpthread.so.2 (0x483ac000)
libc.so.6 => /lib/libc.so.6 (0x483d3000)
libbz2.so.2 => /usr/lib/libbz2.so.2 (0x484cb000)
libz.so.3 => /lib/libz.so.3 (0x484dc000)
ldd on same binary on freebsd 7
libssl.so.5 => /usr/lib/libssl.so.5 (0x28101000)
libcrypto.so.5 => /lib/libcrypto.so.5 (0x28142000)
libcrypt.so.3 => /usr/local/lib/compat/libcrypt.so.3 (0x2829a000)
libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so
(0x282b2000)
libstdc++.so.5 => /usr/local/lib/compat/libstdc++.so.5 (0x282bd000)
libm.so.4 => /usr/local/lib/compat/libm.so.4 (0x28388000)
libpthread.so.2 => /usr/local/lib/compat/libpthread.so.2 (0x2839e000)
libc.so.6 => /usr/local/lib/compat/libc.so.6 (0x283c3000)
libc.so.7 => /lib/libc.so.7 (0x284a9000)
libbz2.so.3 => /usr/lib/libbz2.so.3 (0x285a4000)
libz.so.4 => /lib/libz.so.4 (0x285b4000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x285c6000)
libm.so.5 => /lib/libm.so.5 (0x286ba000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x286cf000)
libthr.so.3 => /lib/libthr.so.3 (0x286da000)
The binary doesnt run on the freebsd 7 either it coredumps even tho I
have compat6x installed, typically in the past I have had no problems
at all using binaries compiled on old freebsd versions on newer
versions I just had to install the appropriate compat package.
Here is the ldd when compiled on the freebsd 7 box
libssl.so.5 => /usr/lib/libssl.so.5 (0x2810f000)
libcrypto.so.5 => /lib/libcrypto.so.5 (0x28150000)
libcrypt.so.4 => /lib/libcrypt.so.4 (0x282a8000)
libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so
(0x282c1000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x282cc000)
libm.so.5 => /lib/libm.so.5 (0x283c0000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x283d5000)
libthr.so.3 => /lib/libthr.so.3 (0x283e0000)
libc.so.7 => /lib/libc.so.7 (0x283f3000)
libbz2.so.3 => /usr/lib/libbz2.so.3 (0x284ee000)
libz.so.4 => /lib/libz.so.4 (0x284fe000)
Is the output for the ssl libraries skewed because both the local
filenames and the base filenames are the same? as there is a
/usr/local/lib/libssl.so.5 and a /usr/lib/libssl.so.5. Or does this
mean that it really is linked against the base libraries as I am
wondering how that is possible when the same binary is linked against
different libraries on a different machine.
Chris
On Tue, Mar 04, 2008 at 12:05:22PM +0000, Chris wrote:> I have a freebsd 6.3 server and a freebsd 7.0 server, I have a binary > I run of the freebsd 7 box but has recently been crashing, the binary > in question is ezbounce. > > It was previously running for weeks no problems at all and then during > the past 2 weeks or so its had problems. > > Like many shell programs it has a configure script and you then run > make or gmake to compile the binary.You know there's /usr/ports/irc/ezbounce, right? Well, I suppose that only applies if you actually maintain/run the servers in question. But apparently you do (see below).> So I then decided to move the binary I compiled on freebsd 6 over to > the freebsd 7 box and when I ran ldd on the binary to my surprise it > is using the base libraries on freebsd 7.This doesn't come as much of a surprise. The binary actually references libraries by names such as libXXX.so, not libXXX.so.NUMBER.> ldd on binary on freebsd 6 > > libssl.so.5 => /usr/local/lib/libssl.so.5 (0x48102000) > libcrypto.so.5 => /usr/local/lib/libcrypto.so.5 (0x48143000) > libcrypt.so.3 => /lib/libcrypt.so.3 (0x4829f000) > libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so (0x482b8000) > libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x482c0000) > libm.so.4 => /lib/libm.so.4 (0x48396000) > libpthread.so.2 => /lib/libpthread.so.2 (0x483ac000) > libc.so.6 => /lib/libc.so.6 (0x483d3000) > libbz2.so.2 => /usr/lib/libbz2.so.2 (0x484cb000) > libz.so.3 => /lib/libz.so.3 (0x484dc000) > > ldd on same binary on freebsd 7 > > libssl.so.5 => /usr/lib/libssl.so.5 (0x28101000) > libcrypto.so.5 => /lib/libcrypto.so.5 (0x28142000)The libssl.so being used by the ezbounce binary you have, on RELENG_7, is using the base system's version. This is NOT compatible, AFAIK, as the libssl.so on RELENG_6. The same issue applies to libcrypto.so.> libcrypt.so.3 => /usr/local/lib/compat/libcrypt.so.3 (0x2829a000) > libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so (0x282b2000) > libstdc++.so.5 => /usr/local/lib/compat/libstdc++.so.5 (0x282bd000) > libm.so.4 => /usr/local/lib/compat/libm.so.4 (0x28388000) > libpthread.so.2 => /usr/local/lib/compat/libpthread.so.2 (0x2839e000) > libc.so.6 => /usr/local/lib/compat/libc.so.6 (0x283c3000) > libc.so.7 => /lib/libc.so.7 (0x284a9000) > libbz2.so.3 => /usr/lib/libbz2.so.3 (0x285a4000) > libz.so.4 => /lib/libz.so.4 (0x285b4000) > libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x285c6000) > libm.so.5 => /lib/libm.so.5 (0x286ba000) > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x286cf000) > libthr.so.3 => /lib/libthr.so.3 (0x286da000)I can't explain what's with the "double-linking" of some libraries, e.g. two linked in-versions of libc, libm, libstdc++, and others. It's possible this is "normal", but it seems like it would cause a crash. I do know that FreeBSD has some sort of internal version numbering for symbols in shared libraries, but I don't know if it was introduced in RELENG_7, or if RELENG_6 had it. If it's new as of RELENG_7, then I can see how a binary built on RELENG_6 _might_ call the wrong "version" of a function. nm(1) output would be able to help with this, I think.> The binary doesnt run on the freebsd 7 either it coredumps even tho I > have compat6x installed, typically in the past I have had no problems > at all using binaries compiled on old freebsd versions on newer > versions I just had to install the appropriate compat package.I would strongly recommend against relying on compat6x for anything that can be recompiled. I have never trusted the compat libraries, because I don't like to play risky business with multiple versions of a shared library on a machine (see below). You did not provide a crash dump or gdb output of the program, so it's hard to say where the actual crash (SSL, libc, libm, etc.) occurred. But then again, is it worth debugging when.....> Here is the ldd when compiled on the freebsd 7 box.....you can recompile it? :-) You should be doing this anyways. So what's the problem -- or is this more of a "I'm curious how ld.so works" inquiry?> Is the output for the ssl libraries skewed because both the local > filenames and the base filenames are the same? as there is a > /usr/local/lib/libssl.so.5 and a /usr/lib/libssl.so.5. Or does this > mean that it really is linked against the base libraries as I am > wondering how that is possible when the same binary is linked against > different libraries on a different machine.I indirectly answered this in my 2nd paragraph. Welcome to the UNIX equivalent of "DLL Hell" on Windows -- and why you should *always* recompile programs when the major version of a shared library (.so) changes. I cannot stress this enough. All that said: you might be able to get around the problem in question by setting LD_LIBRARY_PATH="/usr/local/lib/compat" when running the RELENG_6-compiled ezbounce binary on RELENG_7, e.g.: sh# LD_LIBRARY_PATH="/usr/local/lib/compat" ezbounce You could also do (with possibly mixed results, since ldd is linked to libc.so, so this might cause /usr/bin/ldd to try and use the compat6x version of libc.so): sh# LD_LIBRARY_PATH="/usr/local/lib/compat" ldd ezbounce -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
On 2008.03.04 12:05:22 +0000, Chris wrote:> On freebsd 6 it picks up /usr/local ssl libaries no problem and in > fact uses them without even haveing to specify the directory it auto > detects them over the base ssl. On freebsd 7 it uses the base > libraries even when telling it to search in /usr/local.That sounds either like there is a bug in the program build system... it might also happen if the base system and and ports libraries have the same version number... I never tried that as I always use base system OpenSSL.> So I then decided to move the binary I compiled on freebsd 6 over to > the freebsd 7 box and when I ran ldd on the binary to my surprise it > is using the base libraries on freebsd 7.Note that we do not guarentee at all that you can do that. I'm not even sure that the .so version number in the port and in the base system match.> ldd on binary on freebsd 6 > > libssl.so.5 => /usr/local/lib/libssl.so.5 (0x48102000) > libcrypto.so.5 => /usr/local/lib/libcrypto.so.5 (0x48143000) > libcrypt.so.3 => /lib/libcrypt.so.3 (0x4829f000) > libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so > (0x482b8000) > libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x482c0000) > libm.so.4 => /lib/libm.so.4 (0x48396000) > libpthread.so.2 => /lib/libpthread.so.2 (0x483ac000) > libc.so.6 => /lib/libc.so.6 (0x483d3000) > libbz2.so.2 => /usr/lib/libbz2.so.2 (0x484cb000) > libz.so.3 => /lib/libz.so.3 (0x484dc000) > > ldd on same binary on freebsd 7 > > libssl.so.5 => /usr/lib/libssl.so.5 (0x28101000) > libcrypto.so.5 => /lib/libcrypto.so.5 (0x28142000) > libcrypt.so.3 => /usr/local/lib/compat/libcrypt.so.3 (0x2829a000) > libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so > (0x282b2000) > libstdc++.so.5 => /usr/local/lib/compat/libstdc++.so.5 (0x282bd000) > libm.so.4 => /usr/local/lib/compat/libm.so.4 (0x28388000) > libpthread.so.2 => /usr/local/lib/compat/libpthread.so.2 (0x2839e000) > libc.so.6 => /usr/local/lib/compat/libc.so.6 (0x283c3000) > libc.so.7 => /lib/libc.so.7 (0x284a9000)Uhh, not good. If you link against two versions of libc bad things are bound to happen. That can happen if you have a old binary which links against a new lib... or something like that. If you want to this to work aither compile the binary statically or get all the 6.x libs and do some LDCONFIGPATH (or whatever the env var is called) to make sure those libs override the 7.x libs.> libboost_iostreams.so => /usr/local/lib/libboost_iostreams.so (0x282c1000) > libbz2.so.3 => /usr/lib/libbz2.so.3 (0x284ee000) > libc.so.7 => /lib/libc.so.7 (0x283f3000) > libcrypt.so.4 => /lib/libcrypt.so.4 (0x282a8000) > libcrypto.so.5 => /lib/libcrypto.so.5 (0x28150000) > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x283d5000) > libm.so.5 => /lib/libm.so.5 (0x283c0000) > libssl.so.5 => /usr/lib/libssl.so.5 (0x2810f000) > libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x282cc000) > libthr.so.3 => /lib/libthr.so.3 (0x283e0000) > libz.so.4 => /lib/libz.so.4 (0x284fe000)That looks correct (at least no duplicate libs). Unfortunatly I have no idea why it crashes on 7 naively compiled. -- Simon L. Nielsen