thr3ads.net - freebsd stable - libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest) [Feb 2019]

If this information is useful, please help other people find it:
Share via:

Harry Schmalzbauer

2019-Feb-20 16:51 UTC

libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest)

Hello,

I'm tryint to upgrade a bhyve guest from stable/11 to stable/12.

pkg(8) for example crashes with signal 11.

I looked for other binaries affected by
ldd /usr/sbin/* | & grep 'signal 11$'
wich gives
/usr/sbin/auditdistd: signal 11
/usr/sbin/bhyve: signal 11
/usr/sbin/bsnmpd: signal
/usr/sbin/gssd: signal 11
/usr/sbin/hostapd: signal 11
/usr/sbin/iprop-log: signal 11
/usr/sbin/keyserv: signal 11
/usr/sbin/kstash: signal 11
/usr/sbin/ktutil: signal 11
/usr/sbin/local-unbound: signal 11
/usr/sbin/local-unbound-anchor: signal 11
/usr/sbin/local-unbound-checkconf: signal 11
/usr/sbin/local-unbound-control: signal 11
/usr/sbin/ntp-keygen: signal 11
/usr/sbin/ntpd: signal 11
/usr/sbin/ntpdate: signal 11
/usr/sbin/ntpdc: signal 11
/usr/sbin/pkg: signal 11
/usr/sbin/ppp: signal 11
/usr/sbin/sntp: signal 11
/usr/sbin/sshd: signal 11
/usr/sbin/tcpdump: signal 11
/usr/sbin/uefisign: signal 11
/usr/sbin/wpa_supplicant: signal 11

They all seem to have in common beeing linked against 
'/lib/libcrypto.so.111'

truss /usr/sbin/auditdistd
:
close(3)???????????????????????????????????????? = 0 (0x0)
openat(AT_FDCWD,"/lib/libcrypto.so.111",O_RDONLY|O_CLOEXEC|O_VERIFY,00)
= 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=150033332,size=3006464,blksize=4096 }) 
= 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 
34362249216 (0x800265000)
mmap(0x0,3104768,PROT_NONE,MAP_GUARD,-1,0x0)???? = 34362347520 (0x80027d000)
mmap(0x80027d000,1138688,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0)
= 34362347520 (0x80027d000)
mmap(0x800393000,1757184,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x116000)
= 34363486208 (0x800393000)
mmap(0x800540000,196608,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x2c3000)
= 34365243392 (0x800540000) SIGNAL 11 (SIGSEGV) code=SEGV_ACCERR 
trapno=12 addr=0x80056f790
process killed, signal = 11 (core dumped)

I have no idea how to analyze further or what the reason could be (like 
mentioned, all binaries listed dump core after opening lib/libcrypto.so.111

gdb shows:
Core was generated by `/usr/sbin/auditdistd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libutil.so.9...Reading symbols from 
/usr/lib/debug//lib/libutil.so.9.debug...done.
done.
Loaded symbols for /lib/libutil.so.9
Reading symbols from /libexec/ld-elf.so.1...Reading symbols from 
/usr/lib/debug//libexec/ld-elf.so.1.debug...done.
done.
Loaded symbols for /libexec/ld-elf.so.1
#0? memset (dest=0x80056f790, c=0, len=<value optimized out>)
 ??? at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
5624??????????????????? ((char *)dest)[i] = c;
(gdb) bt
#0? memset (dest=0x80056f790, c=0, len=<value optimized out>)
 ??? at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
#1? 0x0000000800235b07 in map_object (fd=3, path=0x800246140 
"/lib/libcrypto.so.111",
 ??? sb=0x7fffffffd4a8)
 ??? at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/map_object.c:249
#2? 0x0000000800230806 in load_object (name=0x201dba
"libcrypto.so.111",
fd_u=-1,
 ??? refobj=0x800248000, flags=<value optimized out>)
 ??? at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2493
#3? 0x0000000800229972 in _rtld (sp=<value optimized out>, 
exit_proc=0x7fffffffea30,
 ??? objp=0x7fffffffea38)
 ??? at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2315
#4? 0x0000000800228019 in .rtld_start ()
 ??? at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/amd64/rtld_start.S:39
#5? 0x0000000000000000 in ?? ()
Current language:? auto; currently minimal

Any help highly appreciated.

This is with a live CD (amd64), compiled with stable/12 from today (so 
clang 7.01).
The bhyve guest has 2GB hardwired and ran stable/11 beforehand, which 
compiled the live CD.
bhyve host is 11.2.? But that shouldn't play a role, does it?

-harry

Harry Schmalzbauer

2019-Feb-21 08:24 UTC

head link

Strange rtld-elf failure on stable/12 [Was: libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest)]

Am 20.02.2019 um 17:51 schrieb Harry Schmalzbauer:> Hello,
>
?> gdb shows:
> Core was generated by `/usr/sbin/auditdistd'.
> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /lib/libutil.so.9...Reading symbols from 
> /usr/lib/debug//lib/libutil.so.9.debug...done.
> done.
> Loaded symbols for /lib/libutil.so.9
> Reading symbols from /libexec/ld-elf.so.1...Reading symbols from 
> /usr/lib/debug//libexec/ld-elf.so.1.debug...done.
> done.
> Loaded symbols for /libexec/ld-elf.so.1
> #0? memset (dest=0x80056f790, c=0, len=<value optimized out>)
> ??? at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
> 5624??????????????????? ((char *)dest)[i] = c;
> (gdb) bt
> #0? memset (dest=0x80056f790, c=0, len=<value optimized out>)
> ??? at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
> #1? 0x0000000800235b07 in map_object (fd=3, path=0x800246140 
> "/lib/libcrypto.so.111",
> ??? sb=0x7fffffffd4a8)
> ??? at 
>
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/map_object.c:249
> #2? 0x0000000800230806 in load_object (name=0x201dba 
> "libcrypto.so.111", fd_u=-1,
> ??? refobj=0x800248000, flags=<value optimized out>)
> ??? at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2493
> #3? 0x0000000800229972 in _rtld (sp=<value optimized out>, 
> exit_proc=0x7fffffffea30,
> ??? objp=0x7fffffffea38)
> ??? at 
> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2315
> #4? 0x0000000800228019 in .rtld_start ()
> ??? at 
>
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/amd64/rtld_start.S:39
> #5? 0x0000000000000000 in ?? ()
> Current language:? auto; currently minimal
>
> Any help highly appreciated.
>
> This is with a live CD (amd64), compiled with stable/12 from today (so 
> clang 7.01).
> The bhyve guest has 2GB hardwired and ran stable/11 beforehand, which 
> compiled the live CD.
> bhyve host is 11.2.? But that shouldn't play a role, does it?
I'm really interested what happens here.
I built stable/11 in that bhyve guest and updated that guest to 
stable/11 from yesterday.
To my surpise llvm 7.01 was also merged to stable/11.? Thank you for 
that great supprt!
No problems with any binary in the stable/11 bhyve guest.

Then I built stable/12 in that re-built stable/11 guest.
As result, again all binaries linked to /lib/libcrypto.so.111 crash 
(signal 11) with the stable/12 iso in the same bhyve guest.

Here the example from ntpq:
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libedit.so.7...Reading symbols from 
/usr/lib/debug//lib/libedit.so.7.debug...done.
done.
Loaded symbols for /lib/libedit.so.7
Reading symbols from /lib/libm.so.5...Reading symbols from 
/usr/lib/debug//lib/libm.so.5.debug...done.
done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /libexec/ld-elf.so.1...Reading symbols from 
/usr/lib/debug//libexec/ld-elf.so.1.debug...done.
done.
#0? memset (dest=0x8005ef790, c=0, len=<value optimized out>) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
5624??????????????????? ((char *)dest)[i] = c;
(gdb) bt
#0? memset (dest=0x8005ef790, c=0, len=<value optimized out>) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624
#1? 0x000000080025db07 in map_object (fd=3, path=0x80026e1a0 
"/lib/libcrypto.so.111", sb=0x7fffffffd4c8) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/map_object.c:249
#2? 0x0000000800258806 in load_object (name=0x201b40
"libcrypto.so.111",
fd_u=-1, refobj=0x800270000, flags=<value optimized out>) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2493
#3? 0x0000000800251972 in _rtld (sp=<value optimized out>, 
exit_proc=0x7fffffffea50, objp=0x7fffffffea58) at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2315
#4? 0x0000000800250019 in .rtld_start () at 
/usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/amd64/rtld_start.S:39
#5? 0x0000000000000000 in ?? ()

So please correct me if I'm comletely wrong, but the problem here seems 
to be reproducably rtld-elf related.
Unfortunately I don't know anything about object files and linkers and 
the related fundamental stuff.
But maybe someone else has an idea what's going wrong here?

Thanks,

-Harry

freebsd stable - Feb 2019 - libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest)

libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest)

Strange rtld-elf failure on stable/12 [Was: libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest)]