Thanks for providing the patch! I got around to testing it this morning and it appears it fixes compilation, but produces a non-working system. I know that's kinda vague and I'll have more details soon, including sample binaries. I at least wanted to give a status update so you didn't think you were being ignored. Thanks, -- Shawn Webb Cofounder and Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera at is.a.hacker.sx GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE On Wed, Nov 28, 2018 at 09:16:24PM -0800, Peter Collingbourne wrote:> https://reviews.llvm.org/D55046 fixes your reproducer here. Let me know if > that works for you. > > Note that I had to rebuild one of your bitcode files to be a regular object > file, this is because LTO doesn't want the .eh_frame terminator to live in > a bitcode file. > > $ mv ./usr/lib/crtendS.o ./usr/lib/crtendS.o.bak > $ ~/l4/ra/bin/llc -o ./usr/lib/crtendS.o ./usr/lib/crtendS.o.bak > -filetype=obj -relocation-model=pic > > Peter > > On Wed, Nov 28, 2018 at 5:35 PM Shawn Webb <shawn.webb at hardenedbsd.org> > wrote: > > > Hey Peter, > > > > Here you go! > > > > https://hardenedbsd.org/~shawn/2018-11-28_reproduce-01.tar > > > > Thanks, > > > > -- > > Shawn Webb > > Cofounder and Security Engineer > > HardenedBSD > > > > Tor-ified Signal: +1 443-546-8752 > > Tor+XMPP+OTR: lattera at is.a.hacker.sx > > GPG Key ID: 0x6A84658F52456EEE > > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > > > > On Wed, Nov 28, 2018 at 05:30:57PM -0800, Peter Collingbourne wrote: > > > Hi Shawn, > > > > > > Can you please create a reproducer tarball (using ld.lld --reproduce) so > > > that we don't need to install HardenedBSD in order to reproduce? > > > > > > Peter > > > > > > On Wed, Nov 28, 2018 at 5:16 PM Shawn Webb via llvm-dev < > > > llvm-dev at lists.llvm.org> wrote: > > > > > > > Hey LLVM folks, > > > > > > > > I've run into an interesting assertion. In one of HardenedBSD's > > > > feature branches, we're working on integration llvm's Cross-DSO CFI > > > > implementation. Using Cross-DSO CFI requires building libs with LTO, > > > > which causes clang to emit LLVM IR intermediate object files rather > > > > than ELF intermediate object files. > > > > > > > > I've found that with lld, attempting to link LLVM IR intermediate > > > > object files hits an assert in lld. I've created a reproduction test > > > > case in this tiny little repo: https://github.com/lattera/ifunc_repro > > > > > > > > The assertion I hit is detailed in the commit message of the initial > > > > commit: > > > > > > > > > > https://github.com/lattera/ifunc_repro/commit/0be98f9e81a1c91e80b135da6bb8d073d7a0c6f7 > > > > > > > > HardenedBSD's Cross-DSO CFI feature branch uses clang/llvm/lld 7.0.1. > > > > I'm more than happy to test out patches to help address this issue. > > > > > > > > Please let me know if you have any questions, comments, or concerns. > > > > > > > > Thanks, > > > > > > > > -- > > > > Shawn Webb > > > > Cofounder and Security Engineer > > > > HardenedBSD > > > > > > > > Tor-ified Signal: +1 443-546-8752 > > > > Tor+XMPP+OTR: lattera at is.a.hacker.sx > > > > GPG Key ID: 0x6A84658F52456EEE > > > > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > > > > _______________________________________________ > > > > LLVM Developers mailing list > > > > llvm-dev at lists.llvm.org > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > > > > > > > > > > -- > > > -- > > > Peter > > > > > -- > -- > Peter-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181201/bb6e3df4/attachment.sig>
It looks like this commit breaks CSU initialization with
statically-compiled applications.
With a very simple application at [1], compiled with:
cc -g -O0 -flto -static -o pid pid.c
The application segfaults:
# lldb ./pid
(lldb) target create "./pid"
Current executable set to './pid' (x86_64).
(lldb) run
Process 84115 launching
Process 84115 launched: '/root/pid' (x86_64)
Process 84115 stopped
* thread #1, name = 'pid', stop reason = signal SIGSEGV: invalid address
(fault address: 0x0)
frame #0: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 [inlined]
tsd_fetch_impl(init=true, minimal=false) at tsd.h:265
(lldb) bt
* thread #1, name = 'pid', stop reason = signal SIGSEGV: invalid address
(fault address: 0x0)
* frame #0: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 [inlined]
tsd_fetch_impl(init=true, minimal=false) at tsd.h:265
frame #1: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 [inlined] tsd_fetch
at tsd.h:292
frame #2: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 at jemalloc_tsd.c:266
frame #3: 0x0000000000225581 pid`malloc_init [inlined] malloc_init_hard at
jemalloc_jemalloc.c:1527
frame #4: 0x00000000002254ca pid`malloc_init at jemalloc_jemalloc.c:221
frame #5: 0x000000000021b208 pid`handle_static_init(argc=1,
argv=0x00006f9d260072f8, env=0x00006f9d26007308) at ignore_init.c:124
frame #6: 0x000000000021b103 pid`_start(ap=<unavailable>,
cleanup=<unavailable>) at crt1.c:75
(lldb)
[1]: https://gist.github.com/lattera/758b28c1e315cd70e670dd5211388864
The CSU can be found here:
https://github.com/HardenedBSD/hardenedBSD/tree/hardened/current/master/lib/csu
I'm working on amd64 (so crt1.c would be at lib/csu/amd64/crt1.c). The
handle_static_init function is here:
https://github.com/HardenedBSD/hardenedBSD/blob/hardened/current/master/lib/csu/common/ignore_init.c
Thanks,
--
Shawn Webb
Cofounder and Security Engineer
HardenedBSD
Tor-ified Signal: +1 443-546-8752
Tor+XMPP+OTR: lattera at is.a.hacker.sx
GPG Key ID: 0x6A84658F52456EEE
GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE
On Sat, Dec 01, 2018 at 08:56:16AM -0500, Shawn Webb
wrote:> Thanks for providing the patch! I got around to testing it this
> morning and it appears it fixes compilation, but produces a
> non-working system.
>
> I know that's kinda vague and I'll have more details soon,
including
> sample binaries. I at least wanted to give a status update so you
> didn't think you were being ignored.
>
> Thanks,
>
> --
> Shawn Webb
> Cofounder and Security Engineer
> HardenedBSD
>
> Tor-ified Signal: +1 443-546-8752
> Tor+XMPP+OTR: lattera at is.a.hacker.sx
> GPG Key ID: 0x6A84658F52456EEE
> GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE
>
> On Wed, Nov 28, 2018 at 09:16:24PM -0800, Peter Collingbourne wrote:
> > https://reviews.llvm.org/D55046 fixes your reproducer here. Let me
know if
> > that works for you.
> >
> > Note that I had to rebuild one of your bitcode files to be a regular
object
> > file, this is because LTO doesn't want the .eh_frame terminator to
live in
> > a bitcode file.
> >
> > $ mv ./usr/lib/crtendS.o ./usr/lib/crtendS.o.bak
> > $ ~/l4/ra/bin/llc -o ./usr/lib/crtendS.o ./usr/lib/crtendS.o.bak
> > -filetype=obj -relocation-model=pic
> >
> > Peter
> >
> > On Wed, Nov 28, 2018 at 5:35 PM Shawn Webb <shawn.webb at
hardenedbsd.org>
> > wrote:
> >
> > > Hey Peter,
> > >
> > > Here you go!
> > >
> > > https://hardenedbsd.org/~shawn/2018-11-28_reproduce-01.tar
> > >
> > > Thanks,
> > >
> > > --
> > > Shawn Webb
> > > Cofounder and Security Engineer
> > > HardenedBSD
> > >
> > > Tor-ified Signal: +1 443-546-8752
> > > Tor+XMPP+OTR: lattera at is.a.hacker.sx
> > > GPG Key ID: 0x6A84658F52456EEE
> > > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F
5245 6EEE
> > >
> > > On Wed, Nov 28, 2018 at 05:30:57PM -0800, Peter Collingbourne
wrote:
> > > > Hi Shawn,
> > > >
> > > > Can you please create a reproducer tarball (using ld.lld
--reproduce) so
> > > > that we don't need to install HardenedBSD in order to
reproduce?
> > > >
> > > > Peter
> > > >
> > > > On Wed, Nov 28, 2018 at 5:16 PM Shawn Webb via llvm-dev <
> > > > llvm-dev at lists.llvm.org> wrote:
> > > >
> > > > > Hey LLVM folks,
> > > > >
> > > > > I've run into an interesting assertion. In one of
HardenedBSD's
> > > > > feature branches, we're working on integration
llvm's Cross-DSO CFI
> > > > > implementation. Using Cross-DSO CFI requires building
libs with LTO,
> > > > > which causes clang to emit LLVM IR intermediate object
files rather
> > > > > than ELF intermediate object files.
> > > > >
> > > > > I've found that with lld, attempting to link LLVM
IR intermediate
> > > > > object files hits an assert in lld. I've created a
reproduction test
> > > > > case in this tiny little repo:
https://github.com/lattera/ifunc_repro
> > > > >
> > > > > The assertion I hit is detailed in the commit message
of the initial
> > > > > commit:
> > > > >
> > > > >
> > >
https://github.com/lattera/ifunc_repro/commit/0be98f9e81a1c91e80b135da6bb8d073d7a0c6f7
> > > > >
> > > > > HardenedBSD's Cross-DSO CFI feature branch uses
clang/llvm/lld 7.0.1.
> > > > > I'm more than happy to test out patches to help
address this issue.
> > > > >
> > > > > Please let me know if you have any questions, comments,
or concerns.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > --
> > > > > Shawn Webb
> > > > > Cofounder and Security Engineer
> > > > > HardenedBSD
> > > > >
> > > > > Tor-ified Signal: +1 443-546-8752
> > > > > Tor+XMPP+OTR: lattera at is.a.hacker.sx
> > > > > GPG Key ID: 0x6A84658F52456EEE
> > > > > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E
6A84 658F 5245 6EEE
> > > > > _______________________________________________
> > > > > LLVM Developers mailing list
> > > > > llvm-dev at lists.llvm.org
> > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > > > >
> > > >
> > > >
> > > > --
> > > > --
> > > > Peter
> > >
> >
> >
> > --
> > --
> > Peter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190107/05a622f9/attachment.sig>
It's at this point where I think about filing a full bug report with llvm. Any hints before I do? On Mon, Jan 07, 2019 at 04:00:02PM -0500, Shawn Webb wrote:> It looks like this commit breaks CSU initialization with > statically-compiled applications. > > With a very simple application at [1], compiled with: > cc -g -O0 -flto -static -o pid pid.c > > The application segfaults: > > # lldb ./pid > (lldb) target create "./pid" > Current executable set to './pid' (x86_64). > (lldb) run > Process 84115 launching > Process 84115 launched: '/root/pid' (x86_64) > Process 84115 stopped > * thread #1, name = 'pid', stop reason = signal SIGSEGV: invalid address (fault address: 0x0) > frame #0: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 [inlined] tsd_fetch_impl(init=true, minimal=false) at tsd.h:265 > (lldb) bt > * thread #1, name = 'pid', stop reason = signal SIGSEGV: invalid address (fault address: 0x0) > * frame #0: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 [inlined] tsd_fetch_impl(init=true, minimal=false) at tsd.h:265 > frame #1: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 [inlined] tsd_fetch at tsd.h:292 > frame #2: 0x000000000024e6d0 pid`__je_malloc_tsd_boot0 at jemalloc_tsd.c:266 > frame #3: 0x0000000000225581 pid`malloc_init [inlined] malloc_init_hard at jemalloc_jemalloc.c:1527 > frame #4: 0x00000000002254ca pid`malloc_init at jemalloc_jemalloc.c:221 > frame #5: 0x000000000021b208 pid`handle_static_init(argc=1, argv=0x00006f9d260072f8, env=0x00006f9d26007308) at ignore_init.c:124 > frame #6: 0x000000000021b103 pid`_start(ap=<unavailable>, cleanup=<unavailable>) at crt1.c:75 > (lldb) > > [1]: https://gist.github.com/lattera/758b28c1e315cd70e670dd5211388864 > > The CSU can be found here: > https://github.com/HardenedBSD/hardenedBSD/tree/hardened/current/master/lib/csu > > I'm working on amd64 (so crt1.c would be at lib/csu/amd64/crt1.c). The > handle_static_init function is here: > https://github.com/HardenedBSD/hardenedBSD/blob/hardened/current/master/lib/csu/common/ignore_init.c > > Thanks, > > -- > Shawn Webb > Cofounder and Security Engineer > HardenedBSD > > Tor-ified Signal: +1 443-546-8752 > Tor+XMPP+OTR: lattera at is.a.hacker.sx > GPG Key ID: 0x6A84658F52456EEE > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > > On Sat, Dec 01, 2018 at 08:56:16AM -0500, Shawn Webb wrote: > > Thanks for providing the patch! I got around to testing it this > > morning and it appears it fixes compilation, but produces a > > non-working system. > > > > I know that's kinda vague and I'll have more details soon, including > > sample binaries. I at least wanted to give a status update so you > > didn't think you were being ignored. > > > > Thanks, > > > > -- > > Shawn Webb > > Cofounder and Security Engineer > > HardenedBSD > > > > Tor-ified Signal: +1 443-546-8752 > > Tor+XMPP+OTR: lattera at is.a.hacker.sx > > GPG Key ID: 0x6A84658F52456EEE > > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > > > > On Wed, Nov 28, 2018 at 09:16:24PM -0800, Peter Collingbourne wrote: > > > https://reviews.llvm.org/D55046 fixes your reproducer here. Let me know if > > > that works for you. > > > > > > Note that I had to rebuild one of your bitcode files to be a regular object > > > file, this is because LTO doesn't want the .eh_frame terminator to live in > > > a bitcode file. > > > > > > $ mv ./usr/lib/crtendS.o ./usr/lib/crtendS.o.bak > > > $ ~/l4/ra/bin/llc -o ./usr/lib/crtendS.o ./usr/lib/crtendS.o.bak > > > -filetype=obj -relocation-model=pic > > > > > > Peter > > > > > > On Wed, Nov 28, 2018 at 5:35 PM Shawn Webb <shawn.webb at hardenedbsd.org> > > > wrote: > > > > > > > Hey Peter, > > > > > > > > Here you go! > > > > > > > > https://hardenedbsd.org/~shawn/2018-11-28_reproduce-01.tar > > > > > > > > Thanks, > > > > > > > > -- > > > > Shawn Webb > > > > Cofounder and Security Engineer > > > > HardenedBSD > > > > > > > > Tor-ified Signal: +1 443-546-8752 > > > > Tor+XMPP+OTR: lattera at is.a.hacker.sx > > > > GPG Key ID: 0x6A84658F52456EEE > > > > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > > > > > > > > On Wed, Nov 28, 2018 at 05:30:57PM -0800, Peter Collingbourne wrote: > > > > > Hi Shawn, > > > > > > > > > > Can you please create a reproducer tarball (using ld.lld --reproduce) so > > > > > that we don't need to install HardenedBSD in order to reproduce? > > > > > > > > > > Peter > > > > > > > > > > On Wed, Nov 28, 2018 at 5:16 PM Shawn Webb via llvm-dev < > > > > > llvm-dev at lists.llvm.org> wrote: > > > > > > > > > > > Hey LLVM folks, > > > > > > > > > > > > I've run into an interesting assertion. In one of HardenedBSD's > > > > > > feature branches, we're working on integration llvm's Cross-DSO CFI > > > > > > implementation. Using Cross-DSO CFI requires building libs with LTO, > > > > > > which causes clang to emit LLVM IR intermediate object files rather > > > > > > than ELF intermediate object files. > > > > > > > > > > > > I've found that with lld, attempting to link LLVM IR intermediate > > > > > > object files hits an assert in lld. I've created a reproduction test > > > > > > case in this tiny little repo: https://github.com/lattera/ifunc_repro > > > > > > > > > > > > The assertion I hit is detailed in the commit message of the initial > > > > > > commit: > > > > > > > > > > > > > > > > https://github.com/lattera/ifunc_repro/commit/0be98f9e81a1c91e80b135da6bb8d073d7a0c6f7 > > > > > > > > > > > > HardenedBSD's Cross-DSO CFI feature branch uses clang/llvm/lld 7.0.1. > > > > > > I'm more than happy to test out patches to help address this issue. > > > > > > > > > > > > Please let me know if you have any questions, comments, or concerns. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > -- > > > > > > Shawn Webb > > > > > > Cofounder and Security Engineer > > > > > > HardenedBSD > > > > > > > > > > > > Tor-ified Signal: +1 443-546-8752 > > > > > > Tor+XMPP+OTR: lattera at is.a.hacker.sx > > > > > > GPG Key ID: 0x6A84658F52456EEE > > > > > > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > > > > > > _______________________________________________ > > > > > > LLVM Developers mailing list > > > > > > llvm-dev at lists.llvm.org > > > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > > > > > > > > > > > > > > > > > > -- > > > > > -- > > > > > Peter > > > > > > > > > > > > > -- > > > -- > > > Peter > >-- Shawn Webb Cofounder and Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera at is.a.hacker.sx GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190109/5cb2e4df/attachment.sig>