llvm dev - Feb 2020 - Demotion of shared symbols resolved from the dynamic linker.

On PowerPC we have a failing address sanitizer test when linking with LLD.
The issue is that the symbol '__libc_stack_end' is resolved to an
undefined
weak symbol when linking with LLD but not when linking (with the exact same
command/input) with other linkers. Tracing the symbol I see it is resolved
to a definition in the dynamic linker as expected:

/home/sfertile/LLVM_MonoRepo/build/lib/clang/11.0.0/lib/linux/libclang_rt.asan-powerpc64le.a(sanitizer_linux.cpp.o):
reference to __libc_stack_end
/lib/powerpc64le-linux-gnu/libpthread.so.0: reference to __libc_stack_end
/lib/powerpc64le-linux-gnu/ld64.so.2: shared definition of __libc_stack_end
<internal>: reference to __libc_stack_end

The last line in the trace output shows where we demote the shared
definition to an undefined symbol here:
https://github.com/llvm/llvm-project/blob/c8bfed05e21f945b5ac71cd01d62e2854a8ddcf9/lld/ELF/Driver.cpp#L1505

I'm guessing the fix is that if `needsInterpSection()` is true then the
dynamic linker should be marked as needed. Its going to end up in the
DT_NEEDED anyway so the symbols can't become dangling references. In my
case then, the demotion won't happen and everything works as expected. Is
this the right direction?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200228/e8612768/attachment.html>

On Fri, Feb 28, 2020 at 7:24 AM Sean Fertile via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On PowerPC we have a failing address sanitizer test when linking with LLD.
> The issue is that the symbol '__libc_stack_end' is resolved to an
undefined
> weak symbol when linking with LLD but not when linking (with the exact same
> command/input) with other linkers. Tracing the symbol I see it is resolved
> to a definition in the dynamic linker as expected:
>
>
/home/sfertile/LLVM_MonoRepo/build/lib/clang/11.0.0/lib/linux/libclang_rt.asan-powerpc64le.a(sanitizer_linux.cpp.o):
> reference to __libc_stack_end
> /lib/powerpc64le-linux-gnu/libpthread.so.0: reference to __libc_stack_end
> /lib/powerpc64le-linux-gnu/ld64.so.2: shared definition of __libc_stack_end
> <internal>: reference to __libc_stack_end
>
> The last line in the trace output shows where we demote the shared
> definition to an undefined symbol here:
>
https://github.com/llvm/llvm-project/blob/c8bfed05e21f945b5ac71cd01d62e2854a8ddcf9/lld/ELF/Driver.cpp#L1505
>
> I'm guessing the fix is that if `needsInterpSection()` is true then the
> dynamic linker should be marked as needed. Its going to end up in the
> DT_NEEDED anyway so the symbols can't become dangling references. In my
> case then, the demotion won't happen and everything works as expected.
Is
> this the right direction?
>
Can you name the target and upload a reproduce file (--reproduce=) ?

I have checked the x86-64 case: clang -fsanitize=address -fuse-ld=lld a.c
-o a -Wl,-y,__libc_stack_end
The demotion works as expected. ld.so is linked because of AS_NEEDED(...)
in libc.so (a linker script).
It is dropped from DT_NEEDED entries because it only provides definitions
resolving weak references.

--dynamic-linker= pointing to ld.so does not mean ld.so will be added to
DT_NEEDED.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200228/fdfb9b94/attachment.html>

The target is specifically PowerPC64, I've created a bugzilla with the
reproducer: https://bugs.llvm.org/show_bug.cgi?id=45076 and an explanation
of what is causing the failure when linking with LLD.

It is dropped from DT_NEEDED entries because it only provides
definitions
> resolving weak references.
>
You are right it is not added to DT_NEEDED by any of the linkers, I was
looking at the output from ldd which shows it as a dependency and I didn't
realize that it wasn't marked as needed. There is still a behavior
difference between what lld does: demotes the symbol to undefined and emits
a got entry with no dynamic relocation, and what both gold and bfd do: emit
the relocation for the got entry. Is this difference intended?

On Fri, Feb 28, 2020 at 3:44 PM Fāng-ruì Sòng <maskray at google.com>
wrote:
> On Fri, Feb 28, 2020 at 7:24 AM Sean Fertile via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On PowerPC we have a failing address sanitizer test when linking with
>> LLD. The issue is that the symbol '__libc_stack_end' is
resolved to an
>> undefined weak symbol when linking with LLD but not when linking (with
the
>> exact same command/input) with other linkers. Tracing the symbol I see
it
>> is resolved to a definition in the dynamic linker as expected:
>>
>>
/home/sfertile/LLVM_MonoRepo/build/lib/clang/11.0.0/lib/linux/libclang_rt.asan-powerpc64le.a(sanitizer_linux.cpp.o):
>> reference to __libc_stack_end
>> /lib/powerpc64le-linux-gnu/libpthread.so.0: reference to
__libc_stack_end
>> /lib/powerpc64le-linux-gnu/ld64.so.2: shared definition of
>> __libc_stack_end
>> <internal>: reference to __libc_stack_end
>>
>> The last line in the trace output shows where we demote the shared
>> definition to an undefined symbol here:
>>
https://github.com/llvm/llvm-project/blob/c8bfed05e21f945b5ac71cd01d62e2854a8ddcf9/lld/ELF/Driver.cpp#L1505
>>
>> I'm guessing the fix is that if `needsInterpSection()` is true then
the
>> dynamic linker should be marked as needed. Its going to end up in the
>> DT_NEEDED anyway so the symbols can't become dangling references.
In my
>> case then, the demotion won't happen and everything works as
expected. Is
>> this the right direction?
>>
>
> Can you name the target and upload a reproduce file (--reproduce=) ?
>
> I have checked the x86-64 case: clang -fsanitize=address -fuse-ld=lld a.c
> -o a -Wl,-y,__libc_stack_end
> The demotion works as expected. ld.so is linked because of AS_NEEDED(...)
> in libc.so (a linker script).
> It is dropped from DT_NEEDED entries because it only provides definitions
> resolving weak references.
>
> --dynamic-linker= pointing to ld.so does not mean ld.so will be added to
> DT_NEEDED.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200302/c8b49296/attachment.html>