llvm dev - Oct 2020 - LLJIT: __{math}_finite symbols not resolved ?

Hello,
Right now I am just using a Generator to look for symbols in my process
(which links dynamically against libc / libm).
It seems to have no trouble finding every other libc / libm / libc++ / ...
symbol so I assumed that it was not necessary to specifically link against
libm where these __finite symbols reside:

  $ nm -D /usr/lib/libm.so.6 |  grep finite
  0000000000050540 T __acosf128_finite at GLIBC_2.26
  0000000000042f70 T __acosf_finite at GLIBC_2.15
  0000000000026940 i __acos_finite at GLIBC_2.15
  0000000000051000 T __acoshf128_finite at GLIBC_2.26
  0000000000043240 T __acoshf_finite at GLIBC_2.15
)
but maybe it needs some help on that regard ?

Thanks for your quick answer,

Jean-Michaël


On Mon, Oct 5, 2020 at 7:53 PM Lang Hames <lhames at gmail.com> wrote:
> Hi Jean-Michaël,
>
> How are you trying to provide those symbols to the JIT? Are you using a
> DynamicLibrarySearchGenerator to reflect process symbols (or this specific
> library's symbols) into the JIT?
>
> I haven't looked at ELF symbol indirection before -- I'll need to
read up
> on that before I can provide a sensible answer. It's quite likely that
> RuntimeDyld doesn't support it yet though. Depending on what is
required we
> can either try to implement it there, or aim to fix it in the newer JITLink
> linker -- a few people are working on an initial implementation of that at
> the moment.
>
> -- Lang.
>
> On Mon, Oct 5, 2020 at 12:52 AM Jean-Michaël Celerier via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hello,
>> when building code with -Ofast -ffinite-math-only -ffast-math, clang
>> generates calls to "finite" variants of math functions.
>>
>> This has been the source of a fair amount of issues in a
"normal",
>> non-JIT pipeline, which seem to have been fixed over time - a simple
fix
>> being recompiling the target app against the new glibc.
>> - https://bugs.llvm.org/show_bug.cgi?id=44842
>> - https://github.com/cms-sw/cmssw/issues/24935
>> - https://github.com/google/filament/issues/2146
>>
>> But when going through LLJIT (tested with LLVM-10 & LLVM-11, on
>> ArchLinux, glibc-2.32) I still get
>>
>>      Symbols not found: [ __log_finite, __exp2_finite ]
>>
>> when trying to materialize my code.
>>
>> What could be done for that ? "Recompiling" doesn't seem
to fix anything
>> in this case so it looks like LLJIT lacks the mechanism to understand
the
>> ELF symbol indirection.
>>
>> Thanks,
>> Jean-Michaël
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201005/eaf9659f/attachment.html>

Hi Jean-Michaël,

Ok -- if you're linking against other symbols without issue then your setup
sounds good.

My first take is that if you're set up correctly then this should "just
work", and this failure should be considered a bug, but I need to
understand more about ELF indirect / versioned symbols before I can say
that definitively. I usually develop on MacOS, but I'll set up a VM and see
if I can reproduce this locally to get some more insight here.

In the meantime one workaround would be to define absoluteSymbol entries
for these functions:

auto Err = J->getMainJITDylib().define(
  absoluteSymbols({
    { J->mangleAndIntern("__log_finite"),
pointerToJITTargetAddress(&__log_finite) },
    { J->mangleAndIntern("__exp2_finite"),
pointerToJITTargetAddress(&__exp2_finite) }
 }));

-- Lang.

On Mon, Oct 5, 2020 at 12:31 PM Jean-Michaël Celerier <
jeanmichael.celerier at gmail.com> wrote:
> Hello,
> Right now I am just using a Generator to look for symbols in my process
> (which links dynamically against libc / libm).
> It seems to have no trouble finding every other libc / libm / libc++ / ...
> symbol so I assumed that it was not necessary to specifically link against
> libm where these __finite symbols reside:
>
>   $ nm -D /usr/lib/libm.so.6 |  grep finite
>   0000000000050540 T __acosf128_finite at GLIBC_2.26
>   0000000000042f70 T __acosf_finite at GLIBC_2.15
>   0000000000026940 i __acos_finite at GLIBC_2.15
>   0000000000051000 T __acoshf128_finite at GLIBC_2.26
>   0000000000043240 T __acoshf_finite at GLIBC_2.15
> )
> but maybe it needs some help on that regard ?
>
> Thanks for your quick answer,
>
> Jean-Michaël
>
>
> On Mon, Oct 5, 2020 at 7:53 PM Lang Hames <lhames at gmail.com>
wrote:
>
>> Hi Jean-Michaël,
>>
>> How are you trying to provide those symbols to the JIT? Are you using a
>> DynamicLibrarySearchGenerator to reflect process symbols (or this
specific
>> library's symbols) into the JIT?
>>
>> I haven't looked at ELF symbol indirection before -- I'll need
to read up
>> on that before I can provide a sensible answer. It's quite likely
that
>> RuntimeDyld doesn't support it yet though. Depending on what is
required we
>> can either try to implement it there, or aim to fix it in the newer
JITLink
>> linker -- a few people are working on an initial implementation of that
at
>> the moment.
>>
>> -- Lang.
>>
>> On Mon, Oct 5, 2020 at 12:52 AM Jean-Michaël Celerier via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hello,
>>> when building code with -Ofast -ffinite-math-only -ffast-math,
clang
>>> generates calls to "finite" variants of math functions.
>>>
>>> This has been the source of a fair amount of issues in a
"normal",
>>> non-JIT pipeline, which seem to have been fixed over time - a
simple fix
>>> being recompiling the target app against the new glibc.
>>> - https://bugs.llvm.org/show_bug.cgi?id=44842
>>> - https://github.com/cms-sw/cmssw/issues/24935
>>> - https://github.com/google/filament/issues/2146
>>>
>>> But when going through LLJIT (tested with LLVM-10 & LLVM-11, on
>>> ArchLinux, glibc-2.32) I still get
>>>
>>>      Symbols not found: [ __log_finite, __exp2_finite ]
>>>
>>> when trying to materialize my code.
>>>
>>> What could be done for that ? "Recompiling" doesn't
seem to fix anything
>>> in this case so it looks like LLJIT lacks the mechanism to
understand the
>>> ELF symbol indirection.
>>>
>>> Thanks,
>>> Jean-Michaël
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201005/c86a8204/attachment.html>

Hello,
here is a repro which runs in a docker image.
https://we.tl/t-O1EhIAOeOF

To see the issue, run repro.sh
It will first download a (big, sorry) centos:7 docker image with my build
of LLVM-11 and build a simple lljit-based example.

This example is called with some trivial .cpp which calls `cos`.
When ran from *within the container* it works.
When the same example, with the same bitcode input, runs from outside the
container, it does not find this symbol,
likely because the host (in my case Arch, I think you need a glibc-2.31 at
least for that behaviour to be visible)'s glibc symbols
became versioned.
Removing either the -fmath-errno or -ffinite-math-only flag for the clang
cpp -> bitcode invocation in build.sh fixes the issue
(at the expense of potentially slower code).

<http://www.jcelerier.name>
Thanks for the hint, sadly it's not possible to take the address of
__log_finite : what happens is that you call the function e.g. log()
in your code, and either clang or some magic glibc header transforms that
into __log_finite further down the pipeline
(see e.g. the discussion in https://reviews.llvm.org/D74712 - sadly in my
case I can't "upgrade" the headers used by my JIT SDK to
glibc-2.31+
as it would mean that only people with very very recent distros would be
able to run the code that's being jit-compiled.

Thanks !

Jean-Michaël

On Mon, Oct 5, 2020 at 10:11 PM Lang Hames <lhames at gmail.com> wrote:
> Hi Jean-Michaël,
>
> Ok -- if you're linking against other symbols without issue then your
> setup sounds good.
>
> My first take is that if you're set up correctly then this should
"just
> work", and this failure should be considered a bug, but I need to
> understand more about ELF indirect / versioned symbols before I can say
> that definitively. I usually develop on MacOS, but I'll set up a VM and
see
> if I can reproduce this locally to get some more insight here.
>
> In the meantime one workaround would be to define absoluteSymbol entries
> for these functions:
>
> auto Err = J->getMainJITDylib().define(
>   absoluteSymbols({
>     { J->mangleAndIntern("__log_finite"),
> pointerToJITTargetAddress(&__log_finite) },
>     { J->mangleAndIntern("__exp2_finite"),
> pointerToJITTargetAddress(&__exp2_finite) }
>  }));
>
> -- Lang.
>
> On Mon, Oct 5, 2020 at 12:31 PM Jean-Michaël Celerier <
> jeanmichael.celerier at gmail.com> wrote:
>
>> Hello,
>> Right now I am just using a Generator to look for symbols in my process
>> (which links dynamically against libc / libm).
>> It seems to have no trouble finding every other libc / libm / libc++ /
>> ... symbol so I assumed that it was not necessary to specifically link
>> against libm where these __finite symbols reside:
>>
>>   $ nm -D /usr/lib/libm.so.6 |  grep finite
>>   0000000000050540 T __acosf128_finite at GLIBC_2.26
>>   0000000000042f70 T __acosf_finite at GLIBC_2.15
>>   0000000000026940 i __acos_finite at GLIBC_2.15
>>   0000000000051000 T __acoshf128_finite at GLIBC_2.26
>>   0000000000043240 T __acoshf_finite at GLIBC_2.15
>> )
>> but maybe it needs some help on that regard ?
>>
>> Thanks for your quick answer,
>>
>> Jean-Michaël
>>
>>
>> On Mon, Oct 5, 2020 at 7:53 PM Lang Hames <lhames at gmail.com>
wrote:
>>
>>> Hi Jean-Michaël,
>>>
>>> How are you trying to provide those symbols to the JIT? Are you
using a
>>> DynamicLibrarySearchGenerator to reflect process symbols (or this
specific
>>> library's symbols) into the JIT?
>>>
>>> I haven't looked at ELF symbol indirection before -- I'll
need to read
>>> up on that before I can provide a sensible answer. It's quite
likely that
>>> RuntimeDyld doesn't support it yet though. Depending on what is
required we
>>> can either try to implement it there, or aim to fix it in the newer
JITLink
>>> linker -- a few people are working on an initial implementation of
that at
>>> the moment.
>>>
>>> -- Lang.
>>>
>>> On Mon, Oct 5, 2020 at 12:52 AM Jean-Michaël Celerier via llvm-dev
<
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> Hello,
>>>> when building code with -Ofast -ffinite-math-only -ffast-math,
clang
>>>> generates calls to "finite" variants of math
functions.
>>>>
>>>> This has been the source of a fair amount of issues in a
"normal",
>>>> non-JIT pipeline, which seem to have been fixed over time - a
simple fix
>>>> being recompiling the target app against the new glibc.
>>>> - https://bugs.llvm.org/show_bug.cgi?id=44842
>>>> - https://github.com/cms-sw/cmssw/issues/24935
>>>> - https://github.com/google/filament/issues/2146
>>>>
>>>> But when going through LLJIT (tested with LLVM-10 &
LLVM-11, on
>>>> ArchLinux, glibc-2.32) I still get
>>>>
>>>>      Symbols not found: [ __log_finite, __exp2_finite ]
>>>>
>>>> when trying to materialize my code.
>>>>
>>>> What could be done for that ? "Recompiling"
doesn't seem to fix
>>>> anything in this case so it looks like LLJIT lacks the
mechanism to
>>>> understand the ELF symbol indirection.
>>>>
>>>> Thanks,
>>>> Jean-Michaël
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201006/43399f58/attachment.html>