llvm dev - Jun 2018 - Since MCJIT I can't get libm functions to work

I am upgrading our JIT application to use LLVM 6.0.0, and with this 
transition I am making the move to use the new MCJIT facility.

A problem I am encountering is that the math functions from libm are not 
resolved/found. I am using the lambda resolver from the KaleidoscopeJIT 
class which first searches the present modules and, if that is 
unsuccessful, continues the search in the whole process space.

The generated modules would declare external functions like

declare float @cosf(float)

and would call it like:

   %17 = call float @cosf(float %16)

The datalayout is set to

target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"

The code snippet that adds the module is pretty much from the JIT tutorial:

       auto Resolver = llvm::orc::createLambdaResolver(
                               [&](const std::string &Name) {
                             if (auto Sym = 
CompileLayer.findSymbol(Name, false))
                               return Sym;
                             return llvm::JITSymbol(nullptr);
                               },
                               [](const std::string &Name) {
                             if (auto SymAddr
llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
                               return llvm::JITSymbol(SymAddr, 
llvm::JITSymbolFlags::Exported);
                             return llvm::JITSymbol(nullptr);
                               });

       cantFail(CompileLayer.addModule(std::move(M),
                       std::move(Resolver)));

When running the program I receive the following error:

LLVM ERROR: Program used external function 'cosf' which could not be 
resolved!

This is on an Intel i7 CPU with LLVM targets configured as "X86".

Adding the '-lm' and '-ldl' option to the linker command that
links the
final program doesn't help. (I even called the 'cosf' function by
hand
in the host code to make sure it is mapped - didn't change the behavior 
of the MCJIT resolver.)

Any ideas?


Best wishes,

Frank

Hi Frank,

This is on Linux, right? If so, you will need to pass --export-dynamic to
ld to export symbols from the main binary.

For example, given source file foo.c:

#include <stdio.h>
#include <dlfcn.h>

void bar() {}

int main(int argc, char *argv[]) {
  printf("bar is %p\n", dlsym(RTLD_DEFAULT, "bar"));
  return 0;
}

You will see:

$ clang -O3 -D_GNU_SOURCE=1 -o foo foo.c -ldl
$ ./foo
bar is (nil)
$ clang -O3 -D_GNU_SOURCE=1 -Wl,--export-dynamic -o foo foo.c -ldl
$ ./foo
bar is 0x400820

Hope this helps!

Cheers,
Lang.


On Thu, Jun 28, 2018 at 12:40 PM Frank Winter via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I am upgrading our JIT application to use LLVM 6.0.0, and with this
> transition I am making the move to use the new MCJIT facility.
>
> A problem I am encountering is that the math functions from libm are not
> resolved/found. I am using the lambda resolver from the KaleidoscopeJIT
> class which first searches the present modules and, if that is
> unsuccessful, continues the search in the whole process space.
>
> The generated modules would declare external functions like
>
> declare float @cosf(float)
>
> and would call it like:
>
>    %17 = call float @cosf(float %16)
>
> The datalayout is set to
>
> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>
> The code snippet that adds the module is pretty much from the JIT tutorial:
>
>        auto Resolver = llvm::orc::createLambdaResolver(
>                                [&](const std::string &Name) {
>                              if (auto Sym >
CompileLayer.findSymbol(Name, false))
>                                return Sym;
>                              return llvm::JITSymbol(nullptr);
>                                },
>                                [](const std::string &Name) {
>                              if (auto SymAddr >
llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
>                                return llvm::JITSymbol(SymAddr,
> llvm::JITSymbolFlags::Exported);
>                              return llvm::JITSymbol(nullptr);
>                                });
>
>        cantFail(CompileLayer.addModule(std::move(M),
>                        std::move(Resolver)));
>
> When running the program I receive the following error:
>
> LLVM ERROR: Program used external function 'cosf' which could not
be
> resolved!
>
> This is on an Intel i7 CPU with LLVM targets configured as "X86".
>
> Adding the '-lm' and '-ldl' option to the linker command
that links the
> final program doesn't help. (I even called the 'cosf' function
by hand
> in the host code to make sure it is mapped - didn't change the behavior
> of the MCJIT resolver.)
>
> Any ideas?
>
>
> Best wishes,
>
> Frank
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180628/f89f3a93/attachment.html>

Hi Frank,

If I am not mistaken it doesn't look in the whole process space by default.
Please, try loading all the symbols explicitly:

	sys::DynamicLibrary::LoadLibraryPermanently(nullptr);

If it doesn't help, then you may try compiling your host program with
-rdynamic, otherwise dlsym may not see all the symbols. At least it was the case
for me on Linux.

I hope it helps.

Cheers,
Alex.
> On 28. Jun 2018, at 21:40, Frank Winter via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> I am upgrading our JIT application to use LLVM 6.0.0, and with this
transition I am making the move to use the new MCJIT facility.
> 
> A problem I am encountering is that the math functions from libm are not
resolved/found. I am using the lambda resolver from the KaleidoscopeJIT class
which first searches the present modules and, if that is unsuccessful, continues
the search in the whole process space.
> 
> The generated modules would declare external functions like
> 
> declare float @cosf(float)
> 
> and would call it like:
> 
>   %17 = call float @cosf(float %16)
> 
> The datalayout is set to
> 
> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
> 
> The code snippet that adds the module is pretty much from the JIT tutorial:
> 
>       auto Resolver = llvm::orc::createLambdaResolver(
>                               [&](const std::string &Name) {
>                             if (auto Sym = CompileLayer.findSymbol(Name,
false))
>                               return Sym;
>                             return llvm::JITSymbol(nullptr);
>                               },
>                               [](const std::string &Name) {
>                             if (auto SymAddr >
llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
>                               return llvm::JITSymbol(SymAddr,
llvm::JITSymbolFlags::Exported);
>                             return llvm::JITSymbol(nullptr);
>                               });
> 
>       cantFail(CompileLayer.addModule(std::move(M),
>                       std::move(Resolver)));
> 
> When running the program I receive the following error:
> 
> LLVM ERROR: Program used external function 'cosf' which could not
be resolved!
> 
> This is on an Intel i7 CPU with LLVM targets configured as "X86".
> 
> Adding the '-lm' and '-ldl' option to the linker command
that links the final program doesn't help. (I even called the 'cosf'
function by hand in the host code to make sure it is mapped - didn't change
the behavior of the MCJIT resolver.)
> 
> Any ideas?
> 
> 
> Best wishes,
> 
> Frank
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: Message signed with OpenPGP
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180628/4f5f9f87/attachment.sig>

Hi Alex,

loading the symbols explicitly helped. With this the build process 
doesn't need any additional linker flags. Very nice.

Thanks,
Frank


On 06/28/2018 04:38 PM, Alex Denisov wrote:
> Hi Frank,
>
> If I am not mistaken it doesn't look in the whole process space by
default.
> Please, try loading all the symbols explicitly:
>
> 	sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
>
> If it doesn't help, then you may try compiling your host program with
-rdynamic, otherwise dlsym may not see all the symbols. At least it was the case
for me on Linux.
>
> I hope it helps.
>
> Cheers,
> Alex.
>
>> On 28. Jun 2018, at 21:40, Frank Winter via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>
>> I am upgrading our JIT application to use LLVM 6.0.0, and with this
transition I am making the move to use the new MCJIT facility.
>>
>> A problem I am encountering is that the math functions from libm are
not resolved/found. I am using the lambda resolver from the KaleidoscopeJIT
class which first searches the present modules and, if that is unsuccessful,
continues the search in the whole process space.
>>
>> The generated modules would declare external functions like
>>
>> declare float @cosf(float)
>>
>> and would call it like:
>>
>>    %17 = call float @cosf(float %16)
>>
>> The datalayout is set to
>>
>> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>
>> The code snippet that adds the module is pretty much from the JIT
tutorial:
>>
>>        auto Resolver = llvm::orc::createLambdaResolver(
>>                                [&](const std::string &Name) {
>>                              if (auto Sym =
CompileLayer.findSymbol(Name, false))
>>                                return Sym;
>>                              return llvm::JITSymbol(nullptr);
>>                                },
>>                                [](const std::string &Name) {
>>                              if (auto SymAddr >>
llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
>>                                return llvm::JITSymbol(SymAddr,
llvm::JITSymbolFlags::Exported);
>>                              return llvm::JITSymbol(nullptr);
>>                                });
>>
>>        cantFail(CompileLayer.addModule(std::move(M),
>>                        std::move(Resolver)));
>>
>> When running the program I receive the following error:
>>
>> LLVM ERROR: Program used external function 'cosf' which could
not be resolved!
>>
>> This is on an Intel i7 CPU with LLVM targets configured as
"X86".
>>
>> Adding the '-lm' and '-ldl' option to the linker
command that links the final program doesn't help. (I even called the
'cosf' function by hand in the host code to make sure it is mapped -
didn't change the behavior of the MCJIT resolver.)
>>
>> Any ideas?
>>
>>
>> Best wishes,
>>
>> Frank
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev