llvm dev - Jun 2007 - [LLVMdev] How to call native functions from bytecode run in JIT?

Hi,

I was able to try this on linux again. Unfortunately it doesn't work  
at all (neither using runFunction nor a CallInst). It simply says  
function called get5 not known. Calling printf the same way works,  
though. On linux the function is exported as "get5" from the  
executable while it is called "_get5" on OS X. I could not spot any  
other differences.. any thoughts?

greetings,
Jan

On 12. Jun 2007, at 23:08, Jan Rehders wrote:
> Hi,
>
>> Okay.  If the function exists in your application's address space
>> already,
>> just name the LLVM function the same name as the native function
>> and the
>> JIT should find it an do the right thing.  This is how it finds
>> printf and
>> a variety of other things.  You don't need to call  
>> addGlobalMapping at
>> all.
>
> Looking at the output of "nm codegen1" I realized that
"get5" was a C+
> + function whose name was mangled to "__Z4get5v". Surrounding it
by
> extern "C" helped a lot :) Now the function is found by the JIT
and I
> can call it using EE->runFunction als well as using a CallInst.
>
>> Does this work?
>
> However, one strange effet remains: if I first call the function
> using EE->runFunction and then try to call it using a CallInst inside
> another function I get the old "relocation" error in
PPCJITInfo.cpp,
> again. Using a CallInst first, then runFunction and then a CallInst
> again works, though. For my project this is probably a non-issue but
> it might indicate some problem in either my code or LLVM - any ideas?
>
> Anyway, thank you all for the helpful comments. I'd surely yielded to
> despair without your help :)
>
> - Jan
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On Wed, 13 Jun 2007, Jan Rehders wrote:
> I was able to try this on linux again. Unfortunately it doesn't work
> at all (neither using runFunction nor a CallInst). It simply says
> function called get5 not known. Calling printf the same way works,
> though. On linux the function is exported as "get5" from the
> executable while it is called "_get5" on OS X. I could not spot
any
> other differences.. any thoughts?
The _ shouldn't matter.  You'll have to dig into the code a little bit
and
try to understand what is failing.

-Chris
> On 12. Jun 2007, at 23:08, Jan Rehders wrote:
>
>> Hi,
>>
>>> Okay.  If the function exists in your application's address
space
>>> already,
>>> just name the LLVM function the same name as the native function
>>> and the
>>> JIT should find it an do the right thing.  This is how it finds
>>> printf and
>>> a variety of other things.  You don't need to call
>>> addGlobalMapping at
>>> all.
>>
>> Looking at the output of "nm codegen1" I realized that
"get5" was a C+
>> + function whose name was mangled to "__Z4get5v". Surrounding
it by
>> extern "C" helped a lot :) Now the function is found by the
JIT and I
>> can call it using EE->runFunction als well as using a CallInst.
>>
>>> Does this work?
>>
>> However, one strange effet remains: if I first call the function
>> using EE->runFunction and then try to call it using a CallInst
inside
>> another function I get the old "relocation" error in
PPCJITInfo.cpp,
>> again. Using a CallInst first, then runFunction and then a CallInst
>> again works, though. For my project this is probably a non-issue but
>> it might indicate some problem in either my code or LLVM - any ideas?
>>
>> Anyway, thank you all for the helpful comments. I'd surely yielded
to
>> despair without your help :)
>>
>> - Jan
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-Chris

-- 
http://nondot.org/sabre/
http://llvm.org/

Hi Jan,

In gcc for Linux, you have the -rdynamic option that allows an
executable to dlsym its symbols. Since llvm use dlsym to find the
symbols, you could try with this option. That's what I did. And don't
forget to use the C++ name if you compile with C++.

Cheers,
Nicolas

Jan Rehders wrote:
> Hi,
>
> I was able to try this on linux again. Unfortunately it doesn't work  
> at all (neither using runFunction nor a CallInst). It simply says  
> function called get5 not known. Calling printf the same way works,  
> though. On linux the function is exported as "get5" from the  
> executable while it is called "_get5" on OS X. I could not spot
any
> other differences.. any thoughts?
>
> greetings,
> Jan
>
> On 12. Jun 2007, at 23:08, Jan Rehders wrote:
>
>   
>> Hi,
>>
>>     
>>> Okay.  If the function exists in your application's address
space
>>> already,
>>> just name the LLVM function the same name as the native function
>>> and the
>>> JIT should find it an do the right thing.  This is how it finds
>>> printf and
>>> a variety of other things.  You don't need to call  
>>> addGlobalMapping at
>>> all.
>>>       
>> Looking at the output of "nm codegen1" I realized that
"get5" was a C+
>> + function whose name was mangled to "__Z4get5v". Surrounding
it by
>> extern "C" helped a lot :) Now the function is found by the
JIT and I
>> can call it using EE->runFunction als well as using a CallInst.
>>
>>     
>>> Does this work?
>>>       
>> However, one strange effet remains: if I first call the function
>> using EE->runFunction and then try to call it using a CallInst
inside
>> another function I get the old "relocation" error in
PPCJITInfo.cpp,
>> again. Using a CallInst first, then runFunction and then a CallInst
>> again works, though. For my project this is probably a non-issue but
>> it might indicate some problem in either my code or LLVM - any ideas?
>>
>> Anyway, thank you all for the helpful comments. I'd surely yielded
to
>> despair without your help :)
>>
>> - Jan
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>     
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Hello,

finally I tried a bit more but could not get the native calls to work.

I checked how dlsym works on Linux and OS X. On the latter I can  
dlsym any function in a dynamically loaded library as well as in the  
application. On Linux only functions in loaded dlls can be retrieved  
using dlsym. Adding -rdynamic as Nicolas suggested allows me to load  
functions from the exe, too. The following code works fine on Linux  
and OS X:

void* dllHandle = dlopen( "./codegen1.so", RTLD_LAZY );
void* exeHandle = dlopen( NULL, RTLD_LAZY );

// int get5() is contained in the dll, get6 in the app (both using  
extern "C")
getAndCall( dllHandle, "get5" );
getAndCall( exeHandle, "get6" );

void getAndCall(void* handle, char* name) {
   int (*get)() = (int(*)()) dlsym( handle, name );

   if( dlerror() == NULL && get != NULL )
     printf( "Calling %s() = %d\n", name, get() );
}

However if I use LLVM like described in my previous mails it only  
works on OS X and cannot find the functions on Linux. This is both  
true for functions in the exe as well as in a loaded dll (passing - 
lfoo or foo.so to the linker). Using addGlobalMapping does not help,  
neither.

Until now the dlsym behavior is the only relevant difference I could  
spot between the two systems. However -dynamic should remove this  
difference. Still LLVM does not find any functions apart from printf  
in the system libs. My code is identical on both systems.
> In gcc for Linux, you have the -rdynamic option that allows an
> executable to dlsym its symbols. Since llvm use dlsym to find the
> symbols, you could try with this option. That's what I did. And
don't
> forget to use the C++ name if you compile with C++.
Do you have a working code sample (with LLVM) which I could compare  
to my own code?

Currently I'm pretty much out of ideas. The next thing to try would  
probably to install a debug version of LLVM and dive into the source  
using a debugger. But I fear I don't have enough time for this. I had  
a short look at some of the functions to identify libraries but could  
not make too much sense of it, yet.

greetings,
Jan

Hi,

attached is a small testcase I did. It builds two LLVM functions  
which both call two native functions get5 and get6. The native  
functions are in the exe and in the dll. On OS X it works like a  
charm. On Linux none of the two functions can be called.

Maybe someone can try them or have a look at it to see if there is  
something obviously wrong

greetings,
Jan


-------------- next part --------------
A non-text attachment was scrubbed...
Name: codegen1.cpp
Type: application/octet-stream
Size: 4065 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20070627/54390960/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: foo.c
Type: application/octet-stream
Size: 29 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20070627/54390960/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: makefile
Type: application/octet-stream
Size: 1923 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20070627/54390960/attachment-0002.obj>
-------------- next part --------------

Hello, Nicolas.
> This will force an indirect call, and won't use the jump-size
limitation
> of the bl instruction.
I think we're using some logic to determine, whether call should be
"direct" or "indirect" on x86. Look for
GVRequiresExtraLoad() routine in
the X86Subtarget.cpp

-- 
With best regards, Anton Korobeynikov.

Faculty of Mathematics & Mechanics, Saint Petersburg State University.