llvm dev - Oct 2012 - [LLVMdev] Dynamically loading native code generated from LLVM IR

Kaylor,

do you have some good documented example code which shows the usage of 
the MCJIT ?
This would help a lot ... the sematic of lots of API calls are not 
intuitively understandable.

Best Regards

--Armin



Kaylor, Andrew wrote:
> I'm not sure I understand your use case, but MCJIT (as opposed to the
legacy JIT) does almost exactly what you're asking for.  It generates an
in-memory object file image (using addPassesToEmitMC) and then loads and links
it for execution.
>
> If there's some particular detail you don't like in the way this is
happening, you might be able to generate a file as you have and then use the
RuntimeDyld interface to load it.  The llvm-rtdyld tool does something like
this.
>
> -Andy
>
> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at
cs.uiuc.edu] On Behalf Of Baris Aktemur
> Sent: Thursday, October 11, 2012 11:58 PM
> To: llvmdev at cs.uiuc.edu
> Subject: [LLVMdev] Dynamically loading native code generated from LLVM IR
>
> Hi,
>
> I'm building LLVM IR. I'd like to compile this IR to native code (I
don't want JIT) and immediately load it to execute. So far, I've the
following:
>
> 1) I can emit the IR to native assembly/object file doing the same thing
llc does (using TargetMachine::addPassesToEmitFile).
> 2) I can dynamically load a precompiled .so file (using
llvm::sys::DynamicLibrary::getPermanentLibrary), get a function pointer from
that file, and execute.
>
> I can't dynamically load the .o file I produce in step 1 because
it's a static library. If I could produce a .so file in step 1, my problem
would be solved. llc has a "-relocation-model=pic" option, but the
file produced with that did not dynamically load. I got lost in clang's
options when trying to find where the "-shared" and "-fPIC"
options are used.
>
> So, my question is: Which API should I look at to emit dynamically loadable
native code from LLVM IR?
>
> I would also like to emit code to an in-memory stream instead of a file
because everything happens at runtime, but that's a secondary concern.
>
> Thanks in advance.
>
> -Baris Aktemur
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Take a look at the MCJIT unit tests under unittests/ExecutionEngine/MCJIT

The MCJITTestBase class does the majority of the interactions with the LLVM API
you're referring to.

Good luck,
Dan

On 2012-10-13, at 4:57 AM, "Armin Steinhoff" <as at
steinhoff-automation.com> wrote:
> 
> Kaylor,
> 
> do you have some good documented example code which shows the usage of the
MCJIT ?
> This would help a lot ... the sematic of lots of API calls are not
intuitively understandable.
> 
> Best Regards
> 
> --Armin
> 
> 
> 
> Kaylor, Andrew wrote:
>> I'm not sure I understand your use case, but MCJIT (as opposed to
the legacy JIT) does almost exactly what you're asking for.  It generates an
in-memory object file image (using addPassesToEmitMC) and then loads and links
it for execution.
>> 
>> If there's some particular detail you don't like in the way
this is happening, you might be able to generate a file as you have and then use
the RuntimeDyld interface to load it.  The llvm-rtdyld tool does something like
this.
>> 
>> -Andy
>> 
>> -----Original Message-----
>> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at
cs.uiuc.edu] On Behalf Of Baris Aktemur
>> Sent: Thursday, October 11, 2012 11:58 PM
>> To: llvmdev at cs.uiuc.edu
>> Subject: [LLVMdev] Dynamically loading native code generated from LLVM
IR
>> 
>> Hi,
>> 
>> I'm building LLVM IR. I'd like to compile this IR to native
code (I don't want JIT) and immediately load it to execute. So far, I've
the following:
>> 
>> 1) I can emit the IR to native assembly/object file doing the same
thing llc does (using TargetMachine::addPassesToEmitFile).
>> 2) I can dynamically load a precompiled .so file (using
llvm::sys::DynamicLibrary::getPermanentLibrary), get a function pointer from
that file, and execute.
>> 
>> I can't dynamically load the .o file I produce in step 1 because
it's a static library. If I could produce a .so file in step 1, my problem
would be solved. llc has a "-relocation-model=pic" option, but the
file produced with that did not dynamically load. I got lost in clang's
options when trying to find where the "-shared" and "-fPIC"
options are used.
>> 
>> So, my question is: Which API should I look at to emit dynamically
loadable native code from LLVM IR?
>> 
>> I would also like to emit code to an in-memory stream instead of a file
because everything happens at runtime, but that's a secondary concern.
>> 
>> Thanks in advance.
>> 
>> -Baris Aktemur
>> 
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Daniel,

I didn't find the MCJIT directory under unitests/ExecutionEngine ... 
there is only a directory called JIT.
You mean this directory ?

Many thanks

--Armin



Malea, Daniel wrote:
> Take a look at the MCJIT unit tests under unittests/ExecutionEngine/MCJIT
>
> The MCJITTestBase class does the majority of the interactions with the LLVM
API you're referring to.
>
> Good luck,
> Dan
>
> On 2012-10-13, at 4:57 AM, "Armin Steinhoff" <as at
steinhoff-automation.com> wrote:
>
>> Kaylor,
>>
>> do you have some good documented example code which shows the usage of
the MCJIT ?
>> This would help a lot ... the sematic of lots of API calls are not
intuitively understandable.
>>
>> Best Regards
>>
>> --Armin
>>
>>
>>
>> Kaylor, Andrew wrote:
>>> I'm not sure I understand your use case, but MCJIT (as opposed
to the legacy JIT) does almost exactly what you're asking for.  It generates
an in-memory object file image (using addPassesToEmitMC) and then loads and
links it for execution.
>>>
>>> If there's some particular detail you don't like in the way
this is happening, you might be able to generate a file as you have and then use
the RuntimeDyld interface to load it.  The llvm-rtdyld tool does something like
this.
>>>
>>> -Andy
>>>
>>> -----Original Message-----
>>> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at
cs.uiuc.edu] On Behalf Of Baris Aktemur
>>> Sent: Thursday, October 11, 2012 11:58 PM
>>> To: llvmdev at cs.uiuc.edu
>>> Subject: [LLVMdev] Dynamically loading native code generated from
LLVM IR
>>>
>>> Hi,
>>>
>>> I'm building LLVM IR. I'd like to compile this IR to native
code (I don't want JIT) and immediately load it to execute. So far, I've
the following:
>>>
>>> 1) I can emit the IR to native assembly/object file doing the same
thing llc does (using TargetMachine::addPassesToEmitFile).
>>> 2) I can dynamically load a precompiled .so file (using
llvm::sys::DynamicLibrary::getPermanentLibrary), get a function pointer from
that file, and execute.
>>>
>>> I can't dynamically load the .o file I produce in step 1
because it's a static library. If I could produce a .so file in step 1, my
problem would be solved. llc has a "-relocation-model=pic" option, but
the file produced with that did not dynamically load. I got lost in clang's
options when trying to find where the "-shared" and "-fPIC"
options are used.
>>>
>>> So, my question is: Which API should I look at to emit dynamically
loadable native code from LLVM IR?
>>>
>>> I would also like to emit code to an in-memory stream instead of a
file because everything happens at runtime, but that's a secondary concern.
>>>
>>> Thanks in advance.
>>>
>>> -Baris Aktemur
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev