llvm dev - Jul 2010 - [LLVMdev] Another way to JIT: "dlopen from memory"

On Tue, Jul 6, 2010 at 9:08 AM, Greg Clayton <gclayton at apple.com>
wrote:
>> 2) Can I easily debug JIT'd code with LLDB?
>
> Are you running JIT'd code within another process that can be debugged
(i.e. the simulation binary)? If so, you might want to have LLVM generate a full
blown dylib, not just a JIT'd chunk of code and load the dylib using the
standard shared library load/unload calls. Why? This gets you around the fact
that JIT'd code isn't recognized by the dynamic linker on most systems.
/usr/bin/dyld on Mac OS X has some provisions for JIT'd code, but nothing
that gives debuggers visibility into that code. So going the dylib route has a
few benefits:
> - code will work with any debugger (gdb and lldb) in a standard kind of way
> - dylib's can have debug information generated in the standard way
> - no need to manage memory chunks for JIT'd code
This idea has been kicked around a few times on IRC and elsewhere, and
I think it's pretty clever.  The idea is to JIT by making a valid
dylib/dll/so (heceforth, dylib) with the JITed code in memory, and
then convincing the loader to load it.  This is nice for the reasons
listed above.

I think it would be neat if LLVM could package up this functionality,
since it fits most use cases for the current LLVM JIT.  LLVM
compilation is already heavyweight, so the overhead of linking and
loading the library shouldn't be too bad.

For now, this can be achieved by rolling your own compilation pipeline of:
1. run llc or the equivalent to make a .s in a temp dir
2. assemble it
3. link it to make a dylib in the temp dir
4. dlopen it
5. use dladdr to get the function pointer for your code

1 and 2 can already be done in memory with LLVM.

3 cannot, and doesn't look like it is on anyone's roadmap.

I think 4 may become possible on OS X, and there's this feature
request for glibc that essentially implements "dlopen from memory":
http://sourceware.org/bugzilla/show_bug.cgi?id=11767

5 is trivial.

Even a prototype of this initial JIT model that uses temporaries on
the filesystem would be useful for JIT users doing debugging or
profiling.

I don't really have time to implement this.  I'm just kicking the
proverbial can down the road.  I may file a feature request PR or
write up something in the style of sabre's LLVMNotes:
http://nondot.org/sabre/LLVMNotes/

Reid

Wouldn't it mean each time a function is compiled it would need to be
bundled to its own dylib? How well would that scale?

Félix

Le 2010-07-07 à 15:41:18, Reid Kleckner a écrit :
> On Tue, Jul 6, 2010 at 9:08 AM, Greg Clayton <gclayton at apple.com>
wrote:
>>> 2) Can I easily debug JIT'd code with LLDB?
>> 
>> Are you running JIT'd code within another process that can be
debugged (i.e. the simulation binary)? If so, you might want to have LLVM
generate a full blown dylib, not just a JIT'd chunk of code and load the
dylib using the standard shared library load/unload calls. Why? This gets you
around the fact that JIT'd code isn't recognized by the dynamic linker
on most systems. /usr/bin/dyld on Mac OS X has some provisions for JIT'd
code, but nothing that gives debuggers visibility into that code. So going the
dylib route has a few benefits:
>> - code will work with any debugger (gdb and lldb) in a standard kind of
way
>> - dylib's can have debug information generated in the standard way
>> - no need to manage memory chunks for JIT'd code
> 
> This idea has been kicked around a few times on IRC and elsewhere, and
> I think it's pretty clever.  The idea is to JIT by making a valid
> dylib/dll/so (heceforth, dylib) with the JITed code in memory, and
> then convincing the loader to load it.  This is nice for the reasons
> listed above.
> 
> I think it would be neat if LLVM could package up this functionality,
> since it fits most use cases for the current LLVM JIT.  LLVM
> compilation is already heavyweight, so the overhead of linking and
> loading the library shouldn't be too bad.
> 
> For now, this can be achieved by rolling your own compilation pipeline of:
> 1. run llc or the equivalent to make a .s in a temp dir
> 2. assemble it
> 3. link it to make a dylib in the temp dir
> 4. dlopen it
> 5. use dladdr to get the function pointer for your code
> 
> 1 and 2 can already be done in memory with LLVM.
> 
> 3 cannot, and doesn't look like it is on anyone's roadmap.
> 
> I think 4 may become possible on OS X, and there's this feature
> request for glibc that essentially implements "dlopen from
memory":
> http://sourceware.org/bugzilla/show_bug.cgi?id=11767
> 
> 5 is trivial.
> 
> Even a prototype of this initial JIT model that uses temporaries on
> the filesystem would be useful for JIT users doing debugging or
> profiling.
> 
> I don't really have time to implement this.  I'm just kicking the
> proverbial can down the road.  I may file a feature request PR or
> write up something in the style of sabre's LLVMNotes:
> http://nondot.org/sabre/LLVMNotes/
> 
> Reid
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Probably not well.  But it depends on the application, and probably the loader.

Some applications (OpenCL comes to mind) have the bitcode ready to go
as soon as they start.  The only reason they use the JIT is because
they want to compile the code to run fast on the available hardware.

It really depends on how often the application wants to generate code,
which for some people may not be that often.

This is a new model that doesn't really fit with the current
interface, but it could be made to work.  I just wanted to throw the
idea out for discussion and to gauge interest.

Reid

2010/7/7 Félix Cloutier <felixcca at yahoo.ca>:
> Wouldn't it mean each time a function is compiled it would need to be
bundled to its own dylib? How well would that scale?
>
> Félix
>
> Le 2010-07-07 à 15:41:18, Reid Kleckner a écrit :
>
>> On Tue, Jul 6, 2010 at 9:08 AM, Greg Clayton <gclayton at
apple.com> wrote:
>>>> 2) Can I easily debug JIT'd code with LLDB?
>>>
>>> Are you running JIT'd code within another process that can be
debugged (i.e. the simulation binary)? If so, you might want to have LLVM
generate a full blown dylib, not just a JIT'd chunk of code and load the
dylib using the standard shared library load/unload calls. Why? This gets you
around the fact that JIT'd code isn't recognized by the dynamic linker
on most systems. /usr/bin/dyld on Mac OS X has some provisions for JIT'd
code, but nothing that gives debuggers visibility into that code. So going the
dylib route has a few benefits:
>>> - code will work with any debugger (gdb and lldb) in a standard
kind of way
>>> - dylib's can have debug information generated in the standard
way
>>> - no need to manage memory chunks for JIT'd code
>>
>> This idea has been kicked around a few times on IRC and elsewhere, and
>> I think it's pretty clever.  The idea is to JIT by making a valid
>> dylib/dll/so (heceforth, dylib) with the JITed code in memory, and
>> then convincing the loader to load it.  This is nice for the reasons
>> listed above.
>>
>> I think it would be neat if LLVM could package up this functionality,
>> since it fits most use cases for the current LLVM JIT.  LLVM
>> compilation is already heavyweight, so the overhead of linking and
>> loading the library shouldn't be too bad.
>>
>> For now, this can be achieved by rolling your own compilation pipeline
of:
>> 1. run llc or the equivalent to make a .s in a temp dir
>> 2. assemble it
>> 3. link it to make a dylib in the temp dir
>> 4. dlopen it
>> 5. use dladdr to get the function pointer for your code
>>
>> 1 and 2 can already be done in memory with LLVM.
>>
>> 3 cannot, and doesn't look like it is on anyone's roadmap.
>>
>> I think 4 may become possible on OS X, and there's this feature
>> request for glibc that essentially implements "dlopen from
memory":
>> http://sourceware.org/bugzilla/show_bug.cgi?id=11767
>>
>> 5 is trivial.
>>
>> Even a prototype of this initial JIT model that uses temporaries on
>> the filesystem would be useful for JIT users doing debugging or
>> profiling.
>>
>> I don't really have time to implement this.  I'm just kicking
the
>> proverbial can down the road.  I may file a feature request PR or
>> write up something in the style of sabre's LLVMNotes:
>> http://nondot.org/sabre/LLVMNotes/
>>
>> Reid
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
>