llvm dev - Jun 2011 - [LLVMdev] Understanding resolving external/built-in function references

Hi James,
> It depends on how your bitcode is loaded. It can be loaded either by llc,
> which outputs pure ELF/Mach-O, in which case those function references get
> turned into relocations that the linker (ld, Gold) applies.
> 
> If you're loading bitcode via the JIT, it uses dlsym() to try and
introspect
> your runtime environment to find the symbol.
Ah, that makes sense - thanks.

> The simple answer is that LLVM bitcode cannot be "loaded and run"
as-is - it
> needs to be converted to machine code and relocations are processed at this
> time.
I suspect the way I'm hoping to use LLVM may be slightly
uncommon:

I'd like to use the LLVM C++ API to generate modules and
functions on-the-fly and JIT-execute them, and then I'd
like to add additional functions and modules in a piecemeal
fashion, to the growing system.  Similar in this way to a
Forth or Smalltalk image-based system where one can add,
remove, or recompile function definitions on the fly, in
the context of the whole image in memory.

My impression so far from the LLVM C++ API is that this
would appear to be possible - at least up to a point.

One thing I'm unclear on is whether it's possible to add
a new function and/or module to the system, which can
reference functions in previously defined modules, without
needing to re-generate the bitcode for the whole system.

Is it possible to keep adding bitcode incrementally to the
system, and JIT'ing it, and have the set of functions
defined in the system keep growing, so that the next hunk
of bitcode added can have access to all the functions
defined so far?


Thanks,

Bill

Hi,
> Is it possible to keep adding bitcode incrementally to the
> system, and JIT'ing it, and have the set of functions
> defined in the system keep growing, so that the next hunk
> of bitcode added can have access to all the functions
> defined so far?
Yes. The JIT supports lazy compilation mode where it will generate thunks that
only get compiled/linked when called - see JIT::getPointerToFunctionOrStub *.

The above would solve a bijective requirement that earlier code may rely on
functions defined in later code - if that is not necessary then the JIT should
just work out of the box.

* Note that the lazy compilation mode currently does not work well in a
multithreaded environment.

Cheers,

James
> -----Original Message-----
> From: Bill Kelly [mailto:billk at cts.com]
> Sent: 06 June 2011 13:36
> To: llvmdev at cs.uiuc.edu
> Cc: James Molloy
> Subject: Re: [LLVMdev] Understanding resolving external/built-in
> function references
>
> Hi James,
>
> > It depends on how your bitcode is loaded. It can be loaded either by
> llc,
> > which outputs pure ELF/Mach-O, in which case those function
> references get
> > turned into relocations that the linker (ld, Gold) applies.
> >
> > If you're loading bitcode via the JIT, it uses dlsym() to try and
> introspect
> > your runtime environment to find the symbol.
>
> Ah, that makes sense - thanks.
>
>
> > The simple answer is that LLVM bitcode cannot be "loaded and
run" as-
> is - it
> > needs to be converted to machine code and relocations are processed
> at this
> > time.
> I>  suspect the way I'm hoping to use LLVM may be slightly
> uncommon:
>
> I'd like to use the LLVM C++ API to generate modules and
> functions on-the-fly and JIT-execute them, and then I'd
> like to add additional functions and modules in a piecemeal
> fashion, to the growing system.  Similar in this way to a
> Forth or Smalltalk image-based system where one can add,
> remove, or recompile function definitions on the fly, in
> the context of the whole image in memory.
>
> My impression so far from the LLVM C++ API is that this
> would appear to be possible - at least up to a point.
>
> One thing I'm unclear on is whether it's possible to add
> a new function and/or module to the system, which can
> reference functions in previously defined modules, without
> needing to re-generate the bitcode for the whole system.
>
>
>
> Thanks,
>
> Bill
>

-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium.  Thank you.

James,

James Molloy wrote:
> The JIT supports lazy compilation mode where it will generate thunks
> that only get compiled/linked when called - see
> JIT::getPointerToFunctionOrStub *.
> The above would solve a bijective requirement that earlier code may
> rely on functions defined in later code - if that is not necessary
> then the JIT should just work out of the box.
> * Note that the lazy compilation mode currently does not work well in
> a multithreaded environment.
Thanks!

Sounds promising - lots to learn...


Regards,

Bill