llvm dev - Nov 2018 - ORC JIT api, object files and stackmaps

I have a few questions about the new ORC JIT.

I saw Lang Hames (hi!) excellent talk at the llvm-dev meeting a few weeks
ago. The ORC JIT is undergoing some API changes and I'd like/need to take
advantage of them.

(1) How do I take ownership of the ObjectFile once the ORC JIT has created
it?
    I'd like to take ownership of object files generated by the ORC JIT so
that I can save them to disk and in later sessions reload them.
(2) How would I pass an ObjectFile saved in question#1 back to ORC so that
it will relocate it and generate function pointers?
(3) How do I get access to the relocated ObjectFile sections?
    Currently I subclass SectionMemoryManager and implement
allocateDataSection(...)
    I can get the memory for the "__llvm_stackmaps" section - but I
don't
know when/if the contents have been fully set up with relocated function
pointers.
(4) For the "__llvm_stackmaps" section - will I need to do any
relocation
to obtain the function pointers?

Background:
I'm using llvm.experimental.stackmaps to register one variable in each
stack frame that contains spilled register arguments.
I've figured out how to get access to the stackmaps for code that I load
into my system from dynamic libraries that our compiler generates.
The answers to questions above will help me get access to the stackmaps
from ORC JITted code.

-- 
Christian Schafmeister
Professor, Chemistry Department
Temple University
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181105/9ccc2c04/attachment.html>

I think I found the answer to #3 and #4.
(a) I overloaded the SectionMemoryManager::finalizeMemory(...) method.
(b) I first call the base classes method (maybe not necessary).
(c) At this point the __llvm_stackmaps section that I saved in thread local
memory when allocateDataSection was called appears to be fully set up and
relocated.


On Mon, Nov 5, 2018 at 12:36 PM Christian Schafmeister <meister at
temple.edu>
wrote:
> I have a few questions about the new ORC JIT.
>
> I saw Lang Hames (hi!) excellent talk at the llvm-dev meeting a few weeks
> ago. The ORC JIT is undergoing some API changes and I'd like/need to
take
> advantage of them.
>
> (1) How do I take ownership of the ObjectFile once the ORC JIT has created
> it?
>     I'd like to take ownership of object files generated by the ORC JIT
so
> that I can save them to disk and in later sessions reload them.
> (2) How would I pass an ObjectFile saved in question#1 back to ORC so that
> it will relocate it and generate function pointers?
> (3) How do I get access to the relocated ObjectFile sections?
>     Currently I subclass SectionMemoryManager and implement
> allocateDataSection(...)
>     I can get the memory for the "__llvm_stackmaps" section - but
I don't
> know when/if the contents have been fully set up with relocated function
> pointers.
> (4) For the "__llvm_stackmaps" section - will I need to do any
relocation
> to obtain the function pointers?
>
> Background:
> I'm using llvm.experimental.stackmaps to register one variable in each
> stack frame that contains spilled register arguments.
> I've figured out how to get access to the stackmaps for code that I
load
> into my system from dynamic libraries that our compiler generates.
> The answers to questions above will help me get access to the stackmaps
> from ORC JITted code.
>
> --
> Christian Schafmeister
> Professor, Chemistry Department
> Temple University
>

-- 
Christian Schafmeister
Professor, Chemistry Department
Temple University
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181105/007f0b26/attachment.html>

Hi Christian

Your use case seems to have similar requirements as remote JITing in
ORC. So far I haven't used that part myself and I am sure Lang can tell
you much more about it. However, this comment on the
RemoteObjectClientLayer class sounds promising for your questions (1)
and (2):

/// Sending relocatable objects to the server (rather than fully relocated
/// bits) allows JIT'd code to be cached on the server side and re-used in
/// subsequent JIT sessions.

There are a few tests here that illustrate its usage:
https://github.com/llvm-mirror/llvm/blob/master/unittests/ExecutionEngine/Orc/RemoteObjectLayerTest.cpp

Note that it still uses LegacyJITSymbolResolver, so I guess it may
change any time soon.

Hope it helps for the time being.
Cheers, Stefan

Am 05.11.18 um 19:44 schrieb Christian Schafmeister via
llvm-dev:
> I think I found the answer to #3 and #4.
> (a) I overloaded the SectionMemoryManager::finalizeMemory(...) method. 
> (b) I first call the base classes method (maybe not necessary).
> (c) At this point the __llvm_stackmaps section that I saved in thread
> local memory when allocateDataSection was called appears to be fully
> set up and relocated.
>
>
> On Mon, Nov 5, 2018 at 12:36 PM Christian Schafmeister
> <meister at temple.edu <mailto:meister at temple.edu>> wrote:
>
>     I have a few questions about the new ORC JIT.
>
>     I saw Lang Hames (hi!) excellent talk at the llvm-dev meeting a
>     few weeks ago. The ORC JIT is undergoing some API changes and I'd
>     like/need to take advantage of them.
>
>     (1) How do I take ownership of the ObjectFile once the ORC JIT has
>     created it?
>         I'd like to take ownership of object files generated by the
>     ORC JIT so that I can save them to disk and in later sessions
>     reload them.
>     (2) How would I pass an ObjectFile saved in question#1 back to ORC
>     so that it will relocate it and generate function pointers?
>     (3) How do I get access to the relocated ObjectFile sections?
>         Currently I subclass SectionMemoryManager and implement
>     allocateDataSection(...)
>         I can get the memory for the "__llvm_stackmaps" section -
but
>     I don't know when/if the contents have been fully set up with
>     relocated function pointers.
>     (4) For the "__llvm_stackmaps" section - will I need to do
any
>     relocation to obtain the function pointers?
>
>     Background:
>     I'm using llvm.experimental.stackmaps to register one variable in
>     each stack frame that contains spilled register arguments.
>     I've figured out how to get access to the stackmaps for code that
>     I load into my system from dynamic libraries that our compiler
>     generates.
>     The answers to questions above will help me get access to the
>     stackmaps from ORC JITted code.
>
>     -- 
>     Christian Schafmeister
>     Professor, Chemistry Department
>     Temple University
>
>
>
> -- 
> Christian Schafmeister
> Professor, Chemistry Department
> Temple University
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 
https://weliveindetail.github.io/blog/
https://cryptup.org/pub/stefan.graenitz at gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181105/0d95628a/attachment.html>