llvm dev - Jun 2019 - Questions about moving from MCJIT to Orc JIT

Hi Bjoern,
CC'ing Lang hames

For questions,
1. In short yes, you can replace the memory manager, default one provided
is section memory manager.

2. If you mean by " address of already compiled code", yes you can do
that.
 Like this
 JITDylib.define(absoluteSymbols, ( Your_own_symbol ,
JITTargetAddress(Address of function))), now ORC can resolve all the
references to Your_own_symbol to the provided the Address.

3. Yes, all the class/function with legacy prefix will be removed in
subsequent releases.

4. Could please provide some more context on this? Like it's usage model,
what goal you are aiming for?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/27748ecf/attachment.html>

Hey Praveen,

Thanks for your help! I still have questions and try providing some context.

For questions:
2.) I exactly mean that! Sadly I was not able to understand how to use the
‘define’ function, I tried that:
[cid:image001.png at 01D52CCF.B4668E10]
I had no idea how to deal with that >o<

2.1.) With MCJIT I was actually used to getting the name of a function and then
‘answering’ with an address. Is that behavior not possible with OrcJIT anymore?
I actually tried looking into the ‘setGenerator’ function of the JITDylib. As a
test I came up with this:
[cid:image002.png at 01D52CD0.5B8D6330]
This was pretty cool, because I actually saw the missing function ‘puts’ but I
have no idea how to reply with an  address for ‘puts’. Returning ‘Names’
actually makes the application pause and not go on.

4.) I used the code of Kaleidoscope JIT for my little experiments. If I
understood it correctly, the JIT process is completed after I used the lookup
function to find a symbol in the code. Now I would like to reset the instance of
the Kaleidoscope JIT I had, to jit new modules, which are not related to the
previous modules I jitted.
Kind greetings
Björn

From: Praveen Velliengiri <praveenvelliengiri at gmail.com>
Sent: Donnerstag, 27. Juni 2019 08:21
To: Gaier, Bjoern <Bjoern.Gaier at horiba.com>; Lang Hames <lhames at
gmail.com>
Cc: llvm-dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Questions about moving from MCJIT to Orc JIT

Hi Bjoern,
CC'ing Lang hames

For questions,
1. In short yes, you can replace the memory manager, default one provided is
section memory manager.

2. If you mean by " address of already compiled code", yes you can do
that.
 Like this
 JITDylib.define(absoluteSymbols, ( Your_own_symbol , JITTargetAddress(Address
of function))), now ORC can resolve all the references to Your_own_symbol to the
provided the Address.

3. Yes, all the class/function with legacy prefix will be removed in subsequent
releases.

4. Could please provide some more context on this? Like it's usage model,
what goal you are aiming for?

Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/57c0d9ee/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 8510 bytes
Desc: image001.png
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/57c0d9ee/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 8298 bytes
Desc: image002.png
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/57c0d9ee/attachment-0001.png>

Nice!
Let me try to answer some questions,
Before that I have to mention this is ORC version 2 APIs and this is where
the project is moving forward.
JITDylib is the symbol table, basically for a JIT Symbol it have an
associated materializers, (you can think of it like an entity that generate
the address for that symbol),
Example: compiler are materializers.
So to add symbols to your own JIT you can define them in JITDylib.

For example, Suppose you want to add "your host process (on which JIT runs)
symbols to the JIT".
You can do
JD.setGenerator(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL.getGlobalPrefix()))


Under hood, ORC normally search for your host symbols in the defGenerator
if it can't find symbols anywhere else.
You can look at core.cpp at line 1163 to see how it works!

I highly recommended you to watch "Updating ORC for Concurrency" dev
meeting talks, it is explained in detail,,
For question 4, I don't know maybe If I know after sometime I will write
here :)

Thanks

On Thu, 27 Jun 2019 at 14:00, Gaier, Bjoern <Bjoern.Gaier at horiba.com>
wrote:
> Hey Praveen,
>
>
>
> Thanks for your help! I still have questions and try providing some
> context.
>
>
>
> For questions:
>
> 2.) I exactly mean that! Sadly I was not able to understand how to use the
> ‘define’ function, I tried that:
>
> I had no idea how to deal with that >o<
>
>
>
> 2.1.) With MCJIT I was actually used to getting the name of a function and
> then ‘answering’ with an address. Is that behavior not possible with OrcJIT
> anymore? I actually tried looking into the ‘setGenerator’ function of the
> JITDylib. As a test I came up with this:
>
> This was pretty cool, because I actually saw the missing function ‘puts’
> but I have no idea how to reply with an  address for ‘puts’. Returning
> ‘Names’ actually makes the application pause and not go on.
>
>
>
> 4.) I used the code of Kaleidoscope JIT for my little experiments. If I
> understood it correctly, the JIT process is completed after I used the
> lookup function to find a symbol in the code. Now I would like to reset the
> instance of the Kaleidoscope JIT I had, to jit new modules, which are not
> related to the previous modules I jitted.
>
> Kind greetings
>
> Björn
>
>
>
> *From:* Praveen Velliengiri <praveenvelliengiri at gmail.com>
> *Sent:* Donnerstag, 27. Juni 2019 08:21
> *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com>; Lang Hames <
> lhames at gmail.com>
> *Cc:* llvm-dev <llvm-dev at lists.llvm.org>
> *Subject:* Re: [llvm-dev] Questions about moving from MCJIT to Orc JIT
>
>
>
> Hi Bjoern,
>
> CC'ing Lang hames
>
>
>
> For questions,
>
> 1. In short yes, you can replace the memory manager, default one provided
> is section memory manager.
>
>
>
> 2. If you mean by " address of already compiled code", yes you
can do
> that.
>
>  Like this
>
>  JITDylib.define(absoluteSymbols, ( Your_own_symbol ,
> JITTargetAddress(Address of function))), now ORC can resolve all the
> references to Your_own_symbol to the provided the Address.
>
>
>
> 3. Yes, all the class/function with legacy prefix will be removed in
> subsequent releases.
>
>
>
> 4. Could please provide some more context on this? Like it's usage
model,
> what goal you are aiming for?
>
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima.
> Junichi Tajika
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/5c50509d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 8510 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/5c50509d/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 8298 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/5c50509d/attachment-0003.png>