Praveen Velliengiri via llvm-dev
2019-Jun-27 08:55 UTC
[llvm-dev] Questions about moving from MCJIT to Orc JIT
Nice! Let me try to answer some questions, Before that I have to mention this is ORC version 2 APIs and this is where the project is moving forward. JITDylib is the symbol table, basically for a JIT Symbol it have an associated materializers, (you can think of it like an entity that generate the address for that symbol), Example: compiler are materializers. So to add symbols to your own JIT you can define them in JITDylib. For example, Suppose you want to add "your host process (on which JIT runs) symbols to the JIT". You can do JD.setGenerator(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL.getGlobalPrefix())) Under hood, ORC normally search for your host symbols in the defGenerator if it can't find symbols anywhere else. You can look at core.cpp at line 1163 to see how it works! I highly recommended you to watch "Updating ORC for Concurrency" dev meeting talks, it is explained in detail,, For question 4, I don't know maybe If I know after sometime I will write here :) Thanks On Thu, 27 Jun 2019 at 14:00, Gaier, Bjoern <Bjoern.Gaier at horiba.com> wrote:> Hey Praveen, > > > > Thanks for your help! I still have questions and try providing some > context. > > > > For questions: > > 2.) I exactly mean that! Sadly I was not able to understand how to use the > ‘define’ function, I tried that: > > I had no idea how to deal with that >o< > > > > 2.1.) With MCJIT I was actually used to getting the name of a function and > then ‘answering’ with an address. Is that behavior not possible with OrcJIT > anymore? I actually tried looking into the ‘setGenerator’ function of the > JITDylib. As a test I came up with this: > > This was pretty cool, because I actually saw the missing function ‘puts’ > but I have no idea how to reply with an address for ‘puts’. Returning > ‘Names’ actually makes the application pause and not go on. > > > > 4.) I used the code of Kaleidoscope JIT for my little experiments. If I > understood it correctly, the JIT process is completed after I used the > lookup function to find a symbol in the code. Now I would like to reset the > instance of the Kaleidoscope JIT I had, to jit new modules, which are not > related to the previous modules I jitted. > > Kind greetings > > Björn > > > > *From:* Praveen Velliengiri <praveenvelliengiri at gmail.com> > *Sent:* Donnerstag, 27. Juni 2019 08:21 > *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com>; Lang Hames < > lhames at gmail.com> > *Cc:* llvm-dev <llvm-dev at lists.llvm.org> > *Subject:* Re: [llvm-dev] Questions about moving from MCJIT to Orc JIT > > > > Hi Bjoern, > > CC'ing Lang hames > > > > For questions, > > 1. In short yes, you can replace the memory manager, default one provided > is section memory manager. > > > > 2. If you mean by " address of already compiled code", yes you can do > that. > > Like this > > JITDylib.define(absoluteSymbols, ( Your_own_symbol , > JITTargetAddress(Address of function))), now ORC can resolve all the > references to Your_own_symbol to the provided the Address. > > > > 3. Yes, all the class/function with legacy prefix will be removed in > subsequent releases. > > > > 4. Could please provide some more context on this? Like it's usage model, > what goal you are aiming for? > > > Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, > USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert > Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. > Junichi Tajika >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/5c50509d/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 8510 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/5c50509d/attachment-0002.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 8298 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/5c50509d/attachment-0003.png>
Gaier, Bjoern via llvm-dev
2019-Jun-27 09:47 UTC
[llvm-dev] Questions about moving from MCJIT to Orc JIT
Thanks for the input! I was able to write my own Generator function for the JITDylib! Thank you a lot x3 From: Praveen Velliengiri <praveenvelliengiri at gmail.com> Sent: Donnerstag, 27. Juni 2019 10:55 To: Gaier, Bjoern <Bjoern.Gaier at horiba.com> Cc: Lang Hames <lhames at gmail.com>; llvm-dev <llvm-dev at lists.llvm.org> Subject: Re: [llvm-dev] Questions about moving from MCJIT to Orc JIT Nice! Let me try to answer some questions, Before that I have to mention this is ORC version 2 APIs and this is where the project is moving forward. JITDylib is the symbol table, basically for a JIT Symbol it have an associated materializers, (you can think of it like an entity that generate the address for that symbol), Example: compiler are materializers. So to add symbols to your own JIT you can define them in JITDylib. For example, Suppose you want to add "your host process (on which JIT runs) symbols to the JIT". You can do JD.setGenerator(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL.getGlobalPrefix())) Under hood, ORC normally search for your host symbols in the defGenerator if it can't find symbols anywhere else. You can look at core.cpp at line 1163 to see how it works! I highly recommended you to watch "Updating ORC for Concurrency" dev meeting talks, it is explained in detail,, For question 4, I don't know maybe If I know after sometime I will write here :) Thanks On Thu, 27 Jun 2019 at 14:00, Gaier, Bjoern <Bjoern.Gaier at horiba.com<mailto:Bjoern.Gaier at horiba.com>> wrote: Hey Praveen, Thanks for your help! I still have questions and try providing some context. For questions: 2.) I exactly mean that! Sadly I was not able to understand how to use the ‘define’ function, I tried that: [cid:image001.png at 01D52CDE.1C37ABB0] I had no idea how to deal with that >o< 2.1.) With MCJIT I was actually used to getting the name of a function and then ‘answering’ with an address. Is that behavior not possible with OrcJIT anymore? I actually tried looking into the ‘setGenerator’ function of the JITDylib. As a test I came up with this: [cid:image002.png at 01D52CDE.1C37ABB0] This was pretty cool, because I actually saw the missing function ‘puts’ but I have no idea how to reply with an address for ‘puts’. Returning ‘Names’ actually makes the application pause and not go on. 4.) I used the code of Kaleidoscope JIT for my little experiments. If I understood it correctly, the JIT process is completed after I used the lookup function to find a symbol in the code. Now I would like to reset the instance of the Kaleidoscope JIT I had, to jit new modules, which are not related to the previous modules I jitted. Kind greetings Björn From: Praveen Velliengiri <praveenvelliengiri at gmail.com<mailto:praveenvelliengiri at gmail.com>> Sent: Donnerstag, 27. Juni 2019 08:21 To: Gaier, Bjoern <Bjoern.Gaier at horiba.com<mailto:Bjoern.Gaier at horiba.com>>; Lang Hames <lhames at gmail.com<mailto:lhames at gmail.com>> Cc: llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> Subject: Re: [llvm-dev] Questions about moving from MCJIT to Orc JIT Hi Bjoern, CC'ing Lang hames For questions, 1. In short yes, you can replace the memory manager, default one provided is section memory manager. 2. If you mean by " address of already compiled code", yes you can do that. Like this JITDylib.define(absoluteSymbols, ( Your_own_symbol , JITTargetAddress(Address of function))), now ORC can resolve all the references to Your_own_symbol to the provided the Address. 3. Yes, all the class/function with legacy prefix will be removed in subsequent releases. 4. Could please provide some more context on this? Like it's usage model, what goal you are aiming for? Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/14610d04/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 8510 bytes Desc: image001.png URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/14610d04/attachment.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 8298 bytes Desc: image002.png URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/14610d04/attachment-0001.png>
Praveen Velliengiri via llvm-dev
2019-Jun-27 17:46 UTC
[llvm-dev] Questions about moving from MCJIT to Orc JIT
Awesome! On Thu, 27 Jun 2019 at 15:17, Gaier, Bjoern <Bjoern.Gaier at horiba.com> wrote:> Thanks for the input! I was able to write my own Generator function for > the JITDylib! > Thank you a lot x3 > > > > *From:* Praveen Velliengiri <praveenvelliengiri at gmail.com> > *Sent:* Donnerstag, 27. Juni 2019 10:55 > *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com> > *Cc:* Lang Hames <lhames at gmail.com>; llvm-dev <llvm-dev at lists.llvm.org> > *Subject:* Re: [llvm-dev] Questions about moving from MCJIT to Orc JIT > > > > Nice! > > Let me try to answer some questions, > > Before that I have to mention this is ORC version 2 APIs and this is where > the project is moving forward. > > JITDylib is the symbol table, basically for a JIT Symbol it have an > associated materializers, (you can think of it like an entity that generate > the address for that symbol), > > Example: compiler are materializers. > > So to add symbols to your own JIT you can define them in JITDylib. > > > > For example, Suppose you want to add "your host process (on which JIT > runs) symbols to the JIT". > > You can do > > JD.setGenerator(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL.getGlobalPrefix())) > > > > > Under hood, ORC normally search for your host symbols in the defGenerator > if it can't find symbols anywhere else. > > You can look at core.cpp at line 1163 to see how it works! > > > > I highly recommended you to watch "Updating ORC for Concurrency" dev > meeting talks, it is explained in detail,, > > For question 4, I don't know maybe If I know after sometime I will write > here :) > > > > Thanks > > > > On Thu, 27 Jun 2019 at 14:00, Gaier, Bjoern <Bjoern.Gaier at horiba.com> > wrote: > > Hey Praveen, > > > > Thanks for your help! I still have questions and try providing some > context. > > > > For questions: > > 2.) I exactly mean that! Sadly I was not able to understand how to use the > ‘define’ function, I tried that: > > I had no idea how to deal with that >o< > > > > 2.1.) With MCJIT I was actually used to getting the name of a function and > then ‘answering’ with an address. Is that behavior not possible with OrcJIT > anymore? I actually tried looking into the ‘setGenerator’ function of the > JITDylib. As a test I came up with this: > > This was pretty cool, because I actually saw the missing function ‘puts’ > but I have no idea how to reply with an address for ‘puts’. Returning > ‘Names’ actually makes the application pause and not go on. > > > > 4.) I used the code of Kaleidoscope JIT for my little experiments. If I > understood it correctly, the JIT process is completed after I used the > lookup function to find a symbol in the code. Now I would like to reset the > instance of the Kaleidoscope JIT I had, to jit new modules, which are not > related to the previous modules I jitted. > > Kind greetings > > Björn > > > > *From:* Praveen Velliengiri <praveenvelliengiri at gmail.com> > *Sent:* Donnerstag, 27. Juni 2019 08:21 > *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com>; Lang Hames < > lhames at gmail.com> > *Cc:* llvm-dev <llvm-dev at lists.llvm.org> > *Subject:* Re: [llvm-dev] Questions about moving from MCJIT to Orc JIT > > > > Hi Bjoern, > > CC'ing Lang hames > > > > For questions, > > 1. In short yes, you can replace the memory manager, default one provided > is section memory manager. > > > > 2. If you mean by " address of already compiled code", yes you can do > that. > > Like this > > JITDylib.define(absoluteSymbols, ( Your_own_symbol , > JITTargetAddress(Address of function))), now ORC can resolve all the > references to Your_own_symbol to the provided the Address. > > > > 3. Yes, all the class/function with legacy prefix will be removed in > subsequent releases. > > > > 4. Could please provide some more context on this? Like it's usage model, > what goal you are aiming for? > > > > Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, > USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert > Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. > Junichi Tajika > > Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, > USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert > Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. > Junichi Tajika >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/bcadbf7c/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 8510 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/bcadbf7c/attachment.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 8298 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190627/bcadbf7c/attachment-0001.png>