Lang Hames via llvm-dev
2020-Sep-28 20:56 UTC
[llvm-dev] LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
Hi Geoff, Importing symbols into the JIT from an object file or static library...? Sure! Are you interested in doing this with the C API, LLJIT, or raw OrcV2 components? The high-level answer here (which we can dig into further in the BoF) is: For object files: - For raw OrcV2 components you'll want to create an RTDyldObjectLinkingLayer or ObjectLinkingLayer and use the 'add' method. - For an LLJIT instance you can just call the 'addObjectFile' method. - For the OrcV2 C API you can call the 'LLVMOrcLLJITAddObjectFile' function. For static libraries: - For raw OrcV2 components or an LLJIT instance you can attach a StaticLibraryDefinitionGenerator [1] to the JITDylib that you would like to load the library into. See [2] for an example. - For the OrcV2 C API we will just need to add a method to wrap the operation above. We already have 'LLVMOrcCreateDynamicLibrarySearchGeneratorForProcess'. We should add counterparts to that to load dynamic and static libraries from paths. -- Lang. [1] https://github.com/llvm/llvm-project/blob/5aa56b242951ab0f0181386ea58509f19b09206e/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h#L361 [2] https://github.com/llvm/llvm-project/blob/e7549dafcd33ced4280a81ca1d1ee4cc78ed253f/llvm/tools/lli/lli.cpp#L968 On Mon, Sep 28, 2020 at 7:18 AM Geoff Levner <glevner at gmail.com> wrote:> Importing symbols into the JIT from an object file or static library...? > > On Mon, Sep 28, 2020 at 5:29 AM Lang Hames <lhames at gmail.com> wrote: > >> Hi Andres, >> >> A topic that I'd find interesting is what it'd take to make it easier to >>> use profile guided optimization in the context of JIT. A few years back >>> I crudely hacked this together, but it was fairly ugly. It'd be nice to >>> make that easier. Possibly too complicated for that type of session? >> >> >> We couldn't go into detail, but we could kick off a discussion and >> identify who is interested in working on this. My pitch: We should solve >> this by investing in JITLink and implementing profiling (and debugger >> support) via ObjectLinkingLayer::Plugins. Maybe a brief overview of / >> discussion on how these work would be useful? >> >> -- Lang. >> >> >> On Fri, Sep 25, 2020 at 7:19 PM Andres Freund <andres at anarazel.de> wrote: >> >>> Hi, >>> >>> On 2020-09-25 19:05:42 -0700, Lang Hames wrote: >>> > The 2020 Virtual LLVM Developer's Meeting is coming up the week after >>> next. >>> > I'll be hosting a JIT Birds-of-a-Feather session on Thursday the 8th at >>> > 10:55am PDT (See http://llvm.org/devmtg/2020-09/schedule/). I'm >>> planning to >>> > run this more like a Round Table: Minimal introduction, plenty of time >>> for >>> > discussion. >>> > >>> > Does anyone have any LLVM JIT related topics that they would >>> particularly >>> > like to discuss at this session? Any questions that they think could be >>> > best answered by a round-table style discussion? I'll aim to rope in >>> the >>> > right people and prepare answers to make sure we get the most out of >>> the >>> > session. >>> >>> A topic that I'd find interesting is what it'd take to make it easier to >>> use profile guided optimization in the context of JIT. A few years back >>> I crudely hacked this together, but it was fairly ugly. It'd be nice to >>> make that easier. Possibly too complicated for that type of session? >>> >>> I dimly recall that the main issue was that there is no good way to >>> extract the profile information without going through files / applying >>> the profile data without going through files. >>> >>> Greetings, >>> >>> Andres Freund >>> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200928/00825801/attachment.html>
Geoff Levner via llvm-dev
2020-Sep-28 21:05 UTC
[llvm-dev] LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator work for ELF objects? On Mon, Sep 28, 2020 at 10:57 PM Lang Hames <lhames at gmail.com> wrote:> > Hi Geoff, > >> Importing symbols into the JIT from an object file or static library...? > > > Sure! Are you interested in doing this with the C API, LLJIT, or raw OrcV2 components? > > The high-level answer here (which we can dig into further in the BoF) is: > > For object files: > - For raw OrcV2 components you'll want to create an RTDyldObjectLinkingLayer or ObjectLinkingLayer and use the 'add' method. > - For an LLJIT instance you can just call the 'addObjectFile' method. > - For the OrcV2 C API you can call the 'LLVMOrcLLJITAddObjectFile' function. > > For static libraries: > - For raw OrcV2 components or an LLJIT instance you can attach a StaticLibraryDefinitionGenerator [1] to the JITDylib that you would like to load the library into. See [2] for an example. > - For the OrcV2 C API we will just need to add a method to wrap the operation above. We already have 'LLVMOrcCreateDynamicLibrarySearchGeneratorForProcess'. We should add counterparts to that to load dynamic and static libraries from paths. > > -- Lang. > > [1] https://github.com/llvm/llvm-project/blob/5aa56b242951ab0f0181386ea58509f19b09206e/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h#L361 > [2] https://github.com/llvm/llvm-project/blob/e7549dafcd33ced4280a81ca1d1ee4cc78ed253f/llvm/tools/lli/lli.cpp#L968 > > > On Mon, Sep 28, 2020 at 7:18 AM Geoff Levner <glevner at gmail.com> wrote: >> >> Importing symbols into the JIT from an object file or static library...? >> >> On Mon, Sep 28, 2020 at 5:29 AM Lang Hames <lhames at gmail.com> wrote: >>> >>> Hi Andres, >>> >>>> A topic that I'd find interesting is what it'd take to make it easier to >>>> use profile guided optimization in the context of JIT. A few years back >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice to >>>> make that easier. Possibly too complicated for that type of session? >>> >>> >>> We couldn't go into detail, but we could kick off a discussion and identify who is interested in working on this. My pitch: We should solve this by investing in JITLink and implementing profiling (and debugger support) via ObjectLinkingLayer::Plugins. Maybe a brief overview of / discussion on how these work would be useful? >>> >>> -- Lang. >>> >>> >>> On Fri, Sep 25, 2020 at 7:19 PM Andres Freund <andres at anarazel.de> wrote: >>>> >>>> Hi, >>>> >>>> On 2020-09-25 19:05:42 -0700, Lang Hames wrote: >>>> > The 2020 Virtual LLVM Developer's Meeting is coming up the week after next. >>>> > I'll be hosting a JIT Birds-of-a-Feather session on Thursday the 8th at >>>> > 10:55am PDT (See http://llvm.org/devmtg/2020-09/schedule/). I'm planning to >>>> > run this more like a Round Table: Minimal introduction, plenty of time for >>>> > discussion. >>>> > >>>> > Does anyone have any LLVM JIT related topics that they would particularly >>>> > like to discuss at this session? Any questions that they think could be >>>> > best answered by a round-table style discussion? I'll aim to rope in the >>>> > right people and prepare answers to make sure we get the most out of the >>>> > session. >>>> >>>> A topic that I'd find interesting is what it'd take to make it easier to >>>> use profile guided optimization in the context of JIT. A few years back >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice to >>>> make that easier. Possibly too complicated for that type of session? >>>> >>>> I dimly recall that the main issue was that there is no good way to >>>> extract the profile information without going through files / applying >>>> the profile data without going through files. >>>> >>>> Greetings, >>>> >>>> Andres Freund
Lang Hames via llvm-dev
2020-Sep-28 21:58 UTC
[llvm-dev] LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
Hi Geoff, We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator work> for ELF objects?They do. :) I've not tested StaticLibraryDefinitionGenerator extensively on Linux. but we have a regression test checking basic usage. If you run into any trouble at all please file a bug and assign it to me. -- Lang. On Mon, Sep 28, 2020 at 2:05 PM Geoff Levner <glevner at gmail.com> wrote:> We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator > work for ELF objects? > > On Mon, Sep 28, 2020 at 10:57 PM Lang Hames <lhames at gmail.com> wrote: > > > > Hi Geoff, > > > >> Importing symbols into the JIT from an object file or static library...? > > > > > > Sure! Are you interested in doing this with the C API, LLJIT, or raw > OrcV2 components? > > > > The high-level answer here (which we can dig into further in the BoF) is: > > > > For object files: > > - For raw OrcV2 components you'll want to create an > RTDyldObjectLinkingLayer or ObjectLinkingLayer and use the 'add' method. > > - For an LLJIT instance you can just call the 'addObjectFile' method. > > - For the OrcV2 C API you can call the 'LLVMOrcLLJITAddObjectFile' > function. > > > > For static libraries: > > - For raw OrcV2 components or an LLJIT instance you can attach a > StaticLibraryDefinitionGenerator [1] to the JITDylib that you would like to > load the library into. See [2] for an example. > > - For the OrcV2 C API we will just need to add a method to wrap the > operation above. We already have > 'LLVMOrcCreateDynamicLibrarySearchGeneratorForProcess'. We should add > counterparts to that to load dynamic and static libraries from paths. > > > > -- Lang. > > > > [1] > https://github.com/llvm/llvm-project/blob/5aa56b242951ab0f0181386ea58509f19b09206e/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h#L361 > > [2] > https://github.com/llvm/llvm-project/blob/e7549dafcd33ced4280a81ca1d1ee4cc78ed253f/llvm/tools/lli/lli.cpp#L968 > > > > > > On Mon, Sep 28, 2020 at 7:18 AM Geoff Levner <glevner at gmail.com> wrote: > >> > >> Importing symbols into the JIT from an object file or static library...? > >> > >> On Mon, Sep 28, 2020 at 5:29 AM Lang Hames <lhames at gmail.com> wrote: > >>> > >>> Hi Andres, > >>> > >>>> A topic that I'd find interesting is what it'd take to make it easier > to > >>>> use profile guided optimization in the context of JIT. A few years > back > >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice > to > >>>> make that easier. Possibly too complicated for that type of session? > >>> > >>> > >>> We couldn't go into detail, but we could kick off a discussion and > identify who is interested in working on this. My pitch: We should solve > this by investing in JITLink and implementing profiling (and debugger > support) via ObjectLinkingLayer::Plugins. Maybe a brief overview of / > discussion on how these work would be useful? > >>> > >>> -- Lang. > >>> > >>> > >>> On Fri, Sep 25, 2020 at 7:19 PM Andres Freund <andres at anarazel.de> > wrote: > >>>> > >>>> Hi, > >>>> > >>>> On 2020-09-25 19:05:42 -0700, Lang Hames wrote: > >>>> > The 2020 Virtual LLVM Developer's Meeting is coming up the week > after next. > >>>> > I'll be hosting a JIT Birds-of-a-Feather session on Thursday the > 8th at > >>>> > 10:55am PDT (See http://llvm.org/devmtg/2020-09/schedule/). I'm > planning to > >>>> > run this more like a Round Table: Minimal introduction, plenty of > time for > >>>> > discussion. > >>>> > > >>>> > Does anyone have any LLVM JIT related topics that they would > particularly > >>>> > like to discuss at this session? Any questions that they think > could be > >>>> > best answered by a round-table style discussion? I'll aim to rope > in the > >>>> > right people and prepare answers to make sure we get the most out > of the > >>>> > session. > >>>> > >>>> A topic that I'd find interesting is what it'd take to make it easier > to > >>>> use profile guided optimization in the context of JIT. A few years > back > >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice > to > >>>> make that easier. Possibly too complicated for that type of session? > >>>> > >>>> I dimly recall that the main issue was that there is no good way to > >>>> extract the profile information without going through files / applying > >>>> the profile data without going through files. > >>>> > >>>> Greetings, > >>>> > >>>> Andres Freund >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200928/b101cba6/attachment.html>
Maybe Matching Threads
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
- LLJIT vs. thread-local storage
- LLJIT vs. thread-local storage