Lang Hames via llvm-dev
2020-Sep-28 21:58 UTC
[llvm-dev] LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
Hi Geoff, We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator work> for ELF objects?They do. :) I've not tested StaticLibraryDefinitionGenerator extensively on Linux. but we have a regression test checking basic usage. If you run into any trouble at all please file a bug and assign it to me. -- Lang. On Mon, Sep 28, 2020 at 2:05 PM Geoff Levner <glevner at gmail.com> wrote:> We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator > work for ELF objects? > > On Mon, Sep 28, 2020 at 10:57 PM Lang Hames <lhames at gmail.com> wrote: > > > > Hi Geoff, > > > >> Importing symbols into the JIT from an object file or static library...? > > > > > > Sure! Are you interested in doing this with the C API, LLJIT, or raw > OrcV2 components? > > > > The high-level answer here (which we can dig into further in the BoF) is: > > > > For object files: > > - For raw OrcV2 components you'll want to create an > RTDyldObjectLinkingLayer or ObjectLinkingLayer and use the 'add' method. > > - For an LLJIT instance you can just call the 'addObjectFile' method. > > - For the OrcV2 C API you can call the 'LLVMOrcLLJITAddObjectFile' > function. > > > > For static libraries: > > - For raw OrcV2 components or an LLJIT instance you can attach a > StaticLibraryDefinitionGenerator [1] to the JITDylib that you would like to > load the library into. See [2] for an example. > > - For the OrcV2 C API we will just need to add a method to wrap the > operation above. We already have > 'LLVMOrcCreateDynamicLibrarySearchGeneratorForProcess'. We should add > counterparts to that to load dynamic and static libraries from paths. > > > > -- Lang. > > > > [1] > https://github.com/llvm/llvm-project/blob/5aa56b242951ab0f0181386ea58509f19b09206e/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h#L361 > > [2] > https://github.com/llvm/llvm-project/blob/e7549dafcd33ced4280a81ca1d1ee4cc78ed253f/llvm/tools/lli/lli.cpp#L968 > > > > > > On Mon, Sep 28, 2020 at 7:18 AM Geoff Levner <glevner at gmail.com> wrote: > >> > >> Importing symbols into the JIT from an object file or static library...? > >> > >> On Mon, Sep 28, 2020 at 5:29 AM Lang Hames <lhames at gmail.com> wrote: > >>> > >>> Hi Andres, > >>> > >>>> A topic that I'd find interesting is what it'd take to make it easier > to > >>>> use profile guided optimization in the context of JIT. A few years > back > >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice > to > >>>> make that easier. Possibly too complicated for that type of session? > >>> > >>> > >>> We couldn't go into detail, but we could kick off a discussion and > identify who is interested in working on this. My pitch: We should solve > this by investing in JITLink and implementing profiling (and debugger > support) via ObjectLinkingLayer::Plugins. Maybe a brief overview of / > discussion on how these work would be useful? > >>> > >>> -- Lang. > >>> > >>> > >>> On Fri, Sep 25, 2020 at 7:19 PM Andres Freund <andres at anarazel.de> > wrote: > >>>> > >>>> Hi, > >>>> > >>>> On 2020-09-25 19:05:42 -0700, Lang Hames wrote: > >>>> > The 2020 Virtual LLVM Developer's Meeting is coming up the week > after next. > >>>> > I'll be hosting a JIT Birds-of-a-Feather session on Thursday the > 8th at > >>>> > 10:55am PDT (See http://llvm.org/devmtg/2020-09/schedule/). I'm > planning to > >>>> > run this more like a Round Table: Minimal introduction, plenty of > time for > >>>> > discussion. > >>>> > > >>>> > Does anyone have any LLVM JIT related topics that they would > particularly > >>>> > like to discuss at this session? Any questions that they think > could be > >>>> > best answered by a round-table style discussion? I'll aim to rope > in the > >>>> > right people and prepare answers to make sure we get the most out > of the > >>>> > session. > >>>> > >>>> A topic that I'd find interesting is what it'd take to make it easier > to > >>>> use profile guided optimization in the context of JIT. A few years > back > >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice > to > >>>> make that easier. Possibly too complicated for that type of session? > >>>> > >>>> I dimly recall that the main issue was that there is no good way to > >>>> extract the profile information without going through files / applying > >>>> the profile data without going through files. > >>>> > >>>> Greetings, > >>>> > >>>> Andres Freund >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200928/b101cba6/attachment.html>
Geoff Levner via llvm-dev
2020-Oct-02 11:02 UTC
[llvm-dev] LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
StaticLibraryDefinitionGenerator works as promised on ELF archives! I have only encountered one problem. If a static library has not been compiled with -fPIC and uses symbols from a shared library, LLJIT does not complain, but the code may crash without warning when it is executed. Geoff On Mon, Sep 28, 2020 at 11:58 PM Lang Hames <lhames at gmail.com> wrote:> Hi Geoff, > > We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator work >> for ELF objects? > > > They do. :) > > I've not tested StaticLibraryDefinitionGenerator extensively on Linux. but > we have a regression test checking basic usage. If you run into any trouble > at all please file a bug and assign it to me. > > -- Lang. > > > > On Mon, Sep 28, 2020 at 2:05 PM Geoff Levner <glevner at gmail.com> wrote: > >> We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator >> work for ELF objects? >> >> On Mon, Sep 28, 2020 at 10:57 PM Lang Hames <lhames at gmail.com> wrote: >> > >> > Hi Geoff, >> > >> >> Importing symbols into the JIT from an object file or static >> library...? >> > >> > >> > Sure! Are you interested in doing this with the C API, LLJIT, or raw >> OrcV2 components? >> > >> > The high-level answer here (which we can dig into further in the BoF) >> is: >> > >> > For object files: >> > - For raw OrcV2 components you'll want to create an >> RTDyldObjectLinkingLayer or ObjectLinkingLayer and use the 'add' method. >> > - For an LLJIT instance you can just call the 'addObjectFile' method. >> > - For the OrcV2 C API you can call the 'LLVMOrcLLJITAddObjectFile' >> function. >> > >> > For static libraries: >> > - For raw OrcV2 components or an LLJIT instance you can attach a >> StaticLibraryDefinitionGenerator [1] to the JITDylib that you would like to >> load the library into. See [2] for an example. >> > - For the OrcV2 C API we will just need to add a method to wrap the >> operation above. We already have >> 'LLVMOrcCreateDynamicLibrarySearchGeneratorForProcess'. We should add >> counterparts to that to load dynamic and static libraries from paths. >> > >> > -- Lang. >> > >> > [1] >> https://github.com/llvm/llvm-project/blob/5aa56b242951ab0f0181386ea58509f19b09206e/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h#L361 >> > [2] >> https://github.com/llvm/llvm-project/blob/e7549dafcd33ced4280a81ca1d1ee4cc78ed253f/llvm/tools/lli/lli.cpp#L968 >> > >> > >> > On Mon, Sep 28, 2020 at 7:18 AM Geoff Levner <glevner at gmail.com> wrote: >> >> >> >> Importing symbols into the JIT from an object file or static >> library...? >> >> >> >> On Mon, Sep 28, 2020 at 5:29 AM Lang Hames <lhames at gmail.com> wrote: >> >>> >> >>> Hi Andres, >> >>> >> >>>> A topic that I'd find interesting is what it'd take to make it >> easier to >> >>>> use profile guided optimization in the context of JIT. A few years >> back >> >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice >> to >> >>>> make that easier. Possibly too complicated for that type of session? >> >>> >> >>> >> >>> We couldn't go into detail, but we could kick off a discussion and >> identify who is interested in working on this. My pitch: We should solve >> this by investing in JITLink and implementing profiling (and debugger >> support) via ObjectLinkingLayer::Plugins. Maybe a brief overview of / >> discussion on how these work would be useful? >> >>> >> >>> -- Lang. >> >>> >> >>> >> >>> On Fri, Sep 25, 2020 at 7:19 PM Andres Freund <andres at anarazel.de> >> wrote: >> >>>> >> >>>> Hi, >> >>>> >> >>>> On 2020-09-25 19:05:42 -0700, Lang Hames wrote: >> >>>> > The 2020 Virtual LLVM Developer's Meeting is coming up the week >> after next. >> >>>> > I'll be hosting a JIT Birds-of-a-Feather session on Thursday the >> 8th at >> >>>> > 10:55am PDT (See http://llvm.org/devmtg/2020-09/schedule/). I'm >> planning to >> >>>> > run this more like a Round Table: Minimal introduction, plenty of >> time for >> >>>> > discussion. >> >>>> > >> >>>> > Does anyone have any LLVM JIT related topics that they would >> particularly >> >>>> > like to discuss at this session? Any questions that they think >> could be >> >>>> > best answered by a round-table style discussion? I'll aim to rope >> in the >> >>>> > right people and prepare answers to make sure we get the most out >> of the >> >>>> > session. >> >>>> >> >>>> A topic that I'd find interesting is what it'd take to make it >> easier to >> >>>> use profile guided optimization in the context of JIT. A few years >> back >> >>>> I crudely hacked this together, but it was fairly ugly. It'd be nice >> to >> >>>> make that easier. Possibly too complicated for that type of session? >> >>>> >> >>>> I dimly recall that the main issue was that there is no good way to >> >>>> extract the profile information without going through files / >> applying >> >>>> the profile data without going through files. >> >>>> >> >>>> Greetings, >> >>>> >> >>>> Andres Freund >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201002/85132602/attachment.html>
Lang Hames via llvm-dev
2020-Oct-02 16:33 UTC
[llvm-dev] LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
Hi Geoff, I have only encountered one problem. If a static library has not been> compiled with -fPIC and uses symbols from a shared library, LLJIT does not > complain, but the code may crash without warning when it is executed.Was the static library compiled with large code model too? I think this is probably a RuntimeDyld bug: It's not great at error reporting. A few people in the community are working on an ELF/x86-64 implementation of JITLink at the moment -- Once we have that you should get small code model support and better error handling. -- Lang. On Fri, Oct 2, 2020 at 4:03 AM Geoff Levner <glevner at gmail.com> wrote:> StaticLibraryDefinitionGenerator works as promised on ELF archives! > > I have only encountered one problem. If a static library has not been > compiled with -fPIC and uses symbols from a shared library, LLJIT does not > complain, but the code may crash without warning when it is executed. > > Geoff > > On Mon, Sep 28, 2020 at 11:58 PM Lang Hames <lhames at gmail.com> wrote: > >> Hi Geoff, >> >> We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator >>> work for ELF objects? >> >> >> They do. :) >> >> I've not tested StaticLibraryDefinitionGenerator extensively on Linux. >> but we have a regression test checking basic usage. If you run into any >> trouble at all please file a bug and assign it to me. >> >> -- Lang. >> >> >> >> On Mon, Sep 28, 2020 at 2:05 PM Geoff Levner <glevner at gmail.com> wrote: >> >>> We use LLJIT. Do addObjectFile() and StaticLibraryDefinitionGenerator >>> work for ELF objects? >>> >>> On Mon, Sep 28, 2020 at 10:57 PM Lang Hames <lhames at gmail.com> wrote: >>> > >>> > Hi Geoff, >>> > >>> >> Importing symbols into the JIT from an object file or static >>> library...? >>> > >>> > >>> > Sure! Are you interested in doing this with the C API, LLJIT, or raw >>> OrcV2 components? >>> > >>> > The high-level answer here (which we can dig into further in the BoF) >>> is: >>> > >>> > For object files: >>> > - For raw OrcV2 components you'll want to create an >>> RTDyldObjectLinkingLayer or ObjectLinkingLayer and use the 'add' method. >>> > - For an LLJIT instance you can just call the 'addObjectFile' method. >>> > - For the OrcV2 C API you can call the 'LLVMOrcLLJITAddObjectFile' >>> function. >>> > >>> > For static libraries: >>> > - For raw OrcV2 components or an LLJIT instance you can attach a >>> StaticLibraryDefinitionGenerator [1] to the JITDylib that you would like to >>> load the library into. See [2] for an example. >>> > - For the OrcV2 C API we will just need to add a method to wrap the >>> operation above. We already have >>> 'LLVMOrcCreateDynamicLibrarySearchGeneratorForProcess'. We should add >>> counterparts to that to load dynamic and static libraries from paths. >>> > >>> > -- Lang. >>> > >>> > [1] >>> https://github.com/llvm/llvm-project/blob/5aa56b242951ab0f0181386ea58509f19b09206e/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h#L361 >>> > [2] >>> https://github.com/llvm/llvm-project/blob/e7549dafcd33ced4280a81ca1d1ee4cc78ed253f/llvm/tools/lli/lli.cpp#L968 >>> > >>> > >>> > On Mon, Sep 28, 2020 at 7:18 AM Geoff Levner <glevner at gmail.com> >>> wrote: >>> >> >>> >> Importing symbols into the JIT from an object file or static >>> library...? >>> >> >>> >> On Mon, Sep 28, 2020 at 5:29 AM Lang Hames <lhames at gmail.com> wrote: >>> >>> >>> >>> Hi Andres, >>> >>> >>> >>>> A topic that I'd find interesting is what it'd take to make it >>> easier to >>> >>>> use profile guided optimization in the context of JIT. A few years >>> back >>> >>>> I crudely hacked this together, but it was fairly ugly. It'd be >>> nice to >>> >>>> make that easier. Possibly too complicated for that type of >>> session? >>> >>> >>> >>> >>> >>> We couldn't go into detail, but we could kick off a discussion and >>> identify who is interested in working on this. My pitch: We should solve >>> this by investing in JITLink and implementing profiling (and debugger >>> support) via ObjectLinkingLayer::Plugins. Maybe a brief overview of / >>> discussion on how these work would be useful? >>> >>> >>> >>> -- Lang. >>> >>> >>> >>> >>> >>> On Fri, Sep 25, 2020 at 7:19 PM Andres Freund <andres at anarazel.de> >>> wrote: >>> >>>> >>> >>>> Hi, >>> >>>> >>> >>>> On 2020-09-25 19:05:42 -0700, Lang Hames wrote: >>> >>>> > The 2020 Virtual LLVM Developer's Meeting is coming up the week >>> after next. >>> >>>> > I'll be hosting a JIT Birds-of-a-Feather session on Thursday the >>> 8th at >>> >>>> > 10:55am PDT (See http://llvm.org/devmtg/2020-09/schedule/). I'm >>> planning to >>> >>>> > run this more like a Round Table: Minimal introduction, plenty of >>> time for >>> >>>> > discussion. >>> >>>> > >>> >>>> > Does anyone have any LLVM JIT related topics that they would >>> particularly >>> >>>> > like to discuss at this session? Any questions that they think >>> could be >>> >>>> > best answered by a round-table style discussion? I'll aim to rope >>> in the >>> >>>> > right people and prepare answers to make sure we get the most out >>> of the >>> >>>> > session. >>> >>>> >>> >>>> A topic that I'd find interesting is what it'd take to make it >>> easier to >>> >>>> use profile guided optimization in the context of JIT. A few years >>> back >>> >>>> I crudely hacked this together, but it was fairly ugly. It'd be >>> nice to >>> >>>> make that easier. Possibly too complicated for that type of >>> session? >>> >>>> >>> >>>> I dimly recall that the main issue was that there is no good way to >>> >>>> extract the profile information without going through files / >>> applying >>> >>>> the profile data without going through files. >>> >>>> >>> >>>> Greetings, >>> >>>> >>> >>>> Andres Freund >>> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201002/6cf932ca/attachment.html>
Maybe Matching Threads
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest
- LLVM Developers Meeting JIT BoF -- Request for Topics of Interest