Hi All, In the interests of improving visibility into ORC JIT development I'm going to try writing weekly status updates for the community. I hope they will provide insight into the design and state of development of LLVM's JIT APIs, as well as serving as a convenient space for discussions among LLVM's large and growing community of JIT API users. The length and detail will vary (depending on how much work I've gotten through, and how long I can dedicate to writing the update), but if people find the updates valuable I will make an effort to write at least something. Questions and comments are most welcome (both on the format, and on the content). Since this is the first update, I have also added some highlights from last year, and the plan for 2020. Highlights from 2019: (1) ORCv1 was officially deprecated in LLVM 9. I have left it in for the LLVM 10 branch, but plan to remove it from master in the coming weeks. All development effort is now focused on ORCv2. If you are an ORCv1 client, now's the time to switch over. If you need help please ask on the llvm-dev mailing lists (make sure you CC me) or #llvm on discord. There are also some tips available in https://llvm.org/docs/ORCv2.html . (2) LLVM has a new JIT linker, JITLink, which is intended as an eventual replacement for RuntimeDyld. The new design supports linker plugins (allowing operation on the low-level bits generated by the JIT linker) and native code models (RuntimeDyld required a custom code model on some platforms). Currently JITLink only supports Darwin x86-64 and arm64, but I hope to see support for new platforms added in the future. (3) Google Summer of Code student Praveen Velliengiri demonstrated a basic speculative compilation system built on ORCv2. This system analyses code added to the JIT and triggers early compilation on background threads for code that is likely to be used at runtime. Using this system Praveen was able to demonstrate significant speedups on JIT execution of some SPEC benchmarks. He presented this work at the 2019 LLVM Developer's Meeting in San Jose (see https://preejackie.github.io/GSoC-2019-LLVM). The plan for 2020: - Improve JIT support for static initializers: - Add support for running initializers from object files, which will enable loading and caching of objects containing initializers. - Improve support for platform-specific initializer kinds like Objective-C +load methods. - Switch from a push (“runConstructors”) to a pull model (“getConstructorsToRun”) for initializer execution. This will allow JIT’d code to “dlopen” other JIT’d code and run the initializers on the expected thread, which is important for JIT’ing code that uses threads and locks in initializers. - Improve adherence to static/dynamic linker rules: Weak symbol resolution across JITDylib boundaries is still not handled correctly. - Remove ORCv1. - Bug fixes and documentation improvements. Status report for this week: -- I’ve been working on a generic implementation of the new initialization APIs which I hope to be able to land next week. This will replace the runConstructors/runDestructors API in LLJIT (providing equivalent functionality: initializers will be discovered by scanning llvm.global_ctors/llvm.global_dtors), and will enable the development of platform specific initializer-support implementations. -- There’s a long-ish chat with @drmeister on the discord #llvm channel about RuntimeDyld vs JITLink, and large vs small code model. -- I have added a new LLJIT example program that demonstrates how to use lazy-reexports in ORCv2 to add lazy compilation: llvm/examples/LLJITExamples/LLJITWithLazyReexports. -- COFF support in the JIT still lags ELF and MachO (See e.g. http://llvm.org/PR40074). If there are any COFF experts out there who are interested in helping out with JIT bugs please let me know! Ok -- that’s enough from me for now. If you’re a JIT user, developer, or just casual JIT-development observer (dblaikie), and you have questions, comments, or just feel like introducing yourself: jump on in. :) Lang. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200116/b432c2a6/attachment.html>
Excellent initiative. Thanks, Lang! We are grateful for any high-level JIT-related info you see fit to provide, as it avoids us having to bother people on the mailing list or try to figure out how the source code works... Geoff On Fri, Jan 17, 2020 at 3:00 AM Lang Hames <lhames at gmail.com> wrote:> Hi All, > > In the interests of improving visibility into ORC JIT development I'm > going to try writing weekly status updates for the community. I hope they > will provide insight into the design and state of development of LLVM's JIT > APIs, as well as serving as a convenient space for discussions among LLVM's > large and growing community of JIT API users. The > length and detail will vary (depending on how much work I've gotten > through, and how long I can dedicate to writing the update), but if people > find the updates valuable I will make an effort to write at least > something. Questions and comments are most welcome (both on the format, and > on the content). > > Since this is the first update, I have also added some highlights from > last year, and the plan for 2020. > > *Highlights from 2019*: > > (1) ORCv1 was officially deprecated in LLVM 9. I have left it in for the > LLVM 10 branch, but plan to remove it from master in the coming weeks. All > development effort is now focused on ORCv2. If you are an ORCv1 client, > now's the time to switch over. If you need help please ask on the llvm-dev > mailing lists (make sure you CC me) or #llvm on discord. There are also > some tips available in https://llvm.org/docs/ORCv2.html . > > (2) LLVM has a new JIT linker, JITLink, which is intended as an eventual > replacement for RuntimeDyld. The new design supports linker plugins > (allowing operation on the low-level bits generated by the JIT linker) and > native code models (RuntimeDyld required a custom code model on some > platforms). Currently JITLink only supports Darwin x86-64 and arm64, but I > hope to see support for new platforms added in the future. > > (3) Google Summer of Code student Praveen Velliengiri demonstrated a basic *speculative > compilation* system built on ORCv2. This system analyses code added to > the JIT and triggers early compilation on background threads for code that > is likely to be used at runtime. Using this system Praveen was able to > demonstrate significant speedups on JIT execution of some SPEC benchmarks. > He presented this work at the 2019 LLVM Developer's Meeting in San Jose > (see https://preejackie.github.io/GSoC-2019-LLVM). > > *The plan for 2020*: > > - Improve JIT support for static initializers: > - Add support for running initializers from object files, which will > enable loading and caching of objects containing initializers. > - Improve support for platform-specific initializer kinds like > Objective-C +load methods. > - Switch from a push (“runConstructors”) to a pull model > (“getConstructorsToRun”) for initializer execution. This will allow JIT’d > code to “dlopen” other JIT’d code and run the initializers on the expected > thread, which is important for JIT’ing code that uses threads and locks in > initializers. > > - Improve adherence to static/dynamic linker rules: Weak symbol resolution > across JITDylib boundaries is still not handled correctly. > > - Remove ORCv1. > > - Bug fixes and documentation improvements. > > *Status report for this week*: > > -- I’ve been working on a generic implementation of the new initialization > APIs which I hope to be able to land next week. This will replace the > runConstructors/runDestructors API in LLJIT (providing equivalent > functionality: initializers will be discovered by scanning > llvm.global_ctors/llvm.global_dtors), and will enable the development of > platform specific initializer-support implementations. > > -- There’s a long-ish chat with @drmeister on the discord #llvm channel > about RuntimeDyld vs JITLink, and large vs small code model. > > -- I have added a new LLJIT example program that demonstrates how to use > lazy-reexports in ORCv2 to add lazy compilation: > llvm/examples/LLJITExamples/LLJITWithLazyReexports. > > -- COFF support in the JIT still lags ELF and MachO (See e.g. > http://llvm.org/PR40074 <https://bugs.llvm.org/show_bug.cgi?id=40074>). > If there are any COFF experts out there who are interested in helping out > with JIT bugs please let me know! > > Ok -- that’s enough from me for now. If you’re a JIT user, developer, or > just casual JIT-development observer (dblaikie), and you have questions, > comments, or just feel like introducing yourself: jump on in. :) > > Lang. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200117/ec070fa2/attachment.html>
Hi Lang, Great idea on the status updates, I'll be sure to check them out. Also thank you (and other contributors) for your work on ORC so far, it has been a fun API to tinker with and a great entry point into working with LLVM. I'm for the most part just a lowly user of ORC but I try to contribute here and there when I can. Mostly by nagging about COFF support through bug reports ;-). My use case for ORC is an expression-evaluator library I'm working on called JitCat (www.jitcat.org). It has some built-in reflection features to easily expose C++ functions/variables/classes for use in expressions. LLVM/ORC is used for code generation. Future plans are to extend JitCat into a fully featured scripting language. My own background is in game development, which is also what I use JitCat for myself. Regards, Machiel van Hooren (jcmac on Discord) On 17-Jan-20 03:00, Lang Hames via llvm-dev wrote:> Hi All, > > In the interests of improving visibility into ORC JIT development I'm > going to try writing weekly status updates for the community. I hope > they will provide insight into the design and state of development of > LLVM's JIT APIs, as well as serving as a convenient space for > discussions among LLVM's large and growing community of JIT API users. > The > length and detail will vary (depending on how much work I've gotten > through, and how long I can dedicate to writing the update), but if > people find the updates valuable I will make an effort to write at > least something. Questions and comments are most welcome (both on the > format, and on the content). > > Since this is the first update, I have also added some highlights from > last year, and the plan for 2020. > > /Highlights from 2019/: > > (1) ORCv1 was officially deprecated in LLVM 9. I have left it in for > the LLVM 10 branch, but plan to remove it from master in the coming > weeks. All development effort is now focused on ORCv2. If you are an > ORCv1 client, now's the time to switch over. If you need help please > ask on the llvm-dev mailing lists (make sure you CC me) or #llvm on > discord. There are also some tips available in > https://llvm.org/docs/ORCv2.html . > > (2) LLVM has a new JIT linker, JITLink, which is intended as an > eventual replacement for RuntimeDyld. The new design supports linker > plugins (allowing operation on the low-level bits generated by the JIT > linker) and native code models (RuntimeDyld required a custom code > model on some platforms). Currently JITLink only supports Darwin > x86-64 and arm64, but I hope to see support for new platforms added in > the future. > (3) Google Summer of Code student Praveen Velliengiri demonstrated a > basic /speculative compilation/ system built on ORCv2. This system > analyses code added to the JIT and triggers early compilation on > background threads for code that is likely to be used at runtime. > Using this system Praveen was able to demonstrate significant speedups > on JIT execution of some SPEC benchmarks. He presented this work at > the 2019 LLVM Developer's Meeting in San Jose (see > https://preejackie.github.io/GSoC-2019-LLVM). > > /The plan for 2020/: > > - Improve JIT support for static initializers: > - Add support for running initializers from object files, which will > enable loading and caching of objects containing initializers. > - Improve support for platform-specific initializer kinds like > Objective-C +load methods. > - Switch from a push (“runConstructors”) to a pull model > (“getConstructorsToRun”) for initializer execution. This will allow > JIT’d code to “dlopen” other JIT’d code and run the initializers on > the expected thread, which is important for JIT’ing code that uses > threads and locks in initializers. > > - Improve adherence to static/dynamic linker rules: Weak symbol > resolution across JITDylib boundaries is still not handled correctly. > > - Remove ORCv1. > > - Bug fixes and documentation improvements. > > /Status report for this week/: > > -- I’ve been working on a generic implementation of the new > initialization APIs which I hope to be able to land next week. This > will replace the runConstructors/runDestructors API in LLJIT > (providing equivalent functionality: initializers will be discovered > by scanning llvm.global_ctors/llvm.global_dtors), and will enable the > development of platform specific initializer-support implementations. > > -- There’s a long-ish chat with @drmeister on the discord #llvm > channel about RuntimeDyld vs JITLink, and large vs small code model. > > -- I have added a new LLJIT example program that demonstrates how to > use lazy-reexports in ORCv2 to add lazy compilation: > llvm/examples/LLJITExamples/LLJITWithLazyReexports. > > -- COFF support in the JIT still lags ELF and MachO (See e.g. > http://llvm.org/PR40074 > <https://bugs.llvm.org/show_bug.cgi?id=40074>). If there are any COFF > experts out there who are interested in helping out with JIT bugs > please let me know! > > Ok -- that’s enough from me for now. If you’re a JIT user, developer, > or just casual JIT-development observer (dblaikie), and you have > questions, comments, or just feel like introducing yourself: jump on > in. :) > > Lang. > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200117/eeb9a473/attachment.html>
Thank you for creating weekly updates, they will be quite useful, since previously ORC development seemed rather opaque. One thing that will be useful (and was done to some extent with ORCv1) is to expose ORCv2's API via C-compatible bindings such that code from languages other than C++ (Rust for me) can effectively use it, including things such as the equivalent of -march=native and introspection such that the supported SIMD widths can be detected. I'm planning on using ORC to compile shaders for Kazan, the GPU driver that I'm writing for libre-riscv's hybrid cpu/gpu. Jacob Lifshay -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200117/964df61b/attachment.html>
Hi, Lang As a starter using LLVM JIT to improve OLAP execution engine performance, I'm very glad to hear that. I can't find some useful document help me get start to use the new ORC JIT API quickly. Only can find some examples how to use it, but don't know the internal from low level, and very blurred to design a clearly JIT toolset. Hope more tutorials add in and help ORC JIT more easy to adoption. Big thanks. Jacob Lifshay via llvm-dev <llvm-dev at lists.llvm.org> 于2020年1月17日周五 下午11:38写道：> Thank you for creating weekly updates, they will be quite useful, since > previously ORC development seemed rather opaque. > > One thing that will be useful (and was done to some extent with ORCv1) is > to expose ORCv2's API via C-compatible bindings such that code from > languages other than C++ (Rust for me) can effectively use it, including > things such as the equivalent of -march=native and introspection such that > the supported SIMD widths can be detected. I'm planning on using ORC to > compile shaders for Kazan, the GPU driver that I'm writing for > libre-riscv's hybrid cpu/gpu. > > Jacob Lifshay > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-- Guang-Nan He -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200118/19ee4708/attachment.html>
Hi, On 2020-01-16 18:00:53 -0800, Lang Hames via llvm-dev wrote:> In the interests of improving visibility into ORC JIT development I'm > going to try writing weekly status updates for the community. I hope > they will provide insight into the design and state of development of > LLVM's JIT APIs, as well as serving as a convenient space for > discussions among LLVM's large and growing community of JIT API users.That's a great idea.> Since this is the first update, I have also added some highlights from last year, and the plan for 2020. > > Highlights from 2019: > > (1) ORCv1 was officially deprecated in LLVM 9. I have left it in for > the LLVM 10 branch, but plan to remove it from master in the coming > weeks. All development effort is now focused on ORCv2. If you are an > ORCv1 client, now's the time to switch over. If you need help please > ask on the llvm-dev mailing lists (make sure you CC me) or #llvm on > discord. There are also some tips available in > https://llvm.org/docs/ORCv2.htmlI also want to highlight the necessity of some form of C API, that others already have. Besides just needing something that can be called from languages besides C++, some amount of higher API stability is also important. For users of LLVM with longer support cycles than LLVM (e.g. Postgres has 5 years of back branch maintenance), and which live in a world where vendoring is not allowed (most things going into linux distros), the API churn can be serious problem. It's fine if the set of "somewhat stable" C APIs doesn't provide all the possible features, though. It's easy enough to add a bunch of wrappers or ifdefs hiding some simple signature changes, e.g. LLVMOrcGetSymbolAddress adding a parameter as happened in LLVM 6, but backpatching support for a larger API redesigns, into stable versions, is scary. We do however quickly get complaints if a supported version cannot be compiled due to dependencies, as people tend to upgrade their OS separately from e.g. their database major version.> (2) LLVM has a new JIT linker, JITLink, which is intended as an > eventual replacement for RuntimeDyld. The new design supports linker > plugins (allowing operation on the low-level bits generated by the JIT > linker) and native code models (RuntimeDyld required a custom code > model on some platforms). Currently JITLink only supports Darwin > x86-64 and arm64, but I hope to see support for new platforms added in > the future.What's the capability level of ORCv2 on RuntimeDyld compared to ORCv1? Are there features supported in v1 that are only available on JITLink supported platforms?> - Improve JIT support for static initializers: > - Add support for running initializers from object files, which will enable loading and caching of objects containing initializers.Hm, that's kind of supported for v1, right? Greetings, Andres Freund
Hi Andres, I also want to highlight the necessity of some form of C API, that others> already have. ><snip>> It's fine if the set of "somewhat stable" C APIs doesn't provide all the > possible features, though.Ok. This got me thinking about what a simple LLJIT API should look like. I have posted a sketch of a possible API on http://llvm.org/PR31103 . I don't have time to implement it just yet, but I would be very happy to provide support and review patches if anyone else wants to give it a shot. What's the capability level of ORCv2 on RuntimeDyld compared to ORCv1?> Are there features supported in v1 that are only available on JITLink > supported platforms?At a high level, ORCv2's design allows for basically the same features as ORCv1, plus concurrent compilation. There are still a number of APIs that haven't been hooked up or implemented though. Most prominently: Event listeners and removable code. If you're using either of those features please let me know: I do want to make sure we continue to support them (or provide an equivalent). There are no features supported by ORCv1 that require JITLink under ORCv2.> - Improve JIT support for static initializers: > > - Add support for running initializers from object files, which will > enable loading and caching of objects containing initializers. > Hm, that's kind of supported for v1, right?It's "kind of" supported. MCJIT and ORCv1 provided support for scanning the llvm.global_ctors variable to find the names of static initializers to run. This works fine when (1) you're adding LLVM IR AND (2) you only care initializers described by llvm.global_ctors. On the other hand, if you add object files (or loading them from an ObjectCache), or if you have initializers not described by llvm.global_ctors (e.g. ObjC and Swift, which have additional initializers described by metadata sections) then MCJIT and ORCv1 provide no help out-of-the-box. This problem is further exacerbated by concurrent compilation in ORCv2: You may need to order your initializers (e.g. according to the llvm.global_ctors priority field), but objects may arrive at the JIT linker out of order due to concurrent compilation. The new ORCv2 initializer support aims to make all of this natural: We will provide 'dlopen' and 'dlclose' equivalent calls on JITDylibs. This will trigger compilation and execution of initializers that have not been run already. If you use JITLink, this will include using JITLink-plugins to discover the initializers to run, including initializers in metadata sections. -- Lang. On Mon, Jan 27, 2020 at 10:14 AM Andres Freund <andres at anarazel.de> wrote:> Hi, > > On 2020-01-16 18:00:53 -0800, Lang Hames via llvm-dev wrote: > > In the interests of improving visibility into ORC JIT development I'm > > going to try writing weekly status updates for the community. I hope > > they will provide insight into the design and state of development of > > LLVM's JIT APIs, as well as serving as a convenient space for > > discussions among LLVM's large and growing community of JIT API users. > > That's a great idea. > > > > Since this is the first update, I have also added some highlights from > last year, and the plan for 2020. > > > > Highlights from 2019: > > > > (1) ORCv1 was officially deprecated in LLVM 9. I have left it in for > > the LLVM 10 branch, but plan to remove it from master in the coming > > weeks. All development effort is now focused on ORCv2. If you are an > > ORCv1 client, now's the time to switch over. If you need help please > > ask on the llvm-dev mailing lists (make sure you CC me) or #llvm on > > discord. There are also some tips available in > > https://llvm.org/docs/ORCv2.html > > I also want to highlight the necessity of some form of C API, that > others already have. > > Besides just needing something that can be called from languages besides > C++, some amount of higher API stability is also important. For users of > LLVM with longer support cycles than LLVM (e.g. Postgres has 5 years of > back branch maintenance), and which live in a world where vendoring is > not allowed (most things going into linux distros), the API churn can be > serious problem. It's fine if the set of "somewhat stable" C APIs > doesn't provide all the possible features, though. > > It's easy enough to add a bunch of wrappers or ifdefs hiding some simple > signature changes, e.g. LLVMOrcGetSymbolAddress adding a parameter as > happened in LLVM 6, but backpatching support for a larger API redesigns, > into stable versions, is scary. We do however quickly get complaints if > a supported version cannot be compiled due to dependencies, as people > tend to upgrade their OS separately from e.g. their database major > version. > > > > (2) LLVM has a new JIT linker, JITLink, which is intended as an > > eventual replacement for RuntimeDyld. The new design supports linker > > plugins (allowing operation on the low-level bits generated by the JIT > > linker) and native code models (RuntimeDyld required a custom code > > model on some platforms). Currently JITLink only supports Darwin > > x86-64 and arm64, but I hope to see support for new platforms added in > > the future. > > What's the capability level of ORCv2 on RuntimeDyld compared to ORCv1? > Are there features supported in v1 that are only available on JITLink > supported platforms? > > > > - Improve JIT support for static initializers: > > - Add support for running initializers from object files, which will > enable loading and caching of objects containing initializers. > > Hm, that's kind of supported for v1, right? > > > Greetings, > > Andres Freund >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200128/061ace30/attachment.html>