llvm dev - Jul 2021 - ORC JIT Weekly #36 -- MachO Platform support lands, ELF JITLink backends get easier, C API improvements

Hi All,

The schedule for this work slipped a little, but I've got good news:

First, I'm happy to announce that MachO platform support has landed in [1].
This allows ORC users to load and use MachO support from the ORC runtime
with the following idiom:

if (auto P = MachOPlatform::Create(ES, ObjLayer, EPC, PlatformJD,
                                   OrcRuntimePath))
  ES.setPlatform(std::move(*P));
else
  return P.takeError();

where
  ES is the ExecutionSession,
  ObjLayer is an ObjectLinkingLayer,
  EPC is an ExecutorProcessControl instance,
  PlatformJD is the JITDylib that the MachO platform symbols should be
loaded into, and
  OrcRuntimePath is the path on disk of the ORC runtime archive.

Once the MachOPlatform instance has been created and attached to the
execution session MachO programs should *Just Work* under ORC*,* including
features like static initializers, static destructors, exceptions, native
thread-local-variables, and language runtime support (e.g. for Objective-C
and Swift)*. This support should be independent of the program
representation added (e.g. object file, LLVM IR, or higher-level module
representation), which will enable object-caching schemes to work even in
the presence of these features. The support should also work equally for
both in-process and out-of-process JITs, making it easier for users to
execute JIT'd code in separate processes.

As JITLink ELF support continues to grow, and with the fundamental
infrastructure in place, we're in good shape if anyone wants to start work
on a corresponding ELFPlatform.

Speaking of JITLink ELF support, there's good news on that front too:
I've
added a generic ELFLinkGraphBuilder in [2] and [3]. This should make it
much easier to contribute new ELF backends for JITLink. Generic
section/symbol parsing is now handled for you, and you should only need to
implement relocation parsing/application for new architectures when
bringing up a new ELF backend.

The first new backend that uses the generic ELFLinkGraphBuilder is already
up for review at https://reviews.llvm.org/D105429 -- Thanks very much
to StephenFan for working on that!

Finally, thanks very much to Valentin Churavy for recent improvements to
the ORC C APIs in https://reviews.llvm.org/D104672 and
https://reviews.llvm.org/D104799. Valentin's additions enable lazy
compilation via the new ORC C bindings, including lazy partitioning of
modules. He has provided example code
in llvm/examples/OrcV2Examples/OrcV2CBindingsLazy/,
and llvm/examples/OrcV2Examples/OrcV2CBindingsVeryLazy/. These APIs are
experimental for now (and likely to see some churn as the ORC runtime and
related features settle in) but restore some key functionality that was
lost when the ORCv1 C bindings were removed.

-- Lang.

[1]
https://github.com/llvm/llvm-project/commit/bb5f97e3ad10a0f8a62560890e5a87b4bc2c00bd
[2]
https://github.com/llvm/llvm-project/commit/aff57ff24aca5074b427d6bbc2f3246aa97910c5
[3]
https://github.com/llvm/llvm-project/commit/8e66fc438463e8cf6fa05e88b51ce29b604ce307

* This initial commit omitted TLV and language runtime support to try to
keep the size down (it was already quite large), but I expect support for
those features to land this week.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210719/4918d167/attachment.html>