llvm dev - Apr 2021 - Inclusion of the ORC runtime in compiler-rt.

Hi All,

I'd like to add the ORC runtime library (preview available at
https://github.com/lhames/llvm-project/tree/orc-runtime-preview) to
compiler-rt.

Background:

ORC, like MCJIT, can link JIT'd code either into the current process
("in-process" mode) or into a remote executor process
("cross-process"
mode). Some JIT features require support code in the executor process, but
the existing ORC libraries are only linked into the JIT process. This has
made cross-process mode support for those features (which include static
initializers, thread local variables, exception handling, and others)
awkward or impractical to implement. The ORC runtime library aims to
provide the necessary support code in a form that is loadable by the JIT
itself, which should allow these features to work uniformly in both modes.

My prototype branch of the ORC runtime (available at
https://github.com/lhames/llvm-project/tree/orc-runtime-preview) has
advanced to the point where it can provide uniform support for static
initializers, destructors, exceptions, thread locals, and language
registration for Objective C and Swift code. This support is all
MachO/Darwin only so far, but should be easily adaptable for ELF/Linux/BSD
support.

Proposal:

The proof of concept implementation has been very successful, so I would
like to move future development to the LLVM main branch so that others can
benefit from this.

Before I start posting patches, though:

Does anyone see any problems with including this in compiler-rt?

Does anyone think that there is a more reasonable home for the ORC runtime
within the llvm-project? I considered LLVM itself, or a new top-level
project, but neither seemed as natural a fit as compiler-rt.

Finally, if everyone is happy for it to be included in principle, are there
any volunteers to review ORC runtime patches?

Regards,
Lang.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210412/af1c743b/attachment.html>

I'd like to better understand the structure of the ORC runtime and its
dependencies (both build and runtime). Does it use the C or C++ standard
library? Does it depend on other parts of LLVM? Do you plan on reusing some
of the existing compiler-rt libraries like sanitizer_common?

To give a bit more background on why I'm interested, compiler-rt has grown
fairly organically this has been making the maintenance more and more
difficult, at least from the build perspective. There are some runtimes
that only use C, some that use C++, some that use C++ standard library.
When building compiler-rt together with other runtimes like libc or libc++,
it's difficult to pick up the right order which is why we have several
entry points into compiler-rt's build system to build different subsets and
that's been a maintenance headache.

I've been thinking about this quite a bit recently and I repeatedly came to
the conclusion that compiler-rt would ideally be broken up into several
subprojects, but that should probably be discussed as a separate topic.
However, understanding the build and runtimes dependencies of the ORC
runtime could help us decide whether it should be a part of compiler-rt or
be a separate subproject.

On Mon, Apr 12, 2021 at 12:26 PM Lang Hames <lhames at gmail.com> wrote:
> Hi All,
>
> I'd like to add the ORC runtime library (preview available at
> https://github.com/lhames/llvm-project/tree/orc-runtime-preview) to
> compiler-rt.
>
> Background:
>
> ORC, like MCJIT, can link JIT'd code either into the current process
> ("in-process" mode) or into a remote executor process
("cross-process"
> mode). Some JIT features require support code in the executor process, but
> the existing ORC libraries are only linked into the JIT process. This has
> made cross-process mode support for those features (which include static
> initializers, thread local variables, exception handling, and others)
> awkward or impractical to implement. The ORC runtime library aims to
> provide the necessary support code in a form that is loadable by the JIT
> itself, which should allow these features to work uniformly in both modes.
>
> My prototype branch of the ORC runtime (available at
> https://github.com/lhames/llvm-project/tree/orc-runtime-preview) has
> advanced to the point where it can provide uniform support for static
> initializers, destructors, exceptions, thread locals, and language
> registration for Objective C and Swift code. This support is all
> MachO/Darwin only so far, but should be easily adaptable for ELF/Linux/BSD
> support.
>
> Proposal:
>
> The proof of concept implementation has been very successful, so I would
> like to move future development to the LLVM main branch so that others can
> benefit from this.
>
> Before I start posting patches, though:
>
> Does anyone see any problems with including this in compiler-rt?
>
> Does anyone think that there is a more reasonable home for the ORC runtime
> within the llvm-project? I considered LLVM itself, or a new top-level
> project, but neither seemed as natural a fit as compiler-rt.
>
> Finally, if everyone is happy for it to be included in principle, are
> there any volunteers to review ORC runtime patches?
>
> Regards,
> Lang.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210412/16dd3dad/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3996 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210412/16dd3dad/attachment.bin>