thr3ads.net - llvm dev - [llvm-dev] LLVM Orc Weekly #28 -- ORC Runtime Prototype update [Jan 2021]

If this information is useful, please help other people find it:
Share via:

Lang Hames via llvm-dev

2021-Jan-18 07:55 UTC

[llvm-dev] LLVM Orc Weekly #28 -- ORC Runtime Prototype update

Hi All,

Happy 2021!

I've just posted a new Orc Runtime Preview patch:
https://github.com/lhames/llvm-project/commit/8833a7f24693f1c7a3616438718e7927c6624894

Quick background:

To date, neither ORC nor MCJIT have had their own runtime libraries. This
has limited and complicated the implementation of many features (e.g. jit
re-entry functions, exception handling, JID'd initializers and
de-initializers), and more-or-less prevented the implementation of others
(e.g. native thread local storage).

Late last year I started work on a prototype ORC runtime library to address
this, and with the above commit I've finally got something worth sharing.

The prototype above is simultaneously limited and complex. Limited, in that
it only tackles a small subset of the desired functionality. Complex in
that it's one of the most involved pieces of functionality that I
anticipate supporting, as it requires two-way communication between the
executor and JIT processes. My aim in choosing to tackle the hard part
first was to get a sense of our ultimate requirements for the project,
particularly in regards to *where it should live within the LLVM Project*.
It's not a perfect fit for LLVM proper: there will be lots of target
specific code, including assembly, and it should be easily buildable for
multiple targets (that sounds more like compiler-rt). On the other hand
it's not a perfect fit for compiler-rt: it shares data structures with
LLVM, and it would be very useful to be able to re-use llvm::Error  /
llvm::Expected (that sounds like LLVM). At the moment I think the best way
to square things would be to keep it in compiler-rt, allow inclusion of
header-only code from LLVM in compiler-rt, and then make Error / Expected
header-only (or copy / adapt them for this library). This will be a
discussion for llvm-dev at some point in the near future.

On to the actual functionality though: The prototype makes significant
changes to the MachOPlatform class and introduces an ORC runtime library in
compiler-rt/lib/orc. Together, these changes allow us to emulate the dlopen
/ dlsym / dlclose in the JIT executor process. We can use this to define
what it means to run a *JIT program*, rather than just running a JIT
function (the way TargetProcessControl::runAsMain does):

ORC_RT_INTERFACE int64_t __orc_rt_macho_run_program(int argc, char *argv[])
{
  using MainTy = int (*)(int, char *[]);

  void *H = __orc_rt_macho_jit_dlopen("Main", ORC_RT_RTLD_LAZY);
  if (!H) {
    __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
    return -1;
  }

  auto *Main = reinterpret_cast<MainTy>(__orc_rt_macho_jit_dlsym(H,
"main"
));
  if (!Main) {
    __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
    return -1;
  }

  int Result = Main(argc, argv);

  if (__orc_rt_macho_jit_dlclose(H) == -1)
    __orc_rt_log_error(__orc_rt_macho_jit_dlerror());

  return Result;
}

The functions __orc_rt_macho_jit_dlopen, __orc_rt_macho_jit_dlsym,
and __orc_rt_macho_jit_dlclose behave the same as their dlfcn.h
counterparts (dlopen, dlsym, dlclose), but operate on JITDylibs rather than
regular dylibs. This includes running static initializers and registering
with language runtimes (e.g. ObjC).

While we could run static initializers before (e.g. via
LLJIT::runConstructors), we had to initiate this from the JIT process side,
which has two significant drawbacks: (1) Extra RPC round trips, and (2) in
the out-of-process case: initializers not running on the executor thread
that requested them, since that thread will be blocked waiting for its call
to return. Issue (1) only affects performance, but (2) can affect
correctness if the initializers modify thread local values, or interact
with locks or threads. Interacting with threads from initializers is
generally best avoided, but nonetheless is done by real-world code, so we
want to support it. By using the runtime we can improve both performance
and correctness (or at least consistency with current behavior).

The effect of this is that we can now load C++, Objective-C and Swift
programs in the JIT and expect them to run correctly, at least for simple
cases. This works regardless of whether the JIT'd code runs in-process or
out-of-process. To test all this I have integrated support for the
prototype runtime into llvm-jitlink. You can demo output from this tool
below for two simple input programs: One swift, one C++. All of this is
MachO specific at the moment, but provides a template that could be easily
re-used to support this on ELF platforms, and likely on COFF platforms too.

While the discussion on where the runtime should live plays out I will
continue adding / moving functionality to the prototype runtime. Next up
will be eh-frame registration and resolver functions (both currently in
OrcTargetProcess). After that I'll try to tackle support for native MachO
thread local storage.

As always: Questions and comments are very welcome.

-- Lang.

lhames at Langs-MacBook-Pro scratch % cat foo.swift
class MyClass {
  func foo() {
    print("foo")
  }
}

let m = MyClass()
m.foo();

lhames at Langs-MacBook-Pro scratch % xcrun swiftc -emit-object -o foo.o
foo.swift
lhames at Langs-MacBook-Pro scratch % llvm-jitlink -dlopen
/usr/lib/swift/libswiftCore.dylib foo.o
foo
lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor -dlopen
/usr/lib/swift/libswiftCore.dylib foo.o
foo
lhames at Langs-MacBook-Pro scratch % cat inits.cpp
#include <iostream>

class Foo {
public:
  Foo() { std::cout << "Foo::Foo()\n"; }
  ~Foo() { std::cout << "Foo::~Foo()\n"; }
  void foo() { std::cout << "Foo::foo()\n"; }
};

Foo F;

int main(int argc, char *argv[]) {
  F.foo();
  return 0;
}
lhames at Langs-MacBook-Pro scratch % xcrun clang++ -c -o inits.o inits.cpp
lhames at Langs-MacBook-Pro scratch % llvm-jitlink inits.o

Foo::Foo()
Foo::foo()
Foo::~Foo()
lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor inits.o
Foo::Foo()
Foo::foo()
Foo::~Foo()
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210118/c5fcb0ff/attachment.html>

Stefan Gränitz via llvm-dev

2021-Jan-18 13:49 UTC

head link

[llvm-dev] LLVM Orc Weekly #28 -- ORC Runtime Prototype update

Wow, thanks for the update. One more ORC milestone in a short period of
time!

On macOS I built the C++ example like this:

% cmake -GNinja -DLLVM_TARGETS_TO_BUILD=host
-DLLVM_ENABLE_PROJECTS="clang;compiler-rt" ../llvm
% ninja llvm-jitlink llvm-jitlink-executor
lib/clang/12.0.0/lib/darwin/libclang_rt.orc_osx.a
% clang++ -c -o inits.o inits.cpp

The in-process version works perfectly, but with the out-of-process flag
the examples fails:

% ./bin/llvm-jitlink inits.o
Foo::Foo()
Foo::foo()
Foo::~Foo()
% ./bin/llvm-jitlink -oop-executor inits.o
JIT session error: Symbols not found: [
__ZTIN4llvm6detail14format_adapterE ]

Any idea what could go wrong here? Otherwise I can try to debug it later
this week. (Full error below.)

Best
Stefan

--

JIT session error: Symbols not found: [
__ZTIN4llvm6detail14format_adapterE ]
/Users/staefsn/Develop/LLVM/monorepo/llvm-orc-runtime/build/bin/llvm-jitlink:
Failed to materialize symbols: { (Main, {
___orc_rt_macho_symbol_lookup_remote,
__ZN4llvm3orc6shared21WrapperFunctionResult22destroyWithArrayDeleteE39LLVMOrcSharedCWrapperFunctionResultDatay,
__ZNSt3__113__vector_baseIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEED2Ev,
__ZNK4llvm8ExpectedINS_3orc6shared21WrapperFunctionResultEE22fatalUncheckedExpectedEv,
__ZN4llvm3orc6shared21toWrapperFunctionBlobIJNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEEEENS1_21WrapperFunctionResultEDpRKT_,
__ZN4llvm3orc6shared20VectorRawByteChannelD1Ev,
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJNSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_SA_EE11deserializeISA_JSA_SA_EEENS_5ErrorERS3_RT_DpRT0_,
__ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializers13SectionExtentENS_9allocatorIS5_EEE8__appendEm,
__ZN4llvm3orc6shared21toWrapperFunctionBlobIJyNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEEEENS1_21WrapperFunctionResultEDpRKT_,
__ZN4llvm29VerifyEnableABIBreakingChecksE,
__ZN4llvm15format_providerImvE6formatERKmRNS_11raw_ostreamENS_9StringRefE,
__ZNSt3__114__split_bufferIN4llvm3orc6shared25MachOJITDylibInitializersERNS_9allocatorIS4_EEED2Ev,
__ZTVN4llvm6detail23provider_format_adapterIRmEE,
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEEEE9serializeIRKyJRKSA_EEENS_5ErrorERS3_OT_DpOT0_,
__ZTVN4llvm6detail23provider_format_adapterImEE,
__ZN4llvm6detail23provider_format_adapterImE6formatERNS_11raw_ostreamENS_9StringRefE,
__ZN4llvm6detail23provider_format_adapterIRmED1Ev,
___orc_rt_macho_get_deinitializers_tag,
__ZN4llvm6detail23provider_format_adapterIRmED0Ev,
__ZTVN4llvm3orc6shared14RawByteChannelE,
__ZN6orc_rt12jit_dispatchEPKvN4llvm8ArrayRefIhEE,
__ZN4llvm3orc6shared20VectorRawByteChannelD0Ev,
__ZTVN4llvm3orc6shared20VectorRawByteChannelE,
__ZN4llvm8cantFailENS_5ErrorEPKc,
__ZN4llvm6detail23provider_format_adapterImED1Ev,
__ZTVN4llvm6detail23provider_format_adapterIRjEE,
__ZN4llvm6detail23provider_format_adapterImED0Ev,
__ZN4llvm15format_providerIjvE6formatERKjRNS_11raw_ostreamENS_9StringRefE,
__ZTSN4llvm3orc6shared14RawByteChannelE,
__ZN4llvm6detail23provider_format_adapterIRjE6formatERNS_11raw_ostreamENS_9StringRefE,
__ZN4llvm6detail23provider_format_adapterIRjED0Ev,
__ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__16vectorINS1_25MachOJITDylibInitializersENS4_9allocatorIS6_EEEES9_vE11deserializeERS3_RS9_,
__ZTIN4llvm6detail23provider_format_adapterImEE,
__ZN4llvm6detail15HelperFunctions15consumeHexStyleERNS_9StringRefERNS_13HexPrintStyleE,
__ZTIN4llvm6detail23provider_format_adapterIRmEE,
___orc_rt_macho_get_initializers_tag,
__ZTSN4llvm6detail23provider_format_adapterImEE,
__ZN4llvm3orc6shared20VectorRawByteChannel4sendEv,
__ZN4llvm6detail23provider_format_adapterIRmE6formatERNS_11raw_ostreamENS_9StringRefE,
___orc_rt_macho_symbol_lookup_tag,
__ZTSN4llvm6detail23provider_format_adapterIRmEE,
__ZN4llvm3orc6shared20VectorRawByteChannel11appendBytesEPKcj,
__ZTSN4llvm6detail23provider_format_adapterIRjEE,
__ZN4llvm3orc6shared20VectorRawByteChannel9readBytesEPcj,
__ZTSN4llvm3orc6shared20VectorRawByteChannelE,
__ZN4llvm6detail23provider_format_adapterIRjED1Ev,
__ZN4llvm3orc6shared14RawByteChannelD1Ev,
__ZN4llvm3orc6shared23fromWrapperFunctionBlobIJNSt3__16vectorINS1_25MachOJITDylibInitializersENS3_9allocatorIS5_EEEEEEENS_5ErrorENS_8ArrayRefIhEEDpRT_,
__ZN4llvm3orc6shared14RawByteChannelD0Ev,
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyyNSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_SA_EE11deserializeIyJySA_SA_SA_EEENS_5ErrorERS3_RT_DpRT0_,
__ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEE8__appendEm,
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyyEE11deserializeIyJyEEENS_5ErrorERS3_RT_DpRT0_,
__ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEE6resizeEm,
__ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEESA_vE11deserializeERNS1_14RawByteChannelERSA_,
__ZN4llvm3orc6shared23fromWrapperFunctionBlobIJyEEENS_5ErrorENS_8ArrayRefIhEEDpRT_,
__ZTIN4llvm6detail23provider_format_adapterIRjEE,
__ZN4llvm3orc6shared21WrapperFunctionResult20getAnyOutOfBandErrorEv,
__ZTIN4llvm3orc6shared20VectorRawByteChannelE,
___orc_rt_macho_get_initializers_remote,
__ZTIN4llvm3orc6shared14RawByteChannelE,
__ZNSt3__16vectorIhNS_9allocatorIhEEE26__swap_out_circular_bufferERNS_14__split_bufferIhRS2_EE,
__ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_vE11deserializeERS3_RSA_
}) }
/Users/staefsn/Develop/LLVM/monorepo/llvm-orc-runtime/build/bin/llvm-jitlink-executor:Response
has unknown sequence number 527162

On 18/01/2021 08:55, Lang Hames wrote:> Hi All,
>
> Happy 2021!
>
> I've just posted a new Orc Runtime Preview patch:
>
https://github.com/lhames/llvm-project/commit/8833a7f24693f1c7a3616438718e7927c6624894
>
<https://github.com/lhames/llvm-project/commit/8833a7f24693f1c7a3616438718e7927c6624894>
>
> Quick background:
>
> To date, neither ORC nor MCJIT have had their own runtime libraries.
> This has limited and complicated the implementation of many features
> (e.g. jit re-entry functions, exception handling, JID'd initializers
> and de-initializers), and more-or-less prevented the implementation of
> others (e.g. native thread local storage).
>
> Late last year I started work on a prototype ORC runtime library to
> address this, and with the above commit I've finally got something
> worth sharing.
>
> The prototype above is simultaneously limited and complex. Limited, in
> that it only tackles a small subset of the desired functionality.
> Complex in that it's one of the most involved pieces of functionality
> that I anticipate supporting, as it requires two-way communication
> between the executor and JIT processes. My aim in choosing to tackle
> the hard part first was to get a sense of our ultimate requirements
> for the project, particularly in regards to /where it should live
> within the LLVM Project/. It's not a perfect fit for LLVM proper:
> there will be lots of target specific code, including assembly, and it
> should be easily buildable for multiple targets (that sounds more like
> compiler-rt). On the other hand it's not a perfect fit for
> compiler-rt: it shares data structures with LLVM, and it would be very
> useful to be able to re-use llvm::Error  / llvm::Expected (that sounds
> like LLVM). At the moment I think the best way to square things would
> be to keep it in compiler-rt, allow inclusion of header-only code from
> LLVM in compiler-rt, and then make Error / Expected header-only (or
> copy / adapt them for this library). This will be a discussion for
> llvm-dev at some point in the near future.
>
> On to the actual functionality though: The prototype makes significant
> changes to the MachOPlatform class and introduces an ORC runtime
> library in compiler-rt/lib/orc. Together, these changes allow us to
> emulate the dlopen / dlsym / dlclose in the JIT executor process. We
> can use this to define what it means to run a /JIT program/, rather
> than just running a JIT function (the way
> TargetProcessControl::runAsMain does):
>
> ORC_RT_INTERFACE int64_t __orc_rt_macho_run_program(int argc, char *argv[])
> {
>   using MainTy = int (*)(int, char *[]);
>
>   void *H = __orc_rt_macho_jit_dlopen("Main", ORC_RT_RTLD_LAZY);
>   if (!H) {
>     __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>     return -1;
>   }
>
>
  auto *Main = reinterpret_cast<MainTy>(__orc_rt_macho_jit_dlsym(H, "main"));
>   if (!Main) {
>     __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>     return -1;
>   }
>
>   int Result = Main(argc, argv);
>
>   if (__orc_rt_macho_jit_dlclose(H) == -1)
>     __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>
>   return Result;
> }
>
> The functions __orc_rt_macho_jit_dlopen, __orc_rt_macho_jit_dlsym,
> and __orc_rt_macho_jit_dlclose behave the same as their dlfcn.h
> counterparts (dlopen, dlsym, dlclose), but operate on JITDylibs rather
> than regular dylibs. This includes running static initializers and
> registering with language runtimes (e.g. ObjC).
>
> While we could run static initializers before (e.g. via
> LLJIT::runConstructors), we had to initiate this from the JIT process
> side, which has two significant drawbacks: (1) Extra RPC round trips,
> and (2) in the out-of-process case: initializers not running on the
> executor thread that requested them, since that thread will be blocked
> waiting for its call to return. Issue (1) only affects performance,
> but (2) can affect correctness if the initializers modify thread local
> values, or interact with locks or threads. Interacting with threads
> from initializers is generally best avoided, but nonetheless is done
> by real-world code, so we want to support it. By using the runtime we
> can improve both performance and correctness (or at least consistency
> with current behavior). 
>
> The effect of this is that we can now load C++, Objective-C and Swift
> programs in the JIT and expect them to run correctly, at least for
> simple cases. This works regardless of whether the JIT'd code runs
> in-process or out-of-process. To test all this I have integrated
> support for the prototype runtime into llvm-jitlink. You can demo
> output from this tool below for two simple input programs: One swift,
> one C++. All of this is MachO specific at the moment, but provides a
> template that could be easily re-used to support this on ELF
> platforms, and likely on COFF platforms too.
>
> While the discussion on where the runtime should live plays out I will
> continue adding / moving functionality to the prototype runtime. Next
> up will be eh-frame registration and resolver functions (both
> currently in OrcTargetProcess). After that I'll try to tackle support
> for native MachO thread local storage.
>
> As always: Questions and comments are very welcome.
>
> -- Lang.
>
> lhames at Langs-MacBook-Pro scratch % cat foo.swift
> class MyClass {
>   func foo() {
>     print("foo")
>   }
> }
>
> let m = MyClass()
> m.foo();
>
> lhames at Langs-MacBook-Pro scratch % xcrun swiftc -emit-object -o foo.o
> foo.swift                                    
> lhames at Langs-MacBook-Pro scratch % llvm-jitlink -dlopen
> /usr/lib/swift/libswiftCore.dylib foo.o              
> foo
> lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor -dlopen
> /usr/lib/swift/libswiftCore.dylib foo.o
> foo
> lhames at Langs-MacBook-Pro scratch % cat inits.cpp 
> #include <iostream>
>
> class Foo {
> public:
>   Foo() { std::cout << "Foo::Foo()\n"; }
>   ~Foo() { std::cout << "Foo::~Foo()\n"; }
>   void foo() { std::cout << "Foo::foo()\n"; }
> };
>
> Foo F;
>
> int main(int argc, char *argv[]) {
>   F.foo();
>   return 0;
> }
> lhames at Langs-MacBook-Pro scratch % xcrun clang++ -c -o inits.o inits.cpp
> lhames at Langs-MacBook-Pro scratch % llvm-jitlink inits.o              
>                                        
> Foo::Foo()
> Foo::foo()
> Foo::~Foo()
> lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor inits.o
> Foo::Foo()
> Foo::foo()
> Foo::~Foo()
-- 
https://flowcrypt.com/pub/stefan.graenitz at gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210118/3a42bb39/attachment-0001.html>

llvm dev - Jan 2021 - LLVM Orc Weekly #28 -- ORC Runtime Prototype update

[llvm-dev] LLVM Orc Weekly #28 -- ORC Runtime Prototype update

[llvm-dev] LLVM Orc Weekly #28 -- ORC Runtime Prototype update