[llvm-dev] LLVM Orc Weekly #28 -- ORC Runtime Prototype update

Stefan Gränitz via llvm-dev
Tue, 19 Jan 2021 22:41:18 -0000

Interesting that we don't see the same failure then (Debug is the
default build type).

So I did some debugging:

Actually, the ZTIN prefix of the missing symbol indicates a typeinfo
symbol. This is weird because we don't usually build anything with RTTI
in LLVM right? Anyway I rebuilt orc_rt_macho_dlfcn_remote.cpp.o with an
explicit "-fno-rtti" and voila! That fixes the issue.

It seems that COMPILER_RT_COMMON_CFLAGS doesn't disable RTTI by default.
Or maybe just on my system? Should we add it for clang_rt.orc as long as
we do have the libSupport dependency? I put up a PR here:
https://github.com/lhames/llvm-project/pull/3

Cheers
Stefan

On 18/01/2021 22:45, Lang Hames wrote:
> Hi Stefan,
>
>     % ./bin/llvm-jitlink -oop-executor inits.o
>     JIT session error: Symbols not found: [
>     __ZTIN4llvm6detail14format_adapterE ]
>
>
> I've been testing with a debug build:
>
> % xcrun cmake -GNinja -DCMAKE_BUILD_TYPE=Debug
> -DLLVM_ENABLE_PROJECTS="llvm;clang;compiler-rt" ../llvm
>
> Matching this build might fix the issue, though building with my
> config (if it works) is only a short-term fix. The error that you're
> seeing implies that the runtime is dependending on a symbol from
> libSupport that is not being linked in to the target
> (llvm-jitlink-executor). I'll aim to break these dependencies on
> libSupport in the future. Mostly that means either removing the
> dependence on llvm::Error / llvm::Expected (e.g. by creating stripped
> down versions for the orc runtime), or making those types header-only.
>
> -- Lang.
>
>
>
>
>
> On Tue, Jan 19, 2021 at 12:50 AM Stefan Gränitz
> <stefan.graenitz at gmail.com <mailto:stefan.graenitz at
gmail.com>> wrote:
>
>     Wow, thanks for the update. One more ORC milestone in a short
>     period of time!
>
>     On macOS I built the C++ example like this:
>
>     % cmake -GNinja -DLLVM_TARGETS_TO_BUILD=host
>     -DLLVM_ENABLE_PROJECTS="clang;compiler-rt" ../llvm
>     % ninja llvm-jitlink llvm-jitlink-executor
>     lib/clang/12.0.0/lib/darwin/libclang_rt.orc_osx.a
>     % clang++ -c -o inits.o inits.cpp
>
>     The in-process version works perfectly, but with the
>     out-of-process flag the examples fails:
>
>     % ./bin/llvm-jitlink inits.o
>     Foo::Foo()
>     Foo::foo()
>     Foo::~Foo()
>     % ./bin/llvm-jitlink -oop-executor inits.o
>     JIT session error: Symbols not found: [
>     __ZTIN4llvm6detail14format_adapterE ]
>
>     Any idea what could go wrong here? Otherwise I can try to debug it
>     later this week. (Full error below.)
>
>     Best
>     Stefan
>
>     --
>
>     JIT session error: Symbols not found: [
>     __ZTIN4llvm6detail14format_adapterE ]
>
/Users/staefsn/Develop/LLVM/monorepo/llvm-orc-runtime/build/bin/llvm-jitlink:
>     Failed to materialize symbols: { (Main, {
>     ___orc_rt_macho_symbol_lookup_remote,
>
__ZN4llvm3orc6shared21WrapperFunctionResult22destroyWithArrayDeleteE39LLVMOrcSharedCWrapperFunctionResultDatay,
>
__ZNSt3__113__vector_baseIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEED2Ev,
>
__ZNK4llvm8ExpectedINS_3orc6shared21WrapperFunctionResultEE22fatalUncheckedExpectedEv,
>
__ZN4llvm3orc6shared21toWrapperFunctionBlobIJNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEEEENS1_21WrapperFunctionResultEDpRKT_,
>     __ZN4llvm3orc6shared20VectorRawByteChannelD1Ev,
>
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJNSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_SA_EE11deserializeISA_JSA_SA_EEENS_5ErrorERS3_RT_DpRT0_,
>
__ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializers13SectionExtentENS_9allocatorIS5_EEE8__appendEm,
>
__ZN4llvm3orc6shared21toWrapperFunctionBlobIJyNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEEEENS1_21WrapperFunctionResultEDpRKT_,
>     __ZN4llvm29VerifyEnableABIBreakingChecksE,
>
__ZN4llvm15format_providerImvE6formatERKmRNS_11raw_ostreamENS_9StringRefE,
>
__ZNSt3__114__split_bufferIN4llvm3orc6shared25MachOJITDylibInitializersERNS_9allocatorIS4_EEED2Ev,
>     __ZTVN4llvm6detail23provider_format_adapterIRmEE,
>
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEEEE9serializeIRKyJRKSA_EEENS_5ErrorERS3_OT_DpOT0_,
>     __ZTVN4llvm6detail23provider_format_adapterImEE,
>
__ZN4llvm6detail23provider_format_adapterImE6formatERNS_11raw_ostreamENS_9StringRefE,
>     __ZN4llvm6detail23provider_format_adapterIRmED1Ev,
>     ___orc_rt_macho_get_deinitializers_tag,
>     __ZN4llvm6detail23provider_format_adapterIRmED0Ev,
>     __ZTVN4llvm3orc6shared14RawByteChannelE,
>     __ZN6orc_rt12jit_dispatchEPKvN4llvm8ArrayRefIhEE,
>     __ZN4llvm3orc6shared20VectorRawByteChannelD0Ev,
>     __ZTVN4llvm3orc6shared20VectorRawByteChannelE,
>     __ZN4llvm8cantFailENS_5ErrorEPKc,
>     __ZN4llvm6detail23provider_format_adapterImED1Ev,
>     __ZTVN4llvm6detail23provider_format_adapterIRjEE,
>     __ZN4llvm6detail23provider_format_adapterImED0Ev,
>
__ZN4llvm15format_providerIjvE6formatERKjRNS_11raw_ostreamENS_9StringRefE,
>     __ZTSN4llvm3orc6shared14RawByteChannelE,
>
__ZN4llvm6detail23provider_format_adapterIRjE6formatERNS_11raw_ostreamENS_9StringRefE,
>     __ZN4llvm6detail23provider_format_adapterIRjED0Ev,
>
__ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__16vectorINS1_25MachOJITDylibInitializersENS4_9allocatorIS6_EEEES9_vE11deserializeERS3_RS9_,
>     __ZTIN4llvm6detail23provider_format_adapterImEE,
>
__ZN4llvm6detail15HelperFunctions15consumeHexStyleERNS_9StringRefERNS_13HexPrintStyleE,
>     __ZTIN4llvm6detail23provider_format_adapterIRmEE,
>     ___orc_rt_macho_get_initializers_tag,
>     __ZTSN4llvm6detail23provider_format_adapterImEE,
>     __ZN4llvm3orc6shared20VectorRawByteChannel4sendEv,
>
__ZN4llvm6detail23provider_format_adapterIRmE6formatERNS_11raw_ostreamENS_9StringRefE,
>     ___orc_rt_macho_symbol_lookup_tag,
>     __ZTSN4llvm6detail23provider_format_adapterIRmEE,
>     __ZN4llvm3orc6shared20VectorRawByteChannel11appendBytesEPKcj,
>     __ZTSN4llvm6detail23provider_format_adapterIRjEE,
>     __ZN4llvm3orc6shared20VectorRawByteChannel9readBytesEPcj,
>     __ZTSN4llvm3orc6shared20VectorRawByteChannelE,
>     __ZN4llvm6detail23provider_format_adapterIRjED1Ev,
>     __ZN4llvm3orc6shared14RawByteChannelD1Ev,
>
__ZN4llvm3orc6shared23fromWrapperFunctionBlobIJNSt3__16vectorINS1_25MachOJITDylibInitializersENS3_9allocatorIS5_EEEEEEENS_5ErrorENS_8ArrayRefIhEEDpRT_,
>     __ZN4llvm3orc6shared14RawByteChannelD0Ev,
>
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyyNSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_SA_EE11deserializeIyJySA_SA_SA_EEENS_5ErrorERS3_RT_DpRT0_,
>
__ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEE8__appendEm,
>
__ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyyEE11deserializeIyJyEEENS_5ErrorERS3_RT_DpRT0_,
>
__ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEE6resizeEm,
>
__ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEESA_vE11deserializeERNS1_14RawByteChannelERSA_,
>
__ZN4llvm3orc6shared23fromWrapperFunctionBlobIJyEEENS_5ErrorENS_8ArrayRefIhEEDpRT_,
>     __ZTIN4llvm6detail23provider_format_adapterIRjEE,
>     __ZN4llvm3orc6shared21WrapperFunctionResult20getAnyOutOfBandErrorEv,
>     __ZTIN4llvm3orc6shared20VectorRawByteChannelE,
>     ___orc_rt_macho_get_initializers_remote,
>     __ZTIN4llvm3orc6shared14RawByteChannelE,
>
__ZNSt3__16vectorIhNS_9allocatorIhEEE26__swap_out_circular_bufferERNS_14__split_bufferIhRS2_EE,
>
__ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_vE11deserializeERS3_RSA_
>     }) }
>
/Users/staefsn/Develop/LLVM/monorepo/llvm-orc-runtime/build/bin/llvm-jitlink-executor:Response
>     has unknown sequence number 527162
>
>     On 18/01/2021 08:55, Lang Hames wrote:
>>     Hi All,
>>
>>     Happy 2021!
>>
>>     I've just posted a new Orc Runtime Preview patch:
>>
https://github.com/lhames/llvm-project/commit/8833a7f24693f1c7a3616438718e7927c6624894
>>
<https://github.com/lhames/llvm-project/commit/8833a7f24693f1c7a3616438718e7927c6624894>
>>
>>     Quick background:
>>
>>     To date, neither ORC nor MCJIT have had their own runtime
>>     libraries. This has limited and complicated the implementation of
>>     many features (e.g. jit re-entry functions, exception handling,
>>     JID'd initializers and de-initializers), and more-or-less
>>     prevented the implementation of others (e.g. native thread local
>>     storage).
>>
>>     Late last year I started work on a prototype ORC runtime library
>>     to address this, and with the above commit I've finally got
>>     something worth sharing.
>>
>>     The prototype above is simultaneously limited and complex.
>>     Limited, in that it only tackles a small subset of the desired
>>     functionality. Complex in that it's one of the most involved
>>     pieces of functionality that I anticipate supporting, as it
>>     requires two-way communication between the executor and JIT
>>     processes. My aim in choosing to tackle the hard part first was
>>     to get a sense of our ultimate requirements for the project,
>>     particularly in regards to /where it should live within the LLVM
>>     Project/. It's not a perfect fit for LLVM proper: there will be
>>     lots of target specific code, including assembly, and it should
>>     be easily buildable for multiple targets (that sounds more like
>>     compiler-rt). On the other hand it's not a perfect fit for
>>     compiler-rt: it shares data structures with LLVM, and it would be
>>     very useful to be able to re-use llvm::Error  / llvm::Expected
>>     (that sounds like LLVM). At the moment I think the best way to
>>     square things would be to keep it in compiler-rt, allow inclusion
>>     of header-only code from LLVM in compiler-rt, and then make Error
>>     / Expected header-only (or copy / adapt them for this library).
>>     This will be a discussion for llvm-dev at some point in the near
>>     future.
>>
>>     On to the actual functionality though: The prototype makes
>>     significant changes to the MachOPlatform class and introduces an
>>     ORC runtime library in compiler-rt/lib/orc. Together, these
>>     changes allow us to emulate the dlopen / dlsym / dlclose in the
>>     JIT executor process. We can use this to define what it means to
>>     run a /JIT program/, rather than just running a JIT function (the
>>     way TargetProcessControl::runAsMain does):
>>
>>
ORC_RT_INTERFACE int64_t __orc_rt_macho_run_program(int argc, char *argv[])
>>     {
>>       using MainTy = int (*)(int, char *[]);
>>
>>       void *H = __orc_rt_macho_jit_dlopen("Main",
ORC_RT_RTLD_LAZY);
>>       if (!H) {
>>         __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>>         return -1;
>>       }
>>
>>
  auto *Main = reinterpret_cast<MainTy>(__orc_rt_macho_jit_dlsym(H, "main"));
>>       if (!Main) {
>>         __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>>         return -1;
>>       }
>>
>>       int Result = Main(argc, argv);
>>
>>       if (__orc_rt_macho_jit_dlclose(H) == -1)
>>         __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>>
>>       return Result;
>>     }
>>
>>     The
>>     functions __orc_rt_macho_jit_dlopen, __orc_rt_macho_jit_dlsym,
>>     and __orc_rt_macho_jit_dlclose behave the same as their dlfcn.h
>>     counterparts (dlopen, dlsym, dlclose), but operate on JITDylibs
>>     rather than regular dylibs. This includes running static
>>     initializers and registering with language runtimes (e.g. ObjC).
>>
>>     While we could run static initializers before (e.g. via
>>     LLJIT::runConstructors), we had to initiate this from the JIT
>>     process side, which has two significant drawbacks: (1) Extra RPC
>>     round trips, and (2) in the out-of-process case: initializers not
>>     running on the executor thread that requested them, since that
>>     thread will be blocked waiting for its call to return. Issue (1)
>>     only affects performance, but (2) can affect correctness if the
>>     initializers modify thread local values, or interact with locks
>>     or threads. Interacting with threads from initializers is
>>     generally best avoided, but nonetheless is done by real-world
>>     code, so we want to support it. By using the runtime we can
>>     improve both performance and correctness (or at least consistency
>>     with current behavior).
>>
>>     The effect of this is that we can now load C++, Objective-C and
>>     Swift programs in the JIT and expect them to run correctly, at
>>     least for simple cases. This works regardless of whether the
>>     JIT'd code runs in-process or out-of-process. To test all this
I
>>     have integrated support for the prototype runtime into
>>     llvm-jitlink. You can demo output from this tool below for two
>>     simple input programs: One swift, one C++. All of this is MachO
>>     specific at the moment, but provides a template that could be
>>     easily re-used to support this on ELF platforms, and likely on
>>     COFF platforms too.
>>
>>     While the discussion on where the runtime should live plays out I
>>     will continue adding / moving functionality to the prototype
>>     runtime. Next up will be eh-frame registration and resolver
>>     functions (both currently in OrcTargetProcess). After that I'll
>>     try to tackle support for native MachO thread local storage.
>>
>>     As always: Questions and comments are very welcome.
>>
>>     -- Lang.
>>
>>     lhames at Langs-MacBook-Pro scratch % cat foo.swift
>>     class MyClass {
>>       func foo() {
>>         print("foo")
>>       }
>>     }
>>
>>     let m = MyClass()
>>     m.foo();
>>
>>     lhames at Langs-MacBook-Pro scratch % xcrun swiftc -emit-object -o
>>     foo.o foo.swift
>>     lhames at Langs-MacBook-Pro scratch % llvm-jitlink -dlopen
>>     /usr/lib/swift/libswiftCore.dylib foo.o
>>     foo
>>     lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor
>>     -dlopen /usr/lib/swift/libswiftCore.dylib foo.o
>>     foo
>>     lhames at Langs-MacBook-Pro scratch % cat inits.cpp
>>     #include <iostream>
>>
>>     class Foo {
>>     public:
>>       Foo() { std::cout << "Foo::Foo()\n"; }
>>       ~Foo() { std::cout << "Foo::~Foo()\n"; }
>>       void foo() { std::cout << "Foo::foo()\n"; }
>>     };
>>
>>     Foo F;
>>
>>     int main(int argc, char *argv[]) {
>>       F.foo();
>>       return 0;
>>     }
>>     lhames at Langs-MacBook-Pro scratch % xcrun clang++ -c -o inits.o
>>     inits.cpp
>>     lhames at Langs-MacBook-Pro scratch % llvm-jitlink inits.o

>>
>>     Foo::Foo()
>>     Foo::foo()
>>     Foo::~Foo()
>>     lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor
inits.o
>>     Foo::Foo()
>>     Foo::foo()
>>     Foo::~Foo()
>
>     --
>     https://flowcrypt.com/pub/stefan.graenitz at gmail.com
<https://flowcrypt.com/pub/stefan.graenitz at gmail.com>
>
--
https://flowcrypt.com/pub/stefan.graenitz at gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210119/dbe194be/attachment-0001.html>