[llvm-dev] (Orc)JIT and weak symbol resolution

Lang Hames via llvm-dev
2016-05-02T17:42:54Z

Hi Axel,

Sorry for the delay. Attached is the patch for RuntimeDyld & MCJIT (plus a
proof-of-concept MCJIT setup that uses it). This was just a quick hack to
work around this issue in MCJIT - it needs some polish, and it hasn't been
tested with ORC yet. I think the full solution should look more like this:

1) In loadObject all symbols in the object should be added to the
RuntimeDyld instance's symbol table, each marked weak or not as appropriate.
2) In RuntimeDyldImpl::loadObjectImpl weak symbols should be treated as if
they were externals - i.e. we add symbolic relocations for each of them.
3) In RuntimeDyldImpl::resolveRelocations, for each weak symbol we first
search the logical dylib for an existing strong definition. If we find one
we use that, and remove the weak definition from the local table entirely.
If we don't find an existing strong definition we mark our local definition
as strong and use it.

Are you interested in implementing this?

Cheers,
Lang.


On Fri, Apr 29, 2016 at 9:47 AM, Lang Hames <lhames at gmail.com> wrote:

> Hi Axel,
>
> Yep - RuntimeDyld does not handle weak symbols properly at the moment.
> About a month back I ran into a bug that I thought was due to this lack of
> weak symbol support (but that actually turned out to be something else) and
> wrote up a preliminary patch to fix it. When I get in to the office in a
> couple of hours I'll dig that patch up and see what kind of state
it's in -
> it should serve as a good starting point.
>
> - Lang.
>
> On Fri, Apr 29, 2016 at 5:20 AM, Axel Naumann via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> This is a question on how to resolve weak symbols from the binary for
>> symbols that are also llvm::Module-local. Currently, the JIT seems to
>> favor resolving to module-local symbols over existing symbols:
>>
>> $ cat symbols.cxx
>> extern "C" int printf(const char*,...);
>>
>> template <class T> struct StaticStuff {
>>   static T s_data;
>> };
>> template <class T> T StaticStuff<T>::s_data = 42;
>>
>> int compareAddr(int* interp);
>> #ifdef BUILD_SHARED
>> int compareAddr(int* interp) {
>>   if (interp != &StaticStuff<int>::s_data) {
>>     printf("Wrong address, %ld in shared lib, %ld in
interpreter!\n",
>>            (long)&StaticStuff<int>::s_data,
>>            (long)interp);
>>     // CHECK-NOT: Wrong address
>>     return 1;
>>   }
>>   return 0;
>> }
>> #else
>> # ifdef __CLING__
>> int Symbols() {
>> # else
>> int main()
>> # endif
>> {  return compareAddr(&StaticStuff<int>::s_data);}
>> #endif // ! BUILD_SHARED
>>
>>
>> $ clang++ -shared -fPIC -DBUILD_SHARED symbols.cxx -o libSymbols.so
>> $ clang++ -cc1 -emit-llvm symbols.cxx -o - |  lli -load ./libSymbols.so
-
>> or
>> $ clang++ -lSymbols symbols.cxx && ./a.out
>>
>> Compiled it's happy: the weak symbol used in main gets resolved to
be
>> the one in the binary for both references. That's the behavior
I'd expect.
>>
>> The JIT OTOH is deciding that it shall use the local version of the
>> symbol; RuntimeDyld::getSymbol() doesn't care about the fact that
the
>> symbol is weak and has an existing occurrence.
>>
>> For weak symbols from a different module, RuntimeDyld would use
>> resolveExternalSymbols() and all would be good - but here this
doesn't
>> happen.
>>
>> IIUC, RuntimeDyld::getSymbol() really needs to check whether the local
>> symbol should survive or not, before handing out its address. But that
>> seems to break the whole design of when relocs happen, so I'm sure
I'm
>> wrong. Can I get some hints on how to implement the proper patch?
>>
>> Cheers, Axel.
>>
>> --
>> ROOT - http://root.cern.ch
>> PH-SFT, CERN, 1211 Geneve 23, Switzerland
>> Tel: +41 22 7678225
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160502/8176d5f9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: multi_mcjit_weak_symbols.tgz
Type: application/x-gzip
Size: 1890 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160502/8176d5f9/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rtdyld-weak-symbols-hack.patch
Type: application/octet-stream
Size: 5781 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160502/8176d5f9/attachment.obj>