llvm dev - Apr 2016 - (Orc)JIT and weak symbol resolution

Hi,

This is a question on how to resolve weak symbols from the binary for
symbols that are also llvm::Module-local. Currently, the JIT seems to
favor resolving to module-local symbols over existing symbols:

$ cat symbols.cxx
extern "C" int printf(const char*,...);

template <class T> struct StaticStuff {
  static T s_data;
};
template <class T> T StaticStuff<T>::s_data = 42;

int compareAddr(int* interp);
#ifdef BUILD_SHARED
int compareAddr(int* interp) {
  if (interp != &StaticStuff<int>::s_data) {
    printf("Wrong address, %ld in shared lib, %ld in interpreter!\n",
           (long)&StaticStuff<int>::s_data,
           (long)interp);
    // CHECK-NOT: Wrong address
    return 1;
  }
  return 0;
}
#else
# ifdef __CLING__
int Symbols() {
# else
int main()
# endif
{  return compareAddr(&StaticStuff<int>::s_data);}
#endif // ! BUILD_SHARED


$ clang++ -shared -fPIC -DBUILD_SHARED symbols.cxx -o libSymbols.so
$ clang++ -cc1 -emit-llvm symbols.cxx -o - |  lli -load ./libSymbols.so -
or
$ clang++ -lSymbols symbols.cxx && ./a.out

Compiled it's happy: the weak symbol used in main gets resolved to be
the one in the binary for both references. That's the behavior I'd
expect.

The JIT OTOH is deciding that it shall use the local version of the
symbol; RuntimeDyld::getSymbol() doesn't care about the fact that the
symbol is weak and has an existing occurrence.

For weak symbols from a different module, RuntimeDyld would use
resolveExternalSymbols() and all would be good - but here this doesn't
happen.

IIUC, RuntimeDyld::getSymbol() really needs to check whether the local
symbol should survive or not, before handing out its address. But that
seems to break the whole design of when relocs happen, so I'm sure I'm
wrong. Can I get some hints on how to implement the proper patch?

Cheers, Axel. 

--
ROOT - http://root.cern.ch
PH-SFT, CERN, 1211 Geneve 23, Switzerland
Tel: +41 22 7678225

+Lang, many layered architect of JITs, for JIT questions :)

On Fri, Apr 29, 2016 at 5:20 AM, Axel Naumann via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> This is a question on how to resolve weak symbols from the binary for
> symbols that are also llvm::Module-local. Currently, the JIT seems to
> favor resolving to module-local symbols over existing symbols:
>
> $ cat symbols.cxx
> extern "C" int printf(const char*,...);
>
> template <class T> struct StaticStuff {
>   static T s_data;
> };
> template <class T> T StaticStuff<T>::s_data = 42;
>
> int compareAddr(int* interp);
> #ifdef BUILD_SHARED
> int compareAddr(int* interp) {
>   if (interp != &StaticStuff<int>::s_data) {
>     printf("Wrong address, %ld in shared lib, %ld in
interpreter!\n",
>            (long)&StaticStuff<int>::s_data,
>            (long)interp);
>     // CHECK-NOT: Wrong address
>     return 1;
>   }
>   return 0;
> }
> #else
> # ifdef __CLING__
> int Symbols() {
> # else
> int main()
> # endif
> {  return compareAddr(&StaticStuff<int>::s_data);}
> #endif // ! BUILD_SHARED
>
>
> $ clang++ -shared -fPIC -DBUILD_SHARED symbols.cxx -o libSymbols.so
> $ clang++ -cc1 -emit-llvm symbols.cxx -o - |  lli -load ./libSymbols.so -
> or
> $ clang++ -lSymbols symbols.cxx && ./a.out
>
> Compiled it's happy: the weak symbol used in main gets resolved to be
> the one in the binary for both references. That's the behavior I'd
expect.
>
> The JIT OTOH is deciding that it shall use the local version of the
> symbol; RuntimeDyld::getSymbol() doesn't care about the fact that the
> symbol is weak and has an existing occurrence.
>
> For weak symbols from a different module, RuntimeDyld would use
> resolveExternalSymbols() and all would be good - but here this doesn't
> happen.
>
> IIUC, RuntimeDyld::getSymbol() really needs to check whether the local
> symbol should survive or not, before handing out its address. But that
> seems to break the whole design of when relocs happen, so I'm sure
I'm
> wrong. Can I get some hints on how to implement the proper patch?
>
> Cheers, Axel.
>
> --
> ROOT - http://root.cern.ch
> PH-SFT, CERN, 1211 Geneve 23, Switzerland
> Tel: +41 22 7678225
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160429/2cda54a5/attachment.html>

Hi Axel,

Yep - RuntimeDyld does not handle weak symbols properly at the moment.
About a month back I ran into a bug that I thought was due to this lack of
weak symbol support (but that actually turned out to be something else) and
wrote up a preliminary patch to fix it. When I get in to the office in a
couple of hours I'll dig that patch up and see what kind of state it's
in -
it should serve as a good starting point.

- Lang.

On Fri, Apr 29, 2016 at 5:20 AM, Axel Naumann via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> This is a question on how to resolve weak symbols from the binary for
> symbols that are also llvm::Module-local. Currently, the JIT seems to
> favor resolving to module-local symbols over existing symbols:
>
> $ cat symbols.cxx
> extern "C" int printf(const char*,...);
>
> template <class T> struct StaticStuff {
>   static T s_data;
> };
> template <class T> T StaticStuff<T>::s_data = 42;
>
> int compareAddr(int* interp);
> #ifdef BUILD_SHARED
> int compareAddr(int* interp) {
>   if (interp != &StaticStuff<int>::s_data) {
>     printf("Wrong address, %ld in shared lib, %ld in
interpreter!\n",
>            (long)&StaticStuff<int>::s_data,
>            (long)interp);
>     // CHECK-NOT: Wrong address
>     return 1;
>   }
>   return 0;
> }
> #else
> # ifdef __CLING__
> int Symbols() {
> # else
> int main()
> # endif
> {  return compareAddr(&StaticStuff<int>::s_data);}
> #endif // ! BUILD_SHARED
>
>
> $ clang++ -shared -fPIC -DBUILD_SHARED symbols.cxx -o libSymbols.so
> $ clang++ -cc1 -emit-llvm symbols.cxx -o - |  lli -load ./libSymbols.so -
> or
> $ clang++ -lSymbols symbols.cxx && ./a.out
>
> Compiled it's happy: the weak symbol used in main gets resolved to be
> the one in the binary for both references. That's the behavior I'd
expect.
>
> The JIT OTOH is deciding that it shall use the local version of the
> symbol; RuntimeDyld::getSymbol() doesn't care about the fact that the
> symbol is weak and has an existing occurrence.
>
> For weak symbols from a different module, RuntimeDyld would use
> resolveExternalSymbols() and all would be good - but here this doesn't
> happen.
>
> IIUC, RuntimeDyld::getSymbol() really needs to check whether the local
> symbol should survive or not, before handing out its address. But that
> seems to break the whole design of when relocs happen, so I'm sure
I'm
> wrong. Can I get some hints on how to implement the proper patch?
>
> Cheers, Axel.
>
> --
> ROOT - http://root.cern.ch
> PH-SFT, CERN, 1211 Geneve 23, Switzerland
> Tel: +41 22 7678225
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160429/38de83ba/attachment.html>

Hi Axel,

Sorry for the delay. Attached is the patch for RuntimeDyld & MCJIT (plus a
proof-of-concept MCJIT setup that uses it). This was just a quick hack to
work around this issue in MCJIT - it needs some polish, and it hasn't been
tested with ORC yet. I think the full solution should look more like this:

1) In loadObject all symbols in the object should be added to the
RuntimeDyld instance's symbol table, each marked weak or not as appropriate.
2) In RuntimeDyldImpl::loadObjectImpl weak symbols should be treated as if
they were externals - i.e. we add symbolic relocations for each of them.
3) In RuntimeDyldImpl::resolveRelocations, for each weak symbol we first
search the logical dylib for an existing strong definition. If we find one
we use that, and remove the weak definition from the local table entirely.
If we don't find an existing strong definition we mark our local definition
as strong and use it.

Are you interested in implementing this?

Cheers,
Lang.


On Fri, Apr 29, 2016 at 9:47 AM, Lang Hames <lhames at gmail.com> wrote:
> Hi Axel,
>
> Yep - RuntimeDyld does not handle weak symbols properly at the moment.
> About a month back I ran into a bug that I thought was due to this lack of
> weak symbol support (but that actually turned out to be something else) and
> wrote up a preliminary patch to fix it. When I get in to the office in a
> couple of hours I'll dig that patch up and see what kind of state
it's in -
> it should serve as a good starting point.
>
> - Lang.
>
> On Fri, Apr 29, 2016 at 5:20 AM, Axel Naumann via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> This is a question on how to resolve weak symbols from the binary for
>> symbols that are also llvm::Module-local. Currently, the JIT seems to
>> favor resolving to module-local symbols over existing symbols:
>>
>> $ cat symbols.cxx
>> extern "C" int printf(const char*,...);
>>
>> template <class T> struct StaticStuff {
>>   static T s_data;
>> };
>> template <class T> T StaticStuff<T>::s_data = 42;
>>
>> int compareAddr(int* interp);
>> #ifdef BUILD_SHARED
>> int compareAddr(int* interp) {
>>   if (interp != &StaticStuff<int>::s_data) {
>>     printf("Wrong address, %ld in shared lib, %ld in
interpreter!\n",
>>            (long)&StaticStuff<int>::s_data,
>>            (long)interp);
>>     // CHECK-NOT: Wrong address
>>     return 1;
>>   }
>>   return 0;
>> }
>> #else
>> # ifdef __CLING__
>> int Symbols() {
>> # else
>> int main()
>> # endif
>> {  return compareAddr(&StaticStuff<int>::s_data);}
>> #endif // ! BUILD_SHARED
>>
>>
>> $ clang++ -shared -fPIC -DBUILD_SHARED symbols.cxx -o libSymbols.so
>> $ clang++ -cc1 -emit-llvm symbols.cxx -o - |  lli -load ./libSymbols.so
-
>> or
>> $ clang++ -lSymbols symbols.cxx && ./a.out
>>
>> Compiled it's happy: the weak symbol used in main gets resolved to
be
>> the one in the binary for both references. That's the behavior
I'd expect.
>>
>> The JIT OTOH is deciding that it shall use the local version of the
>> symbol; RuntimeDyld::getSymbol() doesn't care about the fact that
the
>> symbol is weak and has an existing occurrence.
>>
>> For weak symbols from a different module, RuntimeDyld would use
>> resolveExternalSymbols() and all would be good - but here this
doesn't
>> happen.
>>
>> IIUC, RuntimeDyld::getSymbol() really needs to check whether the local
>> symbol should survive or not, before handing out its address. But that
>> seems to break the whole design of when relocs happen, so I'm sure
I'm
>> wrong. Can I get some hints on how to implement the proper patch?
>>
>> Cheers, Axel.
>>
>> --
>> ROOT - http://root.cern.ch
>> PH-SFT, CERN, 1211 Geneve 23, Switzerland
>> Tel: +41 22 7678225
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160502/8176d5f9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: multi_mcjit_weak_symbols.tgz
Type: application/x-gzip
Size: 1890 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160502/8176d5f9/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rtdyld-weak-symbols-hack.patch
Type: application/octet-stream
Size: 5781 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160502/8176d5f9/attachment.obj>