Greetings, LLVM wizards. We are using Clang and Orc JIT (v1) to compile and execute C++ code on the fly. If a C++ module calls functions from external libraries, we add them via DynamicLibrary::LoadLibraryPermanently(). The problem we have run into recently is when a module calls a function from the STL -- in particular this swap() function for input streams: #include <fstream> std::ifstream stream1, stream2; stream1.swap(stream2); When we run the constructors for the module, we get two undefined symbols. And explicitly adding libstdc++ doesn't help. It turns out that the missing symbols are defined not in the runtime DSO but in an archive file: /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.a So my questions are: 1. Is there a simple way to get access to all symbols defined in the STL? Intuitively, it seems like we should not need to know about such compiler magic. 2. If there is no magical solution, is there a way to explicitly add symbols from an archive? Thanks, Geoff -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190827/be00e98c/attachment.html>
HI Did you run the static constructor and destructor? How did you make your process symbols visible to ORC jit? Could you please share us the for what symbols you get undefined references :-) On Aug 27, 2019 8:18 PM, "Geoff Levner via llvm-dev" < llvm-dev at lists.llvm.org> wrote:> Greetings, LLVM wizards. > > We are using Clang and Orc JIT (v1) to compile and execute C++ code on the > fly. If a C++ module calls functions from external libraries, we add them > via DynamicLibrary::LoadLibraryPermanently(). > > The problem we have run into recently is when a module calls a function > from the STL -- in particular this swap() function for input streams: > > #include <fstream> > std::ifstream stream1, stream2; > stream1.swap(stream2); > > When we run the constructors for the module, we get two undefined symbols. > And explicitly adding libstdc++ doesn't help. It turns out that the missing > symbols are defined not in the runtime DSO but in an archive file: > > /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6. > 3.1/libstdc++.a > > So my questions are: > > 1. Is there a simple way to get access to all symbols defined in the STL? > Intuitively, it seems like we should not need to know about such compiler > magic. > > 2. If there is no magical solution, is there a way to explicitly add > symbols from an archive? > > Thanks, > Geoff > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190827/51fefac3/attachment.html>
On Tue, Aug 27, 2019 at 4:56 PM Praveen Velliengiri <praveenvelliengiri at gmail.com> wrote:> > HI > Did you run the static constructor and destructor? How did you make your process symbols visible to ORC jit?Yes. It's the constructor that generates the undefined symbol error. We use DynamicLibrary::LoadLibraryPermanently(nullptr) to add process symbols.> Could you please share us the for what symbols you get undefined references :-)Certainly! Mangled: _ZNSi4swapERSi _ZNSt13basic_filebufIcSt11char_traitsIcEE4swapERS2_ And unmangled: std::basic_istream<char, std::char_traits<char>>::swap(std::basic_istream<char, std::char_traits<char> >&)std::basic_filebuf<char, std::char_traits<char>>::swap(std::basic_filebuf<char, std::char_traits<char> >&)Incidentally, if I call that STL swap() function in the application, to ensure it is in the process symbols, the second symbol is found, but the first is still undefined.> > On Aug 27, 2019 8:18 PM, "Geoff Levner via llvm-dev" <llvm-dev at lists.llvm.org> wrote: >> >> Greetings, LLVM wizards. >> >> We are using Clang and Orc JIT (v1) to compile and execute C++ code on the fly. If a C++ module calls functions from external libraries, we add them via DynamicLibrary::LoadLibraryPermanently(). >> >> The problem we have run into recently is when a module calls a function from the STL -- in particular this swap() function for input streams: >> >> #include <fstream> >> std::ifstream stream1, stream2; >> stream1.swap(stream2); >> >> When we run the constructors for the module, we get two undefined symbols. And explicitly adding libstdc++ doesn't help. It turns out that the missing symbols are defined not in the runtime DSO but in an archive file: >> >> /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.a >> >> So my questions are: >> >> 1. Is there a simple way to get access to all symbols defined in the STL? Intuitively, it seems like we should not need to know about such compiler magic. >> >> 2. If there is no magical solution, is there a way to explicitly add symbols from an archive? >> >> Thanks, >> Geoff >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>
[I am reposting this with a different title and other changes, because I am
fairly confident our problem is related to Red Hat's developer toolset.]
Greetings, LLVM wizards.
We have an application that uses Clang and Orc JIT (v1) to compile and
execute C++ code on the fly. If a C++ module calls functions from external
libraries, we add them via DynamicLibrary::LoadLibraryPermanently().
Recently we moved to gcc 6.3.1 to build our application, using Red Hat's
devtoolset-6 on CentOS, and this seems to create problems when a module
calls functions from the STL. If a module calls this swap() function for
input streams, for example:
#include <fstream>
std::ifstream stream1, stream2;
stream1.swap(stream2);
When we run the constructors for the module, we get undefined STL symbols.
It turns out that the missing symbols are defined not in the runtime DSO
(in /usr/lib64) but in an archive file installed with the developer toolset:
/opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.a
Apparently the linker performs some magic allowing an application compiled
and linked with gcc 6.3.1 to run on a system with an older version of the
STL (from gcc 4.8.5), augmenting the old DSO with stuff that is linked in
statically.
SO my question is: is there any way to reproduce that magical behavior in
an Orc JIT compiler, so that code can link properly with the STL?
Thanks,
Geoff
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190828/58477595/attachment.html>
The libstdc++.so file that comes with devtoolset-6 is actually not a DSO
but a small text file containing this:
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf64-x86-64)
INPUT ( /usr/lib64/libstdc++.so.6 -lstdc++_nonshared )
So the question, I think, is: what exactly does the GNU linker do with this
information, and is there a way to replicate its behavior in Orc JIT?
On Wed, Aug 28, 2019 at 9:16 AM Geoff Levner <glevner at gmail.com> wrote:
> [I am reposting this with a different title and other changes, because I
> am fairly confident our problem is related to Red Hat's developer
toolset.]
>
> Greetings, LLVM wizards.
>
> We have an application that uses Clang and Orc JIT (v1) to compile and
> execute C++ code on the fly. If a C++ module calls functions from external
> libraries, we add them via DynamicLibrary::LoadLibraryPermanently().
>
> Recently we moved to gcc 6.3.1 to build our application, using Red
Hat's
> devtoolset-6 on CentOS, and this seems to create problems when a module
> calls functions from the STL. If a module calls this swap() function for
> input streams, for example:
>
> #include <fstream>
> std::ifstream stream1, stream2;
> stream1.swap(stream2);
>
> When we run the constructors for the module, we get undefined STL symbols.
> It turns out that the missing symbols are defined not in the runtime DSO
> (in /usr/lib64) but in an archive file installed with the developer
toolset:
>
>
> /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.a
>
> Apparently the linker performs some magic allowing an application compiled
> and linked with gcc 6.3.1 to run on a system with an older version of the
> STL (from gcc 4.8.5), augmenting the old DSO with stuff that is linked in
> statically.
>
> SO my question is: is there any way to reproduce that magical behavior in
> an Orc JIT compiler, so that code can link properly with the STL?
>
> Thanks,
> Geoff
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190828/6bcd6c6d/attachment.html>