Scott Smith via llvm-dev
2017-Dec-12 00:04 UTC
[llvm-dev] [lld] bug detecting undefined symbols in shared libraries
With shared libraries, it is always possible that you end up running with a different set of libraries than you actually linked with. However, how often is that actually the case? It seems at the very least there should be an option to tell the user that unless they switch libraries, their code won't be runnable. In most cases, everything is available to the linker to detect the situation (as both ld and gold do). gold has a set of options to vary their behavior - --[no-]allow-shlib-undefined, --no-undefined, -z defs, --warn-unresolved-symbols, --error-unresolved-symbols, --unresolved-symbols ignore-all/report-all/ignore-in-object-files/ignore-in-shared-libs. It seems that lld is implementing --allow-shlib-undefined (test1 builds, test2 fails to build). Is it possible for lld to implement --[no-]allow-shlib-undefined to switch the behavior for undefined symbols in shared libraries? $ make clean && make -k rm -f main.o virtual.o virtual.so test1 test2 clang++-5.0 -fPIC -c main.cc clang++-5.0 -fPIC -c virtual.cc clang++-5.0 -fPIC -fuse-ld=gold -Wl,--allow-shlib-undefined -shared -o virtual.so virtual.o clang++-5.0 -fPIC -fuse-ld=gold -Wl,--allow-shlib-undefined -Wl,-R$ORIGIN. -o test1 main.o virtual.so clang++-5.0 -fPIC -fuse-ld=gold -Wl,--allow-shlib-undefined -Wl,-R$ORIGIN. -o test2 main.o virtual.o virtual.o:virtual.cc:vtable for foo: error: undefined reference to 'foo::bar()' clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [test2] Error 1 make: Target `all' not remade because of errors. On Mon, Dec 11, 2017 at 3:54 PM, Rui Ueyama <ruiu at google.com> wrote:> Hi Scott, > > I think you are saying that lld doesn't report an error for undefined > symbols that exist in a .so which cannot be resolved at link-time. That's > intentional. Trying to resolve undefined symbols in a .so at link-time > doesn't make sense because we can just link fine without resolving such > symbols, and the static linker doesn't really know whether it's actually an > error condition or not. The real failure could only happen at load-time > when the undefined symbols cannot be resolved using other DSOs. So we do > not care about undefined symbols in DSOs in most cases. > > On Mon, Dec 11, 2017 at 3:39 PM, Scott Smith via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> I have a test case where lld-5.0 fails to detect an undefined symbol >> (this bug also happens with clang/lld 4.0). I haven't narrowed down >> exactly all the circumstances when this can occur, but in this case it was >> for a virtual method in a class defined in a shared library. If I build >> the executable with the raw object files, the linker notices the missing >> method (see test1 vs test2). This was tested on Ubuntu 14.04 using the >> llvm/clang-5.0 downloaded from the official apt repository. >> >> If I use gold (from binutils 2.26), it finds the error during link time. >> >> >> ********** DEMO OUTPUT >> $ make clean && make >> rm -f main.o virtual.o virtual.so test1 test2 >> clang++-5.0 -fPIC -c main.cc >> clang++-5.0 -fPIC -c virtual.cc >> clang++-5.0 -fPIC -fuse-ld=lld-5.0 -shared -o virtual.so virtual.o >> clang++-5.0 -fPIC -fuse-ld=lld-5.0 -Wl,-R$ORIGIN. -o test1 main.o >> virtual.so >> clang++-5.0 -fPIC -fuse-ld=lld-5.0 -Wl,-R$ORIGIN. -o test2 main.o >> virtual.o >> /usr/bin/ld.lld-5.0: error: undefined symbol: foo::bar() >> >>> referenced by virtual.cc >> >>> virtual.o:(vtable for foo) >> clang: error: linker command failed with exit code 1 (use -v to see >> invocation) >> make: *** [test2] Error 1 >> >> Note that running test1 results in an error: >> $ ./test1 >> ./test1: symbol lookup error: ./virtual.so: undefined symbol: >> _ZN3foo3barEv >> >> ********** CODE main.cc >> #include "virtual.h" >> int main() { >> foo baz; >> return 0; >> } >> >> ********** CODE virtual.h >> struct foo { >> virtual ~foo(); >> virtual void bar(); >> }; >> >> ********** CODE virtual.cc >> #include "virtual.h" >> foo::~foo() { } >> >> ********* CODE Makefile >> all: test1 test2 >> >> CC = clang++-5.0 -fPIC >> LD = ${CC} -fuse-ld=lld-5.0 >> >> test1: main.o virtual.so >> ${LD} -Wl,-R$$ORIGIN. -o test1 main.o virtual.so >> >> test2: main.o virtual.o >> ${LD} -Wl,-R$$ORIGIN. -o test2 main.o virtual.o >> >> virtual.o: virtual.cc virtual.h >> ${CC} -c virtual.cc >> >> virtual.so: virtual.o >> ${LD} -shared -o virtual.so virtual.o >> >> main.o: main.cc virtual.h >> ${CC} -c main.cc >> >> clean: >> rm -f main.o virtual.o virtual.so test1 test2 >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171211/b8aad5d8/attachment.html>
Rui Ueyama via llvm-dev
2017-Dec-12 00:36 UTC
[llvm-dev] [lld] bug detecting undefined symbols in shared libraries
Sure. I understand why you want it. I don't want to make a conclusion right now, but at least we should evaluate the impact of adding --no-allow-shlib-undefined to lld. Filed your feature request as https://bugs.llvm.org/show_bug.cgi?id=35635 to keep this in my todo list. On Mon, Dec 11, 2017 at 4:04 PM, Scott Smith <scott.smith at purestorage.com> wrote:> With shared libraries, it is always possible that you end up running with > a different set of libraries than you actually linked with. However, how > often is that actually the case? It seems at the very least there should > be an option to tell the user that unless they switch libraries, their code > won't be runnable. In most cases, everything is available to the linker to > detect the situation (as both ld and gold do). > > gold has a set of options to vary their behavior - > --[no-]allow-shlib-undefined, --no-undefined, -z defs, > --warn-unresolved-symbols, --error-unresolved-symbols, --unresolved-symbols > ignore-all/report-all/ignore-in-object-files/ignore-in-shared-libs. It > seems that lld is implementing --allow-shlib-undefined (test1 builds, test2 > fails to build). Is it possible for lld to implement > --[no-]allow-shlib-undefined to switch the behavior for undefined symbols > in shared libraries? > > $ make clean && make -k > rm -f main.o virtual.o virtual.so test1 test2 > clang++-5.0 -fPIC -c main.cc > clang++-5.0 -fPIC -c virtual.cc > clang++-5.0 -fPIC -fuse-ld=gold -Wl,--allow-shlib-undefined -shared -o > virtual.so virtual.o > clang++-5.0 -fPIC -fuse-ld=gold -Wl,--allow-shlib-undefined -Wl,-R$ORIGIN. > -o test1 main.o virtual.so > clang++-5.0 -fPIC -fuse-ld=gold -Wl,--allow-shlib-undefined -Wl,-R$ORIGIN. > -o test2 main.o virtual.o > virtual.o:virtual.cc:vtable for foo: error: undefined reference to > 'foo::bar()' > clang: error: linker command failed with exit code 1 (use -v to see > invocation) > make: *** [test2] Error 1 > make: Target `all' not remade because of errors. > > > On Mon, Dec 11, 2017 at 3:54 PM, Rui Ueyama <ruiu at google.com> wrote: > >> Hi Scott, >> >> I think you are saying that lld doesn't report an error for undefined >> symbols that exist in a .so which cannot be resolved at link-time. That's >> intentional. Trying to resolve undefined symbols in a .so at link-time >> doesn't make sense because we can just link fine without resolving such >> symbols, and the static linker doesn't really know whether it's actually an >> error condition or not. The real failure could only happen at load-time >> when the undefined symbols cannot be resolved using other DSOs. So we do >> not care about undefined symbols in DSOs in most cases. >> >> On Mon, Dec 11, 2017 at 3:39 PM, Scott Smith via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> I have a test case where lld-5.0 fails to detect an undefined symbol >>> (this bug also happens with clang/lld 4.0). I haven't narrowed down >>> exactly all the circumstances when this can occur, but in this case it was >>> for a virtual method in a class defined in a shared library. If I build >>> the executable with the raw object files, the linker notices the missing >>> method (see test1 vs test2). This was tested on Ubuntu 14.04 using the >>> llvm/clang-5.0 downloaded from the official apt repository. >>> >>> If I use gold (from binutils 2.26), it finds the error during link time. >>> >>> >>> ********** DEMO OUTPUT >>> $ make clean && make >>> rm -f main.o virtual.o virtual.so test1 test2 >>> clang++-5.0 -fPIC -c main.cc >>> clang++-5.0 -fPIC -c virtual.cc >>> clang++-5.0 -fPIC -fuse-ld=lld-5.0 -shared -o virtual.so virtual.o >>> clang++-5.0 -fPIC -fuse-ld=lld-5.0 -Wl,-R$ORIGIN. -o test1 main.o >>> virtual.so >>> clang++-5.0 -fPIC -fuse-ld=lld-5.0 -Wl,-R$ORIGIN. -o test2 main.o >>> virtual.o >>> /usr/bin/ld.lld-5.0: error: undefined symbol: foo::bar() >>> >>> referenced by virtual.cc >>> >>> virtual.o:(vtable for foo) >>> clang: error: linker command failed with exit code 1 (use -v to see >>> invocation) >>> make: *** [test2] Error 1 >>> >>> Note that running test1 results in an error: >>> $ ./test1 >>> ./test1: symbol lookup error: ./virtual.so: undefined symbol: >>> _ZN3foo3barEv >>> >>> ********** CODE main.cc >>> #include "virtual.h" >>> int main() { >>> foo baz; >>> return 0; >>> } >>> >>> ********** CODE virtual.h >>> struct foo { >>> virtual ~foo(); >>> virtual void bar(); >>> }; >>> >>> ********** CODE virtual.cc >>> #include "virtual.h" >>> foo::~foo() { } >>> >>> ********* CODE Makefile >>> all: test1 test2 >>> >>> CC = clang++-5.0 -fPIC >>> LD = ${CC} -fuse-ld=lld-5.0 >>> >>> test1: main.o virtual.so >>> ${LD} -Wl,-R$$ORIGIN. -o test1 main.o virtual.so >>> >>> test2: main.o virtual.o >>> ${LD} -Wl,-R$$ORIGIN. -o test2 main.o virtual.o >>> >>> virtual.o: virtual.cc virtual.h >>> ${CC} -c virtual.cc >>> >>> virtual.so: virtual.o >>> ${LD} -shared -o virtual.so virtual.o >>> >>> main.o: main.cc virtual.h >>> ${CC} -c main.cc >>> >>> clean: >>> rm -f main.o virtual.o virtual.so test1 test2 >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171211/ce245e41/attachment.html>
Rafael Avila de Espindola via llvm-dev
2017-Dec-15 04:03 UTC
[llvm-dev] [lld] bug detecting undefined symbols in shared libraries
Scott Smith via llvm-dev <llvm-dev at lists.llvm.org> writes:> With shared libraries, it is always possible that you end up running with a > different set of libraries than you actually linked with. However, how > often is that actually the case? It seems at the very least there should > be an option to tell the user that unless they switch libraries, their code > won't be runnable. In most cases, everything is available to the linker to > detect the situation (as both ld and gold do).Your best option for checking that during link time is to use -z defs when linking the shared library. Cheers, Rafael
Scott Smith via llvm-dev
2017-Dec-15 05:00 UTC
[llvm-dev] [lld] bug detecting undefined symbols in shared libraries
I tried that. It doesn't work in my application due to odd circular references between shared libraries. I need the validation to happen at executable link time, not shared library link time. On Thu, Dec 14, 2017 at 8:03 PM, Rafael Avila de Espindola < rafael.espindola at gmail.com> wrote:> Scott Smith via llvm-dev <llvm-dev at lists.llvm.org> writes: > > > With shared libraries, it is always possible that you end up running > with a > > different set of libraries than you actually linked with. However, how > > often is that actually the case? It seems at the very least there should > > be an option to tell the user that unless they switch libraries, their > code > > won't be runnable. In most cases, everything is available to the linker > to > > detect the situation (as both ld and gold do). > > Your best option for checking that during link time is to use -z defs > when linking the shared library. > > Cheers, > Rafael >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171214/3e8a67a6/attachment-0001.html>