llvm dev - Jul 2020 - How to debug a missing symbol with ThinLTO?

David,

Thanks for looking into this. I did a small reproduction on my machine
outside of my build system. So here is how to reproduce:

Download https://downloads.xiph.org/releases/ogg/libogg-1.3.4.tar.xz
Download llvm-10.0.1 macOS binary

export PATH=<path to llvm/bin>:$PATH
export
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

untar libogg
AR=llvm-ar CC=clang CXX=clang++ CFLAGS="-flto=thin
-mmacosx-version-min=10.9" LDFLAGS=-flto=thin ./configure
--disable-shared --enable-static
make -j20

Then try to link to the library in a small C++ program - I used this:

#include <iostream>
#include "ogg/ogg.h"

using namespace std;

int main()
{
ogg_stream_state os;
if (ogg_stream_init(&os, 123) == 0)
cout << "Initialized stream succesfully" << endl;

return 0;
}

And from the libogg directory I linked to it like this:

clang++ -o test -flto=thin test.cpp src/.libs/libogg.a -I include

undef: _ogg_stream_init
Undefined symbols for architecture x86_64:
"_ogg_stream_init", referenced from:
_main in 0.x86_64.thinlto.o
ld: symbol(s) not found for architecture x86_64
clang-10: error: linker command failed with exit code 1 (use -v to see
invocation)

hope this helps - thanks!

On Wed, Jul 22, 2020 at 9:11 AM David Blaikie <dblaikie at gmail.com>
wrote:
>
> Got a link to the source/build instructions?
>
> This sort of thing happens more often in C++ with templates where one
> object depends (incorrectly) on an implicit instantiation created in
> another object, rather than carrying its own instantiation.
>
> Not sure what might cause it in C code.
>
> On Tue, Jul 21, 2020 at 11:47 PM Tobias Hieta via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > Hello,
> >
> > I am building libogg with clang (10.0.1) on macOS and if I pass
> > "-flto=thin" to C and LDFLAGS it will not link correctly
claiming
> > missing symbols when linking to the archive (libogg.a).
> >
> > undef: _ogg_stream_init
> > Undefined symbols for architecture x86_64:
> > "_ogg_stream_init", referenced from:
> > _main in lto.o
> >
> > Removing lto=thin fixes the problem. Inspecting the AR libs with
> > llvm-nm I see the symbol there (but without address):
> >
> > not working archive:
> > ---------------- T _ogg_stream_init
> >
> > working archive:
> > 0000000000000200 T _ogg_stream_init
> >
> > My guess is that this output is correct since the archive contains
> > bitcode in the thin lto case and otherwise it's the finished
object.
> >
> > It seems to me that the LTO decides to not include this symbol?
It's
> > defined like this:
> >
> > extern int ogg_stream_init(ogg_stream_state *os,int serialno);
> >
> > llvm-ar is used to create the archive.
> >
> > Is there any good way to debug this?
> >
> > Thanks,
> > Tobias
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Adding Steven Wu who can hopefully help as this is MacOS. Unfortunately, I
don't have a way to run the MacOS compiler myself, and it uses the old LTO
API which I am less familiar with. Typically these issues happen if you
don't use llvm-ar, but it looks like you are using that so I'm not sure.

Teresa

On Wed, Jul 22, 2020 at 12:29 AM Tobias Hieta <tobias at plexapp.com>
wrote:
> David,
>
> Thanks for looking into this. I did a small reproduction on my machine
> outside of my build system. So here is how to reproduce:
>
> Download https://downloads.xiph.org/releases/ogg/libogg-1.3.4.tar.xz
> Download llvm-10.0.1 macOS binary
>
> export PATH=<path to llvm/bin>:$PATH
> export
>
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
>
> untar libogg
> AR=llvm-ar CC=clang CXX=clang++ CFLAGS="-flto=thin
> -mmacosx-version-min=10.9" LDFLAGS=-flto=thin ./configure
> --disable-shared --enable-static
> make -j20
>
> Then try to link to the library in a small C++ program - I used this:
>
> #include <iostream>
> #include "ogg/ogg.h"
>
> using namespace std;
>
> int main()
> {
> ogg_stream_state os;
> if (ogg_stream_init(&os, 123) == 0)
> cout << "Initialized stream succesfully" << endl;
>
> return 0;
> }
>
> And from the libogg directory I linked to it like this:
>
> clang++ -o test -flto=thin test.cpp src/.libs/libogg.a -I include
>
> undef: _ogg_stream_init
> Undefined symbols for architecture x86_64:
> "_ogg_stream_init", referenced from:
> _main in 0.x86_64.thinlto.o
> ld: symbol(s) not found for architecture x86_64
> clang-10: error: linker command failed with exit code 1 (use -v to see
> invocation)
>
> hope this helps - thanks!
>
> On Wed, Jul 22, 2020 at 9:11 AM David Blaikie <dblaikie at gmail.com>
wrote:
> >
> > Got a link to the source/build instructions?
> >
> > This sort of thing happens more often in C++ with templates where one
> > object depends (incorrectly) on an implicit instantiation created in
> > another object, rather than carrying its own instantiation.
> >
> > Not sure what might cause it in C code.
> >
> > On Tue, Jul 21, 2020 at 11:47 PM Tobias Hieta via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> > >
> > > Hello,
> > >
> > > I am building libogg with clang (10.0.1) on macOS and if I pass
> > > "-flto=thin" to C and LDFLAGS it will not link
correctly claiming
> > > missing symbols when linking to the archive (libogg.a).
> > >
> > > undef: _ogg_stream_init
> > > Undefined symbols for architecture x86_64:
> > > "_ogg_stream_init", referenced from:
> > > _main in lto.o
> > >
> > > Removing lto=thin fixes the problem. Inspecting the AR libs with
> > > llvm-nm I see the symbol there (but without address):
> > >
> > > not working archive:
> > > ---------------- T _ogg_stream_init
> > >
> > > working archive:
> > > 0000000000000200 T _ogg_stream_init
> > >
> > > My guess is that this output is correct since the archive
contains
> > > bitcode in the thin lto case and otherwise it's the finished
object.
> > >
> > > It seems to me that the LTO decides to not include this symbol?
It's
> > > defined like this:
> > >
> > > extern int ogg_stream_init(ogg_stream_state *os,int serialno);
> > >
> > > llvm-ar is used to create the archive.
> > >
> > > Is there any good way to debug this?
> > >
> > > Thanks,
> > > Tobias
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > llvm-dev at lists.llvm.org
> > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200722/bb601f64/attachment.html>

This is usually a problem that is not using llvm-ar. I cannot reproduce this
problem with either llvm 10.0 or TOT version. Which linker version are you
using? You can also try pass "-Wl,-debug_snapshot" to the command
where the error produces and then locate the "*.ld-snapshot" in /tmp
directory and attach that as a reproducer.

Steven
> On Jul 22, 2020, at 8:41 AM, Teresa Johnson <tejohnson at google.com>
wrote:
> 
> Adding Steven Wu who can hopefully help as this is MacOS. Unfortunately, I
don't have a way to run the MacOS compiler myself, and it uses the old LTO
API which I am less familiar with. Typically these issues happen if you
don't use llvm-ar, but it looks like you are using that so I'm not sure.
> 
> Teresa
> 
> On Wed, Jul 22, 2020 at 12:29 AM Tobias Hieta <tobias at plexapp.com
<mailto:tobias at plexapp.com>> wrote:
> David,
> 
> Thanks for looking into this. I did a small reproduction on my machine
> outside of my build system. So here is how to reproduce:
> 
> Download https://downloads.xiph.org/releases/ogg/libogg-1.3.4.tar.xz
<https://downloads.xiph.org/releases/ogg/libogg-1.3.4.tar.xz>
> Download llvm-10.0.1 macOS binary
> 
> export PATH=<path to llvm/bin>:$PATH
> export
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
> 
> untar libogg
> AR=llvm-ar CC=clang CXX=clang++ CFLAGS="-flto=thin
> -mmacosx-version-min=10.9" LDFLAGS=-flto=thin ./configure
> --disable-shared --enable-static
> make -j20
> 
> Then try to link to the library in a small C++ program - I used this:
> 
> #include <iostream>
> #include "ogg/ogg.h"
> 
> using namespace std;
> 
> int main()
> {
> ogg_stream_state os;
> if (ogg_stream_init(&os, 123) == 0)
> cout << "Initialized stream succesfully" << endl;
> 
> return 0;
> }
> 
> And from the libogg directory I linked to it like this:
> 
> clang++ -o test -flto=thin test.cpp src/.libs/libogg.a -I include
> 
> undef: _ogg_stream_init
> Undefined symbols for architecture x86_64:
> "_ogg_stream_init", referenced from:
> _main in 0.x86_64.thinlto.o
> ld: symbol(s) not found for architecture x86_64
> clang-10: error: linker command failed with exit code 1 (use -v to see
> invocation)
> 
> hope this helps - thanks!
> 
> On Wed, Jul 22, 2020 at 9:11 AM David Blaikie <dblaikie at gmail.com
<mailto:dblaikie at gmail.com>> wrote:
> >
> > Got a link to the source/build instructions?
> >
> > This sort of thing happens more often in C++ with templates where one
> > object depends (incorrectly) on an implicit instantiation created in
> > another object, rather than carrying its own instantiation.
> >
> > Not sure what might cause it in C code.
> >
> > On Tue, Jul 21, 2020 at 11:47 PM Tobias Hieta via llvm-dev
> > <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
> > >
> > > Hello,
> > >
> > > I am building libogg with clang (10.0.1) on macOS and if I pass
> > > "-flto=thin" to C and LDFLAGS it will not link
correctly claiming
> > > missing symbols when linking to the archive (libogg.a).
> > >
> > > undef: _ogg_stream_init
> > > Undefined symbols for architecture x86_64:
> > > "_ogg_stream_init", referenced from:
> > > _main in lto.o
> > >
> > > Removing lto=thin fixes the problem. Inspecting the AR libs with
> > > llvm-nm I see the symbol there (but without address):
> > >
> > > not working archive:
> > > ---------------- T _ogg_stream_init
> > >
> > > working archive:
> > > 0000000000000200 T _ogg_stream_init
> > >
> > > My guess is that this output is correct since the archive
contains
> > > bitcode in the thin lto case and otherwise it's the finished
object.
> > >
> > > It seems to me that the LTO decides to not include this symbol?
It's
> > > defined like this:
> > >
> > > extern int ogg_stream_init(ogg_stream_state *os,int serialno);
> > >
> > > llvm-ar is used to create the archive.
> > >
> > > Is there any good way to debug this?
> > >
> > > Thanks,
> > > Tobias
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
> > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> -- 
> Teresa Johnson |	 Software Engineer |	 tejohnson at google.com
<mailto:tejohnson at google.com> |
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200722/9553e44d/attachment.html>