llvm dev - Jun 2016 - Linker failures in debug build - compiler/linker poll?

This probably just works around a layering violation. Can you provide the
cmake line that produces a broken build?

Cheers,
Rafael
On Jun 18, 2016 7:34 AM, "Nicolai Hähnle" <llvm-dev at
lists.llvm.org> wrote:

Hi,

since recently I'm getting linker failures in debug builds. The root cause
is that -fvisibility-inlines-hidden causes inline functions in explicit
template instantiations to become hidden in certain cases.

The following hunk works around this...

--- a/cmake/modules/HandleLLVMOptions.cmake
+++ b/cmake/modules/HandleLLVMOptions.cmake
@@ -159,7 +159,13 @@ endif()
 if(NOT WIN32 AND NOT CYGWIN)
   # MinGW warns if -fvisibility-inlines-hidden is used.
   check_cxx_compiler_flag("-fvisibility-inlines-hidden"
SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG)
-  append_if(SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG
"-fvisibility-inlines-hidden" CMAKE_CXX_FLAGS)
+
+  # The flag also hides inlines from explicit template instantiations,
which
+  # leads to link failures in debug builds at least with gcc 5.3.1 and
+  # both GNU ld 2.26 or gold 1.11.
+  if (NOT uppercase_CMAKE_BUILD_TYPE STREQUAL "DEBUG")
+    append_if(SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG
"-fvisibility-inlines-hidden" CMAKE_CXX_FLAGS)
+  endif()
 endif()

 if( CMAKE_SIZEOF_VOID_P EQUAL 8 AND NOT WIN32 )

... but before I just wildly commit something, I'd say this does seem more
like a compiler bug to me and so I'd like to poll people about which
compilers they're using to see where this appears.

FWIW, gcc 5.3.1 / ld 2.26 / gold 1.11 is what you get in Ubuntu 16.04.

Also note: This has surfaced only recently, because of how the symbol types
evolve during the different linker stages. PassManager<Function> is not
affected, because it is referenced in libLLVMCore.a outside the .o file
that contains the explicit instantiation, while PassManager<Loop> *is*
affected, because it is referenced in libLLVMAnalysis.a outside the .o file
that contains the explicit instantiation, but that .cpp file including the
reference does *not* include LoopPassManager.h where the template
instantiation is declared extern. This somehow causes the linker to treat
the two cases differently.

Thanks,
Nicolai
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160619/0acbb84a/attachment.html>

On 19.06.2016 16:31, Rafael Espíndola wrote:
> This probably just works around a layering violation. Can you provide
> the cmake line that produces a broken build?
I can't really trace it down to a single cmake line (the linker failure 
happens while linking llc, for example).

And yes, I'm starting to think that this may be something like an 
ODR-rule violation. I've attached a minimal example that has the shape 
of what happens in LLVM. Build with

c++ -fPIC -fvisibility-inlines-hidden -Wall -c a.cpp
c++ -fPIC -fvisibility-inlines-hidden -Wall -c b.cpp
c++ -fPIC -shared a.o b.o -o test.so

Then nm test.so | grep Example says:

0000000000000640 T _Z7TriggerRK7ExampleIiE
000000000000066a W _ZNK7ExampleIdE3getEv
000000000000065a t _ZNK7ExampleIiE3getEv

In b.cpp, both Example<int> and Example<double> are supposed to be 
explicitly instantiated and those instantiations are supposed to be 
visible to users of test.so. However, for Example<int>::get, the fact 
that a.cpp doesn't see the "extern template class
Example<int>" causes
it to instantiate Example<int>::get() a second time, with a hidden 
visibility flag.

The problem isn't visible in an optimized build because the 
Example<int>::get() is inlined while compiling a.cpp.

To translate that back to LLVM, the problem is that LoopPass.cpp uses 
AnalysisManager<Loop> without including LoopPassManager.h, where the 
extern template declaration lives.

I'm not sufficiently expert in the rules of how visibility and explicit 
template instantiations are supposed to interact to be able to tell 
whether this is a bug in the build tools or a bug in the LLVM source code.

In any case, I noticed this morning that somebody else fixed it in the 
other way that I had in mind, which is including LoopPassManager.h in 
LoopPass.cpp.

This does seem pretty fragile to me, though. Perhaps all the "extern 
template" declarations should be in the same header file as the template 
itself?

Cheers,
Nicolai


>
> Cheers,
> Rafael
>
> On Jun 18, 2016 7:34 AM, "Nicolai Hähnle" <llvm-dev at
lists.llvm.org
> <mailto:llvm-dev at lists.llvm.org>> wrote:
>
>     Hi,
>
>     since recently I'm getting linker failures in debug builds. The
root
>     cause is that -fvisibility-inlines-hidden causes inline functions in
>     explicit template instantiations to become hidden in certain cases.
>
>     The following hunk works around this...
>
>     --- a/cmake/modules/HandleLLVMOptions.cmake
>     +++ b/cmake/modules/HandleLLVMOptions.cmake
>     @@ -159,7 +159,13 @@ endif()
>       if(NOT WIN32 AND NOT CYGWIN)
>         # MinGW warns if -fvisibility-inlines-hidden is used.
>         check_cxx_compiler_flag("-fvisibility-inlines-hidden"
>     SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG)
>     -  append_if(SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG
>     "-fvisibility-inlines-hidden" CMAKE_CXX_FLAGS)
>     +
>     +  # The flag also hides inlines from explicit template
>     instantiations, which
>     +  # leads to link failures in debug builds at least with gcc 5.3.1 and
>     +  # both GNU ld 2.26 or gold 1.11.
>     +  if (NOT uppercase_CMAKE_BUILD_TYPE STREQUAL "DEBUG")
>     +    append_if(SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG
>     "-fvisibility-inlines-hidden" CMAKE_CXX_FLAGS)
>     +  endif()
>       endif()
>
>       if( CMAKE_SIZEOF_VOID_P EQUAL 8 AND NOT WIN32 )
>
>     ... but before I just wildly commit something, I'd say this does
>     seem more like a compiler bug to me and so I'd like to poll people
>     about which compilers they're using to see where this appears.
>
>     FWIW, gcc 5.3.1 / ld 2.26 / gold 1.11 is what you get in Ubuntu 16.04.
>
>     Also note: This has surfaced only recently, because of how the
>     symbol types evolve during the different linker stages.
>     PassManager<Function> is not affected, because it is referenced
in
>     libLLVMCore.a outside the .o file that contains the explicit
>     instantiation, while PassManager<Loop> *is* affected, because it
is
>     referenced in libLLVMAnalysis.a outside the .o file that contains
>     the explicit instantiation, but that .cpp file including the
>     reference does *not* include LoopPassManager.h where the template
>     instantiation is declared extern. This somehow causes the linker to
>     treat the two cases differently.
>
>     Thanks,
>     Nicolai
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a.cpp
Type: text/x-c++src
Size: 72 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160620/4039c52a/attachment-0002.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a.h
Type: text/x-chdr
Size: 134 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160620/4039c52a/attachment-0002.h>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: b.cpp
Type: text/x-c++src
Size: 77 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160620/4039c52a/attachment-0003.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: b.h
Type: text/x-chdr
Size: 131 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160620/4039c52a/attachment-0003.h>

> In any case, I noticed this morning that somebody else fixed it in the
other
> way that I had in mind, which is including LoopPassManager.h in
> LoopPass.cpp.
Including LoopPassManager.h in LoopPass.cpp fixes the problem for you?
I think that is the correct fix. I am not sure if c++ requires that an
extern template be visible in all translation units if it is visible
in one, but -fvisibility-inline-hidden is know to not be fully
standards compliant.

Cheers,
Rafael