llvm dev - Dec 2016 - (Thin)LTO llvm build

​Hi again, Teresa.

Looks like I had forgotten to report back with success
when finally building 3.9.0 in ThinLTO linker mode
back in October. Sorry about that and thanks for
helping me out. I know how important it is to get
success reports as well, as a developer myself,
so sorry again :(.

While that worked back then, last weekend I tried to
build 3.9.1 using 3.9.0 as installed from Arch Linux
and ran into a familiar issue but in another module.
I was using the Arch Linux package because it worked
the last time and without something like a chroot,
I cannot build 3.9.1 with personal-3.9.0 and then
overwrite it with the new build because the ninja
scripts hard-code paths to CC and CXX which then
ends in an error.

I tried twice, and before I try another build,
​ ​
I wonder about this:

- Should I avoid any and all CFLAGS, CXXFLAGS, CPPFLAGS,
  LDFLAGS in favor of passing these exclusively as
  cmake's -Dsomething_something=<FLAGS>? Linux distros
  usually have an environment with CFLAGS etc. and use
  that building packages and this didn't interfere in the
  past in my builds, but I'm open to avoid this by unsetting
  any such variables first.

- Is there a configuration in the CI matrix that builds all
  of LLVM with ThinLTO so that I can be reasonably sure
  any error is most likely local to my environment or
  because of operator (silly me) faults?

$ svn info
URL: http://llvm.org/svn/llvm-project/llvm/branches/release_39
Relative URL: ^/llvm/branches/release_39
Repository Root: http://llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 290054
Node Kind: directory
Schedule: normal
Last Changed Author: tstellar
Last Changed Rev: 288847


I also pass these in LDFLAGS but they're not visible
in the error trace below:
    -Wl,-plugin-opt,-function-sections \
    -Wl,-plugin-opt,-data-sections \

Any idea why lldb-argdumper would fail to link reproducably?

$ ninja

[...]

[3480/3688] Linking CXX executable bin/lldb-argdumper
FAILED: bin/lldb-argdumper
: && /usr/bin/clang++ -O2 -pipe
-fstack-protector-strong
--param=ssp-buffer-size=4
-march=x86-64
-mtune=generic
-fPIC
-fvisibility-inlines-hidden
-Wall
-W
-Wno-unused-parameter
-Wwrite-strings
-Wcast-qual
-Wmissing-field-initializers
-pedantic
-Wno-long-long
-Wcovered-switch-default
-Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
-Werror=date-time
-std=c++11
-fcolor-diagnostics
-ffunction-sections
-fdata-sections
-flto=thin
-Wno-deprecated-declarations
-Wno-unknown-pragmas
-Wno-strict-aliasing
-Wno-deprecated-register
-Wno-vla-extension
-O3
-DNDEBUG
-fuse-ld=gold
-flto=thin
-Wl,-allow-shlib-undefined
-Wl,-O3
-Wl,--gc-sections
tools/lldb/tools/argdumper/CMakeFiles/lldb-argdumper.dir/argdumper.cpp.o
-o bin/lldb-argdumper -lpthread lib/liblldb.so.3.9.1 -Wl,--start-group
lib/liblldbBase.a lib/liblldbBreakpoint.a lib/liblldbCommands.a
lib/liblldbDataFormatters.a lib/liblldbHost.a lib/liblldbCore.a
lib/liblldbExpression.a lib/liblldbInitialization.a
lib/liblldbInterpreter.a lib/liblldbSymbol.a lib/liblldbTarget.a
lib/liblldbUtility.a lib/liblldbPluginDisassemblerLLVM.a
lib/liblldbPluginSymbolFileDWARF.a lib/liblldbPluginSymbolFilePDB.a
lib/libLLVMDebugInfoPDB.a lib/libLLVMDebugInfoCodeView.a
lib/liblldbPluginSymbolFileSymtab.a
lib/liblldbPluginDynamicLoaderStatic.a
lib/liblldbPluginDynamicLoaderPosixDYLD.a
lib/liblldbPluginDynamicLoaderHexagonDYLD.a
lib/liblldbPluginDynamicLoaderWindowsDYLD.a
lib/liblldbPluginCPlusPlusLanguage.a lib/liblldbPluginGoLanguage.a
lib/liblldbPluginJavaLanguage.a lib/liblldbPluginObjCLanguage.a
lib/liblldbPluginObjCPlusPlusLanguage.a
lib/liblldbPluginObjectFileELF.a lib/liblldbPluginObjectFileJIT.a
lib/liblldbPluginSymbolVendorELF.a
lib/liblldbPluginObjectContainerBSDArchive.a
lib/liblldbPluginObjectContainerMachOArchive.a
lib/liblldbPluginProcessGDBRemote.a lib/liblldbPluginProcessUtility.a
lib/liblldbPluginPlatformAndroid.a lib/liblldbPluginPlatformGDB.a
lib/liblldbPluginPlatformFreeBSD.a lib/liblldbPluginPlatformKalimba.a
lib/liblldbPluginPlatformLinux.a lib/liblldbPluginPlatformNetBSD.a
lib/liblldbPluginPlatformPOSIX.a lib/liblldbPluginPlatformWindows.a
lib/liblldbPluginPlatformMacOSX.a
lib/liblldbPluginDynamicLoaderMacOSXDYLD.a
lib/liblldbPluginUnwindAssemblyInstEmulation.a
lib/liblldbPluginUnwindAssemblyX86.a
lib/liblldbPluginAppleObjCRuntime.a
lib/liblldbPluginRenderScriptRuntime.a
lib/liblldbPluginLanguageRuntimeGo.a
lib/liblldbPluginLanguageRuntimeJava.a
lib/liblldbPluginCXXItaniumABI.a lib/liblldbPluginABIMacOSX_arm.a
lib/liblldbPluginABIMacOSX_arm64.a lib/liblldbPluginABIMacOSX_i386.a
lib/liblldbPluginABISysV_arm.a lib/liblldbPluginABISysV_arm64.a
lib/liblldbPluginABISysV_i386.a lib/liblldbPluginABISysV_x86_64.a
lib/liblldbPluginABISysV_hexagon.a lib/liblldbPluginABISysV_ppc.a
lib/liblldbPluginABISysV_ppc64.a lib/liblldbPluginABISysV_mips.a
lib/liblldbPluginABISysV_mips64.a lib/liblldbPluginABISysV_s390x.a
lib/liblldbPluginInstructionARM.a lib/liblldbPluginInstructionARM64.a
lib/liblldbPluginInstructionMIPS.a
lib/liblldbPluginInstructionMIPS64.a
lib/liblldbPluginObjectFilePECOFF.a lib/liblldbPluginOSGo.a
lib/liblldbPluginOSPython.a lib/liblldbPluginMemoryHistoryASan.a
lib/liblldbPluginInstrumentationRuntimeAddressSanitizer.a
lib/liblldbPluginInstrumentationRuntimeThreadSanitizer.a
lib/liblldbPluginSystemRuntimeMacOSX.a
lib/liblldbPluginProcessElfCore.a lib/liblldbPluginJITLoaderGDB.a
lib/liblldbPluginExpressionParserClang.a
lib/liblldbPluginExpressionParserGo.a lib/liblldbPluginProcessLinux.a
lib/liblldbPluginProcessPOSIX.a -Wl,--end-group lib/libclangCodeGen.a
lib/libLLVMBitWriter.a lib/libLLVMCoverage.a lib/libLLVMipo.a
lib/libLLVMVectorize.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a
lib/libLLVMInstrumentation.a lib/libLLVMLinker.a
lib/libLLVMObjCARCOpts.a lib/libLLVMObject.a lib/libLLVMScalarOpts.a
lib/libLLVMInstCombine.a lib/libLLVMTarget.a
lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a
lib/libclangRewriteFrontend.a lib/libclangFrontend.a
lib/libclangDriver.a lib/libclangParse.a lib/libLLVMMCParser.a
lib/libLLVMProfileData.a lib/libLLVMOption.a lib/libclangRewrite.a
lib/libclangSerialization.a lib/libclangSema.a lib/libclangAnalysis.a
lib/libclangEdit.a lib/libclangAST.a lib/libclangLex.a
lib/libclangBasic.a lib/libLLVMMC.a lib/libLLVMBitReader.a
lib/libLLVMCore.a lib/libLLVMSupport.a -lrt -ldl -lcurses -lpthread
-lz -lm -Wl,-rpath,"\$ORIGIN/../lib" && :

/usr/bin/ld.gold: bin/lldb-argdumper: hidden symbol `__morestack' in
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o)
is referenced by DSO

/usr/bin/ld.gold: final link failed: Bad value

clang-3.9: error: linker command failed with exit code 1 (use -v to
see invocation)

ninja: build stopped: subcommand failed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161220/f1286d7e/attachment.html>

On Tue, Dec 20, 2016 at 5:49 AM, Carsten Mattner via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hi again, Teresa.
>
> Looks like I had forgotten to report back with success
> when finally building 3.9.0 in ThinLTO linker mode
> back in October. Sorry about that and thanks for
> helping me out. I know how important it is to get
> success reports as well, as a developer myself,
> so sorry again :(.
>
> While that worked back then, last weekend I tried to
> build 3.9.1 using 3.9.0 as installed from Arch Linux
> and ran into a familiar issue but in another module.
> I was using the Arch Linux package because it worked
> the last time and without something like a chroot,
> I cannot build 3.9.1 with personal-3.9.0 and then
> overwrite it with the new build because the ninja
> scripts hard-code paths to CC and CXX which then
> ends in an error.
>
> I tried twice, and before I try another build,
> I wonder about this:
>
> - Should I avoid any and all CFLAGS, CXXFLAGS, CPPFLAGS,
>   LDFLAGS in favor of passing these exclusively as
>   cmake's -Dsomething_something=<FLAGS>? Linux distros
>   usually have an environment with CFLAGS etc. and use
>   that building packages and this didn't interfere in the
>   past in my builds, but I'm open to avoid this by unsetting
>   any such variables first.
>
> - Is there a configuration in the CI matrix that builds all
>   of LLVM with ThinLTO so that I can be reasonably sure
>   any error is most likely local to my environment or
>   because of operator (silly me) faults?
>
> $ svn info
> URL: http://llvm.org/svn/llvm-project/llvm/branches/release_39
> Relative URL: ^/llvm/branches/release_39
> Repository Root: http://llvm.org/svn/llvm-project
> Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
> Revision: 290054
> Node Kind: directory
> Schedule: normal
> Last Changed Author: tstellar
> Last Changed Rev: 288847
>
>
> I also pass these in LDFLAGS but they're not visible
> in the error trace below:
>     -Wl,-plugin-opt,-function-sections \
>     -Wl,-plugin-opt,-data-sections \
>
> Any idea why lldb-argdumper would fail to link reproducably?
>
> $ ninja
>
> [...]
>
> [3480/3688] Linking CXX executable bin/lldb-argdumper
> FAILED: bin/lldb-argdumper
> : && /usr/bin/clang++ -O2 -pipe
> -fstack-protector-strong
> --param=ssp-buffer-size=4
> -march=x86-64
> -mtune=generic
> -fPIC
> -fvisibility-inlines-hidden
> -Wall
> -W
> -Wno-unused-parameter
> -Wwrite-strings
> -Wcast-qual
> -Wmissing-field-initializers
> -pedantic
> -Wno-long-long
> -Wcovered-switch-default
> -Wnon-virtual-dtor
> -Wdelete-non-virtual-dtor
> -Werror=date-time
> -std=c++11
> -fcolor-diagnostics
> -ffunction-sections
> -fdata-sections
> -flto=thin
> -Wno-deprecated-declarations
> -Wno-unknown-pragmas
> -Wno-strict-aliasing
> -Wno-deprecated-register
> -Wno-vla-extension
> -O3
> -DNDEBUG
> -fuse-ld=gold
> -flto=thin
> -Wl,-allow-shlib-undefined
> -Wl,-O3
> -Wl,--gc-sections
> tools/lldb/tools/argdumper/CMakeFiles/lldb-argdumper.dir/argdumper.cpp.o
> -o bin/lldb-argdumper -lpthread lib/liblldb.so.3.9.1 -Wl,--start-group
> lib/liblldbBase.a lib/liblldbBreakpoint.a lib/liblldbCommands.a
> lib/liblldbDataFormatters.a lib/liblldbHost.a lib/liblldbCore.a
> lib/liblldbExpression.a lib/liblldbInitialization.a
> lib/liblldbInterpreter.a lib/liblldbSymbol.a lib/liblldbTarget.a
> lib/liblldbUtility.a lib/liblldbPluginDisassemblerLLVM.a
> lib/liblldbPluginSymbolFileDWARF.a lib/liblldbPluginSymbolFilePDB.a
> lib/libLLVMDebugInfoPDB.a lib/libLLVMDebugInfoCodeView.a
> lib/liblldbPluginSymbolFileSymtab.a
> lib/liblldbPluginDynamicLoaderStatic.a
> lib/liblldbPluginDynamicLoaderPosixDYLD.a
> lib/liblldbPluginDynamicLoaderHexagonDYLD.a
> lib/liblldbPluginDynamicLoaderWindowsDYLD.a
> lib/liblldbPluginCPlusPlusLanguage.a lib/liblldbPluginGoLanguage.a
> lib/liblldbPluginJavaLanguage.a lib/liblldbPluginObjCLanguage.a
> lib/liblldbPluginObjCPlusPlusLanguage.a
> lib/liblldbPluginObjectFileELF.a lib/liblldbPluginObjectFileJIT.a
> lib/liblldbPluginSymbolVendorELF.a
> lib/liblldbPluginObjectContainerBSDArchive.a
> lib/liblldbPluginObjectContainerMachOArchive.a
> lib/liblldbPluginProcessGDBRemote.a lib/liblldbPluginProcessUtility.a
> lib/liblldbPluginPlatformAndroid.a lib/liblldbPluginPlatformGDB.a
> lib/liblldbPluginPlatformFreeBSD.a lib/liblldbPluginPlatformKalimba.a
> lib/liblldbPluginPlatformLinux.a lib/liblldbPluginPlatformNetBSD.a
> lib/liblldbPluginPlatformPOSIX.a lib/liblldbPluginPlatformWindows.a
> lib/liblldbPluginPlatformMacOSX.a
> lib/liblldbPluginDynamicLoaderMacOSXDYLD.a
> lib/liblldbPluginUnwindAssemblyInstEmulation.a
> lib/liblldbPluginUnwindAssemblyX86.a
> lib/liblldbPluginAppleObjCRuntime.a
> lib/liblldbPluginRenderScriptRuntime.a
> lib/liblldbPluginLanguageRuntimeGo.a
> lib/liblldbPluginLanguageRuntimeJava.a
> lib/liblldbPluginCXXItaniumABI.a lib/liblldbPluginABIMacOSX_arm.a
> lib/liblldbPluginABIMacOSX_arm64.a lib/liblldbPluginABIMacOSX_i386.a
> lib/liblldbPluginABISysV_arm.a lib/liblldbPluginABISysV_arm64.a
> lib/liblldbPluginABISysV_i386.a lib/liblldbPluginABISysV_x86_64.a
> lib/liblldbPluginABISysV_hexagon.a lib/liblldbPluginABISysV_ppc.a
> lib/liblldbPluginABISysV_ppc64.a lib/liblldbPluginABISysV_mips.a
> lib/liblldbPluginABISysV_mips64.a lib/liblldbPluginABISysV_s390x.a
> lib/liblldbPluginInstructionARM.a lib/liblldbPluginInstructionARM64.a
> lib/liblldbPluginInstructionMIPS.a
> lib/liblldbPluginInstructionMIPS64.a
> lib/liblldbPluginObjectFilePECOFF.a lib/liblldbPluginOSGo.a
> lib/liblldbPluginOSPython.a lib/liblldbPluginMemoryHistoryASan.a
> lib/liblldbPluginInstrumentationRuntimeAddressSanitizer.a
> lib/liblldbPluginInstrumentationRuntimeThreadSanitizer.a
> lib/liblldbPluginSystemRuntimeMacOSX.a
> lib/liblldbPluginProcessElfCore.a lib/liblldbPluginJITLoaderGDB.a
> lib/liblldbPluginExpressionParserClang.a
> lib/liblldbPluginExpressionParserGo.a lib/liblldbPluginProcessLinux.a
> lib/liblldbPluginProcessPOSIX.a -Wl,--end-group lib/libclangCodeGen.a
> lib/libLLVMBitWriter.a lib/libLLVMCoverage.a lib/libLLVMipo.a
> lib/libLLVMVectorize.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a
> lib/libLLVMInstrumentation.a lib/libLLVMLinker.a
> lib/libLLVMObjCARCOpts.a lib/libLLVMObject.a lib/libLLVMScalarOpts.a
> lib/libLLVMInstCombine.a lib/libLLVMTarget.a
> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a
> lib/libclangRewriteFrontend.a lib/libclangFrontend.a
> lib/libclangDriver.a lib/libclangParse.a lib/libLLVMMCParser.a
> lib/libLLVMProfileData.a lib/libLLVMOption.a lib/libclangRewrite.a
> lib/libclangSerialization.a lib/libclangSema.a lib/libclangAnalysis.a
> lib/libclangEdit.a lib/libclangAST.a lib/libclangLex.a
> lib/libclangBasic.a lib/libLLVMMC.a lib/libLLVMBitReader.a
> lib/libLLVMCore.a lib/libLLVMSupport.a -lrt -ldl -lcurses -lpthread
> -lz -lm -Wl,-rpath,"\$ORIGIN/../lib" && :
>
> /usr/bin/ld.gold: bin/lldb-argdumper: hidden symbol `__morestack' in
> /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o)
> is referenced by DSO
>
> /usr/bin/ld.gold: final link failed: Bad value
>
> clang-3.9: error: linker command failed with exit code 1 (use -v to
> see invocation)
>
> ninja: build stopped: subcommand failed.
>
Can you try and see if this reproduces with lld? You should be able to
pass `-fuse-ld=lld` to the invocation instead of `-fuse-ld=gold`. You
don't need a plugin.

-- 
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare

Hi Carsten,

A few responses below, but first, can you get the link command for
lldb.so.3.9.1? Last time it was the lldb.so build that was using ld.bfd
with the gold plugin which was exposing this issue.

On Tue, Dec 20, 2016 at 5:49 AM, Carsten Mattner <carstenmattner at
gmail.com>
wrote:
> ​Hi again, Teresa.
>
> Looks like I had forgotten to report back with success
> when finally building 3.9.0 in ThinLTO linker mode
> back in October. Sorry about that and thanks for
> helping me out. I know how important it is to get
> success reports as well, as a developer myself,
> so sorry again :(.
>
No problem - I assumed no news was good news! Thanks for the explicit
feedback though!

>
> While that worked back then, last weekend I tried to
> build 3.9.1 using 3.9.0 as installed from Arch Linux
> and ran into a familiar issue but in another module.
> I was using the Arch Linux package because it worked
> the last time and without something like a chroot,
> I cannot build 3.9.1 with personal-3.9.0 and then
> overwrite it with the new build because the ninja
> scripts hard-code paths to CC and CXX which then
> ends in an error.
>
> I tried twice, and before I try another build,
> ​ ​
> I wonder about this:
>
> - Should I avoid any and all CFLAGS, CXXFLAGS, CPPFLAGS,
>   LDFLAGS in favor of passing these exclusively as
>   cmake's -Dsomething_something=<FLAGS>? Linux distros
>   usually have an environment with CFLAGS etc. and use
>   that building packages and this didn't interfere in the
>   past in my builds, but I'm open to avoid this by unsetting
>   any such variables first.
>
The problem I've had in the past setting *FLAGS on the cmake command line
is that sometimes the configure fails because it uses those flags in
various configuration checking compiles. So typically I hand edit the
resulting CMakeCache.txt if I want to pass specific options to the build.
Note that there are various flavors of e.g. LINKER_FLAGS that get passed to
different types of links though (SHARED vs EXE etc). I haven't tried
setting CFLAGS etc env variables before kicking off the ninja build to pass
args.

> - Is there a configuration in the CI matrix that builds all
>   of LLVM with ThinLTO so that I can be reasonably sure
>   any error is most likely local to my environment or
>   because of operator (silly me) faults?
>
There is an lld based ThinLTO buildbot. I've been really remiss in not
setting up a gold based ThinLTO bot...note to self to get that done asap!
But the existing lld will be head not 3.9.
>
> $ svn info
> URL: http://llvm.org/svn/llvm-project/llvm/branches/release_39
> Relative URL: ^/llvm/branches/release_39
> Repository Root: http://llvm.org/svn/llvm-project
> Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
> Revision: 290054
> Node Kind: directory
> Schedule: normal
> Last Changed Author: tstellar
> Last Changed Rev: 288847
>
>
> I also pass these in LDFLAGS but they're not visible
> in the error trace below:
>     -Wl,-plugin-opt,-function-sections \
>     -Wl,-plugin-opt,-data-sections \
>
Hmm, looks like LDFLAGS doesn't work for passing to the resulting link
line? Try setting in the CMakeCache.txt as described above.

>
> Any idea why lldb-argdumper would fail to link reproducably?
>
See suggestion above about looking at lldb.so.3.9.1 link line. Let's make
sure that looks ok first.

>
> $ ninja
>
> [...]
>
> [3480/3688] Linking CXX executable bin/lldb-argdumper
> FAILED: bin/lldb-argdumper
> : && /usr/bin/clang++ -O2 -pipe
> -fstack-protector-strong
> --param=ssp-buffer-size=4
> -march=x86-64
> -mtune=generic
> -fPIC
> -fvisibility-inlines-hidden
> -Wall
> -W
> -Wno-unused-parameter
> -Wwrite-strings
> -Wcast-qual
> -Wmissing-field-initializers
> -pedantic
> -Wno-long-long
> -Wcovered-switch-default
> -Wnon-virtual-dtor
> -Wdelete-non-virtual-dtor
> -Werror=date-time
> -std=c++11
> -fcolor-diagnostics
> -ffunction-sections
> -fdata-sections
> -flto=thin
> -Wno-deprecated-declarations
> -Wno-unknown-pragmas
> -Wno-strict-aliasing
> -Wno-deprecated-register
> -Wno-vla-extension
> -O3
> -DNDEBUG
> -fuse-ld=gold
> -flto=thin
> -Wl,-allow-shlib-undefined
> -Wl,-O3
> -Wl,--gc-sections
> tools/lldb/tools/argdumper/CMakeFiles/lldb-argdumper.dir/argdumper.cpp.o
> -o bin/lldb-argdumper -lpthread lib/liblldb.so.3.9.1 -Wl,--start-group
> lib/liblldbBase.a lib/liblldbBreakpoint.a lib/liblldbCommands.a
> lib/liblldbDataFormatters.a lib/liblldbHost.a lib/liblldbCore.a
> lib/liblldbExpression.a lib/liblldbInitialization.a
> lib/liblldbInterpreter.a lib/liblldbSymbol.a lib/liblldbTarget.a
> lib/liblldbUtility.a lib/liblldbPluginDisassemblerLLVM.a
> lib/liblldbPluginSymbolFileDWARF.a lib/liblldbPluginSymbolFilePDB.a
> lib/libLLVMDebugInfoPDB.a lib/libLLVMDebugInfoCodeView.a
> lib/liblldbPluginSymbolFileSymtab.a
> lib/liblldbPluginDynamicLoaderStatic.a
> lib/liblldbPluginDynamicLoaderPosixDYLD.a
> lib/liblldbPluginDynamicLoaderHexagonDYLD.a
> lib/liblldbPluginDynamicLoaderWindowsDYLD.a
> lib/liblldbPluginCPlusPlusLanguage.a lib/liblldbPluginGoLanguage.a
> lib/liblldbPluginJavaLanguage.a lib/liblldbPluginObjCLanguage.a
> lib/liblldbPluginObjCPlusPlusLanguage.a
> lib/liblldbPluginObjectFileELF.a lib/liblldbPluginObjectFileJIT.a
> lib/liblldbPluginSymbolVendorELF.a
> lib/liblldbPluginObjectContainerBSDArchive.a
> lib/liblldbPluginObjectContainerMachOArchive.a
> lib/liblldbPluginProcessGDBRemote.a lib/liblldbPluginProcessUtility.a
> lib/liblldbPluginPlatformAndroid.a lib/liblldbPluginPlatformGDB.a
> lib/liblldbPluginPlatformFreeBSD.a lib/liblldbPluginPlatformKalimba.a
> lib/liblldbPluginPlatformLinux.a lib/liblldbPluginPlatformNetBSD.a
> lib/liblldbPluginPlatformPOSIX.a lib/liblldbPluginPlatformWindows.a
> lib/liblldbPluginPlatformMacOSX.a
> lib/liblldbPluginDynamicLoaderMacOSXDYLD.a
> lib/liblldbPluginUnwindAssemblyInstEmulation.a
> lib/liblldbPluginUnwindAssemblyX86.a
> lib/liblldbPluginAppleObjCRuntime.a
> lib/liblldbPluginRenderScriptRuntime.a
> lib/liblldbPluginLanguageRuntimeGo.a
> lib/liblldbPluginLanguageRuntimeJava.a
> lib/liblldbPluginCXXItaniumABI.a lib/liblldbPluginABIMacOSX_arm.a
> lib/liblldbPluginABIMacOSX_arm64.a lib/liblldbPluginABIMacOSX_i386.a
> lib/liblldbPluginABISysV_arm.a lib/liblldbPluginABISysV_arm64.a
> lib/liblldbPluginABISysV_i386.a lib/liblldbPluginABISysV_x86_64.a
> lib/liblldbPluginABISysV_hexagon.a lib/liblldbPluginABISysV_ppc.a
> lib/liblldbPluginABISysV_ppc64.a lib/liblldbPluginABISysV_mips.a
> lib/liblldbPluginABISysV_mips64.a lib/liblldbPluginABISysV_s390x.a
> lib/liblldbPluginInstructionARM.a lib/liblldbPluginInstructionARM64.a
> lib/liblldbPluginInstructionMIPS.a
> lib/liblldbPluginInstructionMIPS64.a
> lib/liblldbPluginObjectFilePECOFF.a lib/liblldbPluginOSGo.a
> lib/liblldbPluginOSPython.a lib/liblldbPluginMemoryHistoryASan.a
> lib/liblldbPluginInstrumentationRuntimeAddressSanitizer.a
> lib/liblldbPluginInstrumentationRuntimeThreadSanitizer.a
> lib/liblldbPluginSystemRuntimeMacOSX.a
> lib/liblldbPluginProcessElfCore.a lib/liblldbPluginJITLoaderGDB.a
> lib/liblldbPluginExpressionParserClang.a
> lib/liblldbPluginExpressionParserGo.a lib/liblldbPluginProcessLinux.a
> lib/liblldbPluginProcessPOSIX.a -Wl,--end-group lib/libclangCodeGen.a
> lib/libLLVMBitWriter.a lib/libLLVMCoverage.a lib/libLLVMipo.a
> lib/libLLVMVectorize.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a
> lib/libLLVMInstrumentation.a lib/libLLVMLinker.a
> lib/libLLVMObjCARCOpts.a lib/libLLVMObject.a lib/libLLVMScalarOpts.a
> lib/libLLVMInstCombine.a lib/libLLVMTarget.a
> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a
> lib/libclangRewriteFrontend.a lib/libclangFrontend.a
> lib/libclangDriver.a lib/libclangParse.a lib/libLLVMMCParser.a
> lib/libLLVMProfileData.a lib/libLLVMOption.a lib/libclangRewrite.a
> lib/libclangSerialization.a lib/libclangSema.a lib/libclangAnalysis.a
> lib/libclangEdit.a lib/libclangAST.a lib/libclangLex.a
> lib/libclangBasic.a lib/libLLVMMC.a lib/libLLVMBitReader.a
> lib/libLLVMCore.a lib/libLLVMSupport.a -lrt -ldl -lcurses -lpthread
> -lz -lm -Wl,-rpath,"\$ORIGIN/../lib" && :
>
> /usr/bin/ld.gold: bin/lldb-argdumper: hidden symbol `__morestack' in
> /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o)
> is referenced by DSO
>
> /usr/bin/ld.gold: final link failed: Bad value
>
> clang-3.9: error: linker command failed with exit code 1 (use -v to
> see invocation)
>
> ninja: build stopped: subcommand failed.
>


-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |  408-460-2413
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161220/d8987c4a/attachment-0001.html>

> On Dec 20, 2016, at 5:49 AM, Carsten Mattner via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> ​Hi again, Teresa.
> 
> Looks like I had forgotten to report back with success
> when finally building 3.9.0 in ThinLTO linker mode
> back in October. Sorry about that and thanks for
> helping me out. I know how important it is to get
> success reports as well, as a developer myself,
> so sorry again :(.
> 
> While that worked back then, last weekend I tried to
> build 3.9.1 using 3.9.0 as installed from Arch Linux
> and ran into a familiar issue but in another module.
> I was using the Arch Linux package because it worked
> the last time and without something like a chroot,
> I cannot build 3.9.1 with personal-3.9.0 and then
> overwrite it with the new build because the ninja
> scripts hard-code paths to CC and CXX which then
> ends in an error.
> 
> I tried twice, and before I try another build,​ ​I wonder about this:
> 
> - Should I avoid any and all CFLAGS, CXXFLAGS, CPPFLAGS,
>   LDFLAGS in favor of passing these exclusively as
>   cmake's -Dsomething_something=<FLAGS>? Linux distros
>   usually have an environment with CFLAGS etc. and use
>   that building packages and this didn't interfere in the
>   past in my builds, but I'm open to avoid this by unsetting
>   any such variables first.
> 
> - Is there a configuration in the CI matrix that builds all
>   of LLVM with ThinLTO so that I can be reasonably sure
>   any error is most likely local to my environment or
>   because of operator (silly me) faults?
> 
> $ svn info
> URL: http://llvm.org/svn/llvm-project/llvm/branches/release_39
<http://llvm.org/svn/llvm-project/llvm/branches/release_39>
> Relative URL: ^/llvm/branches/release_39
> Repository Root: http://llvm.org/svn/llvm-project
<http://llvm.org/svn/llvm-project>
> Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
> Revision: 290054
> Node Kind: directory
> Schedule: normal
> Last Changed Author: tstellar
> Last Changed Rev: 288847
> 
> 
> I also pass these in LDFLAGS but they're not visible
> in the error trace below:

What is your exact cmake invocation?
I don’t think cmake accept “LDFLAGS” directly, instead it is using
“CMAKE_MODULE_LINKER_FLAGS”, “CMAKE_SHARED_LINKER_FLAGS”, and
“CMAKE_EXE_LINKER_FLAGS”.

Ideally these would be set automatically when building LLVM with
-DLLVM_ENABLE_LTO=THIN.


Teresa, I think it would be valuable to update
llvm/cmake/modules/HandleLLVMOptions.cmake and make sure 1) either LLD or GOLD
is used, 2) add the relevant plugin flags if needed.


if(uppercase_LLVM_ENABLE_LTO STREQUAL "THIN")
  append("-flto=thin" CMAKE_CXX_FLAGS CMAKE_C_FLAGS
                      CMAKE_EXE_LINKER_FLAGS CMAKE_SHARED_LINKER_FLAGS)
  # On darwin, enable the lto cache. This improves initial build time a little
  # since we re-link a lot of the same objects, and significantly improves
  # incremental build time.
  append_if(APPLE
"-Wl,-cache_path_lto,${PROJECT_BINARY_DIR}/lto.cache"
            CMAKE_EXE_LINKER_FLAGS CMAKE_SHARED_LINKER_FLAGS)


— 
Mehdi


>     -Wl,-plugin-opt,-function-sections \
>     -Wl,-plugin-opt,-data-sections \
> 
> Any idea why lldb-argdumper would fail to link reproducably?
> 
> $ ninja
> 
> [...]
> 
> [3480/3688] Linking CXX executable bin/lldb-argdumper
> FAILED: bin/lldb-argdumper
> : && /usr/bin/clang++ -O2 -pipe
> -fstack-protector-strong
> --param=ssp-buffer-size=4
> -march=x86-64
> -mtune=generic
> -fPIC
> -fvisibility-inlines-hidden
> -Wall
> -W
> -Wno-unused-parameter
> -Wwrite-strings
> -Wcast-qual
> -Wmissing-field-initializers
> -pedantic
> -Wno-long-long
> -Wcovered-switch-default
> -Wnon-virtual-dtor
> -Wdelete-non-virtual-dtor
> -Werror=date-time
> -std=c++11
> -fcolor-diagnostics
> -ffunction-sections
> -fdata-sections
> -flto=thin
> -Wno-deprecated-declarations
> -Wno-unknown-pragmas
> -Wno-strict-aliasing
> -Wno-deprecated-register
> -Wno-vla-extension
> -O3
> -DNDEBUG
> -fuse-ld=gold
> -flto=thin
> -Wl,-allow-shlib-undefined
> -Wl,-O3
> -Wl,--gc-sections
> tools/lldb/tools/argdumper/CMakeFiles/lldb-argdumper.dir/argdumper.cpp.o
> -o bin/lldb-argdumper -lpthread lib/liblldb.so.3.9.1 -Wl,--start-group
> lib/liblldbBase.a lib/liblldbBreakpoint.a lib/liblldbCommands.a
> lib/liblldbDataFormatters.a lib/liblldbHost.a lib/liblldbCore.a
> lib/liblldbExpression.a lib/liblldbInitialization.a
> lib/liblldbInterpreter.a lib/liblldbSymbol.a lib/liblldbTarget.a
> lib/liblldbUtility.a lib/liblldbPluginDisassemblerLLVM.a
> lib/liblldbPluginSymbolFileDWARF.a lib/liblldbPluginSymbolFilePDB.a
> lib/libLLVMDebugInfoPDB.a lib/libLLVMDebugInfoCodeView.a
> lib/liblldbPluginSymbolFileSymtab.a
> lib/liblldbPluginDynamicLoaderStatic.a
> lib/liblldbPluginDynamicLoaderPosixDYLD.a
> lib/liblldbPluginDynamicLoaderHexagonDYLD.a
> lib/liblldbPluginDynamicLoaderWindowsDYLD.a
> lib/liblldbPluginCPlusPlusLanguage.a lib/liblldbPluginGoLanguage.a
> lib/liblldbPluginJavaLanguage.a lib/liblldbPluginObjCLanguage.a
> lib/liblldbPluginObjCPlusPlusLanguage.a
> lib/liblldbPluginObjectFileELF.a lib/liblldbPluginObjectFileJIT.a
> lib/liblldbPluginSymbolVendorELF.a
> lib/liblldbPluginObjectContainerBSDArchive.a
> lib/liblldbPluginObjectContainerMachOArchive.a
> lib/liblldbPluginProcessGDBRemote.a lib/liblldbPluginProcessUtility.a
> lib/liblldbPluginPlatformAndroid.a lib/liblldbPluginPlatformGDB.a
> lib/liblldbPluginPlatformFreeBSD.a lib/liblldbPluginPlatformKalimba.a
> lib/liblldbPluginPlatformLinux.a lib/liblldbPluginPlatformNetBSD.a
> lib/liblldbPluginPlatformPOSIX.a lib/liblldbPluginPlatformWindows.a
> lib/liblldbPluginPlatformMacOSX.a
> lib/liblldbPluginDynamicLoaderMacOSXDYLD.a
> lib/liblldbPluginUnwindAssemblyInstEmulation.a
> lib/liblldbPluginUnwindAssemblyX86.a
> lib/liblldbPluginAppleObjCRuntime.a
> lib/liblldbPluginRenderScriptRuntime.a
> lib/liblldbPluginLanguageRuntimeGo.a
> lib/liblldbPluginLanguageRuntimeJava.a
> lib/liblldbPluginCXXItaniumABI.a lib/liblldbPluginABIMacOSX_arm.a
> lib/liblldbPluginABIMacOSX_arm64.a lib/liblldbPluginABIMacOSX_i386.a
> lib/liblldbPluginABISysV_arm.a lib/liblldbPluginABISysV_arm64.a
> lib/liblldbPluginABISysV_i386.a lib/liblldbPluginABISysV_x86_64.a
> lib/liblldbPluginABISysV_hexagon.a lib/liblldbPluginABISysV_ppc.a
> lib/liblldbPluginABISysV_ppc64.a lib/liblldbPluginABISysV_mips.a
> lib/liblldbPluginABISysV_mips64.a lib/liblldbPluginABISysV_s390x.a
> lib/liblldbPluginInstructionARM.a lib/liblldbPluginInstructionARM64.a
> lib/liblldbPluginInstructionMIPS.a
> lib/liblldbPluginInstructionMIPS64.a
> lib/liblldbPluginObjectFilePECOFF.a lib/liblldbPluginOSGo.a
> lib/liblldbPluginOSPython.a lib/liblldbPluginMemoryHistoryASan.a
> lib/liblldbPluginInstrumentationRuntimeAddressSanitizer.a
> lib/liblldbPluginInstrumentationRuntimeThreadSanitizer.a
> lib/liblldbPluginSystemRuntimeMacOSX.a
> lib/liblldbPluginProcessElfCore.a lib/liblldbPluginJITLoaderGDB.a
> lib/liblldbPluginExpressionParserClang.a
> lib/liblldbPluginExpressionParserGo.a lib/liblldbPluginProcessLinux.a
> lib/liblldbPluginProcessPOSIX.a -Wl,--end-group lib/libclangCodeGen.a
> lib/libLLVMBitWriter.a lib/libLLVMCoverage.a lib/libLLVMipo.a
> lib/libLLVMVectorize.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a
> lib/libLLVMInstrumentation.a lib/libLLVMLinker.a
> lib/libLLVMObjCARCOpts.a lib/libLLVMObject.a lib/libLLVMScalarOpts.a
> lib/libLLVMInstCombine.a lib/libLLVMTarget.a
> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a
> lib/libclangRewriteFrontend.a lib/libclangFrontend.a
> lib/libclangDriver.a lib/libclangParse.a lib/libLLVMMCParser.a
> lib/libLLVMProfileData.a lib/libLLVMOption.a lib/libclangRewrite.a
> lib/libclangSerialization.a lib/libclangSema.a lib/libclangAnalysis.a
> lib/libclangEdit.a lib/libclangAST.a lib/libclangLex.a
> lib/libclangBasic.a lib/libLLVMMC.a lib/libLLVMBitReader.a
> lib/libLLVMCore.a lib/libLLVMSupport.a -lrt -ldl -lcurses -lpthread
> -lz -lm -Wl,-rpath,"\$ORIGIN/../lib" && :
> 
> /usr/bin/ld.gold: bin/lldb-argdumper: hidden symbol `__morestack' in
> /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o)
> is referenced by DSO
> 
> /usr/bin/ld.gold: final link failed: Bad value
> 
> clang-3.9: error: linker command failed with exit code 1 (use -v to
> see invocation)
> 
> ninja: build stopped: subcommand failed.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161220/716d880e/attachment.html>

On Tue, Dec 20, 2016 at 5:52 PM, Mehdi Amini <mehdi.amini at apple.com>
wrote:
> What is your exact cmake invocation?
Same as in October, and this one for the next try, where I have
explicitly overriding all compiler and linker flags, which used to say
just -fuse-ld=gold before to make it work in October and from what I
remember extended it since the pre-set CFLAGS were still respected
back then.

Haven't tries this one yet, but it's what I intend to test next.

cmake \
    -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DLLVM_BINUTILS_INCDIR=/usr/include \
    -DCMAKE_INSTALL_PREFIX=$PREFIX \
    -DLLVM_TARGETS_TO_BUILD="X86" \
    -DLLVM_ENABLE_BACKTRACES=OFF \
    -DLLVM_BUILD_EXAMPLES=OFF \
    -DLLVM_INCLUDE_EXAMPLES=OFF \
    -DLLVM_BUILD_TESTS=OFF \
    -DLLVM_INCLUDE_TESTS=OFF \
    -DLLVM_BUILD_DOCS=OFF \
    -DLLVM_INCLUDE_DOCS=OFF \
    -DLLVM_ENABLE_DOXYGEN=OFF \
    -DLLVM_ENABLE_SPHINX=OFF \
    -DLLDB_DISABLE_PYTHON=ON \
    -DCLANG_PLUGIN_SUPPORT=OFF \
    -DBUILD_SHARED_LIBS=OFF \
    -DCMAKE_AR=`which llvm-ar` \
    -DCMAKE_RANLIB=`which llvm-ranlib` \
    -DCMAKE_C_FLAGS="$CPPFLAGS $CFLAGS" \
    -DCMAKE_CXX_FLAGS="$CPPFLAGS $CXXFLAGS" \
    -DCMAKE_EXE_LINKER_FLAGS=$LDFLAGS \
    -DCMAKE_MODULE_LINKER_FLAGS=$LDFLAGS \
    -DCMAKE_SHARED_LINKER_FLAGS=$LDFLAGS \
    -DLLVM_ENABLE_LTO=Thin \
    -DLLVM_PARALLEL_LINK_JOBS=1

I should probably note that I build clang with polly support,
plus also lldb and libcxx.

I wonder if I should also do this:
-DCMAKE_CXX_FLAGS="$CPPFLAGS $CXXFLAGS -stdlib=libc++

But if I do that, I might also need to know how I can avoid libgcc and
use libunwind and rest of compiler-rt to make it free of any gcc
dependency, as a test and preparation of future defaults in llvm's
build scripts, as suggested by on this list a couple months ago.
> I don’t think cmake accept “LDFLAGS” directly, instead it is using
> “CMAKE_MODULE_LINKER_FLAGS”, “CMAKE_SHARED_LINKER_FLAGS”, and
> “CMAKE_EXE_LINKER_FLAGS”.
It does or it wouldn't otherwise respect what I set, and I've seen it
documented (would need to scour the web for reference) to do so by
design. However, I'm changing the configure invocation to explicitly
set all the CMAKE variables (I could find) explicitly, reusing what's
set in CFLAGS etc. before running cmake.

One interesting observation is that there's no CMake equivalent of
CPPFLAGS (needed for FORTIFY_SOURCE=2 which goes with
stack-protector). I've prepended $CFPPFLAGS to CMAKE_C_FLAGS and
CMAKE_CXX_FLAGS which should hopefully work without causing any
errors. Technically FORTIFY_SOURCE is only for $CPP and neither $CC or
$CXX.

On Tue, Dec 20, 2016 at 3:10 PM, Davide Italiano <davide at freebsd.org>
wrote:
> Can you try and see if this reproduces with lld? You should be able
> to pass `-fuse-ld=lld` to the invocation instead of `-fuse-ld=gold`.
> You don't need a plugin.
Will do in next test, and in 1st test I had used it actually, but I
reverted to gold to use the same as I did for buildign 3.9.0 back
when.

On Tue, Dec 20, 2016 at 5:05 PM, Teresa Johnson <tejohnson at google.com>
wrote:
> Hi Carsten,
>
> A few responses below, but first, can you get the link command for
> lldb.so.3.9.1? Last time it was the lldb.so build that was using
> ld.bfd with the gold plugin which was exposing this issue.
Where would I find it in an otherwise already terminated process?
> On Tue, Dec 20, 2016 at 5:49 AM, Carsten Mattner <carstenmattner at
gmail.com>
> No problem - I assumed no news was good news! Thanks for the
> explicit feedback though!
Sure, but it's important, and I've had my experience with never
hearing back even after asking for confirmation of success. Same as
drive-by patch drops.
> > - Should I avoid any and all CFLAGS, CXXFLAGS, CPPFLAGS, LDFLAGS
> >   in favor of passing these exclusively as cmake's
> >   -Dsomething_something=<FLAGS>? Linux distros usually have an
> >   environment with CFLAGS etc. and use that building packages and
> >   this didn't interfere in the past in my builds, but I'm open
to
> >   avoid this by unsetting any such variables first.
>
>
> The problem I've had in the past setting *FLAGS on the cmake command
> line is that sometimes the configure fails because it uses those
> flags in various configuration checking compiles. So typically I
> hand edit the resulting CMakeCache.txt if I want to pass specific
> options to the build. Note that there are various flavors of e.g.
> LINKER_FLAGS that get passed to different types of links though
> (SHARED vs EXE etc). I haven't tried setting CFLAGS etc env
> variables before kicking off the ninja build to pass args.
I set the variables before running cmake and I didn't expect for it to
be re-considered when running ninja, but it would of course more
closely mirror make's behavior if it does/dit.
> > - Is there a configuration in the CI matrix that builds all
> >   of LLVM with ThinLTO so that I can be reasonably sure
> >   any error is most likely local to my environment or
> >   because of operator (silly me) faults?
>
>
> There is an lld based ThinLTO buildbot. I've been really remiss in
> not setting up a gold based ThinLTO bot...note to self to get that
> done asap! But the existing lld will be head not 3.9.
Please have it build everything that's buildable in the llvm tree.
> > I also pass these in LDFLAGS but they're not visible
> > in the error trace below:
> >     -Wl,-plugin-opt,-function-sections \
> >     -Wl,-plugin-opt,-data-sections \
>
>
> Hmm, looks like LDFLAGS doesn't work for passing to the resulting link
line?
> Try setting in the CMakeCache.txt as described above.
It's a mystery (aka haven't traced/debugged CMake).
> > Any idea why lldb-argdumper would fail to link reproducably?
>
> See suggestion above about looking at lldb.so.3.9.1 link line. Let's
> make sure that looks ok first.
Thanks.