I tried building llvm, clang, lld, lldb from the 3.9 svn release branch with LTO, and some of the results were unexpected. I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I fell back to building with -DLLVM_ENABLE_LTO=On and using the system CC/CXX (gcc 6.1). The resulting installed build is many times bigger than the non-LTO version. Is this to be expected? I thought LTO would reduce size by detecting more unused code. The bindist .tar.xz archives are as follows: LTO=Off: 195MB LTO=On : 953MB Now, I am of course aware that falling back to gcc's lto support is not the same as llvm's lto or thinlto, but I only fell back after it failed to build that way, and I wasn't aware of gcc6 providing almost 5x large code than without lto. Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system, how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this supported, or does it require the use of some libXX too?
> On Sep 10, 2016, at 3:03 AM, Carsten Mattner via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > I tried building llvm, clang, lld, lldb from the 3.9 svn release > branch with LTO, and some of the results were unexpected. > > I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by > providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I > fell back to building with -DLLVM_ENABLE_LTO=On and using the system > CC/CXX (gcc 6.1). > > The resulting installed build is many times bigger than the non-LTO > version. Is this to be expected? I thought LTO would reduce size by > detecting more unused code.LTO also inline more, thus increasing the size. But usually it is quite slight. If you’re concerned about binary size, disabling clang plugin may help: cmake -DCLANG_PLUGIN_SUPPORT=OFF> > The bindist .tar.xz archives are as follows: > > LTO=Off: 195MB > LTO=On : 953MB > > Now, I am of course aware that falling back to gcc's lto support is > not the same as llvm's lto or thinlto, but I only fell back after it > failed to build that way, and I wasn't aware of gcc6 providing almost > 5x large code than without lto.GCC may be using “fat” intermediate objects by default, i.e. it generates both the binary and the IR in the intermediates files and the static archives. What is the size of the final clang binary itself?> > Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system, > how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this > supported, or does it require the use of some libXX too?You need to have Gold installed, but that is the same requirement as for LTO I believe. Then if you’re using 3.9 as a host compiler, it should “just work”. — Mehdi
On Sat, Sep 10, 2016 at 6:25 PM, Mehdi Amini <mehdi.amini at apple.com> wrote:> >> On Sep 10, 2016, at 3:03 AM, Carsten Mattner via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> I tried building llvm, clang, lld, lldb from the 3.9 svn release >> branch with LTO, and some of the results were unexpected. >> >> I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by >> providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I >> fell back to building with -DLLVM_ENABLE_LTO=On and using the system >> CC/CXX (gcc 6.1). >> >> The resulting installed build is many times bigger than the non-LTO >> version. Is this to be expected? I thought LTO would reduce size by >> detecting more unused code. > > LTO also inline more, thus increasing the size. But usually it is quite slight. > If you’re concerned about binary size, disabling clang plugin may help: > > cmake -DCLANG_PLUGIN_SUPPORT=OFFCurious to know why this would result in smaller binaries.>> The bindist .tar.xz archives are as follows: >> >> LTO=Off: 195MB >> LTO=On : 953MB >> >> Now, I am of course aware that falling back to gcc's lto support is >> not the same as llvm's lto or thinlto, but I only fell back after it >> failed to build that way, and I wasn't aware of gcc6 providing almost >> 5x large code than without lto. > > GCC may be using “fat” intermediate objects by default, i.e. it > generates both the binary and the IR in the intermediates files > and the static archives.Are you saying I should strip files?> What is the size of the final clang binary itself?Interestingly the non-LTO clang-3.9 is 49MB while the LTO one if 58MB.>> Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system, >> how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this >> supported, or does it require the use of some libXX too? > > You need to have Gold installed, but that is the same requirement as for LTO I believe. > Then if you’re using 3.9 as a host compiler, it should “just work”.$ c++ -v gcc version 6.2.1 20160830 (GCC) $ ld.gold --version GNU gold (GNU Binutils 2.27) 1.12 $ clang -v clang version 3.9.0 (branches/release_39 279763)
On Sat, Sep 10, 2016 at 3:03 AM, Carsten Mattner via llvm-dev < llvm-dev at lists.llvm.org> wrote:> I tried building llvm, clang, lld, lldb from the 3.9 svn release > branch with LTO, and some of the results were unexpected. > > I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by > providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I >I've been able to do a ThinLTO build of clang for some time with gold. I'm not sure if there was a regression when the 3.9 release was cut? Can you send me the error? Thanks, Teresa> fell back to building with -DLLVM_ENABLE_LTO=On and using the system > CC/CXX (gcc 6.1). > > The resulting installed build is many times bigger than the non-LTO > version. Is this to be expected? I thought LTO would reduce size by > detecting more unused code. > > The bindist .tar.xz archives are as follows: > > LTO=Off: 195MB > LTO=On : 953MB > > Now, I am of course aware that falling back to gcc's lto support is > not the same as llvm's lto or thinlto, but I only fell back after it > failed to build that way, and I wasn't aware of gcc6 providing almost > 5x large code than without lto. > > Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system, > how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this > supported, or does it require the use of some libXX too? > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160910/0d220211/attachment.html>
On Sun, Sep 11, 2016 at 3:36 AM, Teresa Johnson <tejohnson at google.com> wrote:> > > > On Sat, Sep 10, 2016 at 3:03 AM, Carsten Mattner via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> I tried building llvm, clang, lld, lldb from the 3.9 svn release >> branch with LTO, and some of the results were unexpected. >> >> I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by >> providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I > > > I've been able to do a ThinLTO build of clang for some time with gold. > I'm not sure if there was a regression when the 3.9 release was cut? > Can you send me the error?I'll share the result and any reproduceable error (if there are any), after the next build tests.
Cc'ing llvm-dev again in case someone knows what is going on. That error is coming from a polly configure script, which I don't have cloned into my own tree. Not sure why polly would behave differently when configuring for ThinLTO. Does polly configure ok when configuring the same way, but for regular LTO? Teresa On Fri, Sep 16, 2016 at 12:36 PM, Carsten Mattner <carstenmattner at gmail.com> wrote:> Hi Teresa, > > I've just attempted a THIN_LTO build again, and the > error I ran into is as follows, and happens during > cmake. I'm explicitly telling cmake to us llvm-ar and llvm-ranlib > as suggested. > > [...] > -- Performing Test COMPILER_RT_HAS_STATIC_FLAG - Success > -- Performing Test COMPILER_RT_SUPPORTS_ATOMIC_KEYWORD > -- Performing Test COMPILER_RT_SUPPORTS_ATOMIC_KEYWORD - Success > -- Builtin supported architectures: > -- ISL version: isl-0.17.1-164-gcbba1b6 > -- Performing Test HAS_ATTRIBUTE_WARN_UNUSED_RESULT > -- Performing Test HAS_ATTRIBUTE_WARN_UNUSED_RESULT - Failed > -- Performing Test HAVE___ATTRIBUTE__ > -- Performing Test HAVE___ATTRIBUTE__ - Failed > -- Performing Test HAVE_DECL_FFS > -- Performing Test HAVE_DECL_FFS - Failed > -- Performing Test HAVE_DECL___BUILTIN_FFS > -- Performing Test HAVE_DECL___BUILTIN_FFS - Failed > -- Performing Test HAVE_DECL__BITSCANFORWARD > -- Performing Test HAVE_DECL__BITSCANFORWARD - Failed > CMake Error at tools/polly/lib/External/CMakeLists.txt:90 (message): > No ffs implementation found > [...] >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160916/e2bbd6f5/attachment.html>
On Fri, Sep 16, 2016 at 10:18 PM, Teresa Johnson <tejohnson at google.com> wrote:> > Cc'ing llvm-dev again in case someone knows what is going on. > > That error is coming from a polly configure script, which I don't > have cloned into my own tree. Not sure why polly would behave > differently when configuring for ThinLTO. Does polly configure ok > when configuring the same way, but for regular LTO?Yes it does, and I don't remember when I've added it, meaning it's been working when bootstrapping llvm with the host gcc toolchain, for a long time.> On Fri, Sep 16, 2016 at 12:36 PM, Carsten Mattner <carstenmattner at gmail.com> wrote: > > > > Hi Teresa, > > > > I've just attempted a THIN_LTO build again, and the > > error I ran into is as follows, and happens during > > cmake. I'm explicitly telling cmake to us llvm-ar and llvm-ranlib > > as suggested. > > > > [...] > > -- Performing Test COMPILER_RT_HAS_STATIC_FLAG - Success > > -- Performing Test COMPILER_RT_SUPPORTS_ATOMIC_KEYWORD > > -- Performing Test COMPILER_RT_SUPPORTS_ATOMIC_KEYWORD - Success > > -- Builtin supported architectures: > > -- ISL version: isl-0.17.1-164-gcbba1b6 > > -- Performing Test HAS_ATTRIBUTE_WARN_UNUSED_RESULT > > -- Performing Test HAS_ATTRIBUTE_WARN_UNUSED_RESULT - Failed > > -- Performing Test HAVE___ATTRIBUTE__ > > -- Performing Test HAVE___ATTRIBUTE__ - Failed > > -- Performing Test HAVE_DECL_FFS > > -- Performing Test HAVE_DECL_FFS - Failed > > -- Performing Test HAVE_DECL___BUILTIN_FFS > > -- Performing Test HAVE_DECL___BUILTIN_FFS - Failed > > -- Performing Test HAVE_DECL__BITSCANFORWARD > > -- Performing Test HAVE_DECL__BITSCANFORWARD - Failed > > CMake Error at tools/polly/lib/External/CMakeLists.txt:90 (message): > > No ffs implementation found > > [...]