Piotr Padlewski via llvm-dev
2016-Jul-23 17:48 UTC
[llvm-dev] [llvm-toolchain v3.8.1] LTO: Linking clang hangs with ld.gold and LLVMgold.so plugin
How big is your project? LTO eats RAM even faster than chrome. For example linking clang with LTO could take 16GB of ram. Have you tried using LTO on your project on that machine, or is it your first time? Piotr On Sat, Jul 23, 2016 at 2:42 AM, Sedat Dilek via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On Thu, Jul 21, 2016 at 12:01 PM, Sedat Dilek <sedat.dilek at gmail.com> > wrote: > > Hi, > > > > unfortunately, my build somehow hangs when linking clang binary and my > > system is in an unusable state. > > > > My toolchain is clang-3.8, gold-1.11 and LLVMgold.so from binutils > > v2.26.1 (both selfmade) and LTO-flag is enabled. > > My buildsystem uses cmake-3.6.0 and ninja-1.7.1 (both prebuilt). > > I have 52 last steps left in my 3rd build. > > > > My Linux-kernel is v3.13.0-92 from official Ubuntu repositories. > > > > On my Ubuntu/precise AMD64 I have ***4GiB RAM and 256MiB SWAP***. > > Is this not enough RAM/SWAP for building/linking? > > > > Need some additional informations? > > > > Any help appreciated. > > > > Regards, > > - Sedat - > > > > [1] http://llvm.org/docs/GoldPlugin.html > > > > P.S.: More informations... > > > > [ First toolchain build ] > > > > I built a 1st llvm-toolchain v3.8.1 with GCC v4.9.2 and binutils v2.22. > > No speedup settings, no extra-patches applied etc. > > > > [ Selfmade binutils ] > > > > As binutils v2.22 was somehow not able to generate a LLVMgold.so on > > Ubuntu/precise AMD64, I have built binutils v2.26.1 manually and use > > all its binaries for building/linking (GOLD for linking). > > > > [ 2nd llvm-toolchain LLVMgold.so ] > > > > In a 2nd llvm-toolchain v3.8.1 build a LLVMgold.so was generated. > > I placed LLVMgold.so into my selfmade > > /opt/binutils-2.26.1/lib/bfd-plugins/ directory. > > ( I had to create a bfd-plugins subdir manually. ) > > > > [ 3rd llvm-toolchain LTO ] > > > > Now, I am able with backported LTO-flag from upstream to build with > '-flto'. > > > > - EOT - > > Hi, > > in the meantime I tried with Linux v4.4.y LTS and 2GiB swap-space. > So I have 4 GiB RAM and 2GiB SWAP in total 6GiB. > > Paul Rouschal recommended to reduce parallel-compile-jobs from 2 to 1... > > LLVM_PARALLEL_COMPILE_JOBS=1 > LLVM_PARALLEL_LINK_JOBS=1 > > ...but that did not help. > > My Ubuntu/precise hangs and looking at top shows MEM/SWAP to be eaten. > > Anyone has experiences how much RAM or SWAP I need when building a > LTO-optimized llvm-toolchain with Clang, GNU/gold and LLVMgold-Plugin? > > My build-script is attached. > > Thanks. > > Regards, > - Sedat - > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160723/9eae599b/attachment.html>
Sedat Dilek via llvm-dev
2016-Jul-23 20:53 UTC
[llvm-dev] [llvm-toolchain v3.8.1] LTO: Linking clang hangs with ld.gold and LLVMgold.so plugin
On Sat, Jul 23, 2016 at 7:48 PM, Piotr Padlewski <prazek at google.com> wrote:> How big is your project? > LTO eats RAM even faster than chrome. For example linking clang with LTO > could take 16GB of ram. > > Have you tried using LTO on your project on that machine, or is it your > first time? >Wow hu hu - 16GiB of RAM :-). This is my first time dealing with LTO in general. Not sure if it matters if I build with GCC and its LTO-plugin and ld.bfd as linker. Here I tried CLANG with LLVMgold-plugin and ld.gold (v2.26.1). These are the last lines I see... [2114/2157] Linking CXX executable bin/diagtool [2115/2157] Linking CXX executable bin/clang-format [2116/2157] Linking CXX executable bin/clang-3.8 FAILED: bin/clang-3.8 : && /opt/llvm/bin/clang++-3.8 -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections -flto -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -flto -Wl,-allow-shlib-undefined -Wl,--export-dynamic -Wl,-O3 -Wl,--gc-sections tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o -o bin/clang-3.8 lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmPrinter.a lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Info.a lib/libLLVMX86Disassembler.a lib/libLLVMAnalysis.a lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a lib/libLLVMMCParser.a lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a lib/libLLVMScalarOpts.a lib/libLLVMSupport.a lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a lib/libclangBasic.a lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a lib/libclangFrontendTool.a lib/libLLVMAsmPrinter.a lib/libLLVMSelectionDAG.a lib/libLLVMCodeGen.a lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a lib/libLLVMX86Info.a lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMipo.a lib/libLLVMVectorize.a lib/libLLVMInstrumentation.a lib/libLLVMObjCARCOpts.a lib/libLLVMScalarOpts.a lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMBitWriter.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMProfileData.a lib/libLLVMObject.a lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a lib/libclangStaticAnalyzerFrontend.a lib/libclangFrontend.a lib/libclangDriver.a lib/libLLVMOption.a lib/libclangParse.a lib/libLLVMMCParser.a lib/libclangSerialization.a lib/libLLVMBitReader.a lib/libclangSema.a lib/libclangEdit.a lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a lib/libclangAnalysis.a lib/libclangAST.a lib/libclangRewrite.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a lib/libLLVMMC.a lib/libLLVMSupport.a -lrt -ldl -ltinfo -lpthread -lz -lm -Wl,-rpath,"\$ORIGIN/../lib" && : clang-3.8: error: unable to execute command: Killed clang-3.8: error: linker command failed due to signal (use -v to see invocation) ninja: build stopped: subcommand failed. This time I just build and installed ld.gold from binutils v2.26.1 and symlinked system-wide with my ld of my /opt-installation. The generated LLVMgold.so plugin I copied into /usr/lib/bfd-plugins/ directory (bfd-plugins subdir needed to be created manually). - Sedat - P.S.: Output of free and df commands $ free -m total used free shared buffers cached Mem: 3846 2211 1635 0 729 818 -/+ buffers/cache: 662 3183 Swap: 255 143 112 $ df -T | grep ext4 /dev/loop0 ext4 17753424 15997332 831216 96% / - EOT -> Piotr > > > On Sat, Jul 23, 2016 at 2:42 AM, Sedat Dilek via llvm-dev > <llvm-dev at lists.llvm.org> wrote: >> >> On Thu, Jul 21, 2016 at 12:01 PM, Sedat Dilek <sedat.dilek at gmail.com> >> wrote: >> > Hi, >> > >> > unfortunately, my build somehow hangs when linking clang binary and my >> > system is in an unusable state. >> > >> > My toolchain is clang-3.8, gold-1.11 and LLVMgold.so from binutils >> > v2.26.1 (both selfmade) and LTO-flag is enabled. >> > My buildsystem uses cmake-3.6.0 and ninja-1.7.1 (both prebuilt). >> > I have 52 last steps left in my 3rd build. >> > >> > My Linux-kernel is v3.13.0-92 from official Ubuntu repositories. >> > >> > On my Ubuntu/precise AMD64 I have ***4GiB RAM and 256MiB SWAP***. >> > Is this not enough RAM/SWAP for building/linking? >> > >> > Need some additional informations? >> > >> > Any help appreciated. >> > >> > Regards, >> > - Sedat - >> > >> > [1] http://llvm.org/docs/GoldPlugin.html >> > >> > P.S.: More informations... >> > >> > [ First toolchain build ] >> > >> > I built a 1st llvm-toolchain v3.8.1 with GCC v4.9.2 and binutils v2.22. >> > No speedup settings, no extra-patches applied etc. >> > >> > [ Selfmade binutils ] >> > >> > As binutils v2.22 was somehow not able to generate a LLVMgold.so on >> > Ubuntu/precise AMD64, I have built binutils v2.26.1 manually and use >> > all its binaries for building/linking (GOLD for linking). >> > >> > [ 2nd llvm-toolchain LLVMgold.so ] >> > >> > In a 2nd llvm-toolchain v3.8.1 build a LLVMgold.so was generated. >> > I placed LLVMgold.so into my selfmade >> > /opt/binutils-2.26.1/lib/bfd-plugins/ directory. >> > ( I had to create a bfd-plugins subdir manually. ) >> > >> > [ 3rd llvm-toolchain LTO ] >> > >> > Now, I am able with backported LTO-flag from upstream to build with >> > '-flto'. >> > >> > - EOT - >> >> Hi, >> >> in the meantime I tried with Linux v4.4.y LTS and 2GiB swap-space. >> So I have 4 GiB RAM and 2GiB SWAP in total 6GiB. >> >> Paul Rouschal recommended to reduce parallel-compile-jobs from 2 to 1... >> >> LLVM_PARALLEL_COMPILE_JOBS=1 >> LLVM_PARALLEL_LINK_JOBS=1 >> >> ...but that did not help. >> >> My Ubuntu/precise hangs and looking at top shows MEM/SWAP to be eaten. >> >> Anyone has experiences how much RAM or SWAP I need when building a >> LTO-optimized llvm-toolchain with Clang, GNU/gold and LLVMgold-Plugin? >> >> My build-script is attached. >> >> Thanks. >> >> Regards, >> - Sedat - >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >-------------- next part -------------- A non-text attachment was scrubbed... Name: build_llvm-toolchain.sh Type: application/x-sh Size: 6393 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160723/7de8f227/attachment.sh>
Mehdi Amini via llvm-dev
2016-Jul-23 21:59 UTC
[llvm-dev] [llvm-toolchain v3.8.1] LTO: Linking clang hangs with ld.gold and LLVMgold.so plugin
> On Jul 23, 2016, at 1:53 PM, Sedat Dilek via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > > On Sat, Jul 23, 2016 at 7:48 PM, Piotr Padlewski <prazek at google.com <mailto:prazek at google.com>> wrote: >> How big is your project? >> LTO eats RAM even faster than chrome. For example linking clang with LTO >> could take 16GB of ram. >> >> Have you tried using LTO on your project on that machine, or is it your >> first time? >> > > Wow hu hu - 16GiB of RAM :-).It is a bit less now: should be closer to 11GB with Full Debug Info. You may want to try without debug info (or with -gline-tables-only) and it should get under 4GB. — Mehdi> > This is my first time dealing with LTO in general. > > Not sure if it matters if I build with GCC and its LTO-plugin and > ld.bfd as linker. > Here I tried CLANG with LLVMgold-plugin and ld.gold (v2.26.1). > > These are the last lines I see... > > [2114/2157] Linking CXX executable bin/diagtool > [2115/2157] Linking CXX executable bin/clang-format > [2116/2157] Linking CXX executable bin/clang-3.8 > FAILED: bin/clang-3.8 > : && /opt/llvm/bin/clang++-3.8 -fPIC -fvisibility-inlines-hidden > -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual > -Wmissing-field-initializers -pedantic -Wno-long-long > -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor > -std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections > -flto -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 > -flto -Wl,-allow-shlib-undefined -Wl,--export-dynamic -Wl,-O3 > -Wl,--gc-sections > tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o > tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o > tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o -o > bin/clang-3.8 lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmPrinter.a > lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Info.a > lib/libLLVMX86Disassembler.a lib/libLLVMAnalysis.a > lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a > lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a > lib/libLLVMMCParser.a lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a > lib/libLLVMScalarOpts.a lib/libLLVMSupport.a > lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a lib/libclangBasic.a > lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a > lib/libclangFrontendTool.a lib/libLLVMAsmPrinter.a > lib/libLLVMSelectionDAG.a lib/libLLVMCodeGen.a > lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a lib/libLLVMX86Info.a > lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMipo.a > lib/libLLVMVectorize.a lib/libLLVMInstrumentation.a > lib/libLLVMObjCARCOpts.a lib/libLLVMScalarOpts.a > lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMBitWriter.a > lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a > lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a > lib/libLLVMProfileData.a lib/libLLVMObject.a > lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a > lib/libclangStaticAnalyzerFrontend.a lib/libclangFrontend.a > lib/libclangDriver.a lib/libLLVMOption.a lib/libclangParse.a > lib/libLLVMMCParser.a lib/libclangSerialization.a > lib/libLLVMBitReader.a lib/libclangSema.a lib/libclangEdit.a > lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a > lib/libclangAnalysis.a lib/libclangAST.a lib/libclangRewrite.a > lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a > lib/libLLVMMC.a lib/libLLVMSupport.a -lrt -ldl -ltinfo -lpthread -lz > -lm -Wl,-rpath,"\$ORIGIN/../lib" && : > clang-3.8: error: unable to execute command: Killed > clang-3.8: error: linker command failed due to signal (use -v to see invocation) > ninja: build stopped: subcommand failed. > > This time I just build and installed ld.gold from binutils v2.26.1 and > symlinked system-wide with my ld of my /opt-installation. > The generated LLVMgold.so plugin I copied into /usr/lib/bfd-plugins/ > directory (bfd-plugins subdir needed to be created manually). > > - Sedat - > > P.S.: Output of free and df commands > > $ free -m > total used free shared buffers cached > Mem: 3846 2211 1635 0 729 818 > -/+ buffers/cache: 662 3183 > Swap: 255 143 112 > > $ df -T | grep ext4 > /dev/loop0 ext4 17753424 15997332 831216 96% / > > - EOT - > >> Piotr >> >> >> On Sat, Jul 23, 2016 at 2:42 AM, Sedat Dilek via llvm-dev >> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >>> >>> On Thu, Jul 21, 2016 at 12:01 PM, Sedat Dilek <sedat.dilek at gmail.com <mailto:sedat.dilek at gmail.com>> >>> wrote: >>>> Hi, >>>> >>>> unfortunately, my build somehow hangs when linking clang binary and my >>>> system is in an unusable state. >>>> >>>> My toolchain is clang-3.8, gold-1.11 and LLVMgold.so from binutils >>>> v2.26.1 (both selfmade) and LTO-flag is enabled. >>>> My buildsystem uses cmake-3.6.0 and ninja-1.7.1 (both prebuilt). >>>> I have 52 last steps left in my 3rd build. >>>> >>>> My Linux-kernel is v3.13.0-92 from official Ubuntu repositories. >>>> >>>> On my Ubuntu/precise AMD64 I have ***4GiB RAM and 256MiB SWAP***. >>>> Is this not enough RAM/SWAP for building/linking? >>>> >>>> Need some additional informations? >>>> >>>> Any help appreciated. >>>> >>>> Regards, >>>> - Sedat - >>>> >>>> [1] http://llvm.org/docs/GoldPlugin.html <http://llvm.org/docs/GoldPlugin.html> >>>> >>>> P.S.: More informations... >>>> >>>> [ First toolchain build ] >>>> >>>> I built a 1st llvm-toolchain v3.8.1 with GCC v4.9.2 and binutils v2.22. >>>> No speedup settings, no extra-patches applied etc. >>>> >>>> [ Selfmade binutils ] >>>> >>>> As binutils v2.22 was somehow not able to generate a LLVMgold.so on >>>> Ubuntu/precise AMD64, I have built binutils v2.26.1 manually and use >>>> all its binaries for building/linking (GOLD for linking). >>>> >>>> [ 2nd llvm-toolchain LLVMgold.so ] >>>> >>>> In a 2nd llvm-toolchain v3.8.1 build a LLVMgold.so was generated. >>>> I placed LLVMgold.so into my selfmade >>>> /opt/binutils-2.26.1/lib/bfd-plugins/ directory. >>>> ( I had to create a bfd-plugins subdir manually. ) >>>> >>>> [ 3rd llvm-toolchain LTO ] >>>> >>>> Now, I am able with backported LTO-flag from upstream to build with >>>> '-flto'. >>>> >>>> - EOT - >>> >>> Hi, >>> >>> in the meantime I tried with Linux v4.4.y LTS and 2GiB swap-space. >>> So I have 4 GiB RAM and 2GiB SWAP in total 6GiB. >>> >>> Paul Rouschal recommended to reduce parallel-compile-jobs from 2 to 1... >>> >>> LLVM_PARALLEL_COMPILE_JOBS=1 >>> LLVM_PARALLEL_LINK_JOBS=1 >>> >>> ...but that did not help. >>> >>> My Ubuntu/precise hangs and looking at top shows MEM/SWAP to be eaten. >>> >>> Anyone has experiences how much RAM or SWAP I need when building a >>> LTO-optimized llvm-toolchain with Clang, GNU/gold and LLVMgold-Plugin? >>> >>> My build-script is attached. >>> >>> Thanks. >>> >>> Regards, >>> - Sedat - >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >> > <build_llvm-toolchain.sh>_______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160723/ddafe538/attachment-0001.html>