llvm dev - Jun 2019 - Memory overflow during cmake/ninja build

I'm trying to do a simple build from the git 8.0.0 sources. The sources 
seem to build OK but a link step fails from running out of memory. I 
need some clues how to figure out where the bottleneck might be.

The cmake command is:

cmake -G Ninja                                          \
     -DLLVM_TARGETS_TO_BUILD=X86                         \
     -DCMAKE_INSTALL_PREFIX=/home/ron/bin/llvm_project/  \
     ~/llvm-project/llvm

All seems to go well up until this link step:

[8/323] Linking CXX executable bin/llvm-lto
FAILED: bin/llvm-lto
: && /usr/bin/c++  -fPIC -fvisibility-inlines-hidden -Werror=date-time 
-std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings 
-Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long 
-Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess 
-Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment 
-fdiagnostics-color -g -Wl,-allow-shlib-undefined 
-Wl,-rpath-link,/home/ron/llvm-project/build-try3/./lib 
tools/llvm-lto/CMakeFiles/llvm-lto.dir/llvm-lto.cpp.o  -o bin/llvm-lto  
-Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMX86CodeGen.a 
lib/libLLVMX86AsmParser.a lib/libLLVMX86AsmPrinter.a 
lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a 
lib/libLLVMX86Utils.a lib/libLLVMBitReader.a lib/libLLVMBitWriter.a 
lib/libLLVMCore.a lib/libLLVMIRReader.a lib/libLLVMLTO.a lib/libLLVMMC.a 
lib/libLLVMObject.a lib/libLLVMSupport.a lib/libLLVMTarget.a -lpthread 
lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMGlobalISel.a lib/libLLVMSelectionDAG.a 
lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a 
lib/libLLVMMCDisassembler.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a lib/libLLVMTarget.a lib/libLLVMipo.a 
lib/libLLVMBitWriter.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a 
lib/libLLVMLinker.a lib/libLLVMScalarOpts.a 
lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a 
lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a 
lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a 
lib/libLLVMBitReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a 
lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a 
lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a 
lib/libLLVMSupport.a -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a && :
/usr/bin/ld: 
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/librt.so: 
error adding symbols: memory exhausted
collect2: error: ld returned 1 exit status
[9/323] Building CXX object 
tools/bugpoint/CMakeFiles/bugpoint.dir/bugpoint.cpp.o
[10/323] Linking CXX shared library lib/libLTO.so.9svn
ninja: build stopped: subcommand failed.

My hardware is a rather ancient HP a6745f (AMD) with 4GB memory, 200 GB 
HDD. OS is Kubuntu 18.10.

Is my system just too limited or is there something else I should look 
for? What other info can I provide?

Ron



-- 
Ron Brender
Whose favorite airplane is N6119A, a 1979 Cessna T210.

On Fri, 21 Jun 2019 at 15:00, Ron Brender via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> I'm trying to do a simple build from the git 8.0.0 sources. The sources
> seem to build OK but a link step fails from running out of memory. I
> need some clues how to figure out where the bottleneck might be.
>
> The cmake command is:
>
> cmake -G Ninja                                          \
>      -DLLVM_TARGETS_TO_BUILD=X86                         \
>      -DCMAKE_INSTALL_PREFIX=/home/ron/bin/llvm_project/  \
>      ~/llvm-project/llvm
>
> All seems to go well up until this link step:
>
> [8/323] Linking CXX executable bin/llvm-lto
> FAILED: bin/llvm-lto
> : && /usr/bin/c++  -fPIC -fvisibility-inlines-hidden
-Werror=date-time
> -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings
> -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long
> -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess
> -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment
> -fdiagnostics-color -g -Wl,-allow-shlib-undefined
> -Wl,-rpath-link,/home/ron/llvm-project/build-try3/./lib
> tools/llvm-lto/CMakeFiles/llvm-lto.dir/llvm-lto.cpp.o  -o bin/llvm-lto
> -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMX86CodeGen.a
> lib/libLLVMX86AsmParser.a lib/libLLVMX86AsmPrinter.a
> lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a
> lib/libLLVMX86Utils.a lib/libLLVMBitReader.a lib/libLLVMBitWriter.a
> lib/libLLVMCore.a lib/libLLVMIRReader.a lib/libLLVMLTO.a lib/libLLVMMC.a
> lib/libLLVMObject.a lib/libLLVMSupport.a lib/libLLVMTarget.a -lpthread
> lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a
> lib/libLLVMGlobalISel.a lib/libLLVMSelectionDAG.a
> lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a
> lib/libLLVMMCDisassembler.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a
> lib/libLLVMCodeGen.a lib/libLLVMTarget.a lib/libLLVMipo.a
> lib/libLLVMBitWriter.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a
> lib/libLLVMLinker.a lib/libLLVMScalarOpts.a
> lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a
> lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a
> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a
> lib/libLLVMBitReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a
> lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a
> lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a
> lib/libLLVMSupport.a -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a
&& :
> /usr/bin/ld:
> /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/librt.so:
> error adding symbols: memory exhausted
> collect2: error: ld returned 1 exit status
> [9/323] Building CXX object
> tools/bugpoint/CMakeFiles/bugpoint.dir/bugpoint.cpp.o
> [10/323] Linking CXX shared library lib/libLTO.so.9svn
> ninja: build stopped: subcommand failed.
>
> My hardware is a rather ancient HP a6745f (AMD) with 4GB memory, 200 GB
> HDD. OS is Kubuntu 18.10.
>
By default the build system will do a static link including debug, and
will use up all available cores for parallelsim. If you happen to have
a small amount of memory but multiple cores there can be several
multi-gigabyte links going on in parallel. Some cmake flag
suggestions:
Prevent concurrent links:
-DLLVM_PARALLEL_LINK_JOBS=1
If you don't need debug information:
-DCMAKE_BUILD_TYPE=Release
If you do:
-DBUILD_SHARED_LIBS=True

It has been mentioned that ld.bfd can use significantly more memory
than gold or LLD on debug builds, may be worth trying a different
linker. The -DLLVM_USE_LINKER=linker can be used
(https://llvm.org/docs/CMake.html).

Peter
> Is my system just too limited or is there something else I should look
> for? What other info can I provide?
>
> Ron
>
>
>
> --
> Ron Brender
> Whose favorite airplane is N6119A, a 1979 Cessna T210.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 21/06/2019 16:00, Ron Brender via llvm-dev wrote:
> cmake -G Ninja                                          \
>      -DLLVM_TARGETS_TO_BUILD=X86                         \
>      -DCMAKE_INSTALL_PREFIX=/home/ron/bin/llvm_project/  \
>      ~/llvm-project/llvm
try -DLLVM_PARALLEL_LINK_JOBS=1

It has also been suggested to try compiling with clang and linking with 
lld. The gold linker should be already installed and is also recommended 
to reduce memory use.

apt install clang

apt install lld

4G is very tight. You will want to run just the terminal emulator for 
the compile and another running top to see when you go into swap and 
what process is doing that. You will need to avoid running anything else 
that is not absolutely necessary to make the most memory available.

You can control-c out of the compile sequence and restart at any time. 
The sequence will resume.

If you have more than one core and it goes into swap, you might stop the 
compile and use

ninja -j 1

to try to squeak through. You can stop the compile, go back to more 
cores, and restart in more easy stretches.

I used

cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_USE_LINKER=lld 
-DCMAKE_BUILD_TYPE="Release" ../llvm

this morning and it appeared in general to stay below 4G and perhaps 
with enough swap you will make it. But when it goes into heavy swap, you 
are doing a number on your disk with not much forward gain.

If you change compilers such as from GNU cpp to clang after running the 
above cmake you will need to delete the build directory for the new 
compiler to be used.

Change the compiler (after apt above) by adding the following lines to 
/etc/environment and reboot.
export CC=clang
export CXX=clang++

On 6/21/19 8:00 AM, Ron Brender via llvm-dev wrote:
> I'm trying to do a simple build from the git 8.0.0 sources. The 
> sources seem to build OK but a link step fails from running out of 
> memory. I need some clues how to figure out where the bottleneck might 
> be.
>
> The cmake command is:
>
> cmake -G Ninja                                          \
>     -DLLVM_TARGETS_TO_BUILD=X86                         \
>     -DCMAKE_INSTALL_PREFIX=/home/ron/bin/llvm_project/  \
>     ~/llvm-project/llvm
>
> All seems to go well up until this link step:
>
> [8/323] Linking CXX executable bin/llvm-lto
> FAILED: bin/llvm-lto
> : && /usr/bin/c++  -fPIC -fvisibility-inlines-hidden
-Werror=date-time
> -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings 
> -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long 
> -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess 
> -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment 
> -fdiagnostics-color -g -Wl,-allow-shlib-undefined 
> -Wl,-rpath-link,/home/ron/llvm-project/build-try3/./lib 
> tools/llvm-lto/CMakeFiles/llvm-lto.dir/llvm-lto.cpp.o  -o 
> bin/llvm-lto  -Wl,-rpath,"\$ORIGIN/../lib"
lib/libLLVMX86CodeGen.a
> lib/libLLVMX86AsmParser.a lib/libLLVMX86AsmPrinter.a 
> lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a 
> lib/libLLVMX86Utils.a lib/libLLVMBitReader.a lib/libLLVMBitWriter.a 
> lib/libLLVMCore.a lib/libLLVMIRReader.a lib/libLLVMLTO.a 
> lib/libLLVMMC.a lib/libLLVMObject.a lib/libLLVMSupport.a 
> lib/libLLVMTarget.a -lpthread lib/libLLVMAsmPrinter.a 
> lib/libLLVMDebugInfoDWARF.a lib/libLLVMGlobalISel.a 
> lib/libLLVMSelectionDAG.a lib/libLLVMX86AsmPrinter.a 
> lib/libLLVMX86Utils.a lib/libLLVMMCDisassembler.a 
> lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMCodeGen.a 
> lib/libLLVMTarget.a lib/libLLVMipo.a lib/libLLVMBitWriter.a 
> lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a 
> lib/libLLVMScalarOpts.a lib/libLLVMAggressiveInstCombine.a 
> lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a 
> lib/libLLVMVectorize.a lib/libLLVMTransformUtils.a 
> lib/libLLVMAnalysis.a lib/libLLVMObject.a lib/libLLVMBitReader.a 
> lib/libLLVMMCParser.a lib/libLLVMMC.a lib/libLLVMDebugInfoCodeView.a 
> lib/libLLVMDebugInfoMSF.a lib/libLLVMProfileData.a lib/libLLVMCore.a 
> lib/libLLVMBinaryFormat.a lib/libLLVMSupport.a -lrt -ldl -lpthread -lm 
> lib/libLLVMDemangle.a && :
> /usr/bin/ld: 
> /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/librt.so: 
> error adding symbols: memory exhausted
> collect2: error: ld returned 1 exit status
> [9/323] Building CXX object 
> tools/bugpoint/CMakeFiles/bugpoint.dir/bugpoint.cpp.o
> [10/323] Linking CXX shared library lib/libLTO.so.9svn
> ninja: build stopped: subcommand failed.
>
> My hardware is a rather ancient HP a6745f (AMD) with 4GB memory, 200 
> GB HDD. OS is Kubuntu 18.10.
>
> Is my system just too limited or is there something else I should look 
> for? What other info can I provide?
>
> Ron
>
>
>

Peter, Steve, Alex, Neil,

Thanks for the many suggestions. I will try again this weekend...

Ron


On 6/21/2019 9:25 PM, Neil Nelson via llvm-dev wrote:
> It has also been suggested to try compiling with clang and linking 
> with lld. The gold linker should be already installed and is also 
> recommended to reduce memory use.
>
> apt install clang
>
> apt install lld
>
> 4G is very tight. You will want to run just the terminal emulator for 
> the compile and another running top to see when you go into swap and 
> what process is doing that. You will need to avoid running anything 
> else that is not absolutely necessary to make the most memory available.
>
> You can control-c out of the compile sequence and restart at any time. 
> The sequence will resume.
>
> If you have more than one core and it goes into swap, you might stop 
> the compile and use
>
> ninja -j 1
>
> to try to squeak through. You can stop the compile, go back to more 
> cores, and restart in more easy stretches.
>
> I used
>
> cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang"
-DLLVM_USE_LINKER=lld
> -DCMAKE_BUILD_TYPE="Release" ../llvm
>
> this morning and it appeared in general to stay below 4G and perhaps 
> with enough swap you will make it. But when it goes into heavy swap, 
> you are doing a number on your disk with not much forward gain.
>
> If you change compilers such as from GNU cpp to clang after running 
> the above cmake you will need to delete the build directory for the 
> new compiler to be used.
>
> Change the compiler (after apt above) by adding the following lines to 
> /etc/environment and reboot.
> export CC=clang
> export CXX=clang++
>
> On 6/21/19 8:00 AM, Ron Brender via llvm-dev wrote:
>> I'm trying to do a simple build from the git 8.0.0 sources. The 
>> sources seem to build OK but a link step fails from running out of 
>> memory. I need some clues how to figure out where the bottleneck 
>> might be.
>>
>> The cmake command is:
>>
>> cmake -G Ninja                                          \
>>     -DLLVM_TARGETS_TO_BUILD=X86                         \
>>     -DCMAKE_INSTALL_PREFIX=/home/ron/bin/llvm_project/  \
>>     ~/llvm-project/llvm
>>
>> All seems to go well up until this link step:
>>
>> [8/323] Linking CXX executable bin/llvm-lto
>> FAILED: bin/llvm-lto
>> : && /usr/bin/c++  -fPIC -fvisibility-inlines-hidden 
>> -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter 
>> -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic 
>> -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized 
>> -Wno-class-memaccess -Wno-noexcept-type -Wdelete-non-virtual-dtor 
>> -Wno-comment -fdiagnostics-color -g -Wl,-allow-shlib-undefined 
>> -Wl,-rpath-link,/home/ron/llvm-project/build-try3/./lib 
>> tools/llvm-lto/CMakeFiles/llvm-lto.dir/llvm-lto.cpp.o  -o 
>> bin/llvm-lto  -Wl,-rpath,"\$ORIGIN/../lib"
lib/libLLVMX86CodeGen.a
>> lib/libLLVMX86AsmParser.a lib/libLLVMX86AsmPrinter.a 
>> lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a 
>> lib/libLLVMX86Info.a lib/libLLVMX86Utils.a lib/libLLVMBitReader.a 
>> lib/libLLVMBitWriter.a lib/libLLVMCore.a lib/libLLVMIRReader.a 
>> lib/libLLVMLTO.a lib/libLLVMMC.a lib/libLLVMObject.a 
>> lib/libLLVMSupport.a lib/libLLVMTarget.a -lpthread 
>> lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a 
>> lib/libLLVMGlobalISel.a lib/libLLVMSelectionDAG.a 
>> lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a 
>> lib/libLLVMMCDisassembler.a lib/libLLVMObjCARCOpts.a 
>> lib/libLLVMPasses.a lib/libLLVMCodeGen.a lib/libLLVMTarget.a 
>> lib/libLLVMipo.a lib/libLLVMBitWriter.a lib/libLLVMIRReader.a 
>> lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMScalarOpts.a 
>> lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a 
>> lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a 
>> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a 
>> lib/libLLVMBitReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a 
>> lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a 
>> lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a 
>> lib/libLLVMSupport.a -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a
&& :
>> /usr/bin/ld: 
>> /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/librt.so: 
>> error adding symbols: memory exhausted
>> collect2: error: ld returned 1 exit status
>> [9/323] Building CXX object 
>> tools/bugpoint/CMakeFiles/bugpoint.dir/bugpoint.cpp.o
>> [10/323] Linking CXX shared library lib/libLTO.so.9svn
>> ninja: build stopped: subcommand failed.
>>
>> My hardware is a rather ancient HP a6745f (AMD) with 4GB memory, 200 
>> GB HDD. OS is Kubuntu 18.10.
>>
>> Is my system just too limited or is there something else I should 
>> look for? What other info can I provide?
>>
>> Ron
>>
>>
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 
Ron Brender
Whose favorite airplane is N6119A, a 1979 Cessna T210.