thr3ads.net - llvm dev - [llvm-dev] LLD issue on a massively parallel build machine [Mar 2020]

If this information is useful, please help other people find it:
Share via:

Itaru Kitayama via llvm-dev

2020-Mar-28 19:36 UTC

[llvm-dev] LLD issue on a massively parallel build machine

$ free -g
              total        used        free      shared  buff/cache
available
Mem:            376         149          20           1         207
225
Swap:             3           0           3

Too small swap size for a 72-core login machine?

On Sun, Mar 29, 2020 at 4:28 AM Alex Brachet-Mialot <
alexbrachetmialot at gmail.com> wrote:
> Enable threads is for building llvm with or without thread support. To my
> knowledge it has no effect on the build system.
>
> Like was discussed above you could try limiting  the number of processes
> created (assuming your lld was after that commit) with numactl or taskset.
>
> In your case though it looked like you were running out of memory. You
> could try increasing your swap space?
>
> Alex
>
> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama <itaru.kitayama at
gmail.com>
> wrote:
>
>> Hi,
>> My configuration is below:
>> cmake -GNinja -DLLVM_ENABLE_LLD=ON -DLLVM_ENABLE_THREADS=OFF
>> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
>> -DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
>> -DLIBOMPTARGET_ENABLE_DEBUG=ON
>> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
-DCMAKE_C_COMPILER=clang
>> -DCMAKE_C_FLAGS="" -DCMAKE_CXX_COMPILER=clang++
-DCMAKE_CXX_FLAGS=""
>>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
>> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
>> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
-DLLVM_TARGETS_TO_BUILD="all"
>> /tmp/llvm-project/llvm
>>
>> but did not help my case:
>>
>> [...]
>> [4802/5201] Linking CXX executable bin/lld
>> FAILED: bin/lld
>> : && /home/usr4/c74014i/opt/clang/current/bin/clang++ 
-stdlib=libc++
>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
>> -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter
>> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
>> -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
>> -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
>> -Wstring-conversion -fdiagnostics-color -ffunction-sections
-fdata-sections
>> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
>> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o  -o bin/lld
>>  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMSupport.a 
-lpthread
>>  lib/liblldCommon.a  lib/liblldCOFF.a  lib/liblldDriver.a 
lib/liblldELF.a
>>  lib/liblldMinGW.a  lib/liblldWasm.a  lib/liblldMachO.a
>>  lib/liblldReaderWriter.a  lib/liblldYAML.a  lib/liblldCore.a
>>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a  lib/libLLVMLibDriver.a
>>  lib/libLLVMWindowsManifest.a  /usr/lib64/libxml2.so 
lib/liblldCommon.a
>>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
>>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Disassembler.a
>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>>  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUAsmParser.a
>>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUDesc.a
>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a 
lib/libLLVMARMCodeGen.a
>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDisassembler.a
>>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a 
lib/libLLVMAVRDesc.a
>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a 
lib/libLLVMBPFDesc.a
>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>>  lib/libLLVMHexagonDisassembler.a  lib/libLLVMHexagonDesc.a
>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDisassembler.a
>>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a 
lib/libLLVMMipsCodeGen.a
>>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
>>  lib/libLLVMMipsDisassembler.a  lib/libLLVMMipsInfo.a
>>  lib/libLLVMMSP430CodeGen.a  lib/libLLVMMSP430AsmParser.a
>>  lib/libLLVMMSP430Desc.a  lib/libLLVMMSP430Disassembler.a
>>  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXCodeGen.a 
lib/libLLVMNVPTXDesc.a
>>  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCCodeGen.a
>>  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMPowerPCDesc.a
>>  lib/libLLVMPowerPCDisassembler.a  lib/libLLVMPowerPCInfo.a
>>  lib/libLLVMRISCVCodeGen.a  lib/libLLVMRISCVAsmParser.a
>>  lib/libLLVMRISCVDesc.a  lib/libLLVMRISCVDisassembler.a
>>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a 
lib/libLLVMSparcCodeGen.a
>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>>  lib/libLLVMSystemZDisassembler.a  lib/libLLVMSystemZDesc.a
>>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDisassembler.a
>>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a
>>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a 
lib/libLLVMX86Utils.a
>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAsmPrinter.a
>>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>>  lib/libLLVMXCoreDesc.a  lib/libLLVMXCoreDisassembler.a
>>  lib/libLLVMMCDisassembler.a  lib/libLLVMXCoreInfo.a  lib/libLLVMLTO.a
>>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a  lib/libLLVMCodeGen.a
>>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a  lib/libLLVMipo.a
>>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a 
lib/libLLVMVectorize.a
>>  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a
>>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a 
lib/libLLVMIRReader.a
>>  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a
>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a 
lib/libLLVMObject.a
>>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a
>>  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a
>>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a  lib/libLLVMCore.a
>>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
>>  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lz  -lrt  -ldl
>>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a && cd
>> /tmp/build/202003290405/tools/lld/tools/lld &&
>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> create_symlink lld /tmp/build/202003290405/./bin/lld-link && cd
>> /tmp/build/202003290405/tools/lld/tools/lld &&
>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> create_symlink lld /tmp/build/202003290405/./bin/ld.lld && cd
>> /tmp/build/202003290405/tools/lld/tools/lld &&
>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> create_symlink lld /tmp/build/202003290405/./bin/ld64.lld && cd
>> /tmp/build/202003290405/tools/lld/tools/lld &&
>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> create_symlink lld /tmp/build/202003290405/./bin/wasm-ld
>> terminating with uncaught exception of type std::__1::system_error:
>> thread constructor failed: Resource temporarily unavailable
>> PLEASE submit a bug report to https://bugs.llvm.org/ and include the
>> crash backtrace.
>>  #0 0x00000000007924f4 PrintStackTraceSignalHandler(void*)
>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
>>  #1 0x00000000007902ce llvm::sys::RunSignalHandlers()
>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
>>  #2 0x0000000000792bd5 SignalHandler(int)
>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
>>  #3 0x00007fddd54bb630 __restore_rt (/lib64/libpthread.so.0+0xf630)
>>  #4 0x00007fddd3b40377 raise
>>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
>>  #5 0x00007fddd3b41a68 abort
>> /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
>>  #6 0x00007fddd411ae8b
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
>>  #7 0x00007fddd40fdb9f demangling_terminate_handler()
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
>>  #8 0x00007fddd411a0a3 std::__terminate(void (*)())
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
>>  #9 0x00007fddd411ce06
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
>> #10 0x00007fddd411cd9f
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
>> #11 0x00007fddd43bded4 std::__1::__throw_system_error(int, char const*)
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
>> #12 0x0000000002a7a316 void*
>>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
>> std::__1::default_delete<std::__1::__thread_struct> >,
>> llvm::parallel::detail::(anonymous
>>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
>> >(void*) (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
>> #13 0x00007fddd54b3ea5 start_thread
>> /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
>> #14 0x00007fddd3c088cd clone
>>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
>> clang-11: error: unable to execute command: Aborted
>> clang-11: error: linker command failed due to signal (use -v to see
>> invocation)
>>
>> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic <nemanja.i.ibm at
gmail.com>
>> wrote:
>>
>>> The discussion in https://reviews.llvm.org/D76240
>>> may be of interest to the participants of this thread.
Specifically,
>>> Fangrui's planned work on a command line option to limit
threads for LLD.
>>> Until that becomes a reality, you might want to use the CMake
variable
>>> -DLLVM_ENABLE_THREADS=OFF when building the LLD that you are using
to
>>> prevent parallelism.
>>>
>>> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> Alex :
>>>>
>>>> Can you please try `numactl` or `taskset` after
>>>>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
>>>> ?
>>>>
>>>>
>>>>
>>>> There was a tiny bug in how *sched_getaffinity*() was used,
see:
>>>> https://reviews.llvm.org/D75153#1942336
>>>>
>>>>
>>>>
>>>> *De :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
>>>> *Envoyé :* March 28, 2020 12:11 PM
>>>> *À :* Itaru Kitayama <itaru.kitayama at gmail.com>
>>>> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org>; Alexandre
Ganea <
>>>> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng <maskray
at google.com>
>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively parallel
build
>>>> machine
>>>>
>>>>
>>>>
>>>> I also run into problems with lld on machines with lots of
cores (208),
>>>> but I run into rlimits for number of processes I can create,
something
>>>> outside of my control.
>>>>
>>>>
>>>>
>>>> Unfortunately things like numactl or taskset do not limit the
>>>> concurrency of lld currently.
>>>>
>>>>
>>>>
>>>> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via llvm-dev
<
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On a 1296-core Intel machine with 376 GB, setting
>>>>
>>>>
>>>>
>>>> -DLLVM_PARALLEL_LINK_JOB=1
>>>>
>>>>
>>>>
>>>> does not help (switching back to ld scales) see:
>>>>
>>>>
>>>>
>>>> [5085/5201] Linking CXX executable bin/clang-11
>>>> FAILED: bin/clang-11
>>>> : && /home/usr4/c74014i/opt/clang/current/bin/clang++ 
-stdlib=libc++
>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
>>>> -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter
>>>> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers
-pedantic
>>>> -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
>>>> -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
>>>> -Wstring-conversion -fdiagnostics-color -ffunction-sections
-fdata-sections
>>>> -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3
-DNDEBUG
>>>>  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
>>>> tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
>>>> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
>>>> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
>>>>  -o bin/clang-11  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMAArch64CodeGen.a
>>>>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Desc.a
>>>>  lib/libLLVMAArch64Disassembler.a  lib/libLLVMAArch64Info.a
>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>>>>  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a
>>>>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUInfo.a
>>>>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
>>>>  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a 
lib/libLLVMAVRDesc.a
>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a 
lib/libLLVMBPFDesc.a
>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>>>>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>>>>  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonDisassembler.a
>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>>>>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a
>>>>  lib/libLLVMLanaiDisassembler.a  lib/libLLVMLanaiInfo.a
>>>>  lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a
>>>>  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
>>>>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
>>>>  lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a
>>>>  lib/libLLVMMSP430Disassembler.a  lib/libLLVMMSP430Info.a
>>>>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a 
lib/libLLVMNVPTXInfo.a
>>>>  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMPowerPCAsmParser.a
>>>>  lib/libLLVMPowerPCDesc.a  lib/libLLVMPowerPCDisassembler.a
>>>>  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a
>>>>  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a
>>>>  lib/libLLVMRISCVDisassembler.a  lib/libLLVMRISCVInfo.a
>>>>  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>>>>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>>>>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZDisassembler.a
>>>>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDesc.a
>>>>  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyInfo.a
>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a 
lib/libLLVMX86Desc.a
>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a 
lib/libLLVMX86Utils.a
>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
>>>>  lib/libLLVMXCoreDisassembler.a  lib/libLLVMXCoreInfo.a
>>>>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a  lib/libLLVMCore.a
>>>>  lib/libLLVMipo.a  lib/libLLVMAggressiveInstCombine.a
>>>>  lib/libLLVMInstCombine.a  lib/libLLVMInstrumentation.a 
lib/libLLVMMC.a
>>>>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMOption.a
>>>>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a 
lib/libLLVMTransformUtils.a
>>>>  lib/libLLVMVectorize.a  -lpthread  lib/libclangBasic.a
>>>>  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a
>>>>  lib/libclangFrontendTool.a  lib/libclangSerialization.a
>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a 
lib/libLLVMAMDGPUDesc.a
>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a 
lib/libLLVMARMDesc.a
>>>>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a 
lib/libLLVMHexagonDesc.a
>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a 
lib/libLLVMLanaiInfo.a
>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a
>>>>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>>>>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a 
lib/libLLVMAsmPrinter.a
>>>>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>>>>  lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a 
lib/libLLVMCoverage.a
>>>>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMPasses.a
>>>>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a 
lib/libLLVMCoroutines.a
>>>>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a 
lib/libLLVMVectorize.a
>>>>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a 
lib/libLLVMAsmParser.a
>>>>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
>>>>  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a
>>>>  lib/libclangRewriteFrontend.a  lib/libclangARCMigrate.a
>>>>  lib/libclangStaticAnalyzerFrontend.a 
lib/libclangStaticAnalyzerCheckers.a
>>>>  lib/libclangStaticAnalyzerCore.a  lib/libclangCrossTU.a
>>>>  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a
>>>>  lib/libLLVMOption.a  lib/libclangParse.a 
lib/libclangSerialization.a
>>>>  lib/libclangSema.a  lib/libclangAnalysis.a 
lib/libclangASTMatchers.a
>>>>  lib/libclangEdit.a  lib/libclangFormat.a 
lib/libclangToolingInclusions.a
>>>>  lib/libclangToolingCore.a  lib/libclangAST.a 
lib/libLLVMFrontendOpenMP.a
>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
>>>>  lib/libLLVMProfileData.a  lib/libLLVMObject.a 
lib/libLLVMMCParser.a
>>>>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a 
lib/libclangRewrite.a
>>>>  lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMCore.a
>>>>  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a 
lib/libLLVMMC.a
>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMDebugInfoCodeView.a
>>>>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a  -lz  -lrt 
-ldl  -ltinfo
>>>>  -lpthread  -lm  lib/libLLVMDemangle.a && :
>>>> ld.lld: error: failed to open bin/clang-11: Cannot allocate
memory
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200329/a07204dc/attachment-0001.html>

Itaru Kitayama via llvm-dev

2020-Mar-28 19:40 UTC

head link

[llvm-dev] LLD issue on a massively parallel build machine

As I am not the admin, can't change the swap size.

On Sun, Mar 29, 2020 at 4:36 AM Itaru Kitayama <itaru.kitayama at
gmail.com>
wrote:
> $ free -g
>               total        used        free      shared  buff/cache
> available
> Mem:            376         149          20           1         207
>   225
> Swap:             3           0           3
>
> Too small swap size for a 72-core login machine?
>
> On Sun, Mar 29, 2020 at 4:28 AM Alex Brachet-Mialot <
> alexbrachetmialot at gmail.com> wrote:
>
>> Enable threads is for building llvm with or without thread support. To
my
>> knowledge it has no effect on the build system.
>>
>> Like was discussed above you could try limiting  the number of
processes
>> created (assuming your lld was after that commit) with numactl or
taskset.
>>
>> In your case though it looked like you were running out of memory. You
>> could try increasing your swap space?
>>
>> Alex
>>
>> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama <itaru.kitayama at
gmail.com>
>> wrote:
>>
>>> Hi,
>>> My configuration is below:
>>> cmake -GNinja -DLLVM_ENABLE_LLD=ON -DLLVM_ENABLE_THREADS=OFF
>>> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
>>> -DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
>>> -DLIBOMPTARGET_ENABLE_DEBUG=ON
>>> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
-DCMAKE_C_COMPILER=clang
>>> -DCMAKE_C_FLAGS="" -DCMAKE_CXX_COMPILER=clang++
-DCMAKE_CXX_FLAGS=""
>>>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
>>> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
>>> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
-DLLVM_TARGETS_TO_BUILD="all"
>>> /tmp/llvm-project/llvm
>>>
>>> but did not help my case:
>>>
>>> [...]
>>> [4802/5201] Linking CXX executable bin/lld
>>> FAILED: bin/lld
>>> : && /home/usr4/c74014i/opt/clang/current/bin/clang++ 
-stdlib=libc++
>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
>>> -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter
>>> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
>>> -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
>>> -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
>>> -Wstring-conversion -fdiagnostics-color -ffunction-sections
-fdata-sections
>>> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
>>> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o  -o bin/lld
>>>  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMSupport.a 
-lpthread
>>>  lib/liblldCommon.a  lib/liblldCOFF.a  lib/liblldDriver.a 
lib/liblldELF.a
>>>  lib/liblldMinGW.a  lib/liblldWasm.a  lib/liblldMachO.a
>>>  lib/liblldReaderWriter.a  lib/liblldYAML.a  lib/liblldCore.a
>>>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a 
lib/libLLVMLibDriver.a
>>>  lib/libLLVMWindowsManifest.a  /usr/lib64/libxml2.so 
lib/liblldCommon.a
>>>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
>>>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Disassembler.a
>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>>>  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUAsmParser.a
>>>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUDesc.a
>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a 
lib/libLLVMARMCodeGen.a
>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDisassembler.a
>>>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a 
lib/libLLVMAVRDesc.a
>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a 
lib/libLLVMBPFDesc.a
>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>>>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>>>  lib/libLLVMHexagonDisassembler.a  lib/libLLVMHexagonDesc.a
>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>>>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDisassembler.a
>>>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a 
lib/libLLVMMipsCodeGen.a
>>>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
>>>  lib/libLLVMMipsDisassembler.a  lib/libLLVMMipsInfo.a
>>>  lib/libLLVMMSP430CodeGen.a  lib/libLLVMMSP430AsmParser.a
>>>  lib/libLLVMMSP430Desc.a  lib/libLLVMMSP430Disassembler.a
>>>  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXCodeGen.a 
lib/libLLVMNVPTXDesc.a
>>>  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCCodeGen.a
>>>  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMPowerPCDesc.a
>>>  lib/libLLVMPowerPCDisassembler.a  lib/libLLVMPowerPCInfo.a
>>>  lib/libLLVMRISCVCodeGen.a  lib/libLLVMRISCVAsmParser.a
>>>  lib/libLLVMRISCVDesc.a  lib/libLLVMRISCVDisassembler.a
>>>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a 
lib/libLLVMSparcCodeGen.a
>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>>>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>>>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>>>  lib/libLLVMSystemZDisassembler.a  lib/libLLVMSystemZDesc.a
>>>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDisassembler.a
>>>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a 
lib/libLLVMGlobalISel.a
>>>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a 
lib/libLLVMX86Utils.a
>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAsmPrinter.a
>>>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>>>  lib/libLLVMXCoreDesc.a  lib/libLLVMXCoreDisassembler.a
>>>  lib/libLLVMMCDisassembler.a  lib/libLLVMXCoreInfo.a 
lib/libLLVMLTO.a
>>>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a
>>>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a  lib/libLLVMipo.a
>>>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a 
lib/libLLVMVectorize.a
>>>  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a
>>>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a 
lib/libLLVMIRReader.a
>>>  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a
>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a 
lib/libLLVMObject.a
>>>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a
>>>  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a
>>>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a  lib/libLLVMCore.a
>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
>>>  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lz  -lrt 
-ldl
>>>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a && cd
>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>>> create_symlink lld /tmp/build/202003290405/./bin/lld-link
&& cd
>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>>> create_symlink lld /tmp/build/202003290405/./bin/ld.lld &&
cd
>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>>> create_symlink lld /tmp/build/202003290405/./bin/ld64.lld
&& cd
>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>>> create_symlink lld /tmp/build/202003290405/./bin/wasm-ld
>>> terminating with uncaught exception of type std::__1::system_error:
>>> thread constructor failed: Resource temporarily unavailable
>>> PLEASE submit a bug report to https://bugs.llvm.org/ and include
the
>>> crash backtrace.
>>>  #0 0x00000000007924f4 PrintStackTraceSignalHandler(void*)
>>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
>>>  #1 0x00000000007902ce llvm::sys::RunSignalHandlers()
>>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
>>>  #2 0x0000000000792bd5 SignalHandler(int)
>>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
>>>  #3 0x00007fddd54bb630 __restore_rt (/lib64/libpthread.so.0+0xf630)
>>>  #4 0x00007fddd3b40377 raise
>>>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
>>>  #5 0x00007fddd3b41a68 abort
>>> /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
>>>  #6 0x00007fddd411ae8b
>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
>>>  #7 0x00007fddd40fdb9f demangling_terminate_handler()
>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
>>>  #8 0x00007fddd411a0a3 std::__terminate(void (*)())
>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
>>>  #9 0x00007fddd411ce06
>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
>>> #10 0x00007fddd411cd9f
>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
>>> #11 0x00007fddd43bded4 std::__1::__throw_system_error(int, char
const*)
>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
>>> #12 0x0000000002a7a316 void*
>>>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
>>> std::__1::default_delete<std::__1::__thread_struct> >,
>>> llvm::parallel::detail::(anonymous
>>>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
>>> >(void*)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
>>> #13 0x00007fddd54b3ea5 start_thread
>>> /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
>>> #14 0x00007fddd3c088cd clone
>>>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
>>> clang-11: error: unable to execute command: Aborted
>>> clang-11: error: linker command failed due to signal (use -v to see
>>> invocation)
>>>
>>> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic <
>>> nemanja.i.ibm at gmail.com> wrote:
>>>
>>>> The discussion in https://reviews.llvm.org/D76240
>>>> may be of interest to the participants of this thread.
Specifically,
>>>> Fangrui's planned work on a command line option to limit
threads for LLD.
>>>> Until that becomes a reality, you might want to use the CMake
variable
>>>> -DLLVM_ENABLE_THREADS=OFF when building the LLD that you are
using to
>>>> prevent parallelism.
>>>>
>>>> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via llvm-dev
<
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>> Alex :
>>>>>
>>>>> Can you please try `numactl` or `taskset` after
>>>>>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
>>>>> ?
>>>>>
>>>>>
>>>>>
>>>>> There was a tiny bug in how *sched_getaffinity*() was used,
see:
>>>>> https://reviews.llvm.org/D75153#1942336
>>>>>
>>>>>
>>>>>
>>>>> *De :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
>>>>> *Envoyé :* March 28, 2020 12:11 PM
>>>>> *À :* Itaru Kitayama <itaru.kitayama at gmail.com>
>>>>> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org>;
Alexandre Ganea <
>>>>> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng
<maskray at google.com>
>>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively parallel
build
>>>>> machine
>>>>>
>>>>>
>>>>>
>>>>> I also run into problems with lld on machines with lots of
cores
>>>>> (208), but I run into rlimits for number of processes I can
create,
>>>>> something outside of my control.
>>>>>
>>>>>
>>>>>
>>>>> Unfortunately things like numactl or taskset do not limit
the
>>>>> concurrency of lld currently.
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via llvm-dev
<
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> On a 1296-core Intel machine with 376 GB, setting
>>>>>
>>>>>
>>>>>
>>>>> -DLLVM_PARALLEL_LINK_JOB=1
>>>>>
>>>>>
>>>>>
>>>>> does not help (switching back to ld scales) see:
>>>>>
>>>>>
>>>>>
>>>>> [5085/5201] Linking CXX executable bin/clang-11
>>>>> FAILED: bin/clang-11
>>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
>>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
>>>>> -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter
>>>>> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers
-pedantic
>>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
>>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
>>>>> -Wstring-conversion -fdiagnostics-color -ffunction-sections
-fdata-sections
>>>>> -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3
-DNDEBUG
>>>>>  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
>>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
>>>>> tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
>>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
>>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
>>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
>>>>>  -o bin/clang-11  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMAArch64CodeGen.a
>>>>>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Desc.a
>>>>>  lib/libLLVMAArch64Disassembler.a  lib/libLLVMAArch64Info.a
>>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>>>>>  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a
>>>>>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUInfo.a
>>>>>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
>>>>>  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
>>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a 
lib/libLLVMAVRDesc.a
>>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a 
lib/libLLVMBPFDesc.a
>>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>>>>>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>>>>>  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonDisassembler.a
>>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>>>>>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a
>>>>>  lib/libLLVMLanaiDisassembler.a  lib/libLLVMLanaiInfo.a
>>>>>  lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a
>>>>>  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
>>>>>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
>>>>>  lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a
>>>>>  lib/libLLVMMSP430Disassembler.a  lib/libLLVMMSP430Info.a
>>>>>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a 
lib/libLLVMNVPTXInfo.a
>>>>>  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMPowerPCAsmParser.a
>>>>>  lib/libLLVMPowerPCDesc.a  lib/libLLVMPowerPCDisassembler.a
>>>>>  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a
>>>>>  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a
>>>>>  lib/libLLVMRISCVDisassembler.a  lib/libLLVMRISCVInfo.a
>>>>>  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
>>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>>>>>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>>>>>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZDisassembler.a
>>>>>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDesc.a
>>>>>  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyInfo.a
>>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a 
lib/libLLVMX86Desc.a
>>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a 
lib/libLLVMX86Utils.a
>>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
>>>>>  lib/libLLVMXCoreDisassembler.a  lib/libLLVMXCoreInfo.a
>>>>>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a 
lib/libLLVMCore.a
>>>>>  lib/libLLVMipo.a  lib/libLLVMAggressiveInstCombine.a
>>>>>  lib/libLLVMInstCombine.a  lib/libLLVMInstrumentation.a 
lib/libLLVMMC.a
>>>>>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMOption.a
>>>>>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a 
lib/libLLVMTransformUtils.a
>>>>>  lib/libLLVMVectorize.a  -lpthread  lib/libclangBasic.a
>>>>>  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a
>>>>>  lib/libclangFrontendTool.a  lib/libclangSerialization.a
>>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a 
lib/libLLVMAMDGPUDesc.a
>>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a 
lib/libLLVMARMDesc.a
>>>>>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a 
lib/libLLVMHexagonDesc.a
>>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a 
lib/libLLVMLanaiInfo.a
>>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a
>>>>>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>>>>>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a 
lib/libLLVMAsmPrinter.a
>>>>>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>>>>>  lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a 
lib/libLLVMCoverage.a
>>>>>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMPasses.a
>>>>>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a 
lib/libLLVMCoroutines.a
>>>>>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a 
lib/libLLVMVectorize.a
>>>>>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a 
lib/libLLVMAsmParser.a
>>>>>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
>>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
>>>>>  lib/libclangRewriteFrontend.a  lib/libclangARCMigrate.a
>>>>>  lib/libclangStaticAnalyzerFrontend.a 
lib/libclangStaticAnalyzerCheckers.a
>>>>>  lib/libclangStaticAnalyzerCore.a  lib/libclangCrossTU.a
>>>>>  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a
>>>>>  lib/libLLVMOption.a  lib/libclangParse.a 
lib/libclangSerialization.a
>>>>>  lib/libclangSema.a  lib/libclangAnalysis.a 
lib/libclangASTMatchers.a
>>>>>  lib/libclangEdit.a  lib/libclangFormat.a 
lib/libclangToolingInclusions.a
>>>>>  lib/libclangToolingCore.a  lib/libclangAST.a 
lib/libLLVMFrontendOpenMP.a
>>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
>>>>>  lib/libLLVMProfileData.a  lib/libLLVMObject.a 
lib/libLLVMMCParser.a
>>>>>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a 
lib/libclangRewrite.a
>>>>>  lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMCore.a
>>>>>  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a 
lib/libLLVMMC.a
>>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMDebugInfoCodeView.a
>>>>>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a  -lz  -lrt
-ldl  -ltinfo
>>>>>  -lpthread  -lm  lib/libLLVMDemangle.a && :
>>>>> ld.lld: error: failed to open bin/clang-11: Cannot allocate
memory
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200329/5aa67831/attachment-0001.html>

Alexandre Ganea via llvm-dev

2020-Mar-28 19:42 UTC

head link

[llvm-dev] LLD issue on a massively parallel build machine

Would `taskset -c 0-3 ninja check-all -j4` work?

De : Itaru Kitayama <itaru.kitayama at gmail.com>
Envoyé : March 28, 2020 3:37 PM
À : Alex Brachet-Mialot <alexbrachetmialot at gmail.com>
Cc : Alexandre Ganea <alexandre.ganea at ubisoft.com>; LLVM Dev
<llvm-dev at lists.llvm.org>; Nemanja Ivanovic <nemanja.i.ibm at
gmail.com>
Objet : Re: [llvm-dev] LLD issue on a massively parallel build machine

$ free -g
              total        used        free      shared  buff/cache   available
Mem:            376         149          20           1         207         225
Swap:             3           0           3

Too small swap size for a 72-core login machine?

On Sun, Mar 29, 2020 at 4:28 AM Alex Brachet-Mialot <alexbrachetmialot at
gmail.com<mailto:alexbrachetmialot at gmail.com>> wrote:
Enable threads is for building llvm with or without thread support. To my
knowledge it has no effect on the build system.

Like was discussed above you could try limiting  the number of processes created
(assuming your lld was after that commit) with numactl or taskset.

In your case though it looked like you were running out of memory. You could try
increasing your swap space?

Alex

On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama <itaru.kitayama at
gmail.com<mailto:itaru.kitayama at gmail.com>> wrote:
Hi,
My configuration is below:
cmake -GNinja -DLLVM_ENABLE_LLD=ON -DLLVM_ENABLE_THREADS=OFF
-DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
-DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
-DLIBOMPTARGET_ENABLE_DEBUG=ON -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
-DCMAKE_C_COMPILER=clang -DCMAKE_C_FLAGS=""
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS=""
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
-DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
-DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
-DLLVM_TARGETS_TO_BUILD="all" /tmp/llvm-project/llvm

but did not help my case:

[...]
[4802/5201] Linking CXX executable bin/lld
FAILED: bin/lld
: && /home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
-fPIC -fvisibility-inlines-hidden -Werror=date-time
-Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter
-Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
-Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
-Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
-Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -O3
-DNDEBUG  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
-Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o  -o bin/lld 
-Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMSupport.a  -lpthread 
lib/liblldCommon.a  lib/liblldCOFF.a  lib/liblldDriver.a  lib/liblldELF.a 
lib/liblldMinGW.a  lib/liblldWasm.a  lib/liblldMachO.a  lib/liblldReaderWriter.a
lib/liblldYAML.a  lib/liblldCore.a  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a 
lib/libLLVMLibDriver.a  lib/libLLVMWindowsManifest.a  /usr/lib64/libxml2.so 
lib/liblldCommon.a  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a 
lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Disassembler.a 
lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a  lib/libLLVMAArch64Utils.a 
lib/libLLVMAMDGPUCodeGen.a  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUAsmParser.a
lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUDesc.a 
lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a 
lib/libLLVMARMAsmParser.a  lib/libLLVMARMDisassembler.a  lib/libLLVMARMDesc.a 
lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a  lib/libLLVMAVRCodeGen.a 
lib/libLLVMAVRAsmParser.a  lib/libLLVMAVRDesc.a  lib/libLLVMAVRDisassembler.a 
lib/libLLVMAVRInfo.a  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a 
lib/libLLVMBPFDesc.a  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a 
lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a 
lib/libLLVMHexagonDisassembler.a  lib/libLLVMHexagonDesc.a 
lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a  lib/libLLVMLanaiAsmParser.a
lib/libLLVMLanaiDisassembler.a  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a 
lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a 
lib/libLLVMMipsDisassembler.a  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a 
lib/libLLVMMSP430Disassembler.a  lib/libLLVMMSP430Info.a 
lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMNVPTXInfo.a 
lib/libLLVMPowerPCCodeGen.a  lib/libLLVMPowerPCAsmParser.a 
lib/libLLVMPowerPCDesc.a  lib/libLLVMPowerPCDisassembler.a 
lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a  lib/libLLVMRISCVAsmParser.a
lib/libLLVMRISCVDesc.a  lib/libLLVMRISCVDisassembler.a  lib/libLLVMRISCVInfo.a 
lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a  lib/libLLVMSparcAsmParser.a 
lib/libLLVMSparcDesc.a  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a 
lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a 
lib/libLLVMSystemZDisassembler.a  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a 
lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a 
lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a 
lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a  lib/libLLVMX86Disassembler.a 
lib/libLLVMX86Info.a  lib/libLLVMX86Utils.a  lib/libLLVMXCoreCodeGen.a 
lib/libLLVMAsmPrinter.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a 
lib/libLLVMXCoreDesc.a  lib/libLLVMXCoreDisassembler.a 
lib/libLLVMMCDisassembler.a  lib/libLLVMXCoreInfo.a  lib/libLLVMLTO.a 
lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a  lib/libLLVMCodeGen.a 
lib/libLLVMTarget.a  lib/libLLVMCoroutines.a  lib/libLLVMipo.a 
lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a  lib/libLLVMVectorize.a 
lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a 
lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMIRReader.a 
lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a 
lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMObject.a 
lib/libLLVMBitReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a 
lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMTextAPI.a 
lib/libLLVMProfileData.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a 
lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lz 
-lrt  -ldl  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a && cd
/tmp/build/202003290405/tools/lld/tools/lld &&
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld
/tmp/build/202003290405/./bin/lld-link && cd
/tmp/build/202003290405/tools/lld/tools/lld &&
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld
/tmp/build/202003290405/./bin/ld.lld && cd
/tmp/build/202003290405/tools/lld/tools/lld &&
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld
/tmp/build/202003290405/./bin/ld64.lld && cd
/tmp/build/202003290405/tools/lld/tools/lld &&
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld
/tmp/build/202003290405/./bin/wasm-ld
terminating with uncaught exception of type std::__1::system_error: thread
constructor failed: Resource temporarily unavailable
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash
backtrace.
 #0 0x00000000007924f4 PrintStackTraceSignalHandler(void*)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
 #1 0x00000000007902ce llvm::sys::RunSignalHandlers()
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
 #2 0x0000000000792bd5 SignalHandler(int)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
 #3 0x00007fddd54bb630 __restore_rt (/lib64/libpthread.so.0+0xf630)
 #4 0x00007fddd3b40377 raise
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
 #5 0x00007fddd3b41a68 abort
/usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
 #6 0x00007fddd411ae8b
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
 #7 0x00007fddd40fdb9f demangling_terminate_handler()
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
 #8 0x00007fddd411a0a3 std::__terminate(void (*)())
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
 #9 0x00007fddd411ce06
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
#10 0x00007fddd411cd9f
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
#11 0x00007fddd43bded4 std::__1::__throw_system_error(int, char const*)
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
#12 0x0000000002a7a316 void*
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
std::__1::default_delete<std::__1::__thread_struct> >,
llvm::parallel::detail::(anonymous
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
>(void*) (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
#13 0x00007fddd54b3ea5 start_thread
/usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
#14 0x00007fddd3c088cd clone
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
clang-11: error: unable to execute command: Aborted
clang-11: error: linker command failed due to signal (use -v to see invocation)

On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic <nemanja.i.ibm at
gmail.com<mailto:nemanja.i.ibm at gmail.com>> wrote:
The discussion in https://reviews.llvm.org/D76240
may be of interest to the participants of this thread. Specifically,
Fangrui's planned work on a command line option to limit threads for LLD.
Until that becomes a reality, you might want to use the CMake variable
-DLLVM_ENABLE_THREADS=OFF when building the LLD that you are using to prevent
parallelism.

On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Alex :
Can you please try `numactl` or `taskset` after
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
?

There was a tiny bug in how sched_getaffinity() was used, see:
https://reviews.llvm.org/D75153#1942336

De : Alex Brachet-Mialot <alexbrachetmialot at
gmail.com<mailto:alexbrachetmialot at gmail.com>>
Envoyé : March 28, 2020 12:11 PM
À : Itaru Kitayama <itaru.kitayama at gmail.com<mailto:itaru.kitayama at
gmail.com>>
Cc : LLVM Dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at
lists.llvm.org>>; Alexandre Ganea <alexandre.ganea at
ubisoft.com<mailto:alexandre.ganea at ubisoft.com>>; Fāng-ruì Sòng
<maskray at google.com<mailto:maskray at google.com>>
Objet : Re: [llvm-dev] LLD issue on a massively parallel build machine

I also run into problems with lld on machines with lots of cores (208), but I
run into rlimits for number of processes I can create, something outside of my
control.

Unfortunately things like numactl or taskset do not limit the concurrency of lld
currently.

On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hi,
On a 1296-core Intel machine with 376 GB, setting

-DLLVM_PARALLEL_LINK_JOB=1

does not help (switching back to ld scales) see:

[5085/5201] Linking CXX executable bin/clang-11
FAILED: bin/clang-11
: && /home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
-fPIC -fvisibility-inlines-hidden -Werror=date-time
-Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter
-Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
-Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
-Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
-Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections
-fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG 
-stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics -Wl,-allow-shlib-undefined  
-Wl,--export-dynamic  -Wl,-O3
tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o  -o
bin/clang-11  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMAArch64CodeGen.a  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Disassembler.a 
lib/libLLVMAArch64Info.a  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a 
lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a 
lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUInfo.a 
lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a  lib/libLLVMARMAsmParser.a 
lib/libLLVMARMDesc.a  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a 
lib/libLLVMAVRDesc.a  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a 
lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a  lib/libLLVMBPFDesc.a 
lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a  lib/libLLVMHexagonDesc.a 
lib/libLLVMHexagonDisassembler.a  lib/libLLVMHexagonInfo.a 
lib/libLLVMLanaiCodeGen.a  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a 
lib/libLLVMLanaiDisassembler.a  lib/libLLVMLanaiInfo.a  lib/libLLVMMipsCodeGen.a
lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a  lib/libLLVMMSP430AsmParser.a 
lib/libLLVMMSP430Desc.a  lib/libLLVMMSP430Disassembler.a 
lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a 
lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCCodeGen.a 
lib/libLLVMPowerPCAsmParser.a  lib/libLLVMPowerPCDesc.a 
lib/libLLVMPowerPCDisassembler.a  lib/libLLVMPowerPCInfo.a 
lib/libLLVMRISCVCodeGen.a  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a 
lib/libLLVMRISCVDisassembler.a  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a 
lib/libLLVMSparcCodeGen.a  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a 
lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a 
lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a 
lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZDisassembler.a 
lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a 
lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyDisassembler.a  lib/libLLVMWebAssemblyInfo.a 
lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a 
lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a  lib/libLLVMX86Utils.a 
lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a 
lib/libLLVMXCoreDisassembler.a  lib/libLLVMXCoreInfo.a  lib/libLLVMAnalysis.a 
lib/libLLVMCodeGen.a  lib/libLLVMCore.a  lib/libLLVMipo.a 
lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a 
lib/libLLVMInstrumentation.a  lib/libLLVMMC.a  lib/libLLVMMCParser.a 
lib/libLLVMObjCARCOpts.a  lib/libLLVMOption.a  lib/libLLVMScalarOpts.a 
lib/libLLVMSupport.a  lib/libLLVMTransformUtils.a  lib/libLLVMVectorize.a 
-lpthread  lib/libclangBasic.a  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a  lib/libclangFrontendTool.a  lib/libclangSerialization.a 
lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a  lib/libLLVMAArch64Utils.a 
lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUDesc.a  lib/libLLVMAMDGPUInfo.a 
lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonInfo.a 
lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a 
lib/libLLVMAsmPrinter.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a 
lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a  lib/libLLVMCoverage.a 
lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a 
lib/libLLVMipo.a  lib/libLLVMInstrumentation.a  lib/libLLVMVectorize.a 
lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a 
lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a  lib/libLLVMAggressiveInstCombine.a
lib/libLLVMInstCombine.a  lib/libclangRewriteFrontend.a 
lib/libclangARCMigrate.a  lib/libclangStaticAnalyzerFrontend.a 
lib/libclangStaticAnalyzerCheckers.a  lib/libclangStaticAnalyzerCore.a 
lib/libclangCrossTU.a  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a  lib/libLLVMOption.a  lib/libclangParse.a 
lib/libclangSerialization.a  lib/libclangSema.a  lib/libclangAnalysis.a 
lib/libclangASTMatchers.a  lib/libclangEdit.a  lib/libclangFormat.a 
lib/libclangToolingInclusions.a  lib/libclangToolingCore.a  lib/libclangAST.a 
lib/libLLVMFrontendOpenMP.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a 
lib/libLLVMProfileData.a  lib/libLLVMObject.a  lib/libLLVMMCParser.a 
lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a  lib/libclangRewrite.a 
lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMCore.a  lib/libLLVMRemarks.a 
lib/libLLVMBitstreamReader.a  lib/libLLVMMC.a  lib/libLLVMBinaryFormat.a 
lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a 
-lz  -lrt  -ldl  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a && :
ld.lld: error: failed to open bin/clang-11: Cannot allocate memory
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200328/0c816c78/attachment-0001.html>

Itaru Kitayama via llvm-dev

2020-Mar-28 19:57 UTC

head link

[llvm-dev] LLD issue on a massively parallel build machine

That is slowing down the build visibly, for the speed I should stick with
ld at the moment.


On Sun, Mar 29, 2020 at 4:42 AM Alexandre Ganea <alexandre.ganea at
ubisoft.com>
wrote:
> Would `taskset -c 0-3 ninja check-all -j4` work?
>
>
>
> *De :* Itaru Kitayama <itaru.kitayama at gmail.com>
> *Envoyé :* March 28, 2020 3:37 PM
> *À :* Alex Brachet-Mialot <alexbrachetmialot at gmail.com>
> *Cc :* Alexandre Ganea <alexandre.ganea at ubisoft.com>; LLVM Dev
<
> llvm-dev at lists.llvm.org>; Nemanja Ivanovic <nemanja.i.ibm at
gmail.com>
> *Objet :* Re: [llvm-dev] LLD issue on a massively parallel build machine
>
>
>
> $ free -g
>               total        used        free      shared  buff/cache
> available
> Mem:            376         149          20           1         207
>   225
> Swap:             3           0           3
>
>
>
> Too small swap size for a 72-core login machine?
>
>
>
> On Sun, Mar 29, 2020 at 4:28 AM Alex Brachet-Mialot <
> alexbrachetmialot at gmail.com> wrote:
>
> Enable threads is for building llvm with or without thread support. To my
> knowledge it has no effect on the build system.
>
>
>
> Like was discussed above you could try limiting  the number of processes
> created (assuming your lld was after that commit) with numactl or taskset.
>
>
>
> In your case though it looked like you were running out of memory. You
> could try increasing your swap space?
>
>
>
> Alex
>
>
>
> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama <itaru.kitayama at
gmail.com>
> wrote:
>
> Hi,
>
> My configuration is below:
>
> cmake -GNinja -DLLVM_ENABLE_LLD=ON -DLLVM_ENABLE_THREADS=OFF
> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
> -DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
> -DLIBOMPTARGET_ENABLE_DEBUG=ON
> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today} -DCMAKE_C_COMPILER=clang
> -DCMAKE_C_FLAGS="" -DCMAKE_CXX_COMPILER=clang++
-DCMAKE_CXX_FLAGS=""
>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
-DLLVM_TARGETS_TO_BUILD="all"
> /tmp/llvm-project/llvm
>
>
>
> but did not help my case:
>
>
>
> [...]
>
> [4802/5201] Linking CXX executable bin/lld
> FAILED: bin/lld
> : && /home/usr4/c74014i/opt/clang/current/bin/clang++ 
-stdlib=libc++
> -fPIC -fvisibility-inlines-hidden -Werror=date-time
> -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter
> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
> -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
> -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections
> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o  -o bin/lld
>  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMSupport.a  -lpthread
>  lib/liblldCommon.a  lib/liblldCOFF.a  lib/liblldDriver.a  lib/liblldELF.a
>  lib/liblldMinGW.a  lib/liblldWasm.a  lib/liblldMachO.a
>  lib/liblldReaderWriter.a  lib/liblldYAML.a  lib/liblldCore.a
>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a  lib/libLLVMLibDriver.a
>  lib/libLLVMWindowsManifest.a  /usr/lib64/libxml2.so  lib/liblldCommon.a
>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Disassembler.a
>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUAsmParser.a
>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUDesc.a
>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDisassembler.a
>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a  lib/libLLVMAVRDesc.a
>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a  lib/libLLVMBPFDesc.a
>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>  lib/libLLVMHexagonDisassembler.a  lib/libLLVMHexagonDesc.a
>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDisassembler.a
>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a  lib/libLLVMMipsCodeGen.a
>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
>  lib/libLLVMMipsDisassembler.a  lib/libLLVMMipsInfo.a
>  lib/libLLVMMSP430CodeGen.a  lib/libLLVMMSP430AsmParser.a
>  lib/libLLVMMSP430Desc.a  lib/libLLVMMSP430Disassembler.a
>  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a
>  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCCodeGen.a
>  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMPowerPCDesc.a
>  lib/libLLVMPowerPCDisassembler.a  lib/libLLVMPowerPCInfo.a
>  lib/libLLVMRISCVCodeGen.a  lib/libLLVMRISCVAsmParser.a
>  lib/libLLVMRISCVDesc.a  lib/libLLVMRISCVDisassembler.a
>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>  lib/libLLVMSystemZDisassembler.a  lib/libLLVMSystemZDesc.a
>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>  lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMWebAssemblyDisassembler.a
>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a
>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a  lib/libLLVMX86Utils.a
>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAsmPrinter.a
>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>  lib/libLLVMXCoreDesc.a  lib/libLLVMXCoreDisassembler.a
>  lib/libLLVMMCDisassembler.a  lib/libLLVMXCoreInfo.a  lib/libLLVMLTO.a
>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a  lib/libLLVMCodeGen.a
>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a  lib/libLLVMipo.a
>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a  lib/libLLVMVectorize.a
>  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a
>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMIRReader.a
>  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a
>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMObject.a
>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a
>  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a
>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a  lib/libLLVMCore.a
>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
>  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lz  -lrt  -ldl
>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a && cd
> /tmp/build/202003290405/tools/lld/tools/lld &&
> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> create_symlink lld /tmp/build/202003290405/./bin/lld-link && cd
> /tmp/build/202003290405/tools/lld/tools/lld &&
> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> create_symlink lld /tmp/build/202003290405/./bin/ld.lld && cd
> /tmp/build/202003290405/tools/lld/tools/lld &&
> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> create_symlink lld /tmp/build/202003290405/./bin/ld64.lld && cd
> /tmp/build/202003290405/tools/lld/tools/lld &&
> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> create_symlink lld /tmp/build/202003290405/./bin/wasm-ld
> terminating with uncaught exception of type std::__1::system_error: thread
> constructor failed: Resource temporarily unavailable
> PLEASE submit a bug report to https://bugs.llvm.org/ and include the
> crash backtrace.
>  #0 0x00000000007924f4 PrintStackTraceSignalHandler(void*)
> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
>  #1 0x00000000007902ce llvm::sys::RunSignalHandlers()
> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
>  #2 0x0000000000792bd5 SignalHandler(int)
> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
>  #3 0x00007fddd54bb630 __restore_rt (/lib64/libpthread.so.0+0xf630)
>  #4 0x00007fddd3b40377 raise
>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
>  #5 0x00007fddd3b41a68 abort
> /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
>  #6 0x00007fddd411ae8b
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
>  #7 0x00007fddd40fdb9f demangling_terminate_handler()
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
>  #8 0x00007fddd411a0a3 std::__terminate(void (*)())
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
>  #9 0x00007fddd411ce06
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
> #10 0x00007fddd411cd9f
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
> #11 0x00007fddd43bded4 std::__1::__throw_system_error(int, char const*)
> (/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
> #12 0x0000000002a7a316 void*
>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
> std::__1::default_delete<std::__1::__thread_struct> >,
> llvm::parallel::detail::(anonymous
>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
> >(void*) (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
> #13 0x00007fddd54b3ea5 start_thread
> /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
> #14 0x00007fddd3c088cd clone
>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
> clang-11: error: unable to execute command: Aborted
> clang-11: error: linker command failed due to signal (use -v to see
> invocation)
>
>
>
> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic <nemanja.i.ibm at
gmail.com>
> wrote:
>
> The discussion in https://reviews.llvm.org/D76240
>
> may be of interest to the participants of this thread. Specifically,
> Fangrui's planned work on a command line option to limit threads for
LLD.
> Until that becomes a reality, you might want to use the CMake variable
> -DLLVM_ENABLE_THREADS=OFF when building the LLD that you are using to
> prevent parallelism.
>
>
>
> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Alex :
>
> Can you please try `numactl` or `taskset` after
>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
> ?
>
>
>
> There was a tiny bug in how *sched_getaffinity*() was used, see:
> https://reviews.llvm.org/D75153#1942336
>
>
>
> *De :* Alex Brachet-Mialot <alexbrachetmialot at gmail.com>
> *Envoyé :* March 28, 2020 12:11 PM
> *À :* Itaru Kitayama <itaru.kitayama at gmail.com>
> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org>; Alexandre Ganea <
> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng <maskray at
google.com>
> *Objet :* Re: [llvm-dev] LLD issue on a massively parallel build machine
>
>
>
> I also run into problems with lld on machines with lots of cores (208),
> but I run into rlimits for number of processes I can create, something
> outside of my control.
>
>
>
> Unfortunately things like numactl or taskset do not limit the
> concurrency of lld currently.
>
>
>
> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi,
>
> On a 1296-core Intel machine with 376 GB, setting
>
>
>
> -DLLVM_PARALLEL_LINK_JOB=1
>
>
>
> does not help (switching back to ld scales) see:
>
>
>
> [5085/5201] Linking CXX executable bin/clang-11
> FAILED: bin/clang-11
> : && /home/usr4/c74014i/opt/clang/current/bin/clang++ 
-stdlib=libc++
> -fPIC -fvisibility-inlines-hidden -Werror=date-time
> -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter
> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
> -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
> -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections
> -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG
>  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
> tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
>  -o bin/clang-11  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMAArch64CodeGen.a
>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Desc.a
>  lib/libLLVMAArch64Disassembler.a  lib/libLLVMAArch64Info.a
>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a
>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUInfo.a
>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
>  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a  lib/libLLVMAVRDesc.a
>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a  lib/libLLVMBPFDesc.a
>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonDisassembler.a
>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a
>  lib/libLLVMLanaiDisassembler.a  lib/libLLVMLanaiInfo.a
>  lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a
>  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
>  lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a
>  lib/libLLVMMSP430Disassembler.a  lib/libLLVMMSP430Info.a
>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMNVPTXInfo.a
>  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMPowerPCAsmParser.a
>  lib/libLLVMPowerPCDesc.a  lib/libLLVMPowerPCDisassembler.a
>  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a
>  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a
>  lib/libLLVMRISCVDisassembler.a  lib/libLLVMRISCVInfo.a
>  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZDisassembler.a
>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>  lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMWebAssemblyDesc.a
>  lib/libLLVMWebAssemblyDisassembler.a  lib/libLLVMWebAssemblyInfo.a
>  lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a  lib/libLLVMX86Utils.a
>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
>  lib/libLLVMXCoreDisassembler.a  lib/libLLVMXCoreInfo.a
>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a  lib/libLLVMCore.a
>  lib/libLLVMipo.a  lib/libLLVMAggressiveInstCombine.a
>  lib/libLLVMInstCombine.a  lib/libLLVMInstrumentation.a  lib/libLLVMMC.a
>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMOption.a
>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a  lib/libLLVMTransformUtils.a
>  lib/libLLVMVectorize.a  -lpthread  lib/libclangBasic.a
>  lib/libclangCodeGen.a  lib/libclangDriver.a  lib/libclangFrontend.a
>  lib/libclangFrontendTool.a  lib/libclangSerialization.a
>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUDesc.a
>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMDesc.a
>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a  lib/libLLVMHexagonDesc.a
>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a
>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a
>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a  lib/libLLVMAsmPrinter.a
>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>  lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a  lib/libLLVMCoverage.a
>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a
>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a
>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a  lib/libLLVMVectorize.a
>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a
>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
>  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a
>  lib/libclangRewriteFrontend.a  lib/libclangARCMigrate.a
>  lib/libclangStaticAnalyzerFrontend.a  lib/libclangStaticAnalyzerCheckers.a
>  lib/libclangStaticAnalyzerCore.a  lib/libclangCrossTU.a
>  lib/libclangIndex.a  lib/libclangFrontend.a  lib/libclangDriver.a
>  lib/libLLVMOption.a  lib/libclangParse.a  lib/libclangSerialization.a
>  lib/libclangSema.a  lib/libclangAnalysis.a  lib/libclangASTMatchers.a
>  lib/libclangEdit.a  lib/libclangFormat.a  lib/libclangToolingInclusions.a
>  lib/libclangToolingCore.a  lib/libclangAST.a  lib/libLLVMFrontendOpenMP.a
>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
>  lib/libLLVMProfileData.a  lib/libLLVMObject.a  lib/libLLVMMCParser.a
>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a  lib/libclangRewrite.a
>  lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMCore.a
>  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMMC.a
>  lib/libLLVMBinaryFormat.a  lib/libLLVMDebugInfoCodeView.a
>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a  -lz  -lrt  -ldl  -ltinfo
>  -lpthread  -lm  lib/libLLVMDemangle.a && :
> ld.lld: error: failed to open bin/clang-11: Cannot allocate memory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200329/a2a4ccbe/attachment-0001.html>

Reasonably Related Threads

Search for more seemingly similar threads

llvm dev - Mar 2020 - LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

Reasonably Related Threads