thr3ads.net - llvm dev - [llvm-dev] LLD issue on a massively parallel build machine [Apr 2020]

If this information is useful, please help other people find it:
Share via:

Fangrui Song via llvm-dev

2020-Apr-01 16:49 UTC

[llvm-dev] LLD issue on a massively parallel build machine

On 2020-03-29, Nemanja Ivanovic via llvm-dev wrote:>Glad you got it working.
>My suggestion about LLVM_ENABLE_THREADS didn't work because you
didn't
>apply it when building the build linker.
>
>When you don't have the ability to rebuild the build compiler, this
doesn't
>apply. In those cases, I end up doing a dirty hack where I use a wrapper
>script with something like:
>
><path-to-build-compiler> "$@" -Wl,--no-threads
>
>To invoke the build compiler. That way you still get full parallelism while
>compiling and linking, but you prevent LLD from using threads.

https://reviews.llvm.org/D76885 (committed yesterday) changed
--no-threads to --threads={1,2,...} (--no-threads is used rarely (GNU ld
does not support it) so I'd rather not keep the option for
compatibility).

You may try -Wl,--threads=1
>On Sat., Mar. 28, 2020, 4:19 p.m. Itaru Kitayama, <itaru.kitayama at
gmail.com>
>wrote:
>
>> Good news, I was able to use up to 37 cores for LLVM build with LLD.
>> The build speed, did not measure precisely though, is comparable to the
>> build with GNU ld case.
>>
>> Thank you all for your help!
>>
>> On Sun, Mar 29, 2020 at 5:04 AM Alex Brachet-Mialot <
>> alexbrachetmialot at gmail.com> wrote:
>>
>>> Yes unfortunately that would limit you to 4 cores.
>>>
>>> There’s no easy way to use lld with —no-threads with our build
system at
>>> the moment unfortunately.
>>>
>>> I also had just been using ld.gold, but will switch now that
numactl
>>> works for me.
>>>
>>> On Sat, Mar 28, 2020 at 3:58 PM Itaru Kitayama <itaru.kitayama
at gmail.com>
>>> wrote:
>>>
>>>> That is slowing down the build visibly, for the speed I should
stick
>>>> with ld at the moment.
>>>>
>>>>
>>>> On Sun, Mar 29, 2020 at 4:42 AM Alexandre Ganea <
>>>> alexandre.ganea at ubisoft.com> wrote:
>>>>
>>>>> Would `taskset -c 0-3 ninja check-all -j4` work?
>>>>>
>>>>>
>>>>>
>>>>> *De :* Itaru Kitayama <itaru.kitayama at gmail.com>
>>>>> *Envoyé :* March 28, 2020 3:37 PM
>>>>> *À :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
>>>>> *Cc :* Alexandre Ganea <alexandre.ganea at
ubisoft.com>; LLVM Dev <
>>>>> llvm-dev at lists.llvm.org>; Nemanja Ivanovic
<nemanja.i.ibm at gmail.com>
>>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively parallel
build
>>>>> machine
>>>>>
>>>>>
>>>>>
>>>>> $ free -g
>>>>>               total        used        free      shared 
buff/cache
>>>>> available
>>>>> Mem:            376         149          20           1    
207
>>>>>     225
>>>>> Swap:             3           0           3
>>>>>
>>>>>
>>>>>
>>>>> Too small swap size for a 72-core login machine?
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Mar 29, 2020 at 4:28 AM Alex Brachet-Mialot <
>>>>> alexbrachetmialot at gmail.com> wrote:
>>>>>
>>>>> Enable threads is for building llvm with or without thread
support. To
>>>>> my knowledge it has no effect on the build system.
>>>>>
>>>>>
>>>>>
>>>>> Like was discussed above you could try limiting  the number
of
>>>>> processes created (assuming your lld was after that commit)
with numactl or
>>>>> taskset.
>>>>>
>>>>>
>>>>>
>>>>> In your case though it looked like you were running out of
memory. You
>>>>> could try increasing your swap space?
>>>>>
>>>>>
>>>>>
>>>>> Alex
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama <
>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> My configuration is below:
>>>>>
>>>>> cmake -GNinja -DLLVM_ENABLE_LLD=ON
-DLLVM_ENABLE_THREADS=OFF
>>>>> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
>>>>> -DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
>>>>> -DLIBOMPTARGET_ENABLE_DEBUG=ON
>>>>> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
-DCMAKE_C_COMPILER=clang
>>>>> -DCMAKE_C_FLAGS="" -DCMAKE_CXX_COMPILER=clang++
-DCMAKE_CXX_FLAGS=""
>>>>>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
>>>>> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
>>>>> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
-DLLVM_TARGETS_TO_BUILD="all"
>>>>> /tmp/llvm-project/llvm
>>>>>
>>>>>
>>>>>
>>>>> but did not help my case:
>>>>>
>>>>>
>>>>>
>>>>> [...]
>>>>>
>>>>> [4802/5201] Linking CXX executable bin/lld
>>>>> FAILED: bin/lld
>>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
>>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
>>>>> -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter
>>>>> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers
-pedantic
>>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
>>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
>>>>> -Wstring-conversion -fdiagnostics-color -ffunction-sections
-fdata-sections
>>>>> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld
-Wl,--color-diagnostics
>>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
>>>>> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o  -o
bin/lld
>>>>>  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMSupport.a  -lpthread
>>>>>  lib/liblldCommon.a  lib/liblldCOFF.a  lib/liblldDriver.a 
lib/liblldELF.a
>>>>>  lib/liblldMinGW.a  lib/liblldWasm.a  lib/liblldMachO.a
>>>>>  lib/liblldReaderWriter.a  lib/liblldYAML.a 
lib/liblldCore.a
>>>>>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a 
lib/libLLVMLibDriver.a
>>>>>  lib/libLLVMWindowsManifest.a  /usr/lib64/libxml2.so 
lib/liblldCommon.a
>>>>>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
>>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Disassembler.a
>>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>>>>>  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUAsmParser.a
>>>>>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUDesc.a
>>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a 
lib/libLLVMARMCodeGen.a
>>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDisassembler.a
>>>>>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
>>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a 
lib/libLLVMAVRDesc.a
>>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a 
lib/libLLVMBPFDesc.a
>>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>>>>>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>>>>>  lib/libLLVMHexagonDisassembler.a  lib/libLLVMHexagonDesc.a
>>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>>>>>  lib/libLLVMLanaiAsmParser.a 
lib/libLLVMLanaiDisassembler.a
>>>>>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a 
lib/libLLVMMipsCodeGen.a
>>>>>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
>>>>>  lib/libLLVMMipsDisassembler.a  lib/libLLVMMipsInfo.a
>>>>>  lib/libLLVMMSP430CodeGen.a  lib/libLLVMMSP430AsmParser.a
>>>>>  lib/libLLVMMSP430Desc.a  lib/libLLVMMSP430Disassembler.a
>>>>>  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXCodeGen.a 
lib/libLLVMNVPTXDesc.a
>>>>>  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCCodeGen.a
>>>>>  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMPowerPCDesc.a
>>>>>  lib/libLLVMPowerPCDisassembler.a  lib/libLLVMPowerPCInfo.a
>>>>>  lib/libLLVMRISCVCodeGen.a  lib/libLLVMRISCVAsmParser.a
>>>>>  lib/libLLVMRISCVDesc.a  lib/libLLVMRISCVDisassembler.a
>>>>>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a 
lib/libLLVMSparcCodeGen.a
>>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>>>>>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>>>>>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>>>>>  lib/libLLVMSystemZDisassembler.a  lib/libLLVMSystemZDesc.a
>>>>>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDisassembler.a
>>>>>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a 
lib/libLLVMGlobalISel.a
>>>>>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a 
lib/libLLVMX86Utils.a
>>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAsmPrinter.a
>>>>>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>>>>>  lib/libLLVMXCoreDesc.a  lib/libLLVMXCoreDisassembler.a
>>>>>  lib/libLLVMMCDisassembler.a  lib/libLLVMXCoreInfo.a 
lib/libLLVMLTO.a
>>>>>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a
>>>>>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a 
lib/libLLVMipo.a
>>>>>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a 
lib/libLLVMVectorize.a
>>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
>>>>>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a 
lib/libLLVMIRReader.a
>>>>>  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a
>>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a 
lib/libLLVMObject.a
>>>>>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a 
lib/libLLVMMC.a
>>>>>  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a
>>>>>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a 
lib/libLLVMCore.a
>>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
>>>>>  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lz 
-lrt  -ldl
>>>>>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a &&
cd
>>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake
-E
>>>>> create_symlink lld /tmp/build/202003290405/./bin/lld-link
&& cd
>>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake
-E
>>>>> create_symlink lld /tmp/build/202003290405/./bin/ld.lld
&& cd
>>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake
-E
>>>>> create_symlink lld /tmp/build/202003290405/./bin/ld64.lld
&& cd
>>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
>>>>> /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake
-E
>>>>> create_symlink lld /tmp/build/202003290405/./bin/wasm-ld
>>>>> terminating with uncaught exception of type
std::__1::system_error:
>>>>> thread constructor failed: Resource temporarily unavailable
>>>>> PLEASE submit a bug report to https://bugs.llvm.org/ and
include the
>>>>> crash backtrace.
>>>>>  #0 0x00000000007924f4 PrintStackTraceSignalHandler(void*)
>>>>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
>>>>>  #1 0x00000000007902ce llvm::sys::RunSignalHandlers()
>>>>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
>>>>>  #2 0x0000000000792bd5 SignalHandler(int)
>>>>> (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
>>>>>  #3 0x00007fddd54bb630 __restore_rt
(/lib64/libpthread.so.0+0xf630)
>>>>>  #4 0x00007fddd3b40377 raise
>>>>>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
>>>>>  #5 0x00007fddd3b41a68 abort
>>>>> /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
>>>>>  #6 0x00007fddd411ae8b
>>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
>>>>>  #7 0x00007fddd40fdb9f demangling_terminate_handler()
>>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
>>>>>  #8 0x00007fddd411a0a3 std::__terminate(void (*)())
>>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
>>>>>  #9 0x00007fddd411ce06
>>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
>>>>> #10 0x00007fddd411cd9f
>>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
>>>>> #11 0x00007fddd43bded4 std::__1::__throw_system_error(int,
char const*)
>>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
>>>>> #12 0x0000000002a7a316 void*
>>>>>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
>>>>> std::__1::default_delete<std::__1::__thread_struct>
>,
>>>>> llvm::parallel::detail::(anonymous
>>>>>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
>>>>> >(void*)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
>>>>> #13 0x00007fddd54b3ea5 start_thread
>>>>>
/usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
>>>>> #14 0x00007fddd3c088cd clone
>>>>>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
>>>>> clang-11: error: unable to execute command: Aborted
>>>>> clang-11: error: linker command failed due to signal (use
-v to see
>>>>> invocation)
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic <
>>>>> nemanja.i.ibm at gmail.com> wrote:
>>>>>
>>>>> The discussion in https://reviews.llvm.org/D76240
>>>>>
>>>>> may be of interest to the participants of this thread.
Specifically,
>>>>> Fangrui's planned work on a command line option to
limit threads for LLD.
>>>>> Until that becomes a reality, you might want to use the
CMake variable
>>>>> -DLLVM_ENABLE_THREADS=OFF when building the LLD that you
are using to
>>>>> prevent parallelism.
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via
llvm-dev <
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>> Alex :
>>>>>
>>>>> Can you please try `numactl` or `taskset` after
>>>>>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
>>>>> ?
>>>>>
>>>>>
>>>>>
>>>>> There was a tiny bug in how *sched_getaffinity*() was used,
see:
>>>>> https://reviews.llvm.org/D75153#1942336
>>>>>
>>>>>
>>>>>
>>>>> *De :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
>>>>> *Envoyé :* March 28, 2020 12:11 PM
>>>>> *À :* Itaru Kitayama <itaru.kitayama at gmail.com>
>>>>> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org>;
Alexandre Ganea <
>>>>> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng
<maskray at google.com>
>>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively parallel
build
>>>>> machine
>>>>>
>>>>>
>>>>>
>>>>> I also run into problems with lld on machines with lots of
cores (208),
>>>>> but I run into rlimits for number of processes I can
create, something
>>>>> outside of my control.
>>>>>
>>>>>
>>>>>
>>>>> Unfortunately things like numactl or taskset do not limit
the
>>>>> concurrency of lld currently.
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via llvm-dev
<
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> On a 1296-core Intel machine with 376 GB, setting
>>>>>
>>>>>
>>>>>
>>>>> -DLLVM_PARALLEL_LINK_JOB=1
>>>>>
>>>>>
>>>>>
>>>>> does not help (switching back to ld scales) see:
>>>>>
>>>>>
>>>>>
>>>>> [5085/5201] Linking CXX executable bin/clang-11
>>>>> FAILED: bin/clang-11
>>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
>>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
>>>>> -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter
>>>>> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers
-pedantic
>>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
>>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
>>>>> -Wstring-conversion -fdiagnostics-color -ffunction-sections
-fdata-sections
>>>>> -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3
-DNDEBUG
>>>>>  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
>>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
>>>>> tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
>>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
>>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
>>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
>>>>>  -o bin/clang-11  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMAArch64CodeGen.a
>>>>>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Desc.a
>>>>>  lib/libLLVMAArch64Disassembler.a  lib/libLLVMAArch64Info.a
>>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>>>>>  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a
>>>>>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUInfo.a
>>>>>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
>>>>>  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
>>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a 
lib/libLLVMAVRDesc.a
>>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a 
lib/libLLVMBPFDesc.a
>>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>>>>>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>>>>>  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonDisassembler.a
>>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>>>>>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a
>>>>>  lib/libLLVMLanaiDisassembler.a  lib/libLLVMLanaiInfo.a
>>>>>  lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a
>>>>>  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
>>>>>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
>>>>>  lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a
>>>>>  lib/libLLVMMSP430Disassembler.a  lib/libLLVMMSP430Info.a
>>>>>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a 
lib/libLLVMNVPTXInfo.a
>>>>>  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMPowerPCAsmParser.a
>>>>>  lib/libLLVMPowerPCDesc.a  lib/libLLVMPowerPCDisassembler.a
>>>>>  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a
>>>>>  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a
>>>>>  lib/libLLVMRISCVDisassembler.a  lib/libLLVMRISCVInfo.a
>>>>>  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
>>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>>>>>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>>>>>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZDisassembler.a
>>>>>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDesc.a
>>>>>  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyInfo.a
>>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a 
lib/libLLVMX86Desc.a
>>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a 
lib/libLLVMX86Utils.a
>>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
>>>>>  lib/libLLVMXCoreDisassembler.a  lib/libLLVMXCoreInfo.a
>>>>>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a 
lib/libLLVMCore.a
>>>>>  lib/libLLVMipo.a  lib/libLLVMAggressiveInstCombine.a
>>>>>  lib/libLLVMInstCombine.a  lib/libLLVMInstrumentation.a 
lib/libLLVMMC.a
>>>>>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMOption.a
>>>>>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a 
lib/libLLVMTransformUtils.a
>>>>>  lib/libLLVMVectorize.a  -lpthread  lib/libclangBasic.a
>>>>>  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a
>>>>>  lib/libclangFrontendTool.a  lib/libclangSerialization.a
>>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a 
lib/libLLVMAMDGPUDesc.a
>>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a 
lib/libLLVMARMDesc.a
>>>>>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a 
lib/libLLVMHexagonDesc.a
>>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a 
lib/libLLVMLanaiInfo.a
>>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a
>>>>>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>>>>>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a 
lib/libLLVMAsmPrinter.a
>>>>>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>>>>>  lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a 
lib/libLLVMCoverage.a
>>>>>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMPasses.a
>>>>>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a 
lib/libLLVMCoroutines.a
>>>>>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a 
lib/libLLVMVectorize.a
>>>>>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a 
lib/libLLVMAsmParser.a
>>>>>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
>>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
>>>>>  lib/libclangRewriteFrontend.a  lib/libclangARCMigrate.a
>>>>>  lib/libclangStaticAnalyzerFrontend.a 
lib/libclangStaticAnalyzerCheckers.a
>>>>>  lib/libclangStaticAnalyzerCore.a  lib/libclangCrossTU.a
>>>>>  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a
>>>>>  lib/libLLVMOption.a  lib/libclangParse.a 
lib/libclangSerialization.a
>>>>>  lib/libclangSema.a  lib/libclangAnalysis.a 
lib/libclangASTMatchers.a
>>>>>  lib/libclangEdit.a  lib/libclangFormat.a 
lib/libclangToolingInclusions.a
>>>>>  lib/libclangToolingCore.a  lib/libclangAST.a 
lib/libLLVMFrontendOpenMP.a
>>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
>>>>>  lib/libLLVMProfileData.a  lib/libLLVMObject.a 
lib/libLLVMMCParser.a
>>>>>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a 
lib/libclangRewrite.a
>>>>>  lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMCore.a
>>>>>  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a 
lib/libLLVMMC.a
>>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMDebugInfoCodeView.a
>>>>>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a  -lz  -lrt
-ldl  -ltinfo
>>>>>  -lpthread  -lm  lib/libLLVMDemangle.a && :
>>>>> ld.lld: error: failed to open bin/clang-11: Cannot allocate
memory
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>>
>_______________________________________________
>LLVM Developers mailing list
>llvm-dev at lists.llvm.org
>https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Itaru Kitayama via llvm-dev

2020-Apr-01 20:47 UTC

head link

[llvm-dev] LLD issue on a massively parallel build machine

Thanks for the heads up the supercomputer
Is down for maintenance this week.
I’ll give it a try when it gets back.

On Thu, Apr 2, 2020 at 1:49 Fangrui Song <maskray at google.com> wrote:
> On 2020-03-29, Nemanja Ivanovic via llvm-dev wrote:
> >Glad you got it working.
> >My suggestion about LLVM_ENABLE_THREADS didn't work because you
didn't
> >apply it when building the build linker.
> >
> >When you don't have the ability to rebuild the build compiler, this
> doesn't
> >apply. In those cases, I end up doing a dirty hack where I use a
wrapper
> >script with something like:
> >
> ><path-to-build-compiler> "$@" -Wl,--no-threads
> >
> >To invoke the build compiler. That way you still get full parallelism
> while
> >compiling and linking, but you prevent LLD from using threads.
>
>
> https://reviews.llvm.org/D76885 (committed yesterday) changed
> --no-threads to --threads={1,2,...} (--no-threads is used rarely (GNU ld
> does not support it) so I'd rather not keep the option for
> compatibility).
>
> You may try -Wl,--threads=1
>
> >On Sat., Mar. 28, 2020, 4:19 p.m. Itaru Kitayama, <
> itaru.kitayama at gmail.com>
> >wrote:
> >
> >> Good news, I was able to use up to 37 cores for LLVM build with
LLD.
> >> The build speed, did not measure precisely though, is comparable
to the
> >> build with GNU ld case.
> >>
> >> Thank you all for your help!
> >>
> >> On Sun, Mar 29, 2020 at 5:04 AM Alex Brachet-Mialot <
> >> alexbrachetmialot at gmail.com> wrote:
> >>
> >>> Yes unfortunately that would limit you to 4 cores.
> >>>
> >>> There’s no easy way to use lld with —no-threads with our build
system
> at
> >>> the moment unfortunately.
> >>>
> >>> I also had just been using ld.gold, but will switch now that
numactl
> >>> works for me.
> >>>
> >>> On Sat, Mar 28, 2020 at 3:58 PM Itaru Kitayama <
> itaru.kitayama at gmail.com>
> >>> wrote:
> >>>
> >>>> That is slowing down the build visibly, for the speed I
should stick
> >>>> with ld at the moment.
> >>>>
> >>>>
> >>>> On Sun, Mar 29, 2020 at 4:42 AM Alexandre Ganea <
> >>>> alexandre.ganea at ubisoft.com> wrote:
> >>>>
> >>>>> Would `taskset -c 0-3 ninja check-all -j4` work?
> >>>>>
> >>>>>
> >>>>>
> >>>>> *De :* Itaru Kitayama <itaru.kitayama at
gmail.com>
> >>>>> *Envoyé :* March 28, 2020 3:37 PM
> >>>>> *À :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
> >>>>> *Cc :* Alexandre Ganea <alexandre.ganea at
ubisoft.com>; LLVM Dev <
> >>>>> llvm-dev at lists.llvm.org>; Nemanja Ivanovic
<nemanja.i.ibm at gmail.com>
> >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
> >>>>> machine
> >>>>>
> >>>>>
> >>>>>
> >>>>> $ free -g
> >>>>>               total        used        free     
shared  buff/cache
> >>>>> available
> >>>>> Mem:            376         149          20          
1         207
> >>>>>     225
> >>>>> Swap:             3           0           3
> >>>>>
> >>>>>
> >>>>>
> >>>>> Too small swap size for a 72-core login machine?
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sun, Mar 29, 2020 at 4:28 AM Alex Brachet-Mialot
<
> >>>>> alexbrachetmialot at gmail.com> wrote:
> >>>>>
> >>>>> Enable threads is for building llvm with or without
thread support.
> To
> >>>>> my knowledge it has no effect on the build system.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Like was discussed above you could try limiting  the
number of
> >>>>> processes created (assuming your lld was after that
commit) with
> numactl or
> >>>>> taskset.
> >>>>>
> >>>>>
> >>>>>
> >>>>> In your case though it looked like you were running
out of memory.
> You
> >>>>> could try increasing your swap space?
> >>>>>
> >>>>>
> >>>>>
> >>>>> Alex
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama <
> >>>>> itaru.kitayama at gmail.com> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> My configuration is below:
> >>>>>
> >>>>> cmake -GNinja -DLLVM_ENABLE_LLD=ON
-DLLVM_ENABLE_THREADS=OFF
> >>>>> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
> >>>>> -DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
> >>>>> -DLIBOMPTARGET_ENABLE_DEBUG=ON
> >>>>> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
> -DCMAKE_C_COMPILER=clang
> >>>>> -DCMAKE_C_FLAGS=""
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS=""
> >>>>>
>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
> >>>>> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
> >>>>> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
> -DLLVM_TARGETS_TO_BUILD="all"
> >>>>> /tmp/llvm-project/llvm
> >>>>>
> >>>>>
> >>>>>
> >>>>> but did not help my case:
> >>>>>
> >>>>>
> >>>>>
> >>>>> [...]
> >>>>>
> >>>>> [4802/5201] Linking CXX executable bin/lld
> >>>>> FAILED: bin/lld
> >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
> >>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
> >>>>> -Werror=unguarded-availability-new -Wall -Wextra
> -Wno-unused-parameter
> >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
> >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
> >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
> >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections
> -fdata-sections
> >>>>> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld
-Wl,--color-diagnostics
> >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
> >>>>> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o  -o
bin/lld
> >>>>>  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMSupport.a  -lpthread
> >>>>>  lib/liblldCommon.a  lib/liblldCOFF.a 
lib/liblldDriver.a
> lib/liblldELF.a
> >>>>>  lib/liblldMinGW.a  lib/liblldWasm.a 
lib/liblldMachO.a
> >>>>>  lib/liblldReaderWriter.a  lib/liblldYAML.a 
lib/liblldCore.a
> >>>>>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a 
lib/libLLVMLibDriver.a
> >>>>>  lib/libLLVMWindowsManifest.a  /usr/lib64/libxml2.so
> lib/liblldCommon.a
> >>>>>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
> >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Disassembler.a
> >>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
> >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
> >>>>>  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUAsmParser.a
> >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUDesc.a
> >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
> lib/libLLVMARMCodeGen.a
> >>>>>  lib/libLLVMARMAsmParser.a 
lib/libLLVMARMDisassembler.a
> >>>>>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
> >>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a
> lib/libLLVMAVRDesc.a
> >>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
> >>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a
> lib/libLLVMBPFDesc.a
> >>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
> >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
> >>>>>  lib/libLLVMHexagonDisassembler.a 
lib/libLLVMHexagonDesc.a
> >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
> >>>>>  lib/libLLVMLanaiAsmParser.a 
lib/libLLVMLanaiDisassembler.a
> >>>>>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a
> lib/libLLVMMipsCodeGen.a
> >>>>>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
> >>>>>  lib/libLLVMMipsDisassembler.a  lib/libLLVMMipsInfo.a
> >>>>>  lib/libLLVMMSP430CodeGen.a 
lib/libLLVMMSP430AsmParser.a
> >>>>>  lib/libLLVMMSP430Desc.a 
lib/libLLVMMSP430Disassembler.a
> >>>>>  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXCodeGen.a
> lib/libLLVMNVPTXDesc.a
> >>>>>  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCCodeGen.a
> >>>>>  lib/libLLVMPowerPCAsmParser.a 
lib/libLLVMPowerPCDesc.a
> >>>>>  lib/libLLVMPowerPCDisassembler.a 
lib/libLLVMPowerPCInfo.a
> >>>>>  lib/libLLVMRISCVCodeGen.a 
lib/libLLVMRISCVAsmParser.a
> >>>>>  lib/libLLVMRISCVDesc.a 
lib/libLLVMRISCVDisassembler.a
> >>>>>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a
> lib/libLLVMSparcCodeGen.a
> >>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
> >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
> >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
> >>>>>  lib/libLLVMSystemZDisassembler.a 
lib/libLLVMSystemZDesc.a
> >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
> >>>>>  lib/libLLVMWebAssemblyAsmParser.a
> lib/libLLVMWebAssemblyDisassembler.a
> >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
> >>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a
> lib/libLLVMGlobalISel.a
> >>>>>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
> >>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a
> lib/libLLVMX86Utils.a
> >>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAsmPrinter.a
> >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
> >>>>>  lib/libLLVMXCoreDesc.a 
lib/libLLVMXCoreDisassembler.a
> >>>>>  lib/libLLVMMCDisassembler.a  lib/libLLVMXCoreInfo.a
> lib/libLLVMLTO.a
> >>>>>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a
> >>>>>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a 
lib/libLLVMipo.a
> >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a
> lib/libLLVMVectorize.a
> >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
> >>>>>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a
> lib/libLLVMIRReader.a
> >>>>>  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a
> >>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
> lib/libLLVMObject.a
> >>>>>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a 
lib/libLLVMMC.a
> >>>>>  lib/libLLVMDebugInfoCodeView.a 
lib/libLLVMDebugInfoMSF.a
> >>>>>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a 
lib/libLLVMCore.a
> >>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
> >>>>>  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a 
-lz  -lrt  -ldl
> >>>>>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a
&& cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/lld-link && cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld.lld && cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld64.lld && cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/wasm-ld
> >>>>> terminating with uncaught exception of type
std::__1::system_error:
> >>>>> thread constructor failed: Resource temporarily
unavailable
> >>>>> PLEASE submit a bug report to https://bugs.llvm.org/
and include the
> >>>>> crash backtrace.
> >>>>>  #0 0x00000000007924f4
PrintStackTraceSignalHandler(void*)
> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
> >>>>>  #1 0x00000000007902ce llvm::sys::RunSignalHandlers()
> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
> >>>>>  #2 0x0000000000792bd5 SignalHandler(int)
> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
> >>>>>  #3 0x00007fddd54bb630 __restore_rt
(/lib64/libpthread.so.0+0xf630)
> >>>>>  #4 0x00007fddd3b40377 raise
> >>>>>
>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
> >>>>>  #5 0x00007fddd3b41a68 abort
> >>>>> /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
> >>>>>  #6 0x00007fddd411ae8b
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
> >>>>>  #7 0x00007fddd40fdb9f demangling_terminate_handler()
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
> >>>>>  #8 0x00007fddd411a0a3 std::__terminate(void (*)())
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
> >>>>>  #9 0x00007fddd411ce06
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
> >>>>> #10 0x00007fddd411cd9f
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
> >>>>> #11 0x00007fddd43bded4
std::__1::__throw_system_error(int, char
> const*)
> >>>>>
> (/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
> >>>>> #12 0x0000000002a7a316 void*
> >>>>>
>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
> >>>>>
std::__1::default_delete<std::__1::__thread_struct> >,
> >>>>> llvm::parallel::detail::(anonymous
> >>>>>
>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
> >>>>> >(void*)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
> >>>>> #13 0x00007fddd54b3ea5 start_thread
> >>>>>
/usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
> >>>>> #14 0x00007fddd3c088cd clone
> >>>>>
>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
> >>>>> clang-11: error: unable to execute command: Aborted
> >>>>> clang-11: error: linker command failed due to signal
(use -v to see
> >>>>> invocation)
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic <
> >>>>> nemanja.i.ibm at gmail.com> wrote:
> >>>>>
> >>>>> The discussion in https://reviews.llvm.org/D76240
> >>>>>
> >>>>> may be of interest to the participants of this thread.
Specifically,
> >>>>> Fangrui's planned work on a command line option to
limit threads for
> LLD.
> >>>>> Until that becomes a reality, you might want to use
the CMake
> variable
> >>>>> -DLLVM_ENABLE_THREADS=OFF when building the LLD that
you are using to
> >>>>> prevent parallelism.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via
llvm-dev <
> >>>>> llvm-dev at lists.llvm.org> wrote:
> >>>>>
> >>>>> Alex :
> >>>>>
> >>>>> Can you please try `numactl` or `taskset` after
> >>>>>
>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
> >>>>> ?
> >>>>>
> >>>>>
> >>>>>
> >>>>> There was a tiny bug in how *sched_getaffinity*() was
used, see:
> >>>>> https://reviews.llvm.org/D75153#1942336
> >>>>>
> >>>>>
> >>>>>
> >>>>> *De :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
> >>>>> *Envoyé :* March 28, 2020 12:11 PM
> >>>>> *À :* Itaru Kitayama <itaru.kitayama at
gmail.com>
> >>>>> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org>;
Alexandre Ganea <
> >>>>> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng
<maskray at google.com>
> >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
> >>>>> machine
> >>>>>
> >>>>>
> >>>>>
> >>>>> I also run into problems with lld on machines with
lots of cores
> (208),
> >>>>> but I run into rlimits for number of processes I can
create,
> something
> >>>>> outside of my control.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Unfortunately things like numactl or taskset do not
limit the
> >>>>> concurrency of lld currently.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via
llvm-dev <
> >>>>> llvm-dev at lists.llvm.org> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> On a 1296-core Intel machine with 376 GB, setting
> >>>>>
> >>>>>
> >>>>>
> >>>>> -DLLVM_PARALLEL_LINK_JOB=1
> >>>>>
> >>>>>
> >>>>>
> >>>>> does not help (switching back to ld scales) see:
> >>>>>
> >>>>>
> >>>>>
> >>>>> [5085/5201] Linking CXX executable bin/clang-11
> >>>>> FAILED: bin/clang-11
> >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
> >>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
> >>>>> -Werror=unguarded-availability-new -Wall -Wextra
> -Wno-unused-parameter
> >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
> >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
> >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
> >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections
> -fdata-sections
> >>>>> -fno-common -Woverloaded-virtual
-Wno-nested-anon-types -O3 -DNDEBUG
> >>>>>  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
> >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
> >>>>>
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
> >>>>>  -o bin/clang-11 
-Wl,-rpath,"\$ORIGIN/../lib"
> lib/libLLVMAArch64CodeGen.a
> >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Desc.a
> >>>>>  lib/libLLVMAArch64Disassembler.a 
lib/libLLVMAArch64Info.a
> >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
> >>>>>  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a
> >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUInfo.a
> >>>>>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
> >>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
> >>>>>  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a
> lib/libLLVMARMUtils.a
> >>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a
> lib/libLLVMAVRDesc.a
> >>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
> >>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a
> lib/libLLVMBPFDesc.a
> >>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
> >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
> >>>>>  lib/libLLVMHexagonDesc.a 
lib/libLLVMHexagonDisassembler.a
> >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
> >>>>>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a
> >>>>>  lib/libLLVMLanaiDisassembler.a 
lib/libLLVMLanaiInfo.a
> >>>>>  lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a
> >>>>>  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
> >>>>>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
> >>>>>  lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a
> >>>>>  lib/libLLVMMSP430Disassembler.a 
lib/libLLVMMSP430Info.a
> >>>>>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a
> lib/libLLVMNVPTXInfo.a
> >>>>>  lib/libLLVMPowerPCCodeGen.a 
lib/libLLVMPowerPCAsmParser.a
> >>>>>  lib/libLLVMPowerPCDesc.a 
lib/libLLVMPowerPCDisassembler.a
> >>>>>  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a
> >>>>>  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a
> >>>>>  lib/libLLVMRISCVDisassembler.a 
lib/libLLVMRISCVInfo.a
> >>>>>  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
> >>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
> >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
> >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
> >>>>>  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZDisassembler.a
> >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
> >>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDesc.a
> >>>>>  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyInfo.a
> >>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a
> lib/libLLVMX86Desc.a
> >>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a
> lib/libLLVMX86Utils.a
> >>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
> >>>>>  lib/libLLVMXCoreDisassembler.a 
lib/libLLVMXCoreInfo.a
> >>>>>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a 
lib/libLLVMCore.a
> >>>>>  lib/libLLVMipo.a  lib/libLLVMAggressiveInstCombine.a
> >>>>>  lib/libLLVMInstCombine.a 
lib/libLLVMInstrumentation.a
> lib/libLLVMMC.a
> >>>>>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMOption.a
> >>>>>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a
> lib/libLLVMTransformUtils.a
> >>>>>  lib/libLLVMVectorize.a  -lpthread 
lib/libclangBasic.a
> >>>>>  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a
> >>>>>  lib/libclangFrontendTool.a 
lib/libclangSerialization.a
> >>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
> >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a
> lib/libLLVMAMDGPUDesc.a
> >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
> lib/libLLVMARMDesc.a
> >>>>>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
> lib/libLLVMHexagonDesc.a
> >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a
> lib/libLLVMLanaiInfo.a
> >>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a
> >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
> >>>>>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a
> lib/libLLVMAsmPrinter.a
> >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
> >>>>>  lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a
> lib/libLLVMCoverage.a
> >>>>>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMPasses.a
> >>>>>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a 
lib/libLLVMCoroutines.a
> >>>>>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a
> lib/libLLVMVectorize.a
> >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a
> lib/libLLVMAsmParser.a
> >>>>>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
> >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
> >>>>>  lib/libclangRewriteFrontend.a 
lib/libclangARCMigrate.a
> >>>>>  lib/libclangStaticAnalyzerFrontend.a
> lib/libclangStaticAnalyzerCheckers.a
> >>>>>  lib/libclangStaticAnalyzerCore.a 
lib/libclangCrossTU.a
> >>>>>  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a
> >>>>>  lib/libLLVMOption.a  lib/libclangParse.a
> lib/libclangSerialization.a
> >>>>>  lib/libclangSema.a  lib/libclangAnalysis.a
> lib/libclangASTMatchers.a
> >>>>>  lib/libclangEdit.a  lib/libclangFormat.a
> lib/libclangToolingInclusions.a
> >>>>>  lib/libclangToolingCore.a  lib/libclangAST.a
> lib/libLLVMFrontendOpenMP.a
> >>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
> >>>>>  lib/libLLVMProfileData.a  lib/libLLVMObject.a 
lib/libLLVMMCParser.a
> >>>>>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a 
lib/libclangRewrite.a
> >>>>>  lib/libclangLex.a  lib/libclangBasic.a 
lib/libLLVMCore.a
> >>>>>  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a 
lib/libLLVMMC.a
> >>>>>  lib/libLLVMBinaryFormat.a 
lib/libLLVMDebugInfoCodeView.a
> >>>>>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a  -lz 
-lrt  -ldl
> -ltinfo
> >>>>>  -lpthread  -lm  lib/libLLVMDemangle.a && :
> >>>>> ld.lld: error: failed to open bin/clang-11: Cannot
allocate memory
> >>>>>
> >>>>> _______________________________________________
> >>>>> LLVM Developers mailing list
> >>>>> llvm-dev at lists.llvm.org
> >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>>>>
> >>>>> _______________________________________________
> >>>>> LLVM Developers mailing list
> >>>>> llvm-dev at lists.llvm.org
> >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>>>>
> >>>>>
>
> >_______________________________________________
> >LLVM Developers mailing list
> >llvm-dev at lists.llvm.org
> >https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200402/6e5fb02a/attachment-0001.html>

Tom Stellard via llvm-dev

2020-Apr-01 21:10 UTC

head link

[llvm-dev] LLD issue on a massively parallel build machine

On 04/01/2020 01:47 PM, Itaru Kitayama via llvm-dev
wrote:> Thanks for the heads up the supercomputer 
> Is down for maintenance this week.
> I’ll give it a try when it gets back.
> 
This doesn't look to me like it's necessarily an lld issue.  Trying
to build all the sub-projects without limiting the number of ninja jobs
is almost guaranteed to run out of memory on a machine like this with such
a low memory:core ratio.

-Tom
> On Thu, Apr 2, 2020 at 1:49 Fangrui Song <maskray at google.com
<mailto:maskray at google.com>> wrote:
> 
>     On 2020-03-29, Nemanja Ivanovic via llvm-dev wrote:
>     >Glad you got it working.
>     >My suggestion about LLVM_ENABLE_THREADS didn't work because you
didn't
>     >apply it when building the build linker.
>     >
>     >When you don't have the ability to rebuild the build compiler,
this doesn't
>     >apply. In those cases, I end up doing a dirty hack where I use a
wrapper
>     >script with something like:
>     >
>     ><path-to-build-compiler> "$@" -Wl,--no-threads
>     >
>     >To invoke the build compiler. That way you still get full
parallelism while
>     >compiling and linking, but you prevent LLD from using threads.
> 
> 
>     https://reviews.llvm.org/D76885 (committed yesterday) changed
>     --no-threads to --threads={1,2,...} (--no-threads is used rarely (GNU
ld
>     does not support it) so I'd rather not keep the option for
>     compatibility).
> 
>     You may try -Wl,--threads=1
> 
>     >On Sat., Mar. 28, 2020, 4:19 p.m. Itaru Kitayama,
<itaru.kitayama at gmail.com <mailto:itaru.kitayama at gmail.com>>
>     >wrote:
>     >
>     >> Good news, I was able to use up to 37 cores for LLVM build
with LLD.
>     >> The build speed, did not measure precisely though, is
comparable to the
>     >> build with GNU ld case.
>     >>
>     >> Thank you all for your help!
>     >>
>     >> On Sun, Mar 29, 2020 at 5:04 AM Alex Brachet-Mialot <
>     >> alexbrachetmialot at gmail.com <mailto:alexbrachetmialot at
gmail.com>> wrote:
>     >>
>     >>> Yes unfortunately that would limit you to 4 cores.
>     >>>
>     >>> There’s no easy way to use lld with —no-threads with our
build system at
>     >>> the moment unfortunately.
>     >>>
>     >>> I also had just been using ld.gold, but will switch now
that numactl
>     >>> works for me.
>     >>>
>     >>> On Sat, Mar 28, 2020 at 3:58 PM Itaru Kitayama
<itaru.kitayama at gmail.com <mailto:itaru.kitayama at gmail.com>>
>     >>> wrote:
>     >>>
>     >>>> That is slowing down the build visibly, for the speed
I should stick
>     >>>> with ld at the moment.
>     >>>>
>     >>>>
>     >>>> On Sun, Mar 29, 2020 at 4:42 AM Alexandre Ganea <
>     >>>> alexandre.ganea at ubisoft.com
<mailto:alexandre.ganea at ubisoft.com>> wrote:
>     >>>>
>     >>>>> Would `taskset -c 0-3 ninja check-all -j4` work?
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> *De :* Itaru Kitayama <itaru.kitayama at
gmail.com <mailto:itaru.kitayama at gmail.com>>
>     >>>>> *Envoyé :* March 28, 2020 3:37 PM
>     >>>>> *À :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com <mailto:alexbrachetmialot at gmail.com>>
>     >>>>> *Cc :* Alexandre Ganea <alexandre.ganea at
ubisoft.com <mailto:alexandre.ganea at ubisoft.com>>; LLVM Dev <
>     >>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>>; Nemanja Ivanovic <nemanja.i.ibm at gmail.com
<mailto:nemanja.i.ibm at gmail.com>>
>     >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
>     >>>>> machine
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> $ free -g
>     >>>>>               total        used        free     
shared  buff/cache
>     >>>>> available
>     >>>>> Mem:            376         149          20       
1         207
>     >>>>>     225
>     >>>>> Swap:             3           0           3
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> Too small swap size for a 72-core login machine?
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> On Sun, Mar 29, 2020 at 4:28 AM Alex
Brachet-Mialot <
>     >>>>> alexbrachetmialot at gmail.com
<mailto:alexbrachetmialot at gmail.com>> wrote:
>     >>>>>
>     >>>>> Enable threads is for building llvm with or
without thread support. To
>     >>>>> my knowledge it has no effect on the build system.
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> Like was discussed above you could try limiting 
the number of
>     >>>>> processes created (assuming your lld was after
that commit) with numactl or
>     >>>>> taskset.
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> In your case though it looked like you were
running out of memory. You
>     >>>>> could try increasing your swap space?
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> Alex
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama
<
>     >>>>> itaru.kitayama at gmail.com
<mailto:itaru.kitayama at gmail.com>> wrote:
>     >>>>>
>     >>>>> Hi,
>     >>>>>
>     >>>>> My configuration is below:
>     >>>>>
>     >>>>> cmake -GNinja -DLLVM_ENABLE_LLD=ON
-DLLVM_ENABLE_THREADS=OFF
>     >>>>> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
>     >>>>>
-DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
>     >>>>> -DLIBOMPTARGET_ENABLE_DEBUG=ON
>     >>>>> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
-DCMAKE_C_COMPILER=clang
>     >>>>> -DCMAKE_C_FLAGS=""
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS=""
>     >>>>>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
>     >>>>> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
>     >>>>> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
-DLLVM_TARGETS_TO_BUILD="all"
>     >>>>> /tmp/llvm-project/llvm
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> but did not help my case:
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> [...]
>     >>>>>
>     >>>>> [4802/5201] Linking CXX executable bin/lld
>     >>>>> FAILED: bin/lld
>     >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
>     >>>>> -fPIC -fvisibility-inlines-hidden
-Werror=date-time
>     >>>>> -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter
>     >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
>     >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
>     >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
>     >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections -fdata-sections
>     >>>>> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld
-Wl,--color-diagnostics
>     >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
>     >>>>> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o 
-o bin/lld
>     >>>>>  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMSupport.a  -lpthread
>     >>>>>  lib/liblldCommon.a  lib/liblldCOFF.a 
lib/liblldDriver.a  lib/liblldELF.a
>     >>>>>  lib/liblldMinGW.a  lib/liblldWasm.a 
lib/liblldMachO.a
>     >>>>>  lib/liblldReaderWriter.a  lib/liblldYAML.a 
lib/liblldCore.a
>     >>>>>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a 
lib/libLLVMLibDriver.a
>     >>>>>  lib/libLLVMWindowsManifest.a 
/usr/lib64/libxml2.so  lib/liblldCommon.a
>     >>>>>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
>     >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Disassembler.a
>     >>>>>  lib/libLLVMAArch64Desc.a 
lib/libLLVMAArch64Info.a
>     >>>>>  lib/libLLVMAArch64Utils.a 
lib/libLLVMAMDGPUCodeGen.a
>     >>>>>  lib/libLLVMMIRParser.a 
lib/libLLVMAMDGPUAsmParser.a
>     >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUDesc.a
>     >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
lib/libLLVMARMCodeGen.a
>     >>>>>  lib/libLLVMARMAsmParser.a 
lib/libLLVMARMDisassembler.a
>     >>>>>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
>     >>>>>  lib/libLLVMAVRCodeGen.a 
lib/libLLVMAVRAsmParser.a  lib/libLLVMAVRDesc.a
>     >>>>>  lib/libLLVMAVRDisassembler.a 
lib/libLLVMAVRInfo.a
>     >>>>>  lib/libLLVMBPFCodeGen.a 
lib/libLLVMBPFAsmParser.a  lib/libLLVMBPFDesc.a
>     >>>>>  lib/libLLVMBPFDisassembler.a 
lib/libLLVMBPFInfo.a
>     >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
>     >>>>>  lib/libLLVMHexagonDisassembler.a 
lib/libLLVMHexagonDesc.a
>     >>>>>  lib/libLLVMHexagonInfo.a 
lib/libLLVMLanaiCodeGen.a
>     >>>>>  lib/libLLVMLanaiAsmParser.a 
lib/libLLVMLanaiDisassembler.a
>     >>>>>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a 
lib/libLLVMMipsCodeGen.a
>     >>>>>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
>     >>>>>  lib/libLLVMMipsDisassembler.a 
lib/libLLVMMipsInfo.a
>     >>>>>  lib/libLLVMMSP430CodeGen.a 
lib/libLLVMMSP430AsmParser.a
>     >>>>>  lib/libLLVMMSP430Desc.a 
lib/libLLVMMSP430Disassembler.a
>     >>>>>  lib/libLLVMMSP430Info.a 
lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a
>     >>>>>  lib/libLLVMNVPTXInfo.a 
lib/libLLVMPowerPCCodeGen.a
>     >>>>>  lib/libLLVMPowerPCAsmParser.a 
lib/libLLVMPowerPCDesc.a
>     >>>>>  lib/libLLVMPowerPCDisassembler.a 
lib/libLLVMPowerPCInfo.a
>     >>>>>  lib/libLLVMRISCVCodeGen.a 
lib/libLLVMRISCVAsmParser.a
>     >>>>>  lib/libLLVMRISCVDesc.a 
lib/libLLVMRISCVDisassembler.a
>     >>>>>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a 
lib/libLLVMSparcCodeGen.a
>     >>>>>  lib/libLLVMSparcAsmParser.a 
lib/libLLVMSparcDesc.a
>     >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
>     >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
>     >>>>>  lib/libLLVMSystemZDisassembler.a 
lib/libLLVMSystemZDesc.a
>     >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
>     >>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDisassembler.a
>     >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
>     >>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a 
lib/libLLVMGlobalISel.a
>     >>>>>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>     >>>>>  lib/libLLVMX86Disassembler.a 
lib/libLLVMX86Info.a  lib/libLLVMX86Utils.a
>     >>>>>  lib/libLLVMXCoreCodeGen.a 
lib/libLLVMAsmPrinter.a
>     >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
>     >>>>>  lib/libLLVMXCoreDesc.a 
lib/libLLVMXCoreDisassembler.a
>     >>>>>  lib/libLLVMMCDisassembler.a 
lib/libLLVMXCoreInfo.a  lib/libLLVMLTO.a
>     >>>>>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a
>     >>>>>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a 
lib/libLLVMipo.a
>     >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a 
lib/libLLVMVectorize.a
>     >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
>     >>>>>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a 
lib/libLLVMIRReader.a
>     >>>>>  lib/libLLVMAsmParser.a 
lib/libLLVMInstrumentation.a
>     >>>>>  lib/libLLVMTransformUtils.a 
lib/libLLVMAnalysis.a  lib/libLLVMObject.a
>     >>>>>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a 
lib/libLLVMMC.a
>     >>>>>  lib/libLLVMDebugInfoCodeView.a 
lib/libLLVMDebugInfoMSF.a
>     >>>>>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a 
lib/libLLVMCore.a
>     >>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
>     >>>>>  lib/libLLVMBitstreamReader.a 
lib/libLLVMSupport.a  -lz  -lrt  -ldl
>     >>>>>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a
&& cd
>     >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>     >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>     >>>>> create_symlink lld
/tmp/build/202003290405/./bin/lld-link && cd
>     >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>     >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>     >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld.lld && cd
>     >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>     >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>     >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld64.lld && cd
>     >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>     >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>     >>>>> create_symlink lld
/tmp/build/202003290405/./bin/wasm-ld
>     >>>>> terminating with uncaught exception of type
std::__1::system_error:
>     >>>>> thread constructor failed: Resource temporarily
unavailable
>     >>>>> PLEASE submit a bug report to
https://bugs.llvm.org/ and include the
>     >>>>> crash backtrace.
>     >>>>>  #0 0x00000000007924f4
PrintStackTraceSignalHandler(void*)
>     >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
>     >>>>>  #1 0x00000000007902ce
llvm::sys::RunSignalHandlers()
>     >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
>     >>>>>  #2 0x0000000000792bd5 SignalHandler(int)
>     >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
>     >>>>>  #3 0x00007fddd54bb630 __restore_rt
(/lib64/libpthread.so.0+0xf630)
>     >>>>>  #4 0x00007fddd3b40377 raise
>     >>>>>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
>     >>>>>  #5 0x00007fddd3b41a68 abort
>     >>>>>
/usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
>     >>>>>  #6 0x00007fddd411ae8b
>     >>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
>     >>>>>  #7 0x00007fddd40fdb9f
demangling_terminate_handler()
>     >>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
>     >>>>>  #8 0x00007fddd411a0a3 std::__terminate(void
(*)())
>     >>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
>     >>>>>  #9 0x00007fddd411ce06
>     >>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
>     >>>>> #10 0x00007fddd411cd9f
>     >>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
>     >>>>> #11 0x00007fddd43bded4
std::__1::__throw_system_error(int, char const*)
>     >>>>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
>     >>>>> #12 0x0000000002a7a316 void*
>     >>>>>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
>     >>>>>
std::__1::default_delete<std::__1::__thread_struct> >,
>     >>>>> llvm::parallel::detail::(anonymous
>     >>>>>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
>     >>>>> >(void*)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
>     >>>>> #13 0x00007fddd54b3ea5 start_thread
>     >>>>>
/usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
>     >>>>> #14 0x00007fddd3c088cd clone
>     >>>>>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
>     >>>>> clang-11: error: unable to execute command:
Aborted
>     >>>>> clang-11: error: linker command failed due to
signal (use -v to see
>     >>>>> invocation)
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic
<
>     >>>>> nemanja.i.ibm at gmail.com
<mailto:nemanja.i.ibm at gmail.com>> wrote:
>     >>>>>
>     >>>>> The discussion in https://reviews.llvm.org/D76240
>     >>>>>
>     >>>>> may be of interest to the participants of this
thread. Specifically,
>     >>>>> Fangrui's planned work on a command line
option to limit threads for LLD.
>     >>>>> Until that becomes a reality, you might want to
use the CMake variable
>     >>>>> -DLLVM_ENABLE_THREADS=OFF when building the LLD
that you are using to
>     >>>>> prevent parallelism.
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea
via llvm-dev <
>     >>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>     >>>>>
>     >>>>> Alex :
>     >>>>>
>     >>>>> Can you please try `numactl` or `taskset` after
>     >>>>>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
>     >>>>> ?
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> There was a tiny bug in how *sched_getaffinity*()
was used, see:
>     >>>>> https://reviews.llvm.org/D75153#1942336
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> *De :* Alex Brachet-Mialot <alexbrachetmialot
at gmail.com <mailto:alexbrachetmialot at gmail.com>>
>     >>>>> *Envoyé :* March 28, 2020 12:11 PM
>     >>>>> *À :* Itaru Kitayama <itaru.kitayama at
gmail.com <mailto:itaru.kitayama at gmail.com>>
>     >>>>> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>; Alexandre Ganea <
>     >>>>> alexandre.ganea at ubisoft.com
<mailto:alexandre.ganea at ubisoft.com>>; Fāng-ruì Sòng <maskray at
google.com <mailto:maskray at google.com>>
>     >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
>     >>>>> machine
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> I also run into problems with lld on machines with
lots of cores (208),
>     >>>>> but I run into rlimits for number of processes I
can create, something
>     >>>>> outside of my control.
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> Unfortunately things like numactl or taskset do
not limit the
>     >>>>> concurrency of lld currently.
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via
llvm-dev <
>     >>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>     >>>>>
>     >>>>> Hi,
>     >>>>>
>     >>>>> On a 1296-core Intel machine with 376 GB, setting
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> -DLLVM_PARALLEL_LINK_JOB=1
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> does not help (switching back to ld scales) see:
>     >>>>>
>     >>>>>
>     >>>>>
>     >>>>> [5085/5201] Linking CXX executable bin/clang-11
>     >>>>> FAILED: bin/clang-11
>     >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
>     >>>>> -fPIC -fvisibility-inlines-hidden
-Werror=date-time
>     >>>>> -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter
>     >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
>     >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
>     >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
>     >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections -fdata-sections
>     >>>>> -fno-common -Woverloaded-virtual
-Wno-nested-anon-types -O3 -DNDEBUG
>     >>>>>  -stdlib=libc++ -fuse-ld=lld
-Wl,--color-diagnostics
>     >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
>     >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
>     >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
>     >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
>     >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
>     >>>>>  -o bin/clang-11 
-Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64CodeGen.a
>     >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Desc.a
>     >>>>>  lib/libLLVMAArch64Disassembler.a 
lib/libLLVMAArch64Info.a
>     >>>>>  lib/libLLVMAArch64Utils.a 
lib/libLLVMAMDGPUCodeGen.a
>     >>>>>  lib/libLLVMAMDGPUAsmParser.a 
lib/libLLVMAMDGPUDesc.a
>     >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUInfo.a
>     >>>>>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>     >>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
>     >>>>>  lib/libLLVMARMDisassembler.a 
lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
>     >>>>>  lib/libLLVMAVRCodeGen.a 
lib/libLLVMAVRAsmParser.a  lib/libLLVMAVRDesc.a
>     >>>>>  lib/libLLVMAVRDisassembler.a 
lib/libLLVMAVRInfo.a
>     >>>>>  lib/libLLVMBPFCodeGen.a 
lib/libLLVMBPFAsmParser.a  lib/libLLVMBPFDesc.a
>     >>>>>  lib/libLLVMBPFDisassembler.a 
lib/libLLVMBPFInfo.a
>     >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
>     >>>>>  lib/libLLVMHexagonDesc.a 
lib/libLLVMHexagonDisassembler.a
>     >>>>>  lib/libLLVMHexagonInfo.a 
lib/libLLVMLanaiCodeGen.a
>     >>>>>  lib/libLLVMLanaiAsmParser.a 
lib/libLLVMLanaiDesc.a
>     >>>>>  lib/libLLVMLanaiDisassembler.a 
lib/libLLVMLanaiInfo.a
>     >>>>>  lib/libLLVMMipsCodeGen.a 
lib/libLLVMMipsAsmParser.a
>     >>>>>  lib/libLLVMMipsDesc.a 
lib/libLLVMMipsDisassembler.a
>     >>>>>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
>     >>>>>  lib/libLLVMMSP430AsmParser.a 
lib/libLLVMMSP430Desc.a
>     >>>>>  lib/libLLVMMSP430Disassembler.a 
lib/libLLVMMSP430Info.a
>     >>>>>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a
lib/libLLVMNVPTXInfo.a
>     >>>>>  lib/libLLVMPowerPCCodeGen.a 
lib/libLLVMPowerPCAsmParser.a
>     >>>>>  lib/libLLVMPowerPCDesc.a 
lib/libLLVMPowerPCDisassembler.a
>     >>>>>  lib/libLLVMPowerPCInfo.a 
lib/libLLVMRISCVCodeGen.a
>     >>>>>  lib/libLLVMRISCVAsmParser.a 
lib/libLLVMRISCVDesc.a
>     >>>>>  lib/libLLVMRISCVDisassembler.a 
lib/libLLVMRISCVInfo.a
>     >>>>>  lib/libLLVMRISCVUtils.a 
lib/libLLVMSparcCodeGen.a
>     >>>>>  lib/libLLVMSparcAsmParser.a 
lib/libLLVMSparcDesc.a
>     >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
>     >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
>     >>>>>  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZDisassembler.a
>     >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
>     >>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDesc.a
>     >>>>>  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyInfo.a
>     >>>>>  lib/libLLVMX86CodeGen.a 
lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>     >>>>>  lib/libLLVMX86Disassembler.a 
lib/libLLVMX86Info.a  lib/libLLVMX86Utils.a
>     >>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
>     >>>>>  lib/libLLVMXCoreDisassembler.a 
lib/libLLVMXCoreInfo.a
>     >>>>>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a 
lib/libLLVMCore.a
>     >>>>>  lib/libLLVMipo.a 
lib/libLLVMAggressiveInstCombine.a
>     >>>>>  lib/libLLVMInstCombine.a 
lib/libLLVMInstrumentation.a  lib/libLLVMMC.a
>     >>>>>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMOption.a
>     >>>>>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a 
lib/libLLVMTransformUtils.a
>     >>>>>  lib/libLLVMVectorize.a  -lpthread 
lib/libclangBasic.a
>     >>>>>  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a
>     >>>>>  lib/libclangFrontendTool.a 
lib/libclangSerialization.a
>     >>>>>  lib/libLLVMAArch64Desc.a 
lib/libLLVMAArch64Info.a
>     >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a
lib/libLLVMAMDGPUDesc.a
>     >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
lib/libLLVMARMDesc.a
>     >>>>>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a 
lib/libLLVMHexagonDesc.a
>     >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a 
lib/libLLVMLanaiInfo.a
>     >>>>>  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZInfo.a
>     >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
>     >>>>>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a 
lib/libLLVMAsmPrinter.a
>     >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
>     >>>>>  lib/libLLVMMCDisassembler.a 
lib/libclangCodeGen.a  lib/libLLVMCoverage.a
>     >>>>>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMPasses.a
>     >>>>>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a 
lib/libLLVMCoroutines.a
>     >>>>>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a 
lib/libLLVMVectorize.a
>     >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a 
lib/libLLVMAsmParser.a
>     >>>>>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
>     >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
>     >>>>>  lib/libclangRewriteFrontend.a 
lib/libclangARCMigrate.a
>     >>>>>  lib/libclangStaticAnalyzerFrontend.a 
lib/libclangStaticAnalyzerCheckers.a
>     >>>>>  lib/libclangStaticAnalyzerCore.a 
lib/libclangCrossTU.a
>     >>>>>  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a
>     >>>>>  lib/libLLVMOption.a  lib/libclangParse.a 
lib/libclangSerialization.a
>     >>>>>  lib/libclangSema.a  lib/libclangAnalysis.a 
lib/libclangASTMatchers.a
>     >>>>>  lib/libclangEdit.a  lib/libclangFormat.a 
lib/libclangToolingInclusions.a
>     >>>>>  lib/libclangToolingCore.a  lib/libclangAST.a 
lib/libLLVMFrontendOpenMP.a
>     >>>>>  lib/libLLVMTransformUtils.a 
lib/libLLVMAnalysis.a
>     >>>>>  lib/libLLVMProfileData.a  lib/libLLVMObject.a 
lib/libLLVMMCParser.a
>     >>>>>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a 
lib/libclangRewrite.a
>     >>>>>  lib/libclangLex.a  lib/libclangBasic.a 
lib/libLLVMCore.a
>     >>>>>  lib/libLLVMRemarks.a 
lib/libLLVMBitstreamReader.a  lib/libLLVMMC.a
>     >>>>>  lib/libLLVMBinaryFormat.a 
lib/libLLVMDebugInfoCodeView.a
>     >>>>>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a 
-lz  -lrt  -ldl  -ltinfo
>     >>>>>  -lpthread  -lm  lib/libLLVMDemangle.a &&
:
>     >>>>> ld.lld: error: failed to open bin/clang-11: Cannot
allocate memory
>     >>>>>
>     >>>>> _______________________________________________
>     >>>>> LLVM Developers mailing list
>     >>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>     >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>     >>>>>
>     >>>>> _______________________________________________
>     >>>>> LLVM Developers mailing list
>     >>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>     >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>     >>>>>
>     >>>>>
> 
>     >_______________________________________________
>     >LLVM Developers mailing list
>     >llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>     >https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

Itaru Kitayama via llvm-dev

2020-Apr-04 00:03 UTC

head link

[llvm-dev] LLD issue on a massively parallel build machine

Passing -Wl,--threads=1 via -DCMAKE_CXX_FLAGS causes a config error.

-- Check for working CXX compiler:
/home/usr4/c74014i/opt/clang/current/bin/clang++ -- broken
CMake Error at
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake:53
(message):
  The C++ compiler

    "/home/usr4/c74014i/opt/clang/current/bin/clang++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /tmp/202004040902/CMakeFiles/CMakeTmp

    Run Build Command(s):/home/usr4/c74014i/opt/ninja/bin/ninja cmTC_9430a
&& [1/2] Building CXX object
CMakeFiles/cmTC_9430a.dir/testCXXCompiler.cxx.o
    clang-11: warning: -Wl,--threads=1: 'linker' input unused
[-Wunused-command-line-argument]
    [2/2] Linking CXX executable cmTC_9430a
    FAILED: cmTC_9430a
    : && /home/usr4/c74014i/opt/clang/current/bin/clang++ 
-Wl,--threads=1
  CMakeFiles/cmTC_9430a.dir/testCXXCompiler.cxx.o  -o cmTC_9430a   && :
    /usr/bin/ld: unrecognized option '--threads=1'
    /usr/bin/ld: use the --help option for usage information
    clang-11: error: linker command failed with exit code 1 (use -v to see
invocation)
    ninja: build stopped: subcommand failed.

On Thu, Apr 2, 2020 at 1:49 AM Fangrui Song <maskray at google.com> wrote:
> On 2020-03-29, Nemanja Ivanovic via llvm-dev wrote:
> >Glad you got it working.
> >My suggestion about LLVM_ENABLE_THREADS didn't work because you
didn't
> >apply it when building the build linker.
> >
> >When you don't have the ability to rebuild the build compiler, this
> doesn't
> >apply. In those cases, I end up doing a dirty hack where I use a
wrapper
> >script with something like:
> >
> ><path-to-build-compiler> "$@" -Wl,--no-threads
> >
> >To invoke the build compiler. That way you still get full parallelism
> while
> >compiling and linking, but you prevent LLD from using threads.
>
>
> https://reviews.llvm.org/D76885 (committed yesterday) changed
> --no-threads to --threads={1,2,...} (--no-threads is used rarely (GNU ld
> does not support it) so I'd rather not keep the option for
> compatibility).
>
> You may try -Wl,--threads=1
>
> >On Sat., Mar. 28, 2020, 4:19 p.m. Itaru Kitayama, <
> itaru.kitayama at gmail.com>
> >wrote:
> >
> >> Good news, I was able to use up to 37 cores for LLVM build with
LLD.
> >> The build speed, did not measure precisely though, is comparable
to the
> >> build with GNU ld case.
> >>
> >> Thank you all for your help!
> >>
> >> On Sun, Mar 29, 2020 at 5:04 AM Alex Brachet-Mialot <
> >> alexbrachetmialot at gmail.com> wrote:
> >>
> >>> Yes unfortunately that would limit you to 4 cores.
> >>>
> >>> There’s no easy way to use lld with —no-threads with our build
system
> at
> >>> the moment unfortunately.
> >>>
> >>> I also had just been using ld.gold, but will switch now that
numactl
> >>> works for me.
> >>>
> >>> On Sat, Mar 28, 2020 at 3:58 PM Itaru Kitayama <
> itaru.kitayama at gmail.com>
> >>> wrote:
> >>>
> >>>> That is slowing down the build visibly, for the speed I
should stick
> >>>> with ld at the moment.
> >>>>
> >>>>
> >>>> On Sun, Mar 29, 2020 at 4:42 AM Alexandre Ganea <
> >>>> alexandre.ganea at ubisoft.com> wrote:
> >>>>
> >>>>> Would `taskset -c 0-3 ninja check-all -j4` work?
> >>>>>
> >>>>>
> >>>>>
> >>>>> *De :* Itaru Kitayama <itaru.kitayama at
gmail.com>
> >>>>> *Envoyé :* March 28, 2020 3:37 PM
> >>>>> *À :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
> >>>>> *Cc :* Alexandre Ganea <alexandre.ganea at
ubisoft.com>; LLVM Dev <
> >>>>> llvm-dev at lists.llvm.org>; Nemanja Ivanovic
<nemanja.i.ibm at gmail.com>
> >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
> >>>>> machine
> >>>>>
> >>>>>
> >>>>>
> >>>>> $ free -g
> >>>>>               total        used        free     
shared  buff/cache
> >>>>> available
> >>>>> Mem:            376         149          20          
1         207
> >>>>>     225
> >>>>> Swap:             3           0           3
> >>>>>
> >>>>>
> >>>>>
> >>>>> Too small swap size for a 72-core login machine?
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sun, Mar 29, 2020 at 4:28 AM Alex Brachet-Mialot
<
> >>>>> alexbrachetmialot at gmail.com> wrote:
> >>>>>
> >>>>> Enable threads is for building llvm with or without
thread support.
> To
> >>>>> my knowledge it has no effect on the build system.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Like was discussed above you could try limiting  the
number of
> >>>>> processes created (assuming your lld was after that
commit) with
> numactl or
> >>>>> taskset.
> >>>>>
> >>>>>
> >>>>>
> >>>>> In your case though it looked like you were running
out of memory.
> You
> >>>>> could try increasing your swap space?
> >>>>>
> >>>>>
> >>>>>
> >>>>> Alex
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama <
> >>>>> itaru.kitayama at gmail.com> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> My configuration is below:
> >>>>>
> >>>>> cmake -GNinja -DLLVM_ENABLE_LLD=ON
-DLLVM_ENABLE_THREADS=OFF
> >>>>> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
> >>>>> -DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
> >>>>> -DLIBOMPTARGET_ENABLE_DEBUG=ON
> >>>>> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
> -DCMAKE_C_COMPILER=clang
> >>>>> -DCMAKE_C_FLAGS=""
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS=""
> >>>>>
>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
> >>>>> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
> >>>>> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
> -DLLVM_TARGETS_TO_BUILD="all"
> >>>>> /tmp/llvm-project/llvm
> >>>>>
> >>>>>
> >>>>>
> >>>>> but did not help my case:
> >>>>>
> >>>>>
> >>>>>
> >>>>> [...]
> >>>>>
> >>>>> [4802/5201] Linking CXX executable bin/lld
> >>>>> FAILED: bin/lld
> >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
> >>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
> >>>>> -Werror=unguarded-availability-new -Wall -Wextra
> -Wno-unused-parameter
> >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
> >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
> >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
> >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections
> -fdata-sections
> >>>>> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld
-Wl,--color-diagnostics
> >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
> >>>>> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o  -o
bin/lld
> >>>>>  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMSupport.a  -lpthread
> >>>>>  lib/liblldCommon.a  lib/liblldCOFF.a 
lib/liblldDriver.a
> lib/liblldELF.a
> >>>>>  lib/liblldMinGW.a  lib/liblldWasm.a 
lib/liblldMachO.a
> >>>>>  lib/liblldReaderWriter.a  lib/liblldYAML.a 
lib/liblldCore.a
> >>>>>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a 
lib/libLLVMLibDriver.a
> >>>>>  lib/libLLVMWindowsManifest.a  /usr/lib64/libxml2.so
> lib/liblldCommon.a
> >>>>>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
> >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Disassembler.a
> >>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
> >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
> >>>>>  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUAsmParser.a
> >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUDesc.a
> >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
> lib/libLLVMARMCodeGen.a
> >>>>>  lib/libLLVMARMAsmParser.a 
lib/libLLVMARMDisassembler.a
> >>>>>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
> >>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a
> lib/libLLVMAVRDesc.a
> >>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
> >>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a
> lib/libLLVMBPFDesc.a
> >>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
> >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
> >>>>>  lib/libLLVMHexagonDisassembler.a 
lib/libLLVMHexagonDesc.a
> >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
> >>>>>  lib/libLLVMLanaiAsmParser.a 
lib/libLLVMLanaiDisassembler.a
> >>>>>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a
> lib/libLLVMMipsCodeGen.a
> >>>>>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
> >>>>>  lib/libLLVMMipsDisassembler.a  lib/libLLVMMipsInfo.a
> >>>>>  lib/libLLVMMSP430CodeGen.a 
lib/libLLVMMSP430AsmParser.a
> >>>>>  lib/libLLVMMSP430Desc.a 
lib/libLLVMMSP430Disassembler.a
> >>>>>  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXCodeGen.a
> lib/libLLVMNVPTXDesc.a
> >>>>>  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCCodeGen.a
> >>>>>  lib/libLLVMPowerPCAsmParser.a 
lib/libLLVMPowerPCDesc.a
> >>>>>  lib/libLLVMPowerPCDisassembler.a 
lib/libLLVMPowerPCInfo.a
> >>>>>  lib/libLLVMRISCVCodeGen.a 
lib/libLLVMRISCVAsmParser.a
> >>>>>  lib/libLLVMRISCVDesc.a 
lib/libLLVMRISCVDisassembler.a
> >>>>>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a
> lib/libLLVMSparcCodeGen.a
> >>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
> >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
> >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
> >>>>>  lib/libLLVMSystemZDisassembler.a 
lib/libLLVMSystemZDesc.a
> >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
> >>>>>  lib/libLLVMWebAssemblyAsmParser.a
> lib/libLLVMWebAssemblyDisassembler.a
> >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
> >>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a
> lib/libLLVMGlobalISel.a
> >>>>>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
> >>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a
> lib/libLLVMX86Utils.a
> >>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMAsmPrinter.a
> >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
> >>>>>  lib/libLLVMXCoreDesc.a 
lib/libLLVMXCoreDisassembler.a
> >>>>>  lib/libLLVMMCDisassembler.a  lib/libLLVMXCoreInfo.a
> lib/libLLVMLTO.a
> >>>>>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a
> >>>>>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a 
lib/libLLVMipo.a
> >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a
> lib/libLLVMVectorize.a
> >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
> >>>>>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a
> lib/libLLVMIRReader.a
> >>>>>  lib/libLLVMAsmParser.a  lib/libLLVMInstrumentation.a
> >>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
> lib/libLLVMObject.a
> >>>>>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a 
lib/libLLVMMC.a
> >>>>>  lib/libLLVMDebugInfoCodeView.a 
lib/libLLVMDebugInfoMSF.a
> >>>>>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a 
lib/libLLVMCore.a
> >>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
> >>>>>  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a 
-lz  -lrt  -ldl
> >>>>>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a
&& cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/lld-link && cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld.lld && cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld64.lld && cd
> >>>>> /tmp/build/202003290405/tools/lld/tools/lld &&
> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/wasm-ld
> >>>>> terminating with uncaught exception of type
std::__1::system_error:
> >>>>> thread constructor failed: Resource temporarily
unavailable
> >>>>> PLEASE submit a bug report to https://bugs.llvm.org/
and include the
> >>>>> crash backtrace.
> >>>>>  #0 0x00000000007924f4
PrintStackTraceSignalHandler(void*)
> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
> >>>>>  #1 0x00000000007902ce llvm::sys::RunSignalHandlers()
> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
> >>>>>  #2 0x0000000000792bd5 SignalHandler(int)
> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
> >>>>>  #3 0x00007fddd54bb630 __restore_rt
(/lib64/libpthread.so.0+0xf630)
> >>>>>  #4 0x00007fddd3b40377 raise
> >>>>>
>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
> >>>>>  #5 0x00007fddd3b41a68 abort
> >>>>> /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
> >>>>>  #6 0x00007fddd411ae8b
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
> >>>>>  #7 0x00007fddd40fdb9f demangling_terminate_handler()
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
> >>>>>  #8 0x00007fddd411a0a3 std::__terminate(void (*)())
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
> >>>>>  #9 0x00007fddd411ce06
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
> >>>>> #10 0x00007fddd411cd9f
> >>>>>
>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
> >>>>> #11 0x00007fddd43bded4
std::__1::__throw_system_error(int, char
> const*)
> >>>>>
> (/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
> >>>>> #12 0x0000000002a7a316 void*
> >>>>>
>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
> >>>>>
std::__1::default_delete<std::__1::__thread_struct> >,
> >>>>> llvm::parallel::detail::(anonymous
> >>>>>
>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
> >>>>> >(void*)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
> >>>>> #13 0x00007fddd54b3ea5 start_thread
> >>>>>
/usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
> >>>>> #14 0x00007fddd3c088cd clone
> >>>>>
>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
> >>>>> clang-11: error: unable to execute command: Aborted
> >>>>> clang-11: error: linker command failed due to signal
(use -v to see
> >>>>> invocation)
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic <
> >>>>> nemanja.i.ibm at gmail.com> wrote:
> >>>>>
> >>>>> The discussion in https://reviews.llvm.org/D76240
> >>>>>
> >>>>> may be of interest to the participants of this thread.
Specifically,
> >>>>> Fangrui's planned work on a command line option to
limit threads for
> LLD.
> >>>>> Until that becomes a reality, you might want to use
the CMake
> variable
> >>>>> -DLLVM_ENABLE_THREADS=OFF when building the LLD that
you are using to
> >>>>> prevent parallelism.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via
llvm-dev <
> >>>>> llvm-dev at lists.llvm.org> wrote:
> >>>>>
> >>>>> Alex :
> >>>>>
> >>>>> Can you please try `numactl` or `taskset` after
> >>>>>
>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
> >>>>> ?
> >>>>>
> >>>>>
> >>>>>
> >>>>> There was a tiny bug in how *sched_getaffinity*() was
used, see:
> >>>>> https://reviews.llvm.org/D75153#1942336
> >>>>>
> >>>>>
> >>>>>
> >>>>> *De :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
> >>>>> *Envoyé :* March 28, 2020 12:11 PM
> >>>>> *À :* Itaru Kitayama <itaru.kitayama at
gmail.com>
> >>>>> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org>;
Alexandre Ganea <
> >>>>> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng
<maskray at google.com>
> >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
> >>>>> machine
> >>>>>
> >>>>>
> >>>>>
> >>>>> I also run into problems with lld on machines with
lots of cores
> (208),
> >>>>> but I run into rlimits for number of processes I can
create,
> something
> >>>>> outside of my control.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Unfortunately things like numactl or taskset do not
limit the
> >>>>> concurrency of lld currently.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via
llvm-dev <
> >>>>> llvm-dev at lists.llvm.org> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> On a 1296-core Intel machine with 376 GB, setting
> >>>>>
> >>>>>
> >>>>>
> >>>>> -DLLVM_PARALLEL_LINK_JOB=1
> >>>>>
> >>>>>
> >>>>>
> >>>>> does not help (switching back to ld scales) see:
> >>>>>
> >>>>>
> >>>>>
> >>>>> [5085/5201] Linking CXX executable bin/clang-11
> >>>>> FAILED: bin/clang-11
> >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
> >>>>> -fPIC -fvisibility-inlines-hidden -Werror=date-time
> >>>>> -Werror=unguarded-availability-new -Wall -Wextra
> -Wno-unused-parameter
> >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
> >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
> >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
> >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections
> -fdata-sections
> >>>>> -fno-common -Woverloaded-virtual
-Wno-nested-anon-types -O3 -DNDEBUG
> >>>>>  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
> >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
> >>>>>
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
> >>>>>  -o bin/clang-11 
-Wl,-rpath,"\$ORIGIN/../lib"
> lib/libLLVMAArch64CodeGen.a
> >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Desc.a
> >>>>>  lib/libLLVMAArch64Disassembler.a 
lib/libLLVMAArch64Info.a
> >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
> >>>>>  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a
> >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUInfo.a
> >>>>>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
> >>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
> >>>>>  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a
> lib/libLLVMARMUtils.a
> >>>>>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a
> lib/libLLVMAVRDesc.a
> >>>>>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
> >>>>>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a
> lib/libLLVMBPFDesc.a
> >>>>>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
> >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
> >>>>>  lib/libLLVMHexagonDesc.a 
lib/libLLVMHexagonDisassembler.a
> >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
> >>>>>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a
> >>>>>  lib/libLLVMLanaiDisassembler.a 
lib/libLLVMLanaiInfo.a
> >>>>>  lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a
> >>>>>  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
> >>>>>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
> >>>>>  lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a
> >>>>>  lib/libLLVMMSP430Disassembler.a 
lib/libLLVMMSP430Info.a
> >>>>>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a
> lib/libLLVMNVPTXInfo.a
> >>>>>  lib/libLLVMPowerPCCodeGen.a 
lib/libLLVMPowerPCAsmParser.a
> >>>>>  lib/libLLVMPowerPCDesc.a 
lib/libLLVMPowerPCDisassembler.a
> >>>>>  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a
> >>>>>  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a
> >>>>>  lib/libLLVMRISCVDisassembler.a 
lib/libLLVMRISCVInfo.a
> >>>>>  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
> >>>>>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
> >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
> >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
> >>>>>  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZDisassembler.a
> >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
> >>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDesc.a
> >>>>>  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyInfo.a
> >>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a
> lib/libLLVMX86Desc.a
> >>>>>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a
> lib/libLLVMX86Utils.a
> >>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
> >>>>>  lib/libLLVMXCoreDisassembler.a 
lib/libLLVMXCoreInfo.a
> >>>>>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a 
lib/libLLVMCore.a
> >>>>>  lib/libLLVMipo.a  lib/libLLVMAggressiveInstCombine.a
> >>>>>  lib/libLLVMInstCombine.a 
lib/libLLVMInstrumentation.a
> lib/libLLVMMC.a
> >>>>>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMOption.a
> >>>>>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a
> lib/libLLVMTransformUtils.a
> >>>>>  lib/libLLVMVectorize.a  -lpthread 
lib/libclangBasic.a
> >>>>>  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a
> >>>>>  lib/libclangFrontendTool.a 
lib/libclangSerialization.a
> >>>>>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
> >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a
> lib/libLLVMAMDGPUDesc.a
> >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
> lib/libLLVMARMDesc.a
> >>>>>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
> lib/libLLVMHexagonDesc.a
> >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a
> lib/libLLVMLanaiInfo.a
> >>>>>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a
> >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
> >>>>>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a
> lib/libLLVMAsmPrinter.a
> >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
> >>>>>  lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a
> lib/libLLVMCoverage.a
> >>>>>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMPasses.a
> >>>>>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a 
lib/libLLVMCoroutines.a
> >>>>>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a
> lib/libLLVMVectorize.a
> >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a
> lib/libLLVMAsmParser.a
> >>>>>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
> >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
> >>>>>  lib/libclangRewriteFrontend.a 
lib/libclangARCMigrate.a
> >>>>>  lib/libclangStaticAnalyzerFrontend.a
> lib/libclangStaticAnalyzerCheckers.a
> >>>>>  lib/libclangStaticAnalyzerCore.a 
lib/libclangCrossTU.a
> >>>>>  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a
> >>>>>  lib/libLLVMOption.a  lib/libclangParse.a
> lib/libclangSerialization.a
> >>>>>  lib/libclangSema.a  lib/libclangAnalysis.a
> lib/libclangASTMatchers.a
> >>>>>  lib/libclangEdit.a  lib/libclangFormat.a
> lib/libclangToolingInclusions.a
> >>>>>  lib/libclangToolingCore.a  lib/libclangAST.a
> lib/libLLVMFrontendOpenMP.a
> >>>>>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
> >>>>>  lib/libLLVMProfileData.a  lib/libLLVMObject.a 
lib/libLLVMMCParser.a
> >>>>>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a 
lib/libclangRewrite.a
> >>>>>  lib/libclangLex.a  lib/libclangBasic.a 
lib/libLLVMCore.a
> >>>>>  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a 
lib/libLLVMMC.a
> >>>>>  lib/libLLVMBinaryFormat.a 
lib/libLLVMDebugInfoCodeView.a
> >>>>>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a  -lz 
-lrt  -ldl
> -ltinfo
> >>>>>  -lpthread  -lm  lib/libLLVMDemangle.a && :
> >>>>> ld.lld: error: failed to open bin/clang-11: Cannot
allocate memory
> >>>>>
> >>>>> _______________________________________________
> >>>>> LLVM Developers mailing list
> >>>>> llvm-dev at lists.llvm.org
> >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>>>>
> >>>>> _______________________________________________
> >>>>> LLVM Developers mailing list
> >>>>> llvm-dev at lists.llvm.org
> >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>>>>
> >>>>>
>
> >_______________________________________________
> >LLVM Developers mailing list
> >llvm-dev at lists.llvm.org
> >https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200404/79031556/attachment-0001.html>

Fāng-ruì Sòng via llvm-dev

2020-Apr-04 00:29 UTC

head link

[llvm-dev] LLD issue on a massively parallel build machine

On Fri, Apr 3, 2020 at 5:04 PM Itaru Kitayama <itaru.kitayama at
gmail.com>
wrote:
>
> Passing -Wl,--threads=1 via -DCMAKE_CXX_FLAGS causes a config error.
>
> -- Check for working CXX compiler:
> /home/usr4/c74014i/opt/clang/current/bin/clang++ -- broken
> CMake Error at
>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake:53
> (message):
>   The C++ compiler
>
>     "/home/usr4/c74014i/opt/clang/current/bin/clang++"
>
>   is not able to compile a simple test program.
>
>   It fails with the following output:
>
>     Change Dir: /tmp/202004040902/CMakeFiles/CMakeTmp
>
>     Run Build Command(s):/home/usr4/c74014i/opt/ninja/bin/ninja cmTC_9430a
> && [1/2] Building CXX object
CMakeFiles/cmTC_9430a.dir/testCXXCompiler.cxx.o
>     clang-11: warning: -Wl,--threads=1: 'linker' input unused
> [-Wunused-command-line-argument]
>     [2/2] Linking CXX executable cmTC_9430a
>     FAILED: cmTC_9430a
>     : && /home/usr4/c74014i/opt/clang/current/bin/clang++ 
-Wl,--threads=1
>   CMakeFiles/cmTC_9430a.dir/testCXXCompiler.cxx.o  -o cmTC_9430a  
&& :
>     /usr/bin/ld: unrecognized option '--threads=1'
>     /usr/bin/ld: use the --help option for usage information
>     clang-11: error: linker command failed with exit code 1 (use -v to see
> invocation)
>     ninja: build stopped: subcommand failed.
>
-DCMAKE_EXE_LINKER_FLAGS=-Wl,--threads=1
-DCMAKE_SHARED_LINKER_FLAGS=-Wl,--threads=1
-Wl, should only be used at link time, not compile time.

> On Thu, Apr 2, 2020 at 1:49 AM Fangrui Song <maskray at google.com>
wrote:
>
>> On 2020-03-29, Nemanja Ivanovic via llvm-dev wrote:
>> >Glad you got it working.
>> >My suggestion about LLVM_ENABLE_THREADS didn't work because you
didn't
>> >apply it when building the build linker.
>> >
>> >When you don't have the ability to rebuild the build compiler,
this
>> doesn't
>> >apply. In those cases, I end up doing a dirty hack where I use a
wrapper
>> >script with something like:
>> >
>> ><path-to-build-compiler> "$@" -Wl,--no-threads
>> >
>> >To invoke the build compiler. That way you still get full
parallelism
>> while
>> >compiling and linking, but you prevent LLD from using threads.
>>
>>
>> https://reviews.llvm.org/D76885 (committed yesterday) changed
>> --no-threads to --threads={1,2,...} (--no-threads is used rarely (GNU
ld
>> does not support it) so I'd rather not keep the option for
>> compatibility).
>>
>> You may try -Wl,--threads=1
>>
>> >On Sat., Mar. 28, 2020, 4:19 p.m. Itaru Kitayama, <
>> itaru.kitayama at gmail.com>
>> >wrote:
>> >
>> >> Good news, I was able to use up to 37 cores for LLVM build
with LLD.
>> >> The build speed, did not measure precisely though, is
comparable to the
>> >> build with GNU ld case.
>> >>
>> >> Thank you all for your help!
>> >>
>> >> On Sun, Mar 29, 2020 at 5:04 AM Alex Brachet-Mialot <
>> >> alexbrachetmialot at gmail.com> wrote:
>> >>
>> >>> Yes unfortunately that would limit you to 4 cores.
>> >>>
>> >>> There’s no easy way to use lld with —no-threads with our
build system
>> at
>> >>> the moment unfortunately.
>> >>>
>> >>> I also had just been using ld.gold, but will switch now
that numactl
>> >>> works for me.
>> >>>
>> >>> On Sat, Mar 28, 2020 at 3:58 PM Itaru Kitayama <
>> itaru.kitayama at gmail.com>
>> >>> wrote:
>> >>>
>> >>>> That is slowing down the build visibly, for the speed
I should stick
>> >>>> with ld at the moment.
>> >>>>
>> >>>>
>> >>>> On Sun, Mar 29, 2020 at 4:42 AM Alexandre Ganea <
>> >>>> alexandre.ganea at ubisoft.com> wrote:
>> >>>>
>> >>>>> Would `taskset -c 0-3 ninja check-all -j4` work?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> *De :* Itaru Kitayama <itaru.kitayama at
gmail.com>
>> >>>>> *Envoyé :* March 28, 2020 3:37 PM
>> >>>>> *À :* Alex Brachet-Mialot <alexbrachetmialot at
gmail.com>
>> >>>>> *Cc :* Alexandre Ganea <alexandre.ganea at
ubisoft.com>; LLVM Dev <
>> >>>>> llvm-dev at lists.llvm.org>; Nemanja Ivanovic
<nemanja.i.ibm at gmail.com
>> >
>> >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
>> >>>>> machine
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> $ free -g
>> >>>>>               total        used        free     
shared  buff/cache
>> >>>>> available
>> >>>>> Mem:            376         149          20       
1         207
>> >>>>>     225
>> >>>>> Swap:             3           0           3
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Too small swap size for a 72-core login machine?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Sun, Mar 29, 2020 at 4:28 AM Alex
Brachet-Mialot <
>> >>>>> alexbrachetmialot at gmail.com> wrote:
>> >>>>>
>> >>>>> Enable threads is for building llvm with or
without thread support.
>> To
>> >>>>> my knowledge it has no effect on the build system.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Like was discussed above you could try limiting 
the number of
>> >>>>> processes created (assuming your lld was after
that commit) with
>> numactl or
>> >>>>> taskset.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> In your case though it looked like you were
running out of memory.
>> You
>> >>>>> could try increasing your swap space?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Alex
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Sat, Mar 28, 2020 at 3:15 PM Itaru Kitayama
<
>> >>>>> itaru.kitayama at gmail.com> wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> My configuration is below:
>> >>>>>
>> >>>>> cmake -GNinja -DLLVM_ENABLE_LLD=ON
-DLLVM_ENABLE_THREADS=OFF
>> >>>>> -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release
>> >>>>>
-DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/
>> >>>>> -DLIBOMPTARGET_ENABLE_DEBUG=ON
>> >>>>> -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today}
>> -DCMAKE_C_COMPILER=clang
>> >>>>> -DCMAKE_C_FLAGS=""
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS=""
>> >>>>>
>>
-DLLVM_ENABLE_PROJECTS="clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind"
>> >>>>> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60
>> >>>>> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60
>> -DLLVM_TARGETS_TO_BUILD="all"
>> >>>>> /tmp/llvm-project/llvm
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> but did not help my case:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> [...]
>> >>>>>
>> >>>>> [4802/5201] Linking CXX executable bin/lld
>> >>>>> FAILED: bin/lld
>> >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++
>> -stdlib=libc++
>> >>>>> -fPIC -fvisibility-inlines-hidden
-Werror=date-time
>> >>>>> -Werror=unguarded-availability-new -Wall -Wextra
>> -Wno-unused-parameter
>> >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
>> >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
>> >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
>> >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections
>> -fdata-sections
>> >>>>> -O3 -DNDEBUG  -stdlib=libc++ -fuse-ld=lld
-Wl,--color-diagnostics
>> >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
>> >>>>> tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o 
-o bin/lld
>> >>>>>  -Wl,-rpath,"\$ORIGIN/../lib" 
lib/libLLVMSupport.a  -lpthread
>> >>>>>  lib/liblldCommon.a  lib/liblldCOFF.a 
lib/liblldDriver.a
>> lib/liblldELF.a
>> >>>>>  lib/liblldMinGW.a  lib/liblldWasm.a 
lib/liblldMachO.a
>> >>>>>  lib/liblldReaderWriter.a  lib/liblldYAML.a 
lib/liblldCore.a
>> >>>>>  lib/liblldCOFF.a  lib/libLLVMDebugInfoPDB.a 
lib/libLLVMLibDriver.a
>> >>>>>  lib/libLLVMWindowsManifest.a 
/usr/lib64/libxml2.so
>> lib/liblldCommon.a
>> >>>>>  lib/libLLVMOption.a  lib/libLLVMAArch64CodeGen.a
>> >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Disassembler.a
>> >>>>>  lib/libLLVMAArch64Desc.a 
lib/libLLVMAArch64Info.a
>> >>>>>  lib/libLLVMAArch64Utils.a 
lib/libLLVMAMDGPUCodeGen.a
>> >>>>>  lib/libLLVMMIRParser.a 
lib/libLLVMAMDGPUAsmParser.a
>> >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUDesc.a
>> >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
>> lib/libLLVMARMCodeGen.a
>> >>>>>  lib/libLLVMARMAsmParser.a 
lib/libLLVMARMDisassembler.a
>> >>>>>  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a 
lib/libLLVMARMUtils.a
>> >>>>>  lib/libLLVMAVRCodeGen.a 
lib/libLLVMAVRAsmParser.a
>> lib/libLLVMAVRDesc.a
>> >>>>>  lib/libLLVMAVRDisassembler.a 
lib/libLLVMAVRInfo.a
>> >>>>>  lib/libLLVMBPFCodeGen.a 
lib/libLLVMBPFAsmParser.a
>> lib/libLLVMBPFDesc.a
>> >>>>>  lib/libLLVMBPFDisassembler.a 
lib/libLLVMBPFInfo.a
>> >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
>> >>>>>  lib/libLLVMHexagonDisassembler.a 
lib/libLLVMHexagonDesc.a
>> >>>>>  lib/libLLVMHexagonInfo.a 
lib/libLLVMLanaiCodeGen.a
>> >>>>>  lib/libLLVMLanaiAsmParser.a 
lib/libLLVMLanaiDisassembler.a
>> >>>>>  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a
>> lib/libLLVMMipsCodeGen.a
>> >>>>>  lib/libLLVMMipsAsmParser.a  lib/libLLVMMipsDesc.a
>> >>>>>  lib/libLLVMMipsDisassembler.a 
lib/libLLVMMipsInfo.a
>> >>>>>  lib/libLLVMMSP430CodeGen.a 
lib/libLLVMMSP430AsmParser.a
>> >>>>>  lib/libLLVMMSP430Desc.a 
lib/libLLVMMSP430Disassembler.a
>> >>>>>  lib/libLLVMMSP430Info.a 
lib/libLLVMNVPTXCodeGen.a
>> lib/libLLVMNVPTXDesc.a
>> >>>>>  lib/libLLVMNVPTXInfo.a 
lib/libLLVMPowerPCCodeGen.a
>> >>>>>  lib/libLLVMPowerPCAsmParser.a 
lib/libLLVMPowerPCDesc.a
>> >>>>>  lib/libLLVMPowerPCDisassembler.a 
lib/libLLVMPowerPCInfo.a
>> >>>>>  lib/libLLVMRISCVCodeGen.a 
lib/libLLVMRISCVAsmParser.a
>> >>>>>  lib/libLLVMRISCVDesc.a 
lib/libLLVMRISCVDisassembler.a
>> >>>>>  lib/libLLVMRISCVInfo.a  lib/libLLVMRISCVUtils.a
>> lib/libLLVMSparcCodeGen.a
>> >>>>>  lib/libLLVMSparcAsmParser.a 
lib/libLLVMSparcDesc.a
>> >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
>> >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
>> >>>>>  lib/libLLVMSystemZDisassembler.a 
lib/libLLVMSystemZDesc.a
>> >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
>> >>>>>  lib/libLLVMWebAssemblyAsmParser.a
>> lib/libLLVMWebAssemblyDisassembler.a
>> >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
>> >>>>>  lib/libLLVMX86CodeGen.a  lib/libLLVMCFGuard.a
>> lib/libLLVMGlobalISel.a
>> >>>>>  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>> >>>>>  lib/libLLVMX86Disassembler.a 
lib/libLLVMX86Info.a
>> lib/libLLVMX86Utils.a
>> >>>>>  lib/libLLVMXCoreCodeGen.a 
lib/libLLVMAsmPrinter.a
>> >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
>> >>>>>  lib/libLLVMXCoreDesc.a 
lib/libLLVMXCoreDisassembler.a
>> >>>>>  lib/libLLVMMCDisassembler.a 
lib/libLLVMXCoreInfo.a
>> lib/libLLVMLTO.a
>> >>>>>  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a 
lib/libLLVMCodeGen.a
>> >>>>>  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a 
lib/libLLVMipo.a
>> >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a
>> lib/libLLVMVectorize.a
>> >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
>> >>>>>  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a
>> lib/libLLVMIRReader.a
>> >>>>>  lib/libLLVMAsmParser.a 
lib/libLLVMInstrumentation.a
>> >>>>>  lib/libLLVMTransformUtils.a 
lib/libLLVMAnalysis.a
>> lib/libLLVMObject.a
>> >>>>>  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a 
lib/libLLVMMC.a
>> >>>>>  lib/libLLVMDebugInfoCodeView.a 
lib/libLLVMDebugInfoMSF.a
>> >>>>>  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a 
lib/libLLVMCore.a
>> >>>>>  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a
>> >>>>>  lib/libLLVMBitstreamReader.a 
lib/libLLVMSupport.a  -lz  -lrt  -ldl
>> >>>>>  -ltinfo  -lpthread  -lm  lib/libLLVMDemangle.a
&& cd
>> >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/lld-link && cd
>> >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld.lld && cd
>> >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/ld64.lld && cd
>> >>>>> /tmp/build/202003290405/tools/lld/tools/lld
&&
>> >>>>>
/home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E
>> >>>>> create_symlink lld
/tmp/build/202003290405/./bin/wasm-ld
>> >>>>> terminating with uncaught exception of type
std::__1::system_error:
>> >>>>> thread constructor failed: Resource temporarily
unavailable
>> >>>>> PLEASE submit a bug report to
https://bugs.llvm.org/ and include
>> the
>> >>>>> crash backtrace.
>> >>>>>  #0 0x00000000007924f4
PrintStackTraceSignalHandler(void*)
>> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
>> >>>>>  #1 0x00000000007902ce
llvm::sys::RunSignalHandlers()
>> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
>> >>>>>  #2 0x0000000000792bd5 SignalHandler(int)
>> >>>>>
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
>> >>>>>  #3 0x00007fddd54bb630 __restore_rt
(/lib64/libpthread.so.0+0xf630)
>> >>>>>  #4 0x00007fddd3b40377 raise
>> >>>>>
>>
/usr/src/debug/glibc-2.17-c758a686/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:55:0
>> >>>>>  #5 0x00007fddd3b41a68 abort
>> >>>>>
/usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
>> >>>>>  #6 0x00007fddd411ae8b
>> >>>>>
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2be8b)
>> >>>>>  #7 0x00007fddd40fdb9f
demangling_terminate_handler()
>> >>>>>
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0xeb9f)
>> >>>>>  #8 0x00007fddd411a0a3 std::__terminate(void
(*)())
>> >>>>>
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2b0a3)
>> >>>>>  #9 0x00007fddd411ce06
>> >>>>>
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2de06)
>> >>>>> #10 0x00007fddd411cd9f
>> >>>>>
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++abi.so.1+0x2dd9f)
>> >>>>> #11 0x00007fddd43bded4
std::__1::__throw_system_error(int, char
>> const*)
>> >>>>>
>>
(/home/usr4/c74014i/opt/clang/202003290200/bin/../lib/libc++.so.1+0x90ed4)
>> >>>>> #12 0x0000000002a7a316 void*
>> >>>>>
>>
std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct,
>> >>>>>
std::__1::default_delete<std::__1::__thread_struct> >,
>> >>>>> llvm::parallel::detail::(anonymous
>> >>>>>
>>
namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()>
>> >>>>> >(void*)
(/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
>> >>>>> #13 0x00007fddd54b3ea5 start_thread
>> >>>>>
/usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
>> >>>>> #14 0x00007fddd3c088cd clone
>> >>>>>
>>
/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
>> >>>>> clang-11: error: unable to execute command:
Aborted
>> >>>>> clang-11: error: linker command failed due to
signal (use -v to see
>> >>>>> invocation)
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Sun, Mar 29, 2020 at 1:44 AM Nemanja Ivanovic
<
>> >>>>> nemanja.i.ibm at gmail.com> wrote:
>> >>>>>
>> >>>>> The discussion in https://reviews.llvm.org/D76240
>> >>>>>
>> >>>>> may be of interest to the participants of this
thread. Specifically,
>> >>>>> Fangrui's planned work on a command line
option to limit threads
>> for LLD.
>> >>>>> Until that becomes a reality, you might want to
use the CMake
>> variable
>> >>>>> -DLLVM_ENABLE_THREADS=OFF when building the LLD
that you are using
>> to
>> >>>>> prevent parallelism.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea
via llvm-dev <
>> >>>>> llvm-dev at lists.llvm.org> wrote:
>> >>>>>
>> >>>>> Alex :
>> >>>>>
>> >>>>> Can you please try `numactl` or `taskset` after
>> >>>>>
>>
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
>> >>>>> ?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> There was a tiny bug in how *sched_getaffinity*()
was used, see:
>> >>>>> https://reviews.llvm.org/D75153#1942336
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> *De :* Alex Brachet-Mialot <alexbrachetmialot
at gmail.com>
>> >>>>> *Envoyé :* March 28, 2020 12:11 PM
>> >>>>> *À :* Itaru Kitayama <itaru.kitayama at
gmail.com>
>> >>>>> *Cc :* LLVM Dev <llvm-dev at
lists.llvm.org>; Alexandre Ganea <
>> >>>>> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng
<maskray at google.com>
>> >>>>> *Objet :* Re: [llvm-dev] LLD issue on a massively
parallel build
>> >>>>> machine
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> I also run into problems with lld on machines with
lots of cores
>> (208),
>> >>>>> but I run into rlimits for number of processes I
can create,
>> something
>> >>>>> outside of my control.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Unfortunately things like numactl or taskset do
not limit the
>> >>>>> concurrency of lld currently.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via
llvm-dev <
>> >>>>> llvm-dev at lists.llvm.org> wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> On a 1296-core Intel machine with 376 GB, setting
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> -DLLVM_PARALLEL_LINK_JOB=1
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> does not help (switching back to ld scales) see:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> [5085/5201] Linking CXX executable bin/clang-11
>> >>>>> FAILED: bin/clang-11
>> >>>>> : &&
/home/usr4/c74014i/opt/clang/current/bin/clang++
>> -stdlib=libc++
>> >>>>> -fPIC -fvisibility-inlines-hidden
-Werror=date-time
>> >>>>> -Werror=unguarded-availability-new -Wall -Wextra
>> -Wno-unused-parameter
>> >>>>> -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic
>> >>>>> -Wno-long-long -Wimplicit-fallthrough
-Wcovered-switch-default
>> >>>>> -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
>> >>>>> -Wstring-conversion -fdiagnostics-color
-ffunction-sections
>> -fdata-sections
>> >>>>> -fno-common -Woverloaded-virtual
-Wno-nested-anon-types -O3 -DNDEBUG
>> >>>>>  -stdlib=libc++ -fuse-ld=lld
-Wl,--color-diagnostics
>> >>>>> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic 
-Wl,-O3
>> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
>> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
>> >>>>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
>> >>>>>
>>
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
>> >>>>>  -o bin/clang-11 
-Wl,-rpath,"\$ORIGIN/../lib"
>> lib/libLLVMAArch64CodeGen.a
>> >>>>>  lib/libLLVMAArch64AsmParser.a 
lib/libLLVMAArch64Desc.a
>> >>>>>  lib/libLLVMAArch64Disassembler.a 
lib/libLLVMAArch64Info.a
>> >>>>>  lib/libLLVMAArch64Utils.a 
lib/libLLVMAMDGPUCodeGen.a
>> >>>>>  lib/libLLVMAMDGPUAsmParser.a 
lib/libLLVMAMDGPUDesc.a
>> >>>>>  lib/libLLVMAMDGPUDisassembler.a 
lib/libLLVMAMDGPUInfo.a
>> >>>>>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>> >>>>>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
>> >>>>>  lib/libLLVMARMDisassembler.a 
lib/libLLVMARMInfo.a
>> lib/libLLVMARMUtils.a
>> >>>>>  lib/libLLVMAVRCodeGen.a 
lib/libLLVMAVRAsmParser.a
>> lib/libLLVMAVRDesc.a
>> >>>>>  lib/libLLVMAVRDisassembler.a 
lib/libLLVMAVRInfo.a
>> >>>>>  lib/libLLVMBPFCodeGen.a 
lib/libLLVMBPFAsmParser.a
>> lib/libLLVMBPFDesc.a
>> >>>>>  lib/libLLVMBPFDisassembler.a 
lib/libLLVMBPFInfo.a
>> >>>>>  lib/libLLVMHexagonCodeGen.a 
lib/libLLVMHexagonAsmParser.a
>> >>>>>  lib/libLLVMHexagonDesc.a 
lib/libLLVMHexagonDisassembler.a
>> >>>>>  lib/libLLVMHexagonInfo.a 
lib/libLLVMLanaiCodeGen.a
>> >>>>>  lib/libLLVMLanaiAsmParser.a 
lib/libLLVMLanaiDesc.a
>> >>>>>  lib/libLLVMLanaiDisassembler.a 
lib/libLLVMLanaiInfo.a
>> >>>>>  lib/libLLVMMipsCodeGen.a 
lib/libLLVMMipsAsmParser.a
>> >>>>>  lib/libLLVMMipsDesc.a 
lib/libLLVMMipsDisassembler.a
>> >>>>>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
>> >>>>>  lib/libLLVMMSP430AsmParser.a 
lib/libLLVMMSP430Desc.a
>> >>>>>  lib/libLLVMMSP430Disassembler.a 
lib/libLLVMMSP430Info.a
>> >>>>>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a
>> lib/libLLVMNVPTXInfo.a
>> >>>>>  lib/libLLVMPowerPCCodeGen.a 
lib/libLLVMPowerPCAsmParser.a
>> >>>>>  lib/libLLVMPowerPCDesc.a 
lib/libLLVMPowerPCDisassembler.a
>> >>>>>  lib/libLLVMPowerPCInfo.a 
lib/libLLVMRISCVCodeGen.a
>> >>>>>  lib/libLLVMRISCVAsmParser.a 
lib/libLLVMRISCVDesc.a
>> >>>>>  lib/libLLVMRISCVDisassembler.a 
lib/libLLVMRISCVInfo.a
>> >>>>>  lib/libLLVMRISCVUtils.a 
lib/libLLVMSparcCodeGen.a
>> >>>>>  lib/libLLVMSparcAsmParser.a 
lib/libLLVMSparcDesc.a
>> >>>>>  lib/libLLVMSparcDisassembler.a 
lib/libLLVMSparcInfo.a
>> >>>>>  lib/libLLVMSystemZCodeGen.a 
lib/libLLVMSystemZAsmParser.a
>> >>>>>  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZDisassembler.a
>> >>>>>  lib/libLLVMSystemZInfo.a 
lib/libLLVMWebAssemblyCodeGen.a
>> >>>>>  lib/libLLVMWebAssemblyAsmParser.a 
lib/libLLVMWebAssemblyDesc.a
>> >>>>>  lib/libLLVMWebAssemblyDisassembler.a 
lib/libLLVMWebAssemblyInfo.a
>> >>>>>  lib/libLLVMX86CodeGen.a 
lib/libLLVMX86AsmParser.a
>> lib/libLLVMX86Desc.a
>> >>>>>  lib/libLLVMX86Disassembler.a 
lib/libLLVMX86Info.a
>> lib/libLLVMX86Utils.a
>> >>>>>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
>> >>>>>  lib/libLLVMXCoreDisassembler.a 
lib/libLLVMXCoreInfo.a
>> >>>>>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a 
lib/libLLVMCore.a
>> >>>>>  lib/libLLVMipo.a 
lib/libLLVMAggressiveInstCombine.a
>> >>>>>  lib/libLLVMInstCombine.a 
lib/libLLVMInstrumentation.a
>> lib/libLLVMMC.a
>> >>>>>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a
>> lib/libLLVMOption.a
>> >>>>>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a
>> lib/libLLVMTransformUtils.a
>> >>>>>  lib/libLLVMVectorize.a  -lpthread 
lib/libclangBasic.a
>> >>>>>  lib/libclangCodeGen.a  lib/libclangDriver.a 
lib/libclangFrontend.a
>> >>>>>  lib/libclangFrontendTool.a 
lib/libclangSerialization.a
>> >>>>>  lib/libLLVMAArch64Desc.a 
lib/libLLVMAArch64Info.a
>> >>>>>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a
>> lib/libLLVMAMDGPUDesc.a
>> >>>>>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a
>> lib/libLLVMARMDesc.a
>> >>>>>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
>> lib/libLLVMHexagonDesc.a
>> >>>>>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a
>> lib/libLLVMLanaiInfo.a
>> >>>>>  lib/libLLVMSystemZDesc.a 
lib/libLLVMSystemZInfo.a
>> >>>>>  lib/libLLVMWebAssemblyDesc.a 
lib/libLLVMWebAssemblyInfo.a
>> >>>>>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a
>> lib/libLLVMAsmPrinter.a
>> >>>>>  lib/libLLVMDebugInfoDWARF.a 
lib/libLLVMSelectionDAG.a
>> >>>>>  lib/libLLVMMCDisassembler.a 
lib/libclangCodeGen.a
>> lib/libLLVMCoverage.a
>> >>>>>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a 
lib/libLLVMPasses.a
>> >>>>>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a 
lib/libLLVMCoroutines.a
>> >>>>>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a
>> lib/libLLVMVectorize.a
>> >>>>>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a
>> lib/libLLVMAsmParser.a
>> >>>>>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
>> >>>>>  lib/libLLVMAggressiveInstCombine.a 
lib/libLLVMInstCombine.a
>> >>>>>  lib/libclangRewriteFrontend.a 
lib/libclangARCMigrate.a
>> >>>>>  lib/libclangStaticAnalyzerFrontend.a
>> lib/libclangStaticAnalyzerCheckers.a
>> >>>>>  lib/libclangStaticAnalyzerCore.a 
lib/libclangCrossTU.a
>> >>>>>  lib/libclangIndex.a  lib/libclangFrontend.a 
lib/libclangDriver.a
>> >>>>>  lib/libLLVMOption.a  lib/libclangParse.a
>> lib/libclangSerialization.a
>> >>>>>  lib/libclangSema.a  lib/libclangAnalysis.a
>> lib/libclangASTMatchers.a
>> >>>>>  lib/libclangEdit.a  lib/libclangFormat.a
>> lib/libclangToolingInclusions.a
>> >>>>>  lib/libclangToolingCore.a  lib/libclangAST.a
>> lib/libLLVMFrontendOpenMP.a
>> >>>>>  lib/libLLVMTransformUtils.a 
lib/libLLVMAnalysis.a
>> >>>>>  lib/libLLVMProfileData.a  lib/libLLVMObject.a
>> lib/libLLVMMCParser.a
>> >>>>>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a 
lib/libclangRewrite.a
>> >>>>>  lib/libclangLex.a  lib/libclangBasic.a 
lib/libLLVMCore.a
>> >>>>>  lib/libLLVMRemarks.a 
lib/libLLVMBitstreamReader.a  lib/libLLVMMC.a
>> >>>>>  lib/libLLVMBinaryFormat.a 
lib/libLLVMDebugInfoCodeView.a
>> >>>>>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a 
-lz  -lrt  -ldl
>> -ltinfo
>> >>>>>  -lpthread  -lm  lib/libLLVMDemangle.a &&
:
>> >>>>> ld.lld: error: failed to open bin/clang-11: Cannot
allocate memory
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> LLVM Developers mailing list
>> >>>>> llvm-dev at lists.llvm.org
>> >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> LLVM Developers mailing list
>> >>>>> llvm-dev at lists.llvm.org
>> >>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> >>>>>
>> >>>>>
>>
>> >_______________________________________________
>> >LLVM Developers mailing list
>> >llvm-dev at lists.llvm.org
>> >https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
-- 
宋方睿
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200403/2a64c29b/attachment.html>

Possibly Parallel Threads

Search for more maybe matching threads

llvm dev - Apr 2020 - LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

[llvm-dev] LLD issue on a massively parallel build machine

Possibly Parallel Threads