thr3ads.net - llvm dev - [llvm-dev] Building with LLVM_PARALLEL_XXX

If this information is useful, please help other people find it:
Share via:

Sedat Dilek via llvm-dev

2016-Mar-18 08:18 UTC

[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS

On Thu, Mar 17, 2016 at 11:45 AM, Sedat Dilek <sedat.dilek at gmail.com>
wrote:> On Thu, Mar 17, 2016 at 10:05 AM, Sedat Dilek <sedat.dilek at
gmail.com> wrote:
>> On Mon, Mar 14, 2016 at 5:30 PM, Chris Bieneman <cbieneman at
apple.com> wrote:
>> [ brutal-snip ]
>> ...
>>> [ TODO#S: Before doing a 2nd build (and in a 3rd run using more
>>> optimized binaries) ]
>>>
>>> How do I anable LTO via CMAKE?
>>>
>>>
>>> LLVM_ENALBLE_LTO=On
>>>
>>
>> [ v4 of my build-script attached ]
>>
>> Hi Chris,
>>
>> thanks for the response!
>>
>> That seems to work (see below).
>>
>> $ cd $BUILD_DIR
>> $ grep -i lto CMakeCache.txt | grep ^[A-Z]
>> [ OUTPUT ]
>> LLVMLTO_LIB_DEPENDS:STATIC>> LLVM_ENALBLE_LTO:UNINITIALIZED=ON
>> LLVM_TOOL_LLVM_LTO_BUILD:BOOL=ON
>> LLVM_TOOL_LTO_BUILD:BOOL=ON
>>
LTO_LIB_DEPENDS:STATIC=general;LLVMX86CodeGen;general;LLVMX86AsmPrinter;general;LLVMX86AsmParser;general;LLVMX86Desc;general;LLVMX86Info;general;LLVMX86Disassembler;general;LLVMCore;general;LLVMLTO;general;LLVMMC;general;LLVMMCDisassembler;general;LLVMSupport;general;LLVMTarget;
>> COMPILER_RT_HAS_FNO_LTO_FLAG:INTERNAL=1
>> LLVM_TOOL_LLVM_LTO_BUILD-ADVANCED:INTERNAL=1
>> LLVM_TOOL_LTO_BUILD-ADVANCED:INTERNAL=1
>> [ /OUTPUT ]
>>
>> But is no more documented in
>> <http://llvm.org/releases/3.8.0/docs/CMake.html> or
>> <http://llvm.org/docs/CMake.html>.
>> I have seen LLVM_ENALBLE_LTO=On in some older docs.
>> Bring it back?
>> AFAICS, "cmake -LA | grep -i $pattern" die not show me this,
but I
>> might be wrong.
>> ( Some weeks ago, I dealt with LTO/PGO. )
>>
>
> OK, I got a typo, nevertheless LLVM_ENABLE_LTO does not work!
>
> [ ... ]
> -- Configuring done
> -- Generating done
> CMake Warning:
>   Manually-specified variables were not used by the project:
>
>     LLVM_ENABLE_LTO
>
>
> -- Build files have been written to:
> /home/wearefam/src/llvm-toolchain/llvm-build
>
> Do you need more input?
>
> Thanks!
>
I found in "http://llvmweekly.org/issue/110"

[ llvm.src/cmake/modules/HandleLLVMOptions.cmake ]

option(LLVM_ENABLE_LTO "Enable link-time optimization" OFF)
append_if(LLVM_ENABLE_LTO "-flto"
CMAKE_CXX_FLAGS
CMAKE_C_FLAGS
CMAKE_EXE_LINKER_FLAGS
CMAKE_SHARED_LINKER_FLAGS)

Searching for...

$ grep LLVM_ENABLE_LTO llvm/cmake/modules/HandleLLVMOptions.cmake
[ OUTPUT EMPTY ]

...gives no output.

So, LLVM_ENABLE_LTO is not available for LLVM/Clang v3.8.0?!

Hmm, maybe I can backport (or git cherry-pick) SVN r259766.

- Sedat -

[1] http://llvmweekly.org/issue/110
[2] http://reviews.llvm.org/rL259766

Sedat Dilek via llvm-dev

2016-Mar-18 08:41 UTC

head link

[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS

On Fri, Mar 18, 2016 at 9:18 AM, Sedat Dilek <sedat.dilek at gmail.com>
wrote:> On Thu, Mar 17, 2016 at 11:45 AM, Sedat Dilek <sedat.dilek at
gmail.com> wrote:
>> On Thu, Mar 17, 2016 at 10:05 AM, Sedat Dilek <sedat.dilek at
gmail.com> wrote:
>>> On Mon, Mar 14, 2016 at 5:30 PM, Chris Bieneman <cbieneman at
apple.com> wrote:
>>> [ brutal-snip ]
>>> ...
>>>> [ TODO#S: Before doing a 2nd build (and in a 3rd run using more
>>>> optimized binaries) ]
>>>>
>>>> How do I anable LTO via CMAKE?
>>>>
>>>>
>>>> LLVM_ENALBLE_LTO=On
>>>>
>>>
>>> [ v4 of my build-script attached ]
>>>
>>> Hi Chris,
>>>
>>> thanks for the response!
>>>
>>> That seems to work (see below).
>>>
>>> $ cd $BUILD_DIR
>>> $ grep -i lto CMakeCache.txt | grep ^[A-Z]
>>> [ OUTPUT ]
>>> LLVMLTO_LIB_DEPENDS:STATIC>>>
LLVM_ENALBLE_LTO:UNINITIALIZED=ON
>>> LLVM_TOOL_LLVM_LTO_BUILD:BOOL=ON
>>> LLVM_TOOL_LTO_BUILD:BOOL=ON
>>>
LTO_LIB_DEPENDS:STATIC=general;LLVMX86CodeGen;general;LLVMX86AsmPrinter;general;LLVMX86AsmParser;general;LLVMX86Desc;general;LLVMX86Info;general;LLVMX86Disassembler;general;LLVMCore;general;LLVMLTO;general;LLVMMC;general;LLVMMCDisassembler;general;LLVMSupport;general;LLVMTarget;
>>> COMPILER_RT_HAS_FNO_LTO_FLAG:INTERNAL=1
>>> LLVM_TOOL_LLVM_LTO_BUILD-ADVANCED:INTERNAL=1
>>> LLVM_TOOL_LTO_BUILD-ADVANCED:INTERNAL=1
>>> [ /OUTPUT ]
>>>
>>> But is no more documented in
>>> <http://llvm.org/releases/3.8.0/docs/CMake.html> or
>>> <http://llvm.org/docs/CMake.html>.
>>> I have seen LLVM_ENALBLE_LTO=On in some older docs.
>>> Bring it back?
>>> AFAICS, "cmake -LA | grep -i $pattern" die not show me
this, but I
>>> might be wrong.
>>> ( Some weeks ago, I dealt with LTO/PGO. )
>>>
>>
>> OK, I got a typo, nevertheless LLVM_ENABLE_LTO does not work!
>>
>> [ ... ]
>> -- Configuring done
>> -- Generating done
>> CMake Warning:
>>   Manually-specified variables were not used by the project:
>>
>>     LLVM_ENABLE_LTO
>>
>>
>> -- Build files have been written to:
>> /home/wearefam/src/llvm-toolchain/llvm-build
>>
>> Do you need more input?
>>
>> Thanks!
>>
>
> I found in "http://llvmweekly.org/issue/110"
>
> [ llvm.src/cmake/modules/HandleLLVMOptions.cmake ]
>
> option(LLVM_ENABLE_LTO "Enable link-time optimization" OFF)
> append_if(LLVM_ENABLE_LTO "-flto"
> CMAKE_CXX_FLAGS
> CMAKE_C_FLAGS
> CMAKE_EXE_LINKER_FLAGS
> CMAKE_SHARED_LINKER_FLAGS)
>
> Searching for...
>
> $ grep LLVM_ENABLE_LTO llvm/cmake/modules/HandleLLVMOptions.cmake
> [ OUTPUT EMPTY ]
>
> ...gives no output.
>
> So, LLVM_ENABLE_LTO is not available for LLVM/Clang v3.8.0?!
>
> Hmm, maybe I can backport (or git cherry-pick) SVN r259766.
>
> - Sedat -
>
> [1] http://llvmweekly.org/issue/110
> [2] http://reviews.llvm.org/rL259766
Backport-patch and v6 of my build-script attached.

- Sedat -
-------------- next part --------------
--- llvm.orig/cmake/modules/HandleLLVMOptions.cmake
+++ llvm/cmake/modules/HandleLLVMOptions.cmake
@@ -635,6 +635,13 @@ append_if(LLVM_BUILD_INSTRUMENTED "-fpro
   CMAKE_EXE_LINKER_FLAGS
   CMAKE_SHARED_LINKER_FLAGS)
 
+option(LLVM_ENABLE_LTO "Enable link-time optimization" OFF)
+append_if(LLVM_ENABLE_LTO "-flto"
+  CMAKE_CXX_FLAGS
+  CMAKE_C_FLAGS
+  CMAKE_EXE_LINKER_FLAGS
+  CMAKE_SHARED_LINKER_FLAGS)
+
 # Plugin support
 # FIXME: Make this configurable.
 if(WIN32 OR CYGWIN)
--- llvm.orig/docs/CMake.rst
+++ llvm/docs/CMake.rst
@@ -347,6 +347,10 @@ LLVM-specific variables
   are ``Address``, ``Memory``, ``MemoryWithOrigins``, ``Undefined``,
``Thread``,
   and ``Address;Undefined``. Defaults to empty string.
 
+**LLVM_ENABLE_LTO**:BOOL
+  Add the ``-flto`` flag to the compile and link command lines,
+  enabling link-time optimization. Defaults to OFF.
+
 **LLVM_PARALLEL_COMPILE_JOBS**:STRING
   Define the maximum number of concurrent compilation jobs.
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build_llvm-toolchain_clang-cmake-ninja-gold_v6.sh
Type: application/x-sh
Size: 5264 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160318/e369a606/attachment.sh>

Sedat Dilek via llvm-dev

2016-Mar-20 08:33 UTC

head link

[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS

On Fri, Mar 18, 2016 at 9:41 AM, Sedat Dilek <sedat.dilek at gmail.com>
wrote:> On Fri, Mar 18, 2016 at 9:18 AM, Sedat Dilek <sedat.dilek at
gmail.com> wrote:
>> On Thu, Mar 17, 2016 at 11:45 AM, Sedat Dilek <sedat.dilek at
gmail.com> wrote:
>>> On Thu, Mar 17, 2016 at 10:05 AM, Sedat Dilek <sedat.dilek at
gmail.com> wrote:
>>>> On Mon, Mar 14, 2016 at 5:30 PM, Chris Bieneman <cbieneman
at apple.com> wrote:
>>>> [ brutal-snip ]
>>>> ...
>>>>> [ TODO#S: Before doing a 2nd build (and in a 3rd run using
more
>>>>> optimized binaries) ]
>>>>>
>>>>> How do I anable LTO via CMAKE?
>>>>>
>>>>>
>>>>> LLVM_ENALBLE_LTO=On
>>>>>
>>>>
>>>> [ v4 of my build-script attached ]
>>>>
>>>> Hi Chris,
>>>>
>>>> thanks for the response!
>>>>
>>>> That seems to work (see below).
>>>>
>>>> $ cd $BUILD_DIR
>>>> $ grep -i lto CMakeCache.txt | grep ^[A-Z]
>>>> [ OUTPUT ]
>>>> LLVMLTO_LIB_DEPENDS:STATIC>>>>
LLVM_ENALBLE_LTO:UNINITIALIZED=ON
>>>> LLVM_TOOL_LLVM_LTO_BUILD:BOOL=ON
>>>> LLVM_TOOL_LTO_BUILD:BOOL=ON
>>>>
LTO_LIB_DEPENDS:STATIC=general;LLVMX86CodeGen;general;LLVMX86AsmPrinter;general;LLVMX86AsmParser;general;LLVMX86Desc;general;LLVMX86Info;general;LLVMX86Disassembler;general;LLVMCore;general;LLVMLTO;general;LLVMMC;general;LLVMMCDisassembler;general;LLVMSupport;general;LLVMTarget;
>>>> COMPILER_RT_HAS_FNO_LTO_FLAG:INTERNAL=1
>>>> LLVM_TOOL_LLVM_LTO_BUILD-ADVANCED:INTERNAL=1
>>>> LLVM_TOOL_LTO_BUILD-ADVANCED:INTERNAL=1
>>>> [ /OUTPUT ]
>>>>
>>>> But is no more documented in
>>>> <http://llvm.org/releases/3.8.0/docs/CMake.html> or
>>>> <http://llvm.org/docs/CMake.html>.
>>>> I have seen LLVM_ENALBLE_LTO=On in some older docs.
>>>> Bring it back?
>>>> AFAICS, "cmake -LA | grep -i $pattern" die not show
me this, but I
>>>> might be wrong.
>>>> ( Some weeks ago, I dealt with LTO/PGO. )
>>>>
>>>
>>> OK, I got a typo, nevertheless LLVM_ENABLE_LTO does not work!
>>>
>>> [ ... ]
>>> -- Configuring done
>>> -- Generating done
>>> CMake Warning:
>>>   Manually-specified variables were not used by the project:
>>>
>>>     LLVM_ENABLE_LTO
>>>
>>>
>>> -- Build files have been written to:
>>> /home/wearefam/src/llvm-toolchain/llvm-build
>>>
>>> Do you need more input?
>>>
>>> Thanks!
>>>
>>
>> I found in "http://llvmweekly.org/issue/110"
>>
>> [ llvm.src/cmake/modules/HandleLLVMOptions.cmake ]
>>
>> option(LLVM_ENABLE_LTO "Enable link-time optimization" OFF)
>> append_if(LLVM_ENABLE_LTO "-flto"
>> CMAKE_CXX_FLAGS
>> CMAKE_C_FLAGS
>> CMAKE_EXE_LINKER_FLAGS
>> CMAKE_SHARED_LINKER_FLAGS)
>>
>> Searching for...
>>
>> $ grep LLVM_ENABLE_LTO llvm/cmake/modules/HandleLLVMOptions.cmake
>> [ OUTPUT EMPTY ]
>>
>> ...gives no output.
>>
>> So, LLVM_ENABLE_LTO is not available for LLVM/Clang v3.8.0?!
>>
>> Hmm, maybe I can backport (or git cherry-pick) SVN r259766.
>>
>> - Sedat -
>>
>> [1] http://llvmweekly.org/issue/110
>> [2] http://reviews.llvm.org/rL259766
>
> Backport-patch and v6 of my build-script attached.
>
With the backported patch I get...

[ build-log ]
...
[164/1813] Linking CXX executable bin/llvm-tblgen
[165/1813] Building CXX object lib/MC/CMakeFiles/LLVMMC.dir/MCAsmInfoELF.cpp.o
[166/1813] Building CXX object lib/MC/CMakeFiles/LLVMMC.dir/MCAsmStreamer.cpp.o
FAILED: : && /opt/llvm/bin/clang++-3.8   -fPIC
-fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter
-Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
-Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor -std=c++11 -fcolor-diagnostics
-ffunction-sections -fdata-sections -flto -O3  -flto
-Wl,-allow-shlib-undefined    -Wl,-O3 -Wl,--gc-sections
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/AsmMatcherEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/AsmWriterEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/AsmWriterInst.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/Attributes.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CallingConvEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeEmitterGen.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeGenDAGPatterns.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeGenInstruction.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeGenMapTable.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeGenRegisters.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeGenSchedule.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeGenTarget.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/DAGISelEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcherEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcherGen.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcherOpt.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcher.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/DFAPacketizerEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/DisassemblerEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/FastISelEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/FixedLenDecoderEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/InstrInfoEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/IntrinsicEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/OptParserEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/PseudoLoweringEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/RegisterInfoEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/SubtargetEmitter.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/TableGen.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/X86DisassemblerTables.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/X86ModRMFilters.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/X86RecognizableInstr.cpp.o
utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CTagsEmitter.cpp.o  -o
bin/llvm-tblgen  lib/libLLVMSupport.a lib/libLLVMTableGen.a
lib/libLLVMSupport.a -lrt -ldl -ltinfo -lpthread -lz -lm
-Wl,-rpath,"\$ORIGIN/../lib" && :
/usr/bin/ld: /opt/llvm-toolchain-3.8.0/bin/../lib/LLVMgold.so: error
loading plugin
/usr/bin/ld: /opt/llvm-toolchain-3.8.0/bin/../lib/LLVMgold.so: error
in plugin cleanup (ignored)
clang-3.8: error: linker command failed with exit code 1 (use -v to
see invocation)
ninja: build stopped: subcommand failed.

Not sure what's going on or missing?

Any help appreciated.
Thanks.

- sed@ -

llvm dev - Mar 2016 - Building with LLVM_PARALLEL_XXX_JOBS

[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS

[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS

[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS