[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS

Chris Bieneman via llvm-dev
Tue, 01 Mar 2016 17:57:47 -0000

There are a few notes I'd like to add to this thread.

(1) we have a number of places throughout out CMake build where we use features
from newer CMakes gated by version checks. Most of these features are
performance or usability related. None of them are correctness. Using the latest
CMake release will often result in faster builds, so I encourage it.

(2) CMake's "install" target will pretty much always be slower
from clean than the old autoconf/make "install" target. This is
because in CMake "install" depends on "all", and our CMake
builds more stuff in "all" than autoconf did. To help with this or
CMake system has lots of convenient "install-${name}" targets that
support component-based installation. Not every component has one of these
rules, but if one you need is missing let me know. I also recently (r261681)
added a new option (LLVM_DISTRIBUTION_COMPONENTS) that allows you to specify a
list of components that have custom install targets. It then creates a new
"install-distribution" target that wraps just the components you want.
For Apple this is almost a 40% speed up over "ninja install".

-Chris

On Feb 25, 2016, at 11:08 AM, Sedat Dilek via llvm-dev <llvm-dev at
lists.llvm.org> wrote:

>> Which combination of cmake/ninja versions are you using (latest are
>> v3.4.3 and v1.6.0)?
>
> With this combination I could reduce build-time down from approx. 3h
> down to 01h20m.
>
> $ egrep -i 'jobs|ninja' llvm-build/CMakeCache.txt
> //Program used to build from build.ninja files.
> CMAKE_MAKE_PROGRAM:FILEPATH=/opt/cmake/bin/ninja
> //Define the maximum number of concurrent compilation jobs.
> LLVM_PARALLEL_COMPILE_JOBS:STRING=3
> //Define the maximum number of concurrent link jobs.
> LLVM_PARALLEL_LINK_JOBS:STRING=1
> CMAKE_GENERATOR:INTERNAL=Ninja
>
> $ LC_ALL=C ls -alt logs/3.8.0rc3_clang-3-8-0-rc3_cmake-3-4-3_ninja-1-6-0/
> total 360
> drwxr-xr-x 2 wearefam wearefam   4096 Feb 25 19:58 .
> drwxr-xr-x 6 wearefam wearefam   4096 Feb 25 19:58 ..
> -rw-r--r-- 1 wearefam wearefam 130196 Feb 25 19:54
> install-log_llvm-toolchain-3.8.0rc3.txt
> -rw-r--r-- 1 wearefam wearefam 205762 Feb 25 19:51
> build-log_llvm-toolchain-3.8.0rc3.txt
> -rw-r--r-- 1 wearefam wearefam  14331 Feb 25 18:30
> configure-log_llvm-toolchain-3.8.0rc3.txt
>
> $ LC_ALL=C du -s -m llvm* /opt/llvm-toolchain-3.8.0rc3
> 315     llvm
> 941     llvm-build
> 609     /opt/llvm-toolchain-3.8.0rc3
>
> - Sedat -
>
> [1] https://cmake.org/files/v3.5/cmake-3.5.0-rc3-Linux-x86_64.tar.gz
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev