llvm dev - Sep 2016 - (Thin)LTO llvm build

So, when I embark on the next ThinLTO try build, probably this Sunday,
should I append -Wl,-plugin-opt,jobs=NUM_PHYS_CORES to LDFLAGS
and run ninja without -j or -jNUM_PHYS_CORES?

> On Sep 17, 2016, at 3:19 PM, Carsten Mattner <carstenmattner at
gmail.com> wrote:
> 
> So, when I embark on the next ThinLTO try build, probably this Sunday,
> should I append -Wl,-plugin-opt,jobs=NUM_PHYS_CORES to LDFLAGS
> and run ninja without -j or -jNUM_PHYS_CORES?

ThinLTO is fairly lean on memory: It should not consume more memory per thread
than if you launch the same number of clang process in parallel to process C++
files.

For example when linking the clang binary itself, without debug info it consumes
0.6GB with 8 threads, 0.9GB with 16 threads, and 1.4GB with 32 threads.
With full debug info, we still have room for improvement, right now it consumes
2.3GB with 8 threads, 3.5GB with 16 threads, and 6.5GB with 32 threads.

So I believe that configuring with -DDLLVM_PARALLEL_LINK_JOBS=1 should be enough
without other constrains, but your mileage may vary.


— 
Mehdi

On Sun, Sep 18, 2016 at 12:32 AM, Mehdi Amini <mehdi.amini at apple.com>
wrote:
>
>> On Sep 17, 2016, at 3:19 PM, Carsten Mattner <carstenmattner at
gmail.com> wrote:
>>
>> So, when I embark on the next ThinLTO try build, probably this Sunday,
>> should I append -Wl,-plugin-opt,jobs=NUM_PHYS_CORES to LDFLAGS
>> and run ninja without -j or -jNUM_PHYS_CORES?
>
>
> ThinLTO is fairly lean on memory: It should not consume more memory per
thread than if you launch the same number of clang process in parallel to
process C++ files.
>
> For example when linking the clang binary itself, without debug info it
consumes 0.6GB with 8 threads, 0.9GB with 16 threads, and 1.4GB with 32 threads.
> With full debug info, we still have room for improvement, right now it
consumes 2.3GB with 8 threads, 3.5GB with 16 threads, and 6.5GB with 32 threads.
>
> So I believe that configuring with -DDLLVM_PARALLEL_LINK_JOBS=1 should be
enough without other constrains, but your mileage may vary.
Sure, I'll try that to not introduce too many variables into the
configure changes,
though I have to ask if using lld would make it possible to have a common -Wl
that works across platforms, being able to ignore if it's binutils.

If I really wanted to pass that to cmake, overriding LDFLAGS would work, right?