llvm dev - Feb 2021 - Improving llvm-projects build

Hi everyone,

I fairly regularly build LLVM (a forked up-to-date repo containing some
custom LLVM passes). But when the build is large, it takes up almost all of 
my remaining RAM (~4 to 5 gigs). I usually run those builds at nights for it
to complete.

Though I would like the build to not clog up entire RAM and just use some 
portion of it, so that I can also work alongside (and my system won't
freeze). My current CMAKE command is:

```
cmake -G Ninja -DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
    -DLLVM_ENABLE_ASSERTIONS=ON \
    -DCMAKE_C_COMPILER=/usr/bin/clang \
    -DCMAKE_CXX_COMPILER=/usr/bin/clang++ \
    -DCMAKE_BUILD_TYPE=Debug \
    -DLLVM_CCACHE_BUILD=ON \
    -DLLVM_USE_LINKER=gold \
    -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
    ../llvm/
```

I found a SO answer claiming `gold` can improve upon memory usage, but I
found almost no improvement.

Are there ways to restrict memory limit during build? Also, I should
mention that I would like to build a Debug version as it is helpful.

Thank you
-- 
Manas
CSAM Undergraduate | 2022
IIIT-Delhi, India

Hi,

> On Feb 14, 2021, at 13:15, Manas via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi everyone,
> 
> I fairly regularly build LLVM (a forked up-to-date repo containing some
> custom LLVM passes). But when the build is large, it takes up almost all of
> my remaining RAM (~4 to 5 gigs). I usually run those builds at nights for
it
> to complete.
> 
> Though I would like the build to not clog up entire RAM and just use some 
> portion of it, so that I can also work alongside (and my system won't
> freeze). My current CMAKE command is:
> 
> ```
> cmake -G Ninja -DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
>    -DLLVM_ENABLE_ASSERTIONS=ON \
>    -DCMAKE_C_COMPILER=/usr/bin/clang \
>    -DCMAKE_CXX_COMPILER=/usr/bin/clang++ \
>    -DCMAKE_BUILD_TYPE=Debug \
>    -DLLVM_CCACHE_BUILD=ON \
>    -DLLVM_USE_LINKER=gold \
>    -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
>    ../llvm/
> ```
Building llvm with debug info requires a large amount of RAM. See
https://llvm.org/docs/GettingStarted.html#common-problems
<https://llvm.org/docs/GettingStarted.html#common-problems> for some
pointers.

Cheers,
Florian

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210214/07d4513d/attachment.html>

On 14 Feb 2021, at 14:15, Manas via llvm-dev <llvm-dev at lists.llvm.org>
wrote:
> 
> I fairly regularly build LLVM (a forked up-to-date repo containing some
> custom LLVM passes). But when the build is large, it takes up almost all of
> my remaining RAM (~4 to 5 gigs). I usually run those builds at nights for
it
> to complete.
> 
> Though I would like the build to not clog up entire RAM and just use some
> portion of it, so that I can also work alongside (and my system won't
> freeze). My current CMAKE command is:
> 
> ```
> cmake -G Ninja -DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
>    -DLLVM_ENABLE_ASSERTIONS=ON \
>    -DCMAKE_C_COMPILER=/usr/bin/clang \
>    -DCMAKE_CXX_COMPILER=/usr/bin/clang++ \
>    -DCMAKE_BUILD_TYPE=Debug \
>    -DLLVM_CCACHE_BUILD=ON \
>    -DLLVM_USE_LINKER=gold \
>    -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
>    ../llvm/
> ```
> 
> I found a SO answer claiming `gold` can improve upon memory usage, but I
> found almost no improvement.
> 
> Are there ways to restrict memory limit during build? Also, I should
> mention that I would like to build a Debug version as it is helpful.
Try adding:

-D LLVM_PARALLEL_LINK_JOBS=1

to your CMake invocation. This lowers the number of parallel link jobs, which
(at last in my experience) tend to be the largest RAM consumers during a build.

-Dimitry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 223 bytes
Desc: Message signed with OpenPGP
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210214/5efd2c96/attachment.sig>