llvm dev - Nov 2019 - debug build busts memory

The linking state of a Debug build seems to require a humongous amount of 
memory.  My poor little linux machine with 16G of ram swaps its brains out. 
Waiting for it to finish (if it ever does) is like the old days when you 
submitted your deck of cards and waited until the next day to see the results.

To debug a new backend, is there a way to just get the debug info for the 
Target/Foo directory?

Is there a way to just build "llc"?

Any ideas to speed things up?

Thanks, brian

Hi Bagel,

On Tue, 26 Nov 2019 at 18:15, Bagel via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Is there a way to just build "llc"?
"make llc" should do the trick (though ninja tends to be even faster).
> Any ideas to speed things up?
As you can imagine, this is an issue lots of people hit. Knobs you can
twist to make things better:

1. Use lld, for example with the LLVM_ENABLE_LLD CMake option
(assuming it's installed).
2. Lower the LLVM_PARALLEL_LINK_JOBS CMake option to something your
system can support.
3. The LLVM_USE_SPLIT_DWARF essentially leaves the DWARF info in the
.o files and avoids linking it properly until it's used. This speeds
up linking, at the cost of slower gdb/lldb load, and you need a quite
new version of lldb to debug at all.
4. You can skip building unnecessary targets with
LLVM_TARGETS_TO_BUILD, though that's more significant as a
compile-time saver than link-memory.
5. Even more-so for LLVM_OPTIMIZED_TABLEGEN.
6. There's some option to build .so files instead of .a, which saves
linker memory. But it slows down test runs significantly and is also
annoying for debug so I don't think people tend to use it these days.

I tend to run with the first three on Linux.

Cheers.

Tim.

On Tue, 26 Nov 2019 at 18:27, Tim Northover <t.p.northover at gmail.com>
wrote:
> 2. Lower the LLVM_PARALLEL_LINK_JOBS CMake option to something your
> system can support.
Oh, and this one only works with ninja. So another good reason to
switch if you haven't already.

Cheers.

Tim.

Throw hardware at it.

I build LLVM just fine on my 32GB Ryzen 1700X system.

You don't indicate what your CPU is. At the very least, another 16GB should
not be that expensive these days.  However, if you are serious on doing
LLVM development, consider a modern system with at least 32GB and a
high-thread-count processor. e.g. Ryzen 1700X is cheap these days and
provides 16 threads. Newer AMD processors will be even better, and Intel
has similar offerings.


On Tue, Nov 26, 2019 at 1:15 PM Bagel via llvm-dev <llvm-dev at
lists.llvm.org>
wrote:
> The linking state of a Debug build seems to require a humongous amount of
> memory.  My poor little linux machine with 16G of ram swaps its brains
> out.
> Waiting for it to finish (if it ever does) is like the old days when you
> submitted your deck of cards and waited until the next day to see the
> results.
>
> To debug a new backend, is there a way to just get the debug info for the
> Target/Foo directory?
>
> Is there a way to just build "llc"?
>
> Any ideas to speed things up?
>
> Thanks, brian
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191126/04e8aaae/attachment.html>

On Tue, Nov 26, 2019 at 10:31 AM David Jones via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Throw hardware at it.
>
While I agree that this is good advice with the current state of affairs, I
don't think we should consider the current situation acceptable.

LLVM has a lot of knobs that Tim alluded to, but first time users shouldn't
have to search for them to get a working debug build of LLVM out of the
box. As a community, we need to raise the bar here, and spend some time
making first time build setup easier. I seem to recall that there were
discussions at the dev meeting about simplifying our CMake build, and I
think this can be part of that effort.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191126/a4d625e6/attachment.html>

Hi Brian,
> The linking state of a Debug build seems to require a humongous amount of
> memory.  My poor little linux machine with 16G of ram swaps its brains out.
> Waiting for it to finish (if it ever does) is like the old days when you
> submitted your deck of cards and waited until the next day to see the
results.
This actually something I've experienced myself when trying to build LLVM in
debug mode on a machine with 16G of memory. It really is annoying.
> Any ideas to speed things up?

Beside the obvious remark that a machine with more memory for frequent debug
builds is desirable, let me give some remarks:

1. Usually compiling itself is not the problem but linking is. Using a single
threaded build `ninja -j1` should make the build work but causes really
annoyingly long build times.

2. Depending on your host target using a different linker can really help. The
BFD linker didn't work well for me but replacing it with gold (ELF only) or
lld
really helped. You can use e.g. the gold linker by setting
`LLVM_USE_LINKER=gold` in your CMake configuration.

3. You can specify the number of parallel link jobs explicitly by setting the
CMake option `LLVM_PARALLEL_LINK_JOBS`. Use a higher number of jobs for
compiling and a lower number for linking.

4. This is a general remark and not limited to a debug build: Use ccache! This
drastically improved build times for me. Enable the CMake option
`LLVM_CCACHE_BUILD` and make sure that ccache is installed and the max. cache
size is big enough (check by running `ccache -s`), I personally use 50G. 
> Is there a way to just build "llc"?
Something I've never tested myself: Build e.g. llc in debug mode (`ninja
llc`
in a cmake configuration where the build type is `Debug`), build another
toolchain in non-debug mode and simply replace the non-debug version of llc with
the debug version.
> To debug a new backend, is there a way to just get the debug info for the
> Target/Foo directory?
You could try to modify the step above by building llc for your desired target
only by setting the CMake option LLVM_TARGETS_TO_BUILD="your desired
target"`.
> Thanks, brian
Cheers,
David
> On 26. Nov 2019, at 19:15, Bagel via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> The linking state of a Debug build seems to require a humongous amount of
> memory.  My poor little linux machine with 16G of ram swaps its brains out.
> Waiting for it to finish (if it ever does) is like the old days when you
> submitted your deck of cards and waited until the next day to see the
results.
> 
> To debug a new backend, is there a way to just get the debug info for the
> Target/Foo directory?
> 
> Is there a way to just build "llc"?
> 
> Any ideas to speed things up?
> 
> Thanks, brian
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev