llvm dev - Feb 2017 - System hangs during last stages of LLVM build | Tips on speeding it up ?

Hello,

My system hangs every time during last stages of building LLVM ( starting
at 95% in a CMake build ) using CMake or Ninja, sometime close to the
linking of llvm-dysmutil. Could you please suggest tips that could speed up
the compilation ?

Thank You,
Sanjay
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170223/e5345596/attachment.html>

Try to compile with "-j1". You likely do not have enough RAM and the
system might be swapping.

Best,
Tobias

On Thu, Feb 23, 2017, at 06:21 PM, SANJAY SRIVALLABH SINGAPURAM via
llvm-dev wrote:
> Hello,
> 
> My system hangs every time during last stages of building LLVM ( starting
> at 95% in a CMake build ) using CMake or Ninja, sometime close to the
> linking of llvm-dysmutil. Could you please suggest tips that could speed
> up
> the compilation ?
> 
> Thank You,
> Sanjay
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi Sanjay,

On 23 February 2017 at 09:21, SANJAY SRIVALLABH SINGAPURAM via
llvm-dev <llvm-dev at lists.llvm.org> wrote:
> My system hangs every time during last stages of building LLVM ( starting
at
> 95% in a CMake build ) using CMake or Ninja, sometime close to the linking
> of llvm-dysmutil. Could you please suggest tips that could speed up the
> compilation ?
This is probably because multiple linker processes are using all your
memory and bogging down in swap. There are various things you can do
to improve performance:

  * There's an LLVM_PARALLEL_LINK_JOBS option which you can use to
reduce the number of ld instances running at once (you'll need to be
using Ninja rather than Make for this to work, but that's faster
anyway). This is a slightly more fine-grained approach than Tobias's
"-j 1"; systems can usually support more compile jobs than link.
 * If on Linux, you can switch to ld.gold (or lld if you're feeling
brave!), which is much faster than the default ld.bfd
 * Also on ELF platforms, there's LLVM_USE_SPLIT_DWARF which separates
the debug info into a different file. This really speeds things up,
but you need a recent(ish) gdb to be able to debug the result.

Those are the obvious ways to reduce memory. You might also consider
only building the targets you need, which will speed things up but not
get around swapping usually.

Cheers.

Tim.

On Thu, Feb 23, 2017 at 09:45:07AM -0800, Tim Northover via llvm-dev
wrote:
> Hi Sanjay,
> 
> On 23 February 2017 at 09:21, SANJAY SRIVALLABH SINGAPURAM via
> llvm-dev <llvm-dev at lists.llvm.org> wrote:
> > My system hangs every time during last stages of building LLVM (
starting at
> > 95% in a CMake build ) using CMake or Ninja, sometime close to the
linking
> > of llvm-dysmutil. Could you please suggest tips that could speed up
the
> > compilation ?
> 
> This is probably because multiple linker processes are using all your
> memory and bogging down in swap. There are various things you can do
> to improve performance:
> 
>   * There's an LLVM_PARALLEL_LINK_JOBS option which you can use to
> reduce the number of ld instances running at once (you'll need to be
> using Ninja rather than Make for this to work, but that's faster
> anyway). This is a slightly more fine-grained approach than Tobias's
> "-j 1"; systems can usually support more compile jobs than link.
>  * If on Linux, you can switch to ld.gold (or lld if you're feeling
> brave!), which is much faster than the default ld.bfd
>  * Also on ELF platforms, there's LLVM_USE_SPLIT_DWARF which separates
> the debug info into a different file. This really speeds things up,
> but you need a recent(ish) gdb to be able to debug the result.
> 
> Those are the obvious ways to reduce memory. You might also consider
> only building the targets you need, which will speed things up but not
> get around swapping usually.
And if you can afford it, passing -DBUILD_SHARED_LIBS=ON to cmake makes
the last step blazing fast.

I did try make -j 1. But, the build was slower and yes, there was a lot of
swapping.

On Thu, Feb 23, 2017 at 11:12 PM Tobias Grosser via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Try to compile with "-j1". You likely do not have enough RAM and
the
> system might be swapping.
>
> Best,
> Tobias
>
> On Thu, Feb 23, 2017, at 06:21 PM, SANJAY SRIVALLABH SINGAPURAM via
> llvm-dev wrote:
> > Hello,
> >
> > My system hangs every time during last stages of building LLVM (
starting
> > at 95% in a CMake build ) using CMake or Ninja, sometime close to the
> > linking of llvm-dysmutil. Could you please suggest tips that could
speed
> > up
> > the compilation ?
> >
> > Thank You,
> > Sanjay
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170223/b57f6396/attachment.html>

Hello Tim,

Thanks for the tips !

On Thu, Feb 23, 2017 at 11:15 PM Tim Northover <t.p.northover at
gmail.com>
wrote:
> Hi Sanjay,
>
> On 23 February 2017 at 09:21, SANJAY SRIVALLABH SINGAPURAM via
> llvm-dev <llvm-dev at lists.llvm.org> wrote:
> > My system hangs every time during last stages of building LLVM (
> starting at
> > 95% in a CMake build ) using CMake or Ninja, sometime close to the
> linking
> > of llvm-dysmutil. Could you please suggest tips that could speed up
the
> > compilation ?
>
> This is probably because multiple linker processes are using all your
> memory and bogging down in swap. There are various things you can do
> to improve performance:
>
>   * There's an LLVM_PARALLEL_LINK_JOBS option which you can use to
> reduce the number of ld instances running at once (you'll need to be
> using Ninja rather than Make for this to work, but that's faster
> anyway). This is a slightly more fine-grained approach than Tobias's
> "-j 1"; systems can usually support more compile jobs than link.
>  * If on Linux, you can switch to ld.gold (or lld if you're feeling
> brave!), which is much faster than the default ld.bfd
>
Would complications arise because of a *cyclic dependency*, using lld to
build (a newer) lld, and possibly changing the lld binary while it's being
used to link other files ?
>  * Also on ELF platforms, there's LLVM_USE_SPLIT_DWARF which separates
> the debug info into a different file. This really speeds things up,
> but you need a recent(ish) gdb to be able to debug the result.
>
> Those are the obvious ways to reduce memory. You might also consider
> only building the targets you need, which will speed things up but not
> get around swapping usually.
>
This is my experience after targeting only X86 and NVPTX !! 😆
>
> Cheers.
>
> Tim.
>
Thank You,
Sanjay Srivallabh
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170223/e1c80446/attachment.html>