llvm dev - Nov 2011 - [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

Nico Weber <thakis at chromium.org> writes:
> On Tue, Nov 1, 2011 at 3:09 PM, David A. Greene <greened at
obbligato.org> wrote:
>> Óscar Fuentes <ofv at wanadoo.es> writes:
>>
>>> Okay, we can get rid of recursive make. However, as pointed out
>>> elsewhere, removing recursive make will not make a difference on
the
>>> LLVM build. What David Greene says probably is related to
difference
>>> sizes among his compiler and LLVM.
>
> I doubt that's true: Chromium's empty build is faster than
llvm's
> (only tried the make build, not the cmake build), and chromium's code
> base is considerably larger.
Here's the dirty little secret.  At Cray we incorporated LLVM into our
build system.  We don't use the autoconf+make stuff at all for our
compiler build.  We do use the "official" LLVM way to build the LLVM
tree so we can use the tools (opt, llc, etc.).

And guess what?  A non-recursive build of all the Cray components (and
it's a huge source base, at least as large as LLVM and probably larger)
+ most of LLVM (we don't build all targets) is much faster than a build
of just the LLVM tree using the "official" recursive make way.  I
believe that to be true both for an empty build AND a full rebuild,
though I would have to gather hard data to be sure about the latter.

Autoconf is another matter.  Autoconf is horribly slow.  But we are not
getting rid of autoconf, unfortunately.  :)

                              -Dave

On Tue, Nov 01, 2011 at 05:23:02PM -0500, David A. Greene
wrote:
> 
> Here's the dirty little secret.  At Cray we incorporated LLVM into our
> build system.  We don't use the autoconf+make stuff at all for our
> compiler build.  We do use the "official" LLVM way to build the
LLVM
> tree so we can use the tools (opt, llc, etc.).
So I am maintaing Makefiles for NetBSD for LLVM as well. I know the
FreeBSD guys do the same for them. I use recursive Makefiles. I don't
use shared libraries. Guess what, the only special handling is for the
tblgen build and in total, there are six serialisation points. It's
quite hard to improve on that even with perfect fine grained
dependencies.
> And guess what?  A non-recursive build of all the Cray components (and
> it's a huge source base, at least as large as LLVM and probably larger)
> + most of LLVM (we don't build all targets) is much faster than a build
> of just the LLVM tree using the "official" recursive make way.  I
> believe that to be true both for an empty build AND a full rebuild,
> though I would have to gather hard data to be sure about the latter.
It still doesn't say much. In many ways, GNU make is horrible. It is
quite hard to write a good build system for it where the individual
Makefiles are anywhere near the compactness of the CMakeLists.txt files.
It's hard to control concurrency and still retain the compact
Makefiles. It doesn't invalidate the approach as long as the number of
targets per Makefile and the number of directories is reasonable small.
Both is true for LLVM.

Joerg

On Wed, Nov 02, 2011 at 12:29:33AM +0100, Óscar Fuentes
wrote:
> Joerg Sonnenberger
> <joerg at britannica.bec.de> writes:
> 
> > So I am maintaing Makefiles for NetBSD for LLVM as well. I know the
> > FreeBSD guys do the same for them.
> 
> Maybe it is not useful for the FreeBSD guys, but the makefiles generated
> by cmake are compatible with BSD `make' (unless they changed on the
last
> year.)
Sure, it works. It just doesn't integrated well with the rest of the
build system and that's a bigger concern. For what it is worth, it is
quite easy to maintain an alternative build system based on the
CMakeLists.txt files...

Joerg