llvm dev - Nov 2011 - [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

Óscar Fuentes <ofv at wanadoo.es> writes:
> greened at obbligato.org (David A. Greene) writes:
>
>> Óscar Fuentes <ofv at wanadoo.es> writes:
>>
>>> A good measure of how fast a set of Makefile are is to run the
build
>>> with all targets up-to-date. Both builds takes a few seconds (3 or
so)
>>> on my Linux quad core box. Whatever improvement can be achieved on
this
>>> seems pretty insignifant.
>>
>> Oh, it's significant.  When I build the Cray compiler with only
non-LLVM
>> stuff changed, the actual compiles (the Cray stuff) finish before the
>> LLVM figures out nothing has changed.  This is a sginificant
>> productivity loss.
>
> How is that? Your compiler builds in less than 3 seconds? (IIRC, the
I have never seen an LLVM up-to-date build take only three seconds.  You
must have a much faster machine than I do.
>> The Cray compiler uses a non-recursive make and so
>> gets tons of parallelism the LLVM build simply can't see because
it's
>> recursive.
>
> There is no parallel loss due to recursive calls.
This is absolutely not true.
> The cmake build will happily compile files from different
> libraries/executables as long as there are enough threads available
> and no blocking dependencies. 
But there are blocking dependencies.  They're implicit in the recursive
calls.
> To check, simply run `make -j' and you'll see how dozens of
compiler
> processes are started. AFAIR the same applies to the `make' build.
Yes initially, but then it funnels down to 2-3 processes that everything
else waits for.  GenLibDeps (or whatever that thing is called) is a good
example.
>>> Furthermore, recursive make is necessary for automatic generation
of
>>> header dependencies, among other things. The makefiles generated by
>>> cmake are "partially" recursive for that reason:
>>
>> Eh?  This is not true.  See for example:
>>
>> http://mad-scientist.net/make/autodep.html
>
> LLVM/Clang uses a lot of generated headers. I think that what that page
> says does not apply to our case.
You said header dependencies.  Generated headers are different than
what's presented in that page.  That page covers header dependencies.

That said, we have generated headers in the Cray compiler and we use
non-recursive make just fine.

                             -Dave

greened at obbligato.org (David A. Greene) writes:
>>> Oh, it's significant.  When I build the Cray compiler with only
non-LLVM
>>> stuff changed, the actual compiles (the Cray stuff) finish before
the
>>> LLVM figures out nothing has changed.  This is a sginificant
>>> productivity loss.
>>
>> How is that? Your compiler builds in less than 3 seconds? (IIRC, the
>
> I have never seen an LLVM up-to-date build take only three seconds.  You
> must have a much faster machine than I do.
$ ../llvm/configure && make -j4
...

$ time make -j4
real    0m5.840s
user    0m5.232s
sys     0m1.388s

$ time make -j4
real    0m1.788s
user    0m2.304s
sys     0m0.484s

The first run of `time make -j4' took so much because it updated
LLVMLibDeps.txt, which was unnecessay (a bug?). After that all runs
takes well below 2 seconds. That was LLVM alone, no Clang, although
including it would not increase the build time by much.

Today, my quadcore machine costs less than 400 euros. Any professional
developer on the western countries should have no problem having a much
faster machine.
>>> The Cray compiler uses a non-recursive make and so
>>> gets tons of parallelism the LLVM build simply can't see
because it's
>>> recursive.
>>
>> There is no parallel loss due to recursive calls.
>
> This is absolutely not true.
>
>> The cmake build will happily compile files from different
>> libraries/executables as long as there are enough threads available
>> and no blocking dependencies. 
>
> But there are blocking dependencies.  They're implicit in the recursive
> calls.
>
>> To check, simply run `make -j' and you'll see how dozens of
compiler
>> processes are started. AFAIR the same applies to the `make' build.
>
> Yes initially, but then it funnels down to 2-3 processes that everything
> else waits for.
Just tried it and after two hours had to kill all processes using a
Linux kernel hotkey. It starts with a few processes, because tblgen
needs to be built before proceeding with the rest of libraries, but
after that the number of processes explodes.
> GenLibDeps (or whatever that thing is called) is a good
> example.
That's on purpose. The traditional LLVM build (aka the `make' build)
runs GenLibDeps after building the libraries. That's used for building
the llvm-config script, which in turn is used for determining the
library dependencies for the executables. That's just one
bottleneck. The cmake build used (and uses, AFAIK) an schema that does
not require llvm-config for building the executables, because the
library dependency info is embedded in the build.

Just to be clear: recursive make does not create blocking dependencies.

[snip]

Óscar Fuentes <ofv at wanadoo.es> writes:
> Today, my quadcore machine costs less than 400 euros. Any professional
> developer on the western countries should have no problem having a
> much faster machine.
Processor speed has little to do with this.  An empty recursive make
bottles up due to I/O.  A non-recursive make is more likely to be
processor-bound.

                          -Dave

On 11/01/2011 10:24 PM, Óscar Fuentes wrote:
> greened at obbligato.org (David A. Greene) writes:
>
>>>> Oh, it's significant.  When I build the Cray compiler with
only non-LLVM
>>>> stuff changed, the actual compiles (the Cray stuff) finish
before the
>>>> LLVM figures out nothing has changed.  This is a sginificant
>>>> productivity loss.
>>>
>>> How is that? Your compiler builds in less than 3 seconds? (IIRC,
the
>>
>> I have never seen an LLVM up-to-date build take only three seconds. 
You
>> must have a much faster machine than I do.
>
> $ ../llvm/configure&&  make -j4
> ...
>
> $ time make -j4
> real    0m5.840s
> user    0m5.232s
> sys     0m1.388s
>
> $ time make -j4
> real    0m1.788s
> user    0m2.304s
> sys     0m0.484s
To support Oscar's claims. On a 1 1/2 year old laptop a cmake build that 
contains LLVM, clang and Polly takes less than 3.5 seconds.

$ time make -j4
real	0m3.459s
user	0m3.990s
sys	0m1.450s

I normally just work on one component e.g. opt or Polly. Their build 
times are:

$ time make -j4 opt
real	0m1.246s
user	0m0.720s
sys	0m0.170s

$ time make -j4 LLVMPolly
real	0m0.550s
user	0m0.290s
sys	0m0.130s

Further increasing performance does not seem to be necessary for me and 
I am against it if involves an increase in build system complexity.

 From my point of view, increased performance is definitely not a valid 
argument for changes in the build system.

Cheers
Tobi