llvm dev - Nov 2011 - [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

On Tue, Nov 1, 2011 at 12:30 PM, Óscar Fuentes <ofv at wanadoo.es>
wrote:
> greened at obbligato.org (David A. Greene) writes:
>
>> Óscar Fuentes <ofv at wanadoo.es> writes:
>>
>>> A good measure of how fast a set of Makefile are is to run the
build
>>> with all targets up-to-date. Both builds takes a few seconds (3 or
so)
>>> on my Linux quad core box. Whatever improvement can be achieved on
this
>>> seems pretty insignifant.
>>
>> Oh, it's significant.  When I build the Cray compiler with only
non-LLVM
>> stuff changed, the actual compiles (the Cray stuff) finish before the
>> LLVM figures out nothing has changed.  This is a sginificant
>> productivity loss.
>
> How is that? Your compiler builds in less than 3 seconds? (IIRC, the
> numbers are more like 1.5 seconds with a warm cache on a Linux 4core 2.4
> GHz machine) Maybe the LLVM build is starved because of your compiler's
> build.
>
>> The Cray compiler uses a non-recursive make and so
>> gets tons of parallelism the LLVM build simply can't see because
it's
>> recursive.
>
> There is no parallel loss due to recursive calls. The cmake build will
> happily compile files from different libraries/executables as long as
> there are enough threads available and no blocking dependencies. To
> check, simply run `make -j' and you'll see how dozens of compiler
> processes are started. AFAIR the same applies to the `make' build.
>
>>> Furthermore, recursive make is necessary for automatic generation
of
>>> header dependencies, among other things. The makefiles generated by
>>> cmake are "partially" recursive for that reason:
>>
>> Eh?  This is not true.  See for example:
>>
>> http://mad-scientist.net/make/autodep.html
>
> LLVM/Clang uses a lot of generated headers. I think that what that page
> says does not apply to our case.
(Chromium's build system has to deal with several generated headers as
well and uses a variant of the build system described on that page,
without using recursive make as far as I know.)

(Nico)

Nico Weber <thakis at chromium.org> writes:
>>>> Furthermore, recursive make is necessary for automatic
generation of
>>>> header dependencies, among other things. The makefiles
generated by
>>>> cmake are "partially" recursive for that reason:
>>>
>>> Eh?  This is not true.  See for example:
>>>
>>> http://mad-scientist.net/make/autodep.html
>>
>> LLVM/Clang uses a lot of generated headers. I think that what that page
>> says does not apply to our case.
>
> (Chromium's build system has to deal with several generated headers as
> well and uses a variant of the build system described on that page,
> without using recursive make as far as I know.)
Okay, we can get rid of recursive make. However, as pointed out
elsewhere, removing recursive make will not make a difference on the
LLVM build. What David Greene says probably is related to difference
sizes among his compiler and LLVM.

Óscar Fuentes <ofv at wanadoo.es> writes:
> Okay, we can get rid of recursive make. However, as pointed out
> elsewhere, removing recursive make will not make a difference on the
> LLVM build. What David Greene says probably is related to difference
> sizes among his compiler and LLVM.
Ok, whatever.

I'm done arguing experiences.

                             -Dave