llvm dev - Nov 2011 - [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

On Oct 27, 2011, at 11:28 PM, Óscar Fuentes wrote:
> Chris Lattner <clattner at apple.com> writes:
> 
>> There are several major problems with CMake IMO:
>> 
>> 1. It generates really slow build systems.
> 
> In my Linux box, last time I checked (long time ago) the cmake build was
> a bit faster than the Makefiles. But this is a tricky terrain, because
> they are not identical, not on the features supported (and some have an
> impact on build-time) nor even on the options passed to the compiler.
The makefiles are known to be really slow, among other problems being based on
recursive make.  One goal of this is to get a non-recursive makefile generated. 
We've prototyped this in the past and found it to be substantially faster
than the recursive makefile.
>> 2. The build system generated by cmake is absolute garbage, at least
>> for Xcode.  The build times of it are really bad, and having to work
>> with it in the IDE is even more terrible.
> 
> AFAIK there is a Xcode project file on the LLVM source tree.
Nope, there was once but it was removed a long time ago though.
> Are the
> LLVM makefiles used by the Xcode project?
No, it is generated by Cmake.
> If the Xcode project files
> generated by cmake is not satisfactory, can't they use the Makefiles
> generated by cmake instead?
Xcode can drive a makefile, but it doesn't provide any of the IDE
integration features, e.g. clang code completion.

-Chris

On Fri, Oct 28, 2011 at 3:54 AM, Óscar Fuentes <ofv at wanadoo.es> wrote:
> Anyways, if you wish to avoid duplicating info on both Makefile and
> CMakeLists.txt there is a simple solution: read and parse the Makefile
> from the corresponding CMakeLists.txt. For instance, if you put the
> library dependencies on the Makefile like this:
>
> LLVMLIBDEPS := foo zoo bar
>
> obtaining that info from the CMakeLists.txt and generating the cmake
> library dependencies is very simple, nor even you have to put anything
> new on all those CMakeLists.txt, just modify one of the macros that are
> (indirectly) called from each CMakeLists.txt.
>
>
While eliminating duplication is one of the goals I see in this build system
change, I think the more important ones are a) simplifying the build files
and b) making the build faster.

Adding CMake code (I agree it's a terrible scripting language) to parse
Makefiles will make the build slower and more complicated.

-----

I'm not really a stakeholder, but I agree with Chandler, it's worth
investigating simplifying LLVM's CMake files before making Yet Another build
system generator if you haven't already.  That may not solve the problems
associated with the bad XCode project generation, though.

OTOH, LLVM seems to have invented many of it's own tools (lit, tablegen,
filecheck) and they seem to have worked out great.  I wish our test suite
for DynamoRIO had a tool as flexible as FileCheck.  :(

Reid
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20111028/f1a6039c/attachment.html>

On Oct 28, 2011, at 12:54 AM, Óscar Fuentes wrote:
> A good measure of how fast a set of Makefile are is to run the build
> with all targets up-to-date. Both builds takes a few seconds (3 or so)
> on my Linux quad core box. Whatever improvement can be achieved on this
> seems pretty insignificant.
There are different overheads in different scenarios.  The makefiles get really
poor utilization out of a 8 or 16-way machine because of implicit
synchronization between different sublibraries.
> Furthermore, recursive make is necessary for automatic generation of
> header dependencies, among other things. The makefiles generated by
> cmake are "partially" recursive for that reason:
> 
>
http://www.cmake.org/Wiki/CMake_FAQ#Why_does_CMake_generate_recursive_Makefiles.3F
I don't understand, what does having one makefile per directory have to do
with header file generation?
>>>> 2. The build system generated by cmake is absolute garbage, at
least
>>>> for Xcode.  The build times of it are really bad, and having to
work
>>>> with it in the IDE is even more terrible.
>>> 
>>> AFAIK there is a Xcode project file on the LLVM source tree.
>> 
>> Nope, there was once but it was removed a long time ago though.
>> 
>>> Are the
>>> LLVM makefiles used by the Xcode project?
>> 
>> No, it is generated by Cmake.
> 
> So, the cmake-generated Xcode file was considered good enough or… ?
No, not at all.  It's awful.
> Anyways, if you wish to avoid duplicating info on both Makefile and
> CMakeLists.txt there is a simple solution: read and parse the Makefile
> from the corresponding CMakeLists.txt. For instance, if you put the
> library dependencies on the Makefile like this:
> 
> LLVMLIBDEPS := foo zoo bar
I don't see how that is any better than what is being proposed.  You're
just moving the complexity from one place to the other, and blocking future
improvements that will build on this.

-Chris

Óscar Fuentes <ofv at wanadoo.es> writes:
> A good measure of how fast a set of Makefile are is to run the build
> with all targets up-to-date. Both builds takes a few seconds (3 or so)
> on my Linux quad core box. Whatever improvement can be achieved on this
> seems pretty insignifant.
Oh, it's significant.  When I build the Cray compiler with only non-LLVM
stuff changed, the actual compiles (the Cray stuff) finish before the
LLVM figures out nothing has changed.  This is a sginificant
productivity loss.  The Cray compiler uses a non-recursive make and so
gets tons of parallelism the LLVM build simply can't see because it's
recursive.
> Furthermore, recursive make is necessary for automatic generation of
> header dependencies, among other things. The makefiles generated by
> cmake are "partially" recursive for that reason:
Eh?  This is not true.  See for example:

http://mad-scientist.net/make/autodep.html

                           -Dave