llvm dev - Oct 2011 - [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

On Oct 28, 2011, at 12:54 AM, Óscar Fuentes wrote:
> A good measure of how fast a set of Makefile are is to run the build
> with all targets up-to-date. Both builds takes a few seconds (3 or so)
> on my Linux quad core box. Whatever improvement can be achieved on this
> seems pretty insignificant.
There are different overheads in different scenarios.  The makefiles get really
poor utilization out of a 8 or 16-way machine because of implicit
synchronization between different sublibraries.
> Furthermore, recursive make is necessary for automatic generation of
> header dependencies, among other things. The makefiles generated by
> cmake are "partially" recursive for that reason:
> 
>
http://www.cmake.org/Wiki/CMake_FAQ#Why_does_CMake_generate_recursive_Makefiles.3F
I don't understand, what does having one makefile per directory have to do
with header file generation?
>>>> 2. The build system generated by cmake is absolute garbage, at
least
>>>> for Xcode.  The build times of it are really bad, and having to
work
>>>> with it in the IDE is even more terrible.
>>> 
>>> AFAIK there is a Xcode project file on the LLVM source tree.
>> 
>> Nope, there was once but it was removed a long time ago though.
>> 
>>> Are the
>>> LLVM makefiles used by the Xcode project?
>> 
>> No, it is generated by Cmake.
> 
> So, the cmake-generated Xcode file was considered good enough or… ?
No, not at all.  It's awful.
> Anyways, if you wish to avoid duplicating info on both Makefile and
> CMakeLists.txt there is a simple solution: read and parse the Makefile
> from the corresponding CMakeLists.txt. For instance, if you put the
> library dependencies on the Makefile like this:
> 
> LLVMLIBDEPS := foo zoo bar
I don't see how that is any better than what is being proposed.  You're
just moving the complexity from one place to the other, and blocking future
improvements that will build on this.

-Chris

I'm going to briefly summarize my responses here.

On Fri, Oct 28, 2011 at 9:56 AM, Chris Lattner <clattner at apple.com>
wrote:
> On Oct 28, 2011, at 12:54 AM, Óscar Fuentes wrote:
>> Anyways, if you wish to avoid duplicating info on both Makefile and
>> CMakeLists.txt there is a simple solution: read and parse the Makefile
>> from the corresponding CMakeLists.txt. For instance, if you put the
>> library dependencies on the Makefile like this:
>>
>> LLVMLIBDEPS := foo zoo bar
>
> I don't see how that is any better than what is being proposed.
 You're just moving the complexity from one place to the other, and blocking
future improvements that will build on this.
I believe this is one of the key points.

In LLVM, we have a lot of domain specific structure in the project.
*Any* build system is going to have to codify that information.
Codifying it *informally* inside CMake / Make / anything else is no
different than codifying it in a new system, but:
 (a) a clear, explicit system makes it easier to understand exactly
what is supported
 (b) it provides a good place to hang documentation. Users can clearly
differentiate between the domain specific LLVM structure and the CMake
implementation details or the make implementation details.

The *ONLY* good reason to put this information inside CMake instead of
external files is as a way to avoid a Python dependency at build time.
However, the cost of doing so is incredibly high:
 * code to use this information in CMake must be written in CMake, a
horrible horrible language which I in no way whatsoever believe should
be inflicted on any more LLVM developers than necessary.
 * other tools to use this information must parse CMake files. Either
they parse some incomplete subset (which then must be documented,
etc.) or they have to deal more with the (see above) horrible horrible
CMake language.


The only other argument on this thread I consider to have significant
weight is whether we should standardize on CMake. I am strongly
against this:

 * CMake is a very heavy dependency to standardize on. Adding external
files that clearly codify how to build the project allows anyone to
reimplement tools to build the project, if necessary, but that is not
at all true of dealing with CMake.

 * I don't think CMake is good enough. I agree it solves problems, but
I want to use great tools, not ones that work. In particular:
    (a) The Xcode projects are horrible.
    (b) The language is horrible.
    (c) This doesn't solve any other nice problems:
          (i) It doesn't make it easier to play with other build
systems (like Ninja, or gyp).
          (ii) It doesn't make it easier to write tools for LLVM. For
example, my system would make it easy to add tools to operate on all
the source files. For example, to add tools to run style checkers on
all the code, or add static analyzer build steps.

 * many many large projects end up with meta build systems, and they
usually are happy with it, and they usually have good reasons for
doing so.

 * This is largely a separate (but related) topic. If you really want
to seriously argue for it, please move it to a different thread (where
I will continue to strongly oppose it) and try to convince Chris it is
the Right Thing (tm). Otherwise, we are as today maintaining multiple
build systems. Even if we wanted to move to CMake, lots of users have
dependencies on the current Makefiles so we will still have to keep
them working for some time to come.

Overall, I believe that my system will *reduce the work* and *improve
the experience* of the majority of LLVM developers. The cost we are
paying for this gain is (a) my time, and (b) a python dependency. As I
said before, we already have (b) for running tests, which no one who
has complained about this have given credence to. In my opinion,
although this adds a new system, but it is actually reducing
complexity because the system becomes (a) better documented, (b) more
understandable, and (c) more explicit.

 - Daniel
> -Chris
>
>

On Oct 28, 2011, at 10:25 AM, Daniel Dunbar wrote:
> Overall, I believe that my system will *reduce the work* and *improve
> the experience* of the majority of LLVM developers. The cost we are
> paying for this gain is (a) my time, and (b) a python dependency. As I
> said before, we already have (b) for running tests, which no one who
> has complained about this have given credence to. In my opinion,
> although this adds a new system, but it is actually reducing
> complexity because the system becomes (a) better documented, (b) more
> understandable, and (c) more explicit.
FWIW as "the other guy who works on the build system that isn't
cmake"
I'm all for this. I have no problem with a python dependency and ultimately
replacing two build systems with a single well documented and more
understandable one is such a worthy goal that some short term change is worth
it.

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20111028/7e66e030/attachment.html>

On Fri, Oct 28, 2011 at 10:25 AM, Daniel Dunbar <daniel at zuster.org>
wrote:
>  * I don't think CMake is good enough. I agree it solves problems, but
> I want to use great tools, not ones that work. In particular:
>    (c) This doesn't solve any other nice problems:
>          (i) It doesn't make it easier to play with other build
> systems (like Ninja, or gyp).
Speaking of ninja and gyp, gyp is actually a build system generator
like what you're proposing, while ninja is a replacement `make`
intended for use with build system generators. What are the reasons
you want to write a new generator rather than using gyp?

I believe someone's now written a ninja backend for cmake, although
I'm not sure where they put it, so cmake might actually provide an
easier way to play with ninja than your new system. Similarly, cmake's
xcode generator is nominally open source
(http://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmGlobalXCodeGenerator.cxx;h=32eaef837e2d79c286ea7651d1ee3f69eb5f0f6a;hb=HEAD).
How come there's no interest in improving it?

Not that I really want to defend cmake. I hate its language as much as anyone.

Jeffrey

Chris Lattner <clattner at apple.com> writes:
> On Oct 28, 2011, at 12:54 AM, Óscar Fuentes wrote:
>> A good measure of how fast a set of Makefile are is to run the build
>> with all targets up-to-date. Both builds takes a few seconds (3 or so)
>> on my Linux quad core box. Whatever improvement can be achieved on this
>> seems pretty insignificant.
>
> There are different overheads in different scenarios.  The makefiles
> get really poor utilization out of a 8 or 16-way machine because of
> implicit synchronization between different sublibraries.
BTW, adding explicit library dependencies will make the parallel builds
worse, because you know when a dependency is missing (the build fails)
but you don't know when a dependency is superfluous.
>> Furthermore, recursive make is necessary for automatic generation of
>> header dependencies, among other things. The makefiles generated by
>> cmake are "partially" recursive for that reason:
>> 
>>
http://www.cmake.org/Wiki/CMake_FAQ#Why_does_CMake_generate_recursive_Makefiles.3F
>
> I don't understand, what does having one makefile per directory have
> to do with header file generation?
No one makefile per directory, but recursive make. Automatic header
dependencies are usually supported by recursively invoking `make', which
is the slow part. Having one makefile per directory may require
recursive invocation or not, depending on how the build was written.

[snip]
>> Anyways, if you wish to avoid duplicating info on both Makefile and
>> CMakeLists.txt there is a simple solution: read and parse the Makefile
>> from the corresponding CMakeLists.txt. For instance, if you put the
>> library dependencies on the Makefile like this:
>> 
>> LLVMLIBDEPS := foo zoo bar
>
> I don't see how that is any better than what is being proposed.
> You're just moving the complexity from one place to the other,
What I propose requires about 20 lines in the cmake build. And what Dan
proposes will require changes on the cmake build as well, I'm pretty
sure that those changes would be much more intrusive.
> and blocking future improvements that will build on this.
Dan proposes to have a file with the info and processing that file for
generating stuff for cmake and `make' (after making the necessary
changes on both builds.) What I propose is to read that info from cmake
itself. No python required. How is that blocking future improvements?

Daniel Dunbar <daniel at zuster.org> writes:
> I'm going to briefly summarize my responses here.
>
> On Fri, Oct 28, 2011 at 9:56 AM, Chris Lattner <clattner at
apple.com> wrote:
>> On Oct 28, 2011, at 12:54 AM, Óscar Fuentes wrote:
>>> Anyways, if you wish to avoid duplicating info on both Makefile and
>>> CMakeLists.txt there is a simple solution: read and parse the
Makefile
>>> from the corresponding CMakeLists.txt. For instance, if you put the
>>> library dependencies on the Makefile like this:
>>>
>>> LLVMLIBDEPS := foo zoo bar
>>
>> I don't see how that is any better than what is being
>> proposed.  You're just moving the complexity from one place to the
>> other, and blocking future improvements that will build on this.
>
> I believe this is one of the key points.
>
> In LLVM, we have a lot of domain specific structure in the project.
> *Any* build system is going to have to codify that information.
Both builds systems already support that information. So...?
> Codifying it *informally* inside CMake / Make / anything else is no
> different than codifying it in a new system, but:
>  (a) a clear, explicit system makes it easier to understand exactly
> what is supported
You are assuming that your .ini or json (or, why not? xml!) will be more
clear than what we can put on the CMakeLists.txt or Makefile. I strongly
doubt that.

It is true that `make' has difficulties for handling info unless you
abide to its rules. That's not the case of cmake. Put the info in the
Makefile, or in a external file if you wish, and cmake will handle it
without the help of an external script. No Python needed.
>  (b) it provides a good place to hang documentation. Users can clearly
> differentiate between the domain specific LLVM structure and the CMake
> implementation details or the make implementation details.
There are no such "cmake implementation details" on CMakeLists.txt!
Those files pretty much contains the strictly required information for
describing the sub-project. If you find a case where the CMakeLists.txt
contains annoying "implementation details" while your system would
avoid
them, the situation can be improved from within the cmake build.
> The *ONLY* good reason to put this information inside CMake instead of
> external files is as a way to avoid a Python dependency at build time.
> However, the cost of doing so is incredibly high:
>  * code to use this information in CMake must be written in CMake, a
> horrible horrible language which I in no way whatsoever believe should
> be inflicted on any more LLVM developers than necessary.
So you can't stand the cmake language and love Python. Fair enough. It
is true that the language is far from pretty. Anyways, what I propose
will require less code in cmake (and more simple) than your Python
scripts would need... for interfacing with cmake!
>  * other tools to use this information must parse CMake files. Either
> they parse some incomplete subset (which then must be documented,
> etc.) or they have to deal more with the (see above) horrible horrible
> CMake language.
Please describe the requirements of any of those "other tools".
Appeals
to mythical unspecified entities does not help the discussion.
> The only other argument on this thread I consider to have significant
> weight is whether we should standardize on CMake. I am strongly
> against this:
>
>  * CMake is a very heavy dependency to standardize on. Adding external
> files that clearly codify how to build the project allows anyone to
> reimplement tools to build the project, if necessary, but that is not
> at all true of dealing with CMake.
Again, mythical entities.
>  * I don't think CMake is good enough. I agree it solves problems, but
> I want to use great tools, not ones that work. In particular:
>     (a) The Xcode projects are horrible.
Ok.
>     (b) The language is horrible.
Not pretty, yes, but every time I needed to implement something on it,
the work was done in a surprisingly small amount of time.
>     (c) This doesn't solve any other nice problems:
>           (i) It doesn't make it easier to play with other build
> systems (like Ninja, or gyp).
Do we need them?
>           (ii) It doesn't make it easier to write tools for LLVM. For
> example, my system would make it easy to add tools to operate on all
> the source files. For example, to add tools to run style checkers on
> all the code, or add static analyzer build steps.
Adding support for your hypothetical case is a few lines of cmake
code. And even implemented on a generic way if you wish, for the case
where you want to run an arbitrary command on every file. Although for
your case a one-line bash script will do too.

That's pretty weak.
>  * many many large projects end up with meta build systems, and they
> usually are happy with it, and they usually have good reasons for
> doing so.
And have we good reasons? CMake is not `make'. You can write complex
logic on it. Actually, the LLVM CMake build is the LLVM meta-build
system. Oh, wait, you don't like the CMake language. So, why not file
feature requests and let others do the work? That's what I do when I
need a feature on LLVM ;-)
>  * This is largely a separate (but related) topic. If you really want
> to seriously argue for it, please move it to a different thread (where
> I will continue to strongly oppose it) and try to convince Chris it is
> the Right Thing (tm). Otherwise, we are as today maintaining multiple
> build systems. Even if we wanted to move to CMake, lots of users have
> dependencies on the current Makefiles so we will still have to keep
> them working for some time to come.
>
> Overall, I believe that my system will *reduce the work* and *improve
> the experience* of the majority of LLVM developers.
I strongly disagree.
> The cost we are paying for this gain is (a) my time,
No. It's the time of anybody who wants to work or maintain the build
system(s). For years I maintained the cmake build knowing only
cmake. With your proposal I would need to know your meta-system, be
fluent on Python, know the Makefile build, the cmake build and how
everything fits together.
> and (b) a python
> dependency. As I said before, we already have (b) for running tests,
> which no one who has complained about this have given credence to. In
> my opinion, although this adds a new system, but it is actually
> reducing complexity because the system becomes (a) better documented,
> (b) more understandable, and (c) more explicit.
Strong disagreement on this too.

On Fri, Oct 28, 2011 at 12:43 PM, Óscar Fuentes <ofv at wanadoo.es>
wrote:
> Chris Lattner <clattner at apple.com> writes:
>
>> On Oct 28, 2011, at 12:54 AM, Óscar Fuentes wrote:
>>> A good measure of how fast a set of Makefile are is to run the
build
>>> with all targets up-to-date. Both builds takes a few seconds (3 or
so)
>>> on my Linux quad core box. Whatever improvement can be achieved on
this
>>> seems pretty insignificant.
>>
>> There are different overheads in different scenarios.  The makefiles
>> get really poor utilization out of a 8 or 16-way machine because of
>> implicit synchronization between different sublibraries.
>
> BTW, adding explicit library dependencies will make the parallel builds
> worse, because you know when a dependency is missing (the build fails)
> but you don't know when a dependency is superfluous.
>
>>> Furthermore, recursive make is necessary for automatic generation
of
>>> header dependencies, among other things. The makefiles generated by
>>> cmake are "partially" recursive for that reason:
>>>
>>>
http://www.cmake.org/Wiki/CMake_FAQ#Why_does_CMake_generate_recursive_Makefiles.3F
>>
>> I don't understand, what does having one makefile per directory
have
>> to do with header file generation?
>
> No one makefile per directory, but recursive make. Automatic header
> dependencies are usually supported by recursively invoking `make',
which
> is the slow part. Having one makefile per directory may require
> recursive invocation or not, depending on how the build was written.
>
> [snip]
>
>>> Anyways, if you wish to avoid duplicating info on both Makefile and
>>> CMakeLists.txt there is a simple solution: read and parse the
Makefile
>>> from the corresponding CMakeLists.txt. For instance, if you put the
>>> library dependencies on the Makefile like this:
>>>
>>> LLVMLIBDEPS := foo zoo bar
>>
>> I don't see how that is any better than what is being proposed.
>> You're just moving the complexity from one place to the other,
>
> What I propose requires about 20 lines in the cmake build. And what Dan
> proposes will require changes on the cmake build as well, I'm pretty
> sure that those changes would be much more intrusive.
>
>> and blocking future improvements that will build on this.
>
> Dan proposes to have a file with the info and processing that file for
> generating stuff for cmake and `make' (after making the necessary
> changes on both builds.) What I propose is to read that info from cmake
> itself. No python required. How is that blocking future improvements?
FWIW, I'm fine with reading that information from CMake, but I imagine
that over time as more stuff moves into shared files the burden of
implementing the CMake logic to deal with that data is just not worth
it (as opposed to leveraging a shared tool).

 - Daniel
>

> BTW, adding explicit library dependencies will make the parallel builds
> worse, because you know when a dependency is missing (the build fails)
> but you don't know when a dependency is superfluous.
Computing the dependencies creates a point in the build that depends on 
all libraries and all tools depend on it. The explicit dependencies 
would have to be in a really bad state to be this bad.

Cheers,
Rafael

On Oct 28, 2011, at 2:16 PM, Óscar Fuentes wrote:
> Eric Christopher <echristo at apple.com>
> writes:
> 
>> FWIW as "the other guy who works on the build system that
isn't cmake"
>> I'm all for this. I have no problem with a python dependency and
ultimately
>> replacing two build systems with a single well documented and more
>> understandable one is such a worthy goal that some short term change is
worth it.
> 
> He is not replacing anything. He is *adding* one layer on top of both.
Yes I know. And it's a layer underneath. By build I largely meant
dependencies
etc. My poor choice of words.

-eric

Óscar Fuentes <ofv at wanadoo.es> writes:
> Chris Lattner <clattner at apple.com> writes:
>
>> There are different overheads in different scenarios.  The makefiles
>> get really poor utilization out of a 8 or 16-way machine because of
>> implicit synchronization between different sublibraries.
>
> BTW, adding explicit library dependencies will make the parallel builds
> worse, 
No they won't, for exactly the reason Chris stated.
> because you know when a dependency is missing (the build fails)
> but you don't know when a dependency is superfluous.
That's true but is not a show-stopper IME.  It's easier to remove
superfluous dependencies than try to track down a missing one in a
recursive make build.

                        -Dave