llvm dev - Oct 2011 - [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

I have a very high level comment, and you may be able to directly shed light
on it before I dig into a lot more detail.

Why not simply standardize on CMake? It's not my favorite tool, but it seems
to work well, we have established usage of it, and several people involved
in the project who understand how it works. It doesn't seem like a
significantly more burdensome dependency than Python when developing, and it
remains possible to build installable packages for the casual hacker.

I can see some objections to CMake, but it's not clear to me that they
should carry the day. I'm also probably missing some.

The one I see most clearly is that the CMake build, as it stands, involves
Too Much Magic. I don't at all disagree. That said, I strongly believe this
could be completely addressed.

- If we moved to CMake as the standard build system, numerous kludgy aspects
of the current build would go away. They are often in existence purely to
support interoperation with the old system.

- It would be very straight forward to centralize all of the library
dependencies and descriptions in the single top-level CMakeLists.txt file,
making it easily consumable by your average developer. It would have a
format no harder to edit or understand than the one you propose, and they
would both (at worst) be unfamiliar to existing developers.

- It would likely improve the quality of our CMake builds by ensuring it was
well tested and always in a consistent state.

- It already has a relatively optimized makefile-generation system, so we
wouldn't need to re-invent this wheel again.


The biggest downside to making CMake the standard build system is the
dependence on CMake to my eyes. However, among the cross-platform users of
LLVM, I think CMake is often the preferred build system. I know of folks
using it under xcode, visual studio, mingw, cygwin, and all flavors of
Linux.


Anyways, I'm sure there are more considerations than just these, I just
think it would be beneficial to seriously consider using an existing
meta-build system rather than rolling our own.
-Chandler

On Thu, Oct 27, 2011 at 6:11 PM, Daniel Dunbar <daniel at zuster.org>
wrote:
> Hi all,
>
> As you might have inferred, I'm in the process of working on some
changes
> to the
> way LLVM builds. I have outlined a rough proposal below, unless there are
> any
> major objections I will probably start committing stuff next week.
>
> This message may be verbose, if you want the executive summary, skip
> to 'What This
> Means For Jane "LLVM Developer" Doe' at the bottom.
>
> Motivation
> ----------
>
> We currently maintain two build systems (make, CMake). This has a couple of
> problems:
>
>  * Duplication: the two build systems duplicate a significant amount of
> implicit
>   logic (higher level dependencies), some config files, and other
> miscellaneous
>   features.
>
>  * Maintenance: Some parts of the build systems requires regular
> maintenance
>   (the CMake file lists, CMake dependency lists, module structure). Having
> one
>   more thing to maintain is annoying.
>
>  * Inconsistency: The two build systems behave inconsistently in some ways.
> If
>   we want both to officially supported systems, it would be nice for them
> to
>   behave as identically as possible.
>
>   For example, CMake now uses explicit dependencies which are hard coded
> into
>   the CMake files, but the Makefiles work completely differently.
>
> There are also other general issues with the way LLVM is built now:
>
>  * LLVM has a higher level structure for its organization, but this is not
>   explicit. LLVM is roughly organized into components (libraries, tools,
> etc.)
>   by directory. It would be nice to have this be more explicit.
>
>  * Much of the project build structure is implicit in the Makefiles or
>   CMakeFiles. It is not particularly explicit anywhere that, say, parts of
>   VMCore depend on building the Intrinsics, which depend on building
> tblgen.
>
>  * The Make system is not very efficient. We use recursive Makefiles which
> make
>   the current system (relatively) simple in some ways, but mean the Make
> build
>   is not nearly as scalable as it could be. In particular, the current
>   organization means the built is often serialized on something that is not
> a
>   strict dependency. It also makes it much more likely to do things like a
>   stampeding link of all the tools, even though many tools could have been
>   built earlier.
>
>
> Specific Goals
> --------------
>
>  * Move both build systems to use explicit library dependencies, in a clean
>   fashion. The CMake files do this now, but I don't think it has been
made
>   clear to developers when they are supposed to edit these files, or how
> (other
>   than when something breaks, I guess).
>
>  * Explicitly describe as much of the project structure as necessary to
> support
>   builds. This means explicitly specifying how the project is organized,
> and
>   the dependencies among the components required to build (e.g., Intrinsics
>   before VMCore).
>
>   I believe a number of other projects/users (FreeBSD, fbuild) have
> built there own
>   build system for LLVM. Encoding the project structure clearly should make
> it
>   easier for such projects, or for other future users that want to do the
> same.
>
>   This should make it easier to experiment with the build system, for
> example
>   we could just directly generate good Makefiles for our project, or could
>   experiment with systems like Ninja which expect to be targetted by a
>   generator of some kind.
>
>
> Proposal
> --------
>
> My initial proposal is focused at moving us to use explicit library
> dependencies, but paves the way for centralizing more "build
systemy" stuff
> in
> the future.
>
>  * Every LLVM "component" (roughly corresponds to each place we
have a
> Makefile
>   or CMakeLists.txt currently) will get a 'LLVMBuild.txt' file.
>
>   This file will be an LLVM specific description of that component.
> Initially,
>   it will look something like this::
>
>     [component]
>     # The kind of component this is (currently library, tool, build tool).
> More
>     # types will be defined over time.
>     type = Library
>
>     # The name of the component.
>     name = VMCore
>
>     # The name of the component to logically group this in. This is just
> for
>     # organization purposes.
>     parent = Libraries
>
>     # The names of the library components that are also required when
> linking
>     # with this library. More on this later.
>     required_libraries = Support
>
>   The exact structure of the format is TBD (and easy to change), currently
> the
>   format is INI style but I may decide to change to JSON once all the
> pieces
>   are in place.
>
>   The LLVM web pages will have clear documentation on what these files
> should
>   look like, what is required, what is supported, and so on.
>
>
>  * I will add a new tool in utils/llvm-build which is designed to load and
> work
>   with these files. This tool will be written in Python, and the
> expectation is
>   that it can be run at configure time.
>
>   TO BE CLEAR: I intend to introduce a hard dependency on requiring Python
> in
>   order to build LLVM.
>
>   For the Makefiles, this is no worse than requiring Perl, so I don't
think
>   there is much to argue with.
>
>   For CMake, this is a new dependency. However, I feel this is unavoidable:
>
>     * I see no way to support multiple build systems including CMake
> without
>       either introducing a dependency on some extra tool (which can be
> shared),
>       or duplicating a significant amount of logic in CMake.
>
>       I believe that duplicating logic is worse than adding the Python
>       dependency, and I think we already have more CMake code (a relatively
>       horrible language) than can be expected for most LLVM developers to
> deal
>       with.
>
>   Additionally, we already require Python for running our tests, so anyone
>   doing serious LLVM development should have it.
>
>   The one use case I believe this particularly hurts is users who just want
> to
>   download and play with LLVM, but I believe the Right Answer (tm) for that
>   case would be for us to provide nice installer packages anyway.
>
>
>  * utils/llvm-build will be run during configure time (both for Make and
> CMake),
>   and will:
>
>   * Generate the library dependency information required to link tools, in
>     whatever format makes the most system for the build system in use.
>
>   * Generate a C++ .inc file containing the dependency table for use by
>     llvm-config (which I am going to rewrite in C++).
>
>   * Add dependencies on the LLVMBuild.txt files to the build system, so
> that
>     the build will reconfigure appropriately when the library
> requirements change.
>
>
>  * Remove GenLibDeps.pl, find-cycles.pl, etc.
>
>   We will no longer be using these after llvm-config has moved over.
>
>
>  * Add explicit source file lists to the LLVMBuild.txt files.
> Unfortunately,
>   this is inevitable if we want to support CMake users projects
> automatically
>   reconfiguring themselves when new files get added. I can make it easier
> to
>   manage (for example, provide build targets that will automatically add
> any
>   new files).
>
>
>  * Move both Make and CMake over to using the explicit file lists in the
>   LLVMBuild files. This ensures that both systems get the same behavior
>   (instead of Make users being able to add files and forget to update
>   CMakeLists.txt).
>
>
>  * Add new 'lit' tests to check that the library dependency
> information is correct.
>
>   This seems a nicer place to do the checking which is currently partially
>   handled by find-cycles, and we should also be able to do a better job of
> the
>   checking (for example, verifying that the dependency list is
"exact" --
> only
>   specifies the minimum set of libraries required, and isn't allowed to
> specify
>   libraries which are only indirectly required).
>
>   It would be particularly cool if we could just write these tests using
> our
>   Object libraries.
>
>   This is one piece I haven't prototyped yet. I can obviously do
something
> as
>   good as the current find-cycles.pl, but I hope to do better
> (eventually).
>
>
>  * These are just the first steps, after this I will continue to
> incrementally
>   try and move as much common information out of the Make and CMake systems
> so
>   there is "one source of truth" with regard to the project
definition.
>
>
>  * I assume it is obvious, but when I say "LLVM" here I am
referring to
> both
>   LLVM and Clang. I have not looked at lldb yet, but if it uses the LLVM
> build
>   system (and llvm-config) functionality I will try to make sure it
> continues to work.
>
>
> What This Means For Jane "LLVM Developer" Doe
> ---------------------------------------------
>
> In practice, this means:
>
>  * LLVM requires Python to build.
>
>  * When you add a file to LLVM, you will need to edit LLVMBuild.txt instead
> of
>   CMakeLists.txt, which will be in a slightly different, but otherwise
> totally
>   obvious format.
>
>   If you forget to do this, your file will not be built (which will most
> likely
>   cause a link error eventually). This is better than it being built by
> Make,
>   but causing CMake build failures when you check in.
>
>  * When you add a new library requirement to an existing component, you
> will be
>   required to edit LLVMBuild.txt instead of CMakeLists.txt, which will be
> in a
>   slightly different, but otherwise totally obvious (hopefully) format.
>
>   If you forget to do this, you will either get a link error or a test
>   failure. This is better than library you need transparently getting
> linked in
>   (with make) because it forces you to think about whether you actually
> should
>   be adding that dependency.
>
>   The goal is that this also ensures that if LLVM links and passes tests on
>   your system, then it should for everyone else as well.
>
>  * Developers not actively touching the build system should never need to
> touch
>   a Makefile or a CMake file.
>
> Overall, I believe this should be a quality of life improvement for the
> developer community. The only downside is having to deal with a new
> non-standard
> LLVM specific format, but I plan to solve this through documentation.
>
> Comments?
>
>  - Daniel
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20111027/ac5ff575/attachment.html>

On Oct 27, 2011, at 10:55 PM, Chris Lattner wrote:
> On Oct 27, 2011, at 6:34 PM, Chandler Carruth wrote:
> 
>> I have a very high level comment, and you may be able to directly shed
light on it before I dig into a lot more detail.
>> 
>> Why not simply standardize on CMake? It's not my favorite tool, but
it seems to work well, we have established usage of it, and several people
involved in the project who understand how it works. It doesn't seem like a
significantly more burdensome dependency than Python when developing, and it
remains possible to build installable packages for the casual hacker.
>> 
>> I can see some objections to CMake, but it's not clear to me that
they should carry the day. I'm also probably missing some.
> 
> There are several major problems with CMake IMO:
> 
> 1. It generates really slow build systems.
Outside of Xcode projects (your issue #2), do you have any evidence to support
this claim? We've measured before that the CMake-generated makefiles are
faster than the current makefile system. Anecdotally, CMake's generated
makefiles scaled very nicely to the multi-library monstrosity that is Boost.
(They beat the crap out of Boost.Build, but that's not exactly hard to do).
> 2. The build system generated by cmake is absolute garbage, at least for
Xcode.  The build times of it are really bad, and having to work with it in the
IDE is even more terrible.
CMake's Xcode project generator is quite horrible. Daniel seems to think
that it can't ever be done well, but I disagree: that generator is so crusty
that a couple of hours would likely produce a huge improvement. Alas, I
don't have the spare cycles to prove Daniel wrong.

Regardless, Daniel's proposal doesn't fix this, either. CMake will still
generate crappy Xcode projects with his system.

CMake's Visual Studio and makefile projects work quite well, in my
experience. Why do you think otherwise?
> 3. I'd really like us to get to explicit and principled library
dependencies, where the build horks if you accidentally add a library
dependency.  Without this, I don't see how LLVM will ever scale up.
CMake already has explicit library dependencies, and the CMake build breaks when
they're wrong. So CMake has already made the world a better place :)

By "principled", I assume you mean that all dependencies are spelled
out rather than getting them transitively. This is new functionality that's
fairly easy to add to both CMake and Daniel's tool.
> 4. I'd really like us to move in a direction where LLVM is less
monolithic.  Right now "llvm" contains all the MC stuff, all the
targets, all the mid-level optimizer, etc.  Adding a new LLVM MC-based linker
will cause it to naturally get dropped into the llvm project, which is great but
not helping the monolithicness. :)
CMake can support different project organizations. I'm currently using CMake
with separate projects for Clang and LLVM, and it works just fine. Why do you
think it can't?

So, #3 and #4 aren't CMake issues at all. #1 and #2 seem only to apply to
CMake's Xcode project generator, which is a tiny fraction of what CMake
provides.

	- Doug

+1

 

As have we. We have CMake wrappers and variable injection to make it work with
our internal build system.

 

I also agree with the point that others have made – why reinvent the wheel? You
already have use of a dedicated build system tool – Cmake – that is fit for
purpose and while it may not generate *quite* as good a set of Makefiles as a
hand-crafted solution would, is this really important? Is that 3 seconds of
build time improvement really important?

 

Why not use a well known and respected build system tool like CMake that all
sysadmins know, can hack, and removes the development burden from LLVM?

 

Cheers,

 

James

 

From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
Behalf Of Rotem, Nadav
Sent: 28 October 2011 15:09
To: Chandler Carruth; Daniel Dunbar
Cc: cfe-dev; LLVM Developers Mailing List
Subject: Re: [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

 

+1 

 

We build our OpenCL SDK (for windows and Linux) using CMake.  We’ve  integrated
LLVM’s Cmake hierarchy into our own (customizing LLVM external parameters like
build and install directories, added passes, etc)

 

Migrating LLVM’s build system from CMake to something else would require us to
change the way we currently do things.

 

 

 

From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
Behalf Of Chandler Carruth
Sent: Friday, October 28, 2011 03:35
To: Daniel Dunbar
Cc: cfe-dev; LLVM Developers Mailing List
Subject: Re: [LLVMdev] [cfe-dev] RFC: Upcoming Build System Changes

 

I have a very high level comment, and you may be able to directly shed light on
it before I dig into a lot more detail.

 

Why not simply standardize on CMake? It's not my favorite tool, but it seems
to work well, we have established usage of it, and several people involved in
the project who understand how it works. It doesn't seem like a
significantly more burdensome dependency than Python when developing, and it
remains possible to build installable packages for the casual hacker.

 

I can see some objections to CMake, but it's not clear to me that they
should carry the day. I'm also probably missing some.

 

The one I see most clearly is that the CMake build, as it stands, involves Too
Much Magic. I don't at all disagree. That said, I strongly believe this
could be completely addressed.

 

- If we moved to CMake as the standard build system, numerous kludgy aspects of
the current build would go away. They are often in existence purely to support
interoperation with the old system.

 

- It would be very straight forward to centralize all of the library
dependencies and descriptions in the single top-level CMakeLists.txt file,
making it easily consumable by your average developer. It would have a format no
harder to edit or understand than the one you propose, and they would both (at
worst) be unfamiliar to existing developers.

 

- It would likely improve the quality of our CMake builds by ensuring it was
well tested and always in a consistent state.

 

- It already has a relatively optimized makefile-generation system, so we
wouldn't need to re-invent this wheel again.

 

 

The biggest downside to making CMake the standard build system is the dependence
on CMake to my eyes. However, among the cross-platform users of LLVM, I think
CMake is often the preferred build system. I know of folks using it under xcode,
visual studio, mingw, cygwin, and all flavors of Linux.

 

 

Anyways, I'm sure there are more considerations than just these, I just
think it would be beneficial to seriously consider using an existing meta-build
system rather than rolling our own.

-Chandler

 

On Thu, Oct 27, 2011 at 6:11 PM, Daniel Dunbar <daniel at zuster.org>
wrote:

Hi all,

As you might have inferred, I'm in the process of working on some changes to
the
way LLVM builds. I have outlined a rough proposal below, unless there are any
major objections I will probably start committing stuff next week.

This message may be verbose, if you want the executive summary, skip
to 'What This
Means For Jane "LLVM Developer" Doe' at the bottom.

Motivation
----------

We currently maintain two build systems (make, CMake). This has a couple of
problems:

 * Duplication: the two build systems duplicate a significant amount of implicit
  logic (higher level dependencies), some config files, and other miscellaneous
  features.

 * Maintenance: Some parts of the build systems requires regular maintenance
  (the CMake file lists, CMake dependency lists, module structure). Having one
  more thing to maintain is annoying.

 * Inconsistency: The two build systems behave inconsistently in some ways. If
  we want both to officially supported systems, it would be nice for them to
  behave as identically as possible.

  For example, CMake now uses explicit dependencies which are hard coded into
  the CMake files, but the Makefiles work completely differently.

There are also other general issues with the way LLVM is built now:

 * LLVM has a higher level structure for its organization, but this is not
  explicit. LLVM is roughly organized into components (libraries, tools, etc.)
  by directory. It would be nice to have this be more explicit.

 * Much of the project build structure is implicit in the Makefiles or
  CMakeFiles. It is not particularly explicit anywhere that, say, parts of
  VMCore depend on building the Intrinsics, which depend on building tblgen.

 * The Make system is not very efficient. We use recursive Makefiles which make
  the current system (relatively) simple in some ways, but mean the Make build
  is not nearly as scalable as it could be. In particular, the current
  organization means the built is often serialized on something that is not a
  strict dependency. It also makes it much more likely to do things like a
  stampeding link of all the tools, even though many tools could have been
  built earlier.


Specific Goals
--------------

 * Move both build systems to use explicit library dependencies, in a clean
  fashion. The CMake files do this now, but I don't think it has been made
  clear to developers when they are supposed to edit these files, or how (other
  than when something breaks, I guess).

 * Explicitly describe as much of the project structure as necessary to support
  builds. This means explicitly specifying how the project is organized, and
  the dependencies among the components required to build (e.g., Intrinsics
  before VMCore).

  I believe a number of other projects/users (FreeBSD, fbuild) have
built there own
  build system for LLVM. Encoding the project structure clearly should make it
  easier for such projects, or for other future users that want to do the same.

  This should make it easier to experiment with the build system, for example
  we could just directly generate good Makefiles for our project, or could
  experiment with systems like Ninja which expect to be targetted by a
  generator of some kind.


Proposal
--------

My initial proposal is focused at moving us to use explicit library
dependencies, but paves the way for centralizing more "build systemy"
stuff in
the future.

 * Every LLVM "component" (roughly corresponds to each place we have a
Makefile
  or CMakeLists.txt currently) will get a 'LLVMBuild.txt' file.

  This file will be an LLVM specific description of that component. Initially,
  it will look something like this::

    [component]
    # The kind of component this is (currently library, tool, build tool). More
    # types will be defined over time.
    type = Library

    # The name of the component.
    name = VMCore

    # The name of the component to logically group this in. This is just for
    # organization purposes.
    parent = Libraries

    # The names of the library components that are also required when linking
    # with this library. More on this later.
    required_libraries = Support

  The exact structure of the format is TBD (and easy to change), currently the
  format is INI style but I may decide to change to JSON once all the pieces
  are in place.

  The LLVM web pages will have clear documentation on what these files should
  look like, what is required, what is supported, and so on.


 * I will add a new tool in utils/llvm-build which is designed to load and work
  with these files. This tool will be written in Python, and the expectation is
  that it can be run at configure time.

  TO BE CLEAR: I intend to introduce a hard dependency on requiring Python in
  order to build LLVM.

  For the Makefiles, this is no worse than requiring Perl, so I don't think
  there is much to argue with.

  For CMake, this is a new dependency. However, I feel this is unavoidable:

    * I see no way to support multiple build systems including CMake without
      either introducing a dependency on some extra tool (which can be shared),
      or duplicating a significant amount of logic in CMake.

      I believe that duplicating logic is worse than adding the Python
      dependency, and I think we already have more CMake code (a relatively
      horrible language) than can be expected for most LLVM developers to deal
      with.

  Additionally, we already require Python for running our tests, so anyone
  doing serious LLVM development should have it.

  The one use case I believe this particularly hurts is users who just want to
  download and play with LLVM, but I believe the Right Answer (tm) for that
  case would be for us to provide nice installer packages anyway.


 * utils/llvm-build will be run during configure time (both for Make and CMake),
  and will:

  * Generate the library dependency information required to link tools, in
    whatever format makes the most system for the build system in use.

  * Generate a C++ .inc file containing the dependency table for use by
    llvm-config (which I am going to rewrite in C++).

  * Add dependencies on the LLVMBuild.txt files to the build system, so that
    the build will reconfigure appropriately when the library
requirements change.


 * Remove GenLibDeps.pl, find-cycles.pl, etc.

  We will no longer be using these after llvm-config has moved over.


 * Add explicit source file lists to the LLVMBuild.txt files. Unfortunately,
  this is inevitable if we want to support CMake users projects automatically
  reconfiguring themselves when new files get added. I can make it easier to
  manage (for example, provide build targets that will automatically add any
  new files).


 * Move both Make and CMake over to using the explicit file lists in the
  LLVMBuild files. This ensures that both systems get the same behavior
  (instead of Make users being able to add files and forget to update
  CMakeLists.txt).


 * Add new 'lit' tests to check that the library dependency
information is correct.

  This seems a nicer place to do the checking which is currently partially
  handled by find-cycles, and we should also be able to do a better job of the
  checking (for example, verifying that the dependency list is "exact"
-- only
  specifies the minimum set of libraries required, and isn't allowed to
specify
  libraries which are only indirectly required).

  It would be particularly cool if we could just write these tests using our
  Object libraries.

  This is one piece I haven't prototyped yet. I can obviously do something
as
  good as the current find-cycles.pl, but I hope to do better (eventually).


 * These are just the first steps, after this I will continue to incrementally
  try and move as much common information out of the Make and CMake systems so
  there is "one source of truth" with regard to the project
definition.


 * I assume it is obvious, but when I say "LLVM" here I am referring
to both
  LLVM and Clang. I have not looked at lldb yet, but if it uses the LLVM build
  system (and llvm-config) functionality I will try to make sure it
continues to work.


What This Means For Jane "LLVM Developer" Doe
---------------------------------------------

In practice, this means:

 * LLVM requires Python to build.

 * When you add a file to LLVM, you will need to edit LLVMBuild.txt instead of
  CMakeLists.txt, which will be in a slightly different, but otherwise totally
  obvious format.

  If you forget to do this, your file will not be built (which will most likely
  cause a link error eventually). This is better than it being built by Make,
  but causing CMake build failures when you check in.

 * When you add a new library requirement to an existing component, you will be
  required to edit LLVMBuild.txt instead of CMakeLists.txt, which will be in a
  slightly different, but otherwise totally obvious (hopefully) format.

  If you forget to do this, you will either get a link error or a test
  failure. This is better than library you need transparently getting linked in
  (with make) because it forces you to think about whether you actually should
  be adding that dependency.

  The goal is that this also ensures that if LLVM links and passes tests on
  your system, then it should for everyone else as well.

 * Developers not actively touching the build system should never need to touch
  a Makefile or a CMake file.

Overall, I believe this should be a quality of life improvement for the
developer community. The only downside is having to deal with a new non-standard
LLVM specific format, but I plan to solve this through documentation.

Comments?

 - Daniel
_______________________________________________
cfe-dev mailing list
cfe-dev at cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

 

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20111031/c8f67bfe/attachment.html>

Chris Lattner <clattner at apple.com> writes:
> It seems that all sufficiently large open source projects evolve their
> own meta build systems!
Many do, but that doesn't mean it's right.  I really dislike stuff like
autoconf and CMake where the actual build rules get generated by some
other tool.  IMHO and IME, it's better to keep things in one tool that
can have all of the relevant knowledge.  It's completely possible to
design a build system with autoconf-like capabilities completely in GNU
make, for example.

But that's a rather larger project than what Daniel is proposing.  :)

I do agree that having two build systems is confusing and error-prone.
Thanks Daniel for putting in the work to get everything a bit less
redundant and a bit more consistent!

                        -Dave