llvm dev - Apr 2016 - [cfe-dev] Fwd: Raising CMake minimum version to 3.4.3

On Wed, Apr 27, 2016 at 10:54 AM Renato Golin via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> On 27 April 2016 at 15:39, Rafael Espíndola <rafael.espindola at
gmail.com>
> wrote:
> > Yes. It is a move to put linux developers in the same position as
> > windows and OS X ones, which is a *very* reasonable thing to do.
>
> Apart from the fact that neither Windows nor OSX users have a choice.
>
> We have discussed this before, Rafael, and I don't think doing it
> again will yield different results unless new evidence is brought to
> light.
>
> Unless there is a feature in a newer CMake that we *must* have, I see
> no need to upgrade the version. In the same way we've been holding on
> C++11/14 functionality because MSVC couldn't cope with anything until
> recently.
>
I don't think this is really the right tradeoff.

The folks working on our build infrastructure have limited time. Making
them cope with designing the usage of CMake in such a way that it
gracefully degrades across many major versions of CMake makes their work
substantially harder, and even if it is theoretically possible to do, it
may become sufficiently annoying or slow to do that we simply don't get
improvements to how we build things.

And I think that the same tradeoff holds for C++11 features. We didn't
*need* any of them, and we actually pushed the Windows platform harder than
all of the others because it was the one holding us back. And I think that
was good because it made the developers substantially more productive. In
this case, it's just the build infrastructure and not the entire codebase,
but I think a similar argument holds. If the functionality in CMake 3.4
makes Chris's job on CMake substantially easier, and it at least seems
reasonable to get access to that version of CMake, I'm very supportive of
the change.

>
> Trying to go too fast on the cool tools can create a patchwork of
> versions and functionality that will make it very hard to debug
> buildbot problems.
>
I'm not really sure how pushing towards newer released and stable versions
will cause *more* of a patchwork of versions than following the distro
installed versions. It seems to involve strictly fewer versions of CMake.
But maybe I'm not understanding your concern here?

>
> LLVM is not a toy any more, we can't just expect that everyone can get
> the same environment we have on our own machines, or we'll fall into
> the "works for me" style of closing bugs that is pervasive of
open
> source projects where only a few people ever commit code to.
>
I don't see anyone on the thread arguing that. I think Chris is trying to
argue instead that this version of CMake is sufficiently widely available
that we should adopt it because the benefits outweigh the costs.

But I do think we need to carefully evaluate the availability outside of
Debian and Ubuntu distros, so I did some more research and found:

- Most major Linux distros have support out of the box in their
"current"
version (OpenSUSE, Fedora, Ubuntu, Debian, Arch), and several in a released
version (Ubuntu, Debian, Arch).
- FreeBSD has support out of the box in the latest port snapshot (as Chris
indicated)
- Homebrew and fink (package managers for OSX) have support out of the box,
and CMake provides binaries for OSX that can be readily installed
- Windows has binary installers available from CMake

There are some OSes where this version is unavailable even in the latest
release. The most notable IMO is CentOS which doesn't have it in any
release. The next most notable is OpenSUSE which only has it in its latest
"tumbleweed" rolling update. I just don't think it is reasonable
for us to
continue to hold back our development tools based on these limitations. I
think we regressed several more distros when we switched our requirement to
GCC 4.7.

But clearly we *do* need a fallback option for folks that are on an OS that
doesn't provide CMake or that can't install packages using the package
manager into the primary OS. I'm totally supportive of that use case.

Here is the thing: the requirements to build and *use* CMake (not install!)
are a strict subset of the requirements to build and *use* LLVM itself. So
I see no way that this will be a limiting factor for folks. If it would
help we could even bundle a copy of the CMake source code at a known good
snapshot in the LLVM project and write a wrapper script to bootstrap it
into a known location and then use it for subsequent builds ... but that
seems unnecessary to me.


I'll go back to my analogy: we require a relatively modern C++ compiler (in
fact, it is past due to update our requirements there....). The reason is
that if you are building a compiler from source, it seems entirely
reasonable to expect you to either use a relatively modern OS environment,
or to have the wherewithal to bootstrap things using one of the numerous
options available. Adding CMake to the potential set of things you need to
bootstrap seems like an extremely small incremental burden compared to that
of getting a modern C++ toolchain. (Although clearly they won't always both
be required in the same situations.)


> Now, back to Chris' point:
>
> Is the ExternalProject the only sane way to build compiler-rt and
> libc++? Because this IS a big point.
>
> Will it allow a way to build Compiler-RT and libc++ for *all*
> supported platforms (as in -DLLVM_TARGETS_TO_BUILD)? Would it be
> possible (even if not nice) to do so with an older version of CMake?
> How worse would it be, if possible?
>
> Until we can answer all these questions, build times and personal
> preferences have no impact on such a big decision.
>
I think removing impedance from the development of the CMake build, or
enabling new functionality such as properly bootstrapped runtime libraries,
are absolutely things to weigh against the cost of using a newer tool.

-Chandler
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160427/79ecfb8b/attachment.html>

On 27 April 2016 at 16:29, Chandler Carruth <chandlerc at google.com>
wrote:
> The folks working on our build infrastructure have limited time. Making
them
> cope with designing the usage of CMake in such a way that it gracefully
> degrades across many major versions of CMake makes their work substantially
> harder, and even if it is theoretically possible to do, it may become
> sufficiently annoying or slow to do that we simply don't get
improvements to
> how we build things.
I agree, this is one of the strong points towards the move. Much more
relevant than both personal preferences.

> I'm not really sure how pushing towards newer released and stable
versions
> will cause *more* of a patchwork of versions than following the distro
> installed versions. It seems to involve strictly fewer versions of CMake.
> But maybe I'm not understanding your concern here?
I may digress a little, so bear with me. But just to be sure, I want
you to know that I don't disagree with the upgrade, I just want to
make the reasons clear.

Today, installing CMake (or any dep) from packages means they were
validated and released by people that did the validation.

Right now, I *need* CMake 3.x and Ninja 1.5.* (because of LINK_POOL),
and I have no access on my ageing boards to install them from
packages. so I have gone to the trouble to download a *stable* copy of
the sources, and built from source.

This, however, adds to the maintenance of adding new boards, or
putting them back online when they fail. Work done at those times is
stressful, because we're out of coverage until I can put them back in
production. It's not a lot of work, but it's a very stressful one. But
I'm willing to consciously pay the price and I do so with great care.

But asking people to build CMake encourages them to build whatever
stable, or even unstable versions of CMake. Not that they can't do it
today, but being this the recommended way means people will think less
about that step and just get whatever is easier. That's where unknown
bugs in unstable versions will catch us, because no one will know it's
an unstable CMake version on a buildbot until a lot of time has been
wasted.

I agree, however, that this is not a very strong argument. But it's
stronger than personal preference. That's why I was asking for the
real reasons.

> - Most major Linux distros have support out of the box in their
"current"
> version (OpenSUSE, Fedora, Ubuntu, Debian, Arch), and several in a released
> version (Ubuntu, Debian, Arch).
Note that this is *current* and mostly x86_64. Life on ARM, MIPS and
probably PPC is not as shiny.

At least on ARM, different boards may have different (old) systems
that they were validated with and will *not* upgrade, at least not
officially. Modulo some serious validation, there's no way I can
upgrade to a current version and expect it to be stable.

> But clearly we *do* need a fallback option for folks that are on an OS that
> doesn't provide CMake or that can't install packages using the
package
> manager into the primary OS. I'm totally supportive of that use case.
Right. And I totally want to support Chris and others to make our
build system better.

I am definitely willing to take the hit and upgrade CMake on *all* my
builders (current and future) to a *stable* CMake release if we can
make it clear what we want.

If it's package based, having a minimum version is necessary, since
some new distros don't have the old one. But it's also ok, because
distros validate their packages and control their back-ports, so we
know any update from them is reliable, no matter how new.

If it has to be compiled, than things are a little more complicated.
We need to make sure people understand that *any* production
environment (buildbot/Jenkins, release testing, etc) needs to use a
stable release. We don't control our validation or release test
environments as tight as we should, anyone can add a bot or be a
tester, so having a loose requirement list makes the binaries less
worth, and we do get bug reports on them due to incompatibilities. We
want to decrease that problem, not increase.

I also want to make the releases a community process, where
stakeholders (like Android, Apple, ARM, Sony, etc) can chime in and
validate in their production environments, so that we know all people
that use LLVM have a good, stable and robust base to build upon. Make
it more flaky will counter those goals.

I don't want us to block people from using experimental CMake
versions, but I think we must do so for the CMake scripts that go
upstream. In a nutshell, we should mark a range, not a minimum.
Preferably, a range includes a CMake that exists in Debian Jessie,
since this is going to be the production release of most ARM and
AArch64 machines for the foreseeable future. But stronger arguments
may trump that, I understand.

> If it would help
> we could even bundle a copy of the CMake source code at a known good
> snapshot in the LLVM project and write a wrapper script to bootstrap it
into
> a known location and then use it for subsequent builds ... but that seems
> unnecessary to me.
No, having a range and *requiring* stable releases should be more than enough.

> I think removing impedance from the development of the CMake build, or
> enabling new functionality such as properly bootstrapped runtime libraries,
> are absolutely things to weigh against the cost of using a newer tool.
I absolutely agree, but so far there were no technical arguments to
support any of that.

All version proposals were hand wavy and based on their existence on
this or that OS. I think we need to get a list of the features we
need, match to the *stable* versions available from source, and make
an informed choice. Just like we did with the compilers versions.

cheers,
--renato

As an actual CMake developer (I'm no Brad King but I try to do my part), I
hugely support the move to a much more modern CMake version.  I do believe
that it will pay dividends in the long run to those who develop on the
build system for LLVM.

> I am definitely willing to take the hit and upgrade CMake on *all* my
> builders (current and future) to a *stable* CMake release if we can
> make it clear what we want.
>
The most *stable* CMake releases will probably be the last patch release in
a minor version cycle.  The versioning uses MAJOR.MINOR.PATCH since the 3.x
series (2.8 was so long lived it followed basically the same pattern but
with 2.8 as the effective MAJOR).  MINOR version increments add features
but PATCH increments add bug fixes only.  So, for instance, currently 3.5.2
is the release version and there should be no new features between 3.5.0
and 3.5.2, only bug fixes.  3.5.1, for instance, has a few known crashes
that necessitated a 3.5.2 patch release.  But 3.4.3 is likely the most
stable since it was the last bug-fix patch release in the 3.4.x series.

Just my 2c.

- Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160427/88269902/attachment.html>

Most of the comments I have are actually very well captured by Chandler's
email, so I'm not going to pile on. I do have a few things to add.

Renato, in your most recent email you comment about differentiating *stable* vs
random versions of CMake built from source. I believe as a community our
recommendation should be that people download CMake sources from
https://CMake.org/download/ instead of git. Then they get the source of a known
stable release. If people choose to pull random git hashes or tags, that is
their decision, but since the minimum version I'm proposing is widely
available I think it will be uncommon.

So let's talk about ExternalProject.

Is it the only way? No. There are actually several approaches that could be
taken here, but let me explain why I want to do it this way.

At a very basic level there are really two high-level approaches to solve the
problem of building runtime libraries multiple times:

(1) Do the "Darwin" thing hand hack building multiple targets from the
same CMake configuration.
(2) Configure the runtimes multiple times, once per target.

I've been working to try and un-do the horrible Darwin approach we have
because I think it has some serious problems, but an alternative solution would
be to extend the Darwin approach to every other platform. If you do this you
don't need ExternalProject, but you do need to do some really dirty hacks.
If you want to understand those hacks, just look at CompilerRTDarwinUtils.cmake
in compiler-rt. That is a bit of evil that I’m responsible for, and I’ll be
paying down that debt for a long time to come.

Apart from this we also have another problem. The build system makes
configuration decisions based on the compiler *at configuration time*. The
problem is you really want to configure compiler-rt (and the other runtimes)
*after* you've already built clang so the runtimes are configured and built
with the clang you just built. The only way to do that is with some mechanism
similar to ExternalProject.

We could roll our own. At this point I'm going to put on my
"non-constructive reality" hat. I'm not going to do that. I
don't mean that from a philosophical stance; I love reinventing me some
wheels. I mean that from a "there is no way my boss is going to let me
waste that much of my time solving an already solved problem" stance. Even
using ExternalProject, this work is largely a side-project of mine, so it is
going to take me a while to untangle it.

Replicating ExternalProject would be a lot of work, and it would be a huge
maintenance burden to the community. As a data point, the one place in LLVM
where we don't use ExternalProject (and we should be) is the cross-compiling
goop, and it is a gigantic mess (see: CrossCompile.cmake and every mention of
LLVM_NATIVE_BUILD in TableGen.cmake). It is filled with bugs and in desperate
need of disentangling. This is another one of my sins that I need to repent.

I feel I should also point out that ExternalProject isn't a silver bullet.
It isn't going to solve all our problems, and since it is one of the more
actively changing parts of CMake we may find ourselves re-visiting this
conversation in a year or two talking about a newer version of CMake.

-Chris

> On Apr 27, 2016, at 9:13 AM, Renato Golin <renato.golin at
linaro.org> wrote:
> 
> On 27 April 2016 at 16:29, Chandler Carruth <chandlerc at google.com>
wrote:
>> The folks working on our build infrastructure have limited time. Making
them
>> cope with designing the usage of CMake in such a way that it gracefully
>> degrades across many major versions of CMake makes their work
substantially
>> harder, and even if it is theoretically possible to do, it may become
>> sufficiently annoying or slow to do that we simply don't get
improvements to
>> how we build things.
> 
> I agree, this is one of the strong points towards the move. Much more
> relevant than both personal preferences.
> 
> 
>> I'm not really sure how pushing towards newer released and stable
versions
>> will cause *more* of a patchwork of versions than following the distro
>> installed versions. It seems to involve strictly fewer versions of
CMake.
>> But maybe I'm not understanding your concern here?
> 
> I may digress a little, so bear with me. But just to be sure, I want
> you to know that I don't disagree with the upgrade, I just want to
> make the reasons clear.
> 
> Today, installing CMake (or any dep) from packages means they were
> validated and released by people that did the validation.
> 
> Right now, I *need* CMake 3.x and Ninja 1.5.* (because of LINK_POOL),
> and I have no access on my ageing boards to install them from
> packages. so I have gone to the trouble to download a *stable* copy of
> the sources, and built from source.
> 
> This, however, adds to the maintenance of adding new boards, or
> putting them back online when they fail. Work done at those times is
> stressful, because we're out of coverage until I can put them back in
> production. It's not a lot of work, but it's a very stressful one.
But
> I'm willing to consciously pay the price and I do so with great care.
> 
> But asking people to build CMake encourages them to build whatever
> stable, or even unstable versions of CMake. Not that they can't do it
> today, but being this the recommended way means people will think less
> about that step and just get whatever is easier. That's where unknown
> bugs in unstable versions will catch us, because no one will know it's
> an unstable CMake version on a buildbot until a lot of time has been
> wasted.
> 
> I agree, however, that this is not a very strong argument. But it's
> stronger than personal preference. That's why I was asking for the
> real reasons.
> 
> 
>> - Most major Linux distros have support out of the box in their
"current"
>> version (OpenSUSE, Fedora, Ubuntu, Debian, Arch), and several in a
released
>> version (Ubuntu, Debian, Arch).
> 
> Note that this is *current* and mostly x86_64. Life on ARM, MIPS and
> probably PPC is not as shiny.
> 
> At least on ARM, different boards may have different (old) systems
> that they were validated with and will *not* upgrade, at least not
> officially. Modulo some serious validation, there's no way I can
> upgrade to a current version and expect it to be stable.
> 
> 
>> But clearly we *do* need a fallback option for folks that are on an OS
that
>> doesn't provide CMake or that can't install packages using the
package
>> manager into the primary OS. I'm totally supportive of that use
case.
> 
> Right. And I totally want to support Chris and others to make our
> build system better.
> 
> I am definitely willing to take the hit and upgrade CMake on *all* my
> builders (current and future) to a *stable* CMake release if we can
> make it clear what we want.
> 
> If it's package based, having a minimum version is necessary, since
> some new distros don't have the old one. But it's also ok, because
> distros validate their packages and control their back-ports, so we
> know any update from them is reliable, no matter how new.
> 
> If it has to be compiled, than things are a little more complicated.
> We need to make sure people understand that *any* production
> environment (buildbot/Jenkins, release testing, etc) needs to use a
> stable release. We don't control our validation or release test
> environments as tight as we should, anyone can add a bot or be a
> tester, so having a loose requirement list makes the binaries less
> worth, and we do get bug reports on them due to incompatibilities. We
> want to decrease that problem, not increase.
> 
> I also want to make the releases a community process, where
> stakeholders (like Android, Apple, ARM, Sony, etc) can chime in and
> validate in their production environments, so that we know all people
> that use LLVM have a good, stable and robust base to build upon. Make
> it more flaky will counter those goals.
> 
> I don't want us to block people from using experimental CMake
> versions, but I think we must do so for the CMake scripts that go
> upstream. In a nutshell, we should mark a range, not a minimum.
> Preferably, a range includes a CMake that exists in Debian Jessie,
> since this is going to be the production release of most ARM and
> AArch64 machines for the foreseeable future. But stronger arguments
> may trump that, I understand.
> 
> 
>> If it would help
>> we could even bundle a copy of the CMake source code at a known good
>> snapshot in the LLVM project and write a wrapper script to bootstrap it
into
>> a known location and then use it for subsequent builds ... but that
seems
>> unnecessary to me.
> 
> No, having a range and *requiring* stable releases should be more than
enough.
> 
> 
>> I think removing impedance from the development of the CMake build, or
>> enabling new functionality such as properly bootstrapped runtime
libraries,
>> are absolutely things to weigh against the cost of using a newer tool.
> 
> I absolutely agree, but so far there were no technical arguments to
> support any of that.
> 
> All version proposals were hand wavy and based on their existence on
> this or that OS. I think we need to get a list of the features we
> need, match to the *stable* versions available from source, and make
> an informed choice. Just like we did with the compilers versions.
> 
> cheers,
> --renato