llvm dev - Nov 2019 - RFC: Updating to CMake 3.15.0

> On Oct 30, 2019, at 12:02 PM, Andrew Kelley via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> Here's an argument for why to *lower* the minimum supported CMake
version:
No offense, but that isn't going to happen.
> 
> LLVM is a compiler backend.
LLVM is *much* more than a compiler backend. Using LLVM as a compiler backend is
one use of LLVM. It is also a JIT infrastructure, an object file toolchain, the
underlying bits of a linker and a debugger. It is widely used in a lot of
situations.
> 
> LLVM is already the most difficult dependency to provide for most
> projects that use it as a compiler backend. This is certainly true for
> Zig, for example. This makes the requirements of building LLVM a
> bottleneck in the bootstrapping process.
> 
> Any bump in minimum required CMake version increases the requirements of
> the system to bootstrap a project which depends on LLVM.
Except that modern CMake's build requirements are a subset of LLVMs. Meaning
if you have everything you need to build LLVM except CMake, you have everything
you need to build a modern CMake.
> 
> For programming languages, a more minimal bootstrapping process is a
> feature.
> 
> Therefore, bumping the minimum required CMake version of LLVM *removes a
> feature* from any languages that depend on LLVM. This is O(N) cost where
> N is the number of projects that depend on LLVM.
> 
> Zig has cmake_minimum_required(VERSION 2.8.5). LLVM's higher CMake
> requirement is already the most demanding CMake version of all Zig's
> dependencies. Making it any higher than it already is is strictly worse.
> 
> Why is newer CMake needed? What is LLVM's build process doing that is
so
> complicated that it needs bleeding edge CMake?
CMake wasn't really designed to accurately represent all of the things that
LLVM does in its build process. Here is a list of a few things LLVM projects do
in CMake that we've had to implement hacks to work around in the past:

(1) LLVM builds a DSL compiler then uses that compiler to generate source files.
CMake historically couldn't represent those dependencies correctly, and
still can't in the Xcode generator
(2) The compiler-rt builtins libraries need to be configured and built with a
not fully functioning C toolchain, as Petr Hosek pointed out CMake 3.14 resolves
that by adding CMAKE_TRY_COMPILE_TARGET_TYPE
(3) MSVC support has been constantly evolving since 2.8, and various releases
have made huge strides in making it work
(4) LLDB generates an macOS framework target which was almost completely broken
before CMake 3.8

We've had lots of reasons in the past to update, and virtually none to stay
on old versions. If building CMake from source was a prohibitively difficult
problem, or CMake itself had wild requirements to bootstrap I'd probably
argue differently about this. That just isn't the case. Holding ourselves to
an old version of CMake is a maintenance burden on our community and IMO
provides little benefit to our users.

-Chris
> 
> Regards,
> Andrew
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191103/1778f20b/attachment.html>

Glad we are getting some reasonable justifications for CMake 3.15.0. 
There are two points we may be missing.

Is our plan to just keep updating to the most recent CMake version or 
will our objectives be reached with this version? If we gain what we 
need with this version then after a while we will have the Ubuntu LTS 
aligned.

For my current purposes, LLVM can get wild without much worry here. I 
just clone a VM and run what is useful and then delete it. A bit like 
use once and throw away. This is not for production purposes but for 
getting a better view of software structure that LLVM can provide.

When I was managing IT and the servers for a small company that needed 
to be live 24/7 we used the latest Ubuntu LTS version because we needed 
rock-solid performance as best we could get. The software in the LTS 
version is tested and used by a large user base with necessary updates 
and so we expected to have high reliability.

Those saying how easy it was to obtain the latest CMake have apparently 
not been faced with the absolute need for solid up-time. Installing 
hack-me-now, buggy, bleeding-edge software was severely discouraged. But 
in the LLVM environment, I can see where that is not a strong necessity.

But at the same time, does LLVM strive to be the backbone of 24/7 
software? Or is it more of a cool thing with some interesting code 
analysis properties? I suspect its nature is a bit wild, bleeding edge. 
And that's OK.

Neil Nelson

On 11/3/19 11:02 AM, Chris Bieneman via llvm-dev wrote:
>
>
>> On Oct 30, 2019, at 12:02 PM, Andrew Kelley via llvm-dev 
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>> Here's an argument for why to *lower* the minimum supported CMake 
>> version:
>
> No offense, but that isn't going to happen.
>
>>
>> LLVM is a compiler backend.
>
> LLVM is *much* more than a compiler backend. Using LLVM as a compiler 
> backend is one use of LLVM. It is also a JIT infrastructure, an object 
> file toolchain, the underlying bits of a linker and a debugger. It is 
> widely used in a lot of situations.
>
>>
>> LLVM is already the most difficult dependency to provide for most
>> projects that use it as a compiler backend. This is certainly true for
>> Zig, for example. This makes the requirements of building LLVM a
>> bottleneck in the bootstrapping process.
>>
>> Any bump in minimum required CMake version increases the requirements
of
>> the system to bootstrap a project which depends on LLVM.
>
> Except that modern CMake's build requirements are a subset of LLVMs. 
> Meaning if you have everything you need to build LLVM except CMake, 
> you have everything you need to build a modern CMake.
>
>>
>> For programming languages, a more minimal bootstrapping process is a
>> feature.
>>
>> Therefore, bumping the minimum required CMake version of LLVM *removes
a
>> feature* from any languages that depend on LLVM. This is O(N) cost
where
>> N is the number of projects that depend on LLVM.
>>
>> Zig has cmake_minimum_required(VERSION 2.8.5). LLVM's higher CMake
>> requirement is already the most demanding CMake version of all
Zig's
>> dependencies. Making it any higher than it already is is strictly
worse.
>>
>> Why is newer CMake needed? What is LLVM's build process doing that
is so
>> complicated that it needs bleeding edge CMake?
>
> CMake wasn't really designed to accurately represent all of the things 
> that LLVM does in its build process. Here is a list of a few things 
> LLVM projects do in CMake that we've had to implement hacks to work 
> around in the past:
>
> (1) LLVM builds a DSL compiler then uses that compiler to generate 
> source files. CMake historically couldn't represent those dependencies 
> correctly, and still can't in the Xcode generator
> (2) The compiler-rt builtins libraries need to be configured and built 
> with a not fully functioning C toolchain, as Petr Hosek pointed out 
> CMake 3.14 resolves that by adding CMAKE_TRY_COMPILE_TARGET_TYPE
> (3) MSVC support has been constantly evolving since 2.8, and various 
> releases have made huge strides in making it work
> (4) LLDB generates an macOS framework target which was almost 
> completely broken before CMake 3.8
>
> We've had lots of reasons in the past to update, and virtually none 
> to stay on old versions. If building CMake from source was a 
> prohibitively difficult problem, or CMake itself had wild requirements 
> to bootstrap I'd probably argue differently about this. That just 
> isn't the case. Holding ourselves to an old version of CMake is a 
> maintenance burden on our community and IMO provides little benefit to 
> our users.
>
> -Chris
>
>>
>> Regards,
>> Andrew
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191103/279081e8/attachment.html>

Am So., 3. Nov. 2019 um 18:20 Uhr schrieb Neil Nelson via llvm-dev
<llvm-dev at lists.llvm.org>:
> Glad we are getting some reasonable justifications for CMake 3.15.0. There
are two points we may be missing.
>
> Is our plan to just keep updating to the most recent CMake version or will
our objectives be reached with this version? If we gain what we need with this
version then after a while we will have the Ubuntu LTS aligned.
>
> For my current purposes, LLVM can get wild without much worry here. I just
clone a VM and run what is useful and then delete it. A bit like use once and
throw away. This is not for production purposes but for getting a better view of
software structure that LLVM can provide.
>
> When I was managing IT and the servers for a small company that needed to
be live 24/7 we used the latest Ubuntu LTS version because we needed rock-solid
performance as best we could get. The software in the LTS version is tested and
used by a large user base with necessary updates and so we expected to have high
reliability.
>
> Those saying how easy it was to obtain the latest CMake have apparently not
been faced with the absolute need for solid up-time. Installing hack-me-now,
buggy, bleeding-edge software was severely discouraged. But in the LLVM
environment, I can see where that is not a strong necessity.
>
> But at the same time, does LLVM strive to be the backbone of 24/7 software?
Or is it more of a cool thing with some interesting code analysis properties? I
suspect its nature is a bit wild, bleeding edge. And that's OK.
You are using LTS versions for reliability, but then expect the
development version of LLVM to be just as stable for your 24/7
software?

Michael

Neil Nelson wrote:
> When I was managing IT and the servers for a small company that
> needed to be live 24/7 we used the latest Ubuntu LTS version
> because we needed rock-solid performance as best we could get.
> The software in the LTS version is tested and used by a large
> user base with necessary updates and so we expected to have high
> reliability.
I used to work for Tandem.  I understand that kind of environment.
> Those saying how easy it was to obtain the latest CMake have
> apparently not been faced with the absolute need for solid
> up-time. Installing hack-me-now, buggy, bleeding-edge software
> was severely discouraged. But in the LLVM environment, I can see
> where that is not a strong necessity.
However I think you are creating a false dichotomy. I have no real
insight into Ubuntu's methodology, but as a user, what I observe
is that Ubuntu LTS is NOT a collection of the most recent stable 
well-tested versions of all components, kept up-to-date; instead, 
it is a snapshot of stable, well-tested versions of all components, 
which Ubuntu then commits to maintaining for N years. A given LTS
releases maintenance updates only, not new features.

And it is certainly not the case that every version of every 
component after an Ubuntu LTS is a buggy, bleeding-edge untested 
hacked-up mess.  Ubuntu LTS chooses to stick with certain versions,
not because later versions are awful, but because picking up new
versions is not their goal.

So, when we talk about raising our minimum required CMake version,
it is not to take CMake bleeding-edge HEAD and have everyone build 
their own; it is to take a more recent formal release of CMake.  
Now, the CMake people might not conveniently package binaries for 
everyone's environment, which might mean some people need to pull 
a release's source package and build it themselves; this is the 
nature of open-source.  
--paulr

On Sun, Nov 3, 2019 at 7:20 PM Neil Nelson via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Glad we are getting some reasonable justifications for CMake 3.15.0. There
> are two points we may be missing.
>
> Is our plan to just keep updating to the most recent CMake version or will
> our objectives be reached with this version? If we gain what we need with
> this version then after a while we will have the Ubuntu LTS aligned.
>
> For my current purposes, LLVM can get wild without much worry here. I just
> clone a VM and run what is useful and then delete it. A bit like use once
> and throw away. This is not for production purposes but for getting a
> better view of software structure that LLVM can provide.
>
> When I was managing IT and the servers for a small company that needed to
> be live 24/7 we used the latest Ubuntu LTS version because we needed
> rock-solid performance as best we could get. The software in the LTS
> version is tested and used by a large user base with necessary updates and
> so we expected to have high reliability.
>
> Those saying how easy it was to obtain the latest CMake have apparently
> not been faced with the absolute need for solid up-time. Installing
> hack-me-now, buggy, bleeding-edge software was severely discouraged. But
> in the LLVM environment, I can see where that is not a strong necessity.
>
> But at the same time, does LLVM strive to be the backbone of 24/7
> software? Or is it more of a cool thing with some interesting code analysis
> properties? I suspect its nature is a bit wild, bleeding edge. And
that's
> OK.
>
I'm not really sure what your worry is here -- if you're downloading and
building a development version of LLVM onto this particular production
machine, why is it problematic to also download a newer release version of
CMake to build it with?

Maybe you're worried that you'd have to install the new cmake into
/usr/local/bin? If that's the worry, I'll note that you are not required
to
install cmake into /usr/local/bin/ in order to use it to build LLVM, and
I'd recommend not doing so. You can actually invoke the cmake binary
directly from its build/download directory -- no "make install"
required at
all. Used that way, it doesn't affect anything else on your system -- if
you like, the new cmake can be used ONLY for building LLVM, and affect
literally nothing else other than a tiny amount of disk-space.

Another important point here is that the version of cmake you use to build
llvm doesn't impact users of the llvm libraries or binaries that were built
with the new cmake. (I'd contrast that with llvm requiring a newer version
of a shared library, which _does_ have runtime impact, and thus is of
potentially larger concern.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191104/bbc6b700/attachment.html>