llvm dev - Oct 2019 - RFC: Updating to CMake 3.15.0

At the LLVM Developer Meeting on October 23rd we held a CMake roundtable. One of
the items we discussed was adopting a more regular timeline for CMake upgrades.
During the roundtable there was overwhelming support for upgrading CMake, and
support for treating CMake differently than how we treat upgrading host
compilers.

Historically we've taken into account recent versions of Visual Studio,
Xcode and Linux distribution long term support packages when deciding what CMake
version to require. While this does work, it has held us to old versions of
CMake for long periods of time.

During the roundtable we discussed that the amount of effort involved in
updating to a modern CMake is more or less the same regardless of what version
we choose. We believe this to be true because most OS package managers have
older versions, like the Ubuntu 18.04 LTS which has CMake 3.10.2 which was
released January 18, 2018. Additionally CMake.org provides binary and source
downloads, and building CMake from source has lower system requirements than
LLVM and is very simple.

With all of this in mind at the roundtable we decided to propose raising the
minimum required CMake version for all LLVM sub-projects to 3.15.0. Assuming
there are no objections to this proposal, I propose a cut-over date of December
2, 2019.

Thanks,
-Chris

On 10/29/2019 10:10 AM, Chris Bieneman via llvm-dev
wrote:
> At the LLVM Developer Meeting on October 23rd we held a CMake roundtable.
One of the items we discussed was adopting a more regular timeline for CMake
upgrades. During the roundtable there was overwhelming support for upgrading
CMake, and support for treating CMake differently than how we treat upgrading
host compilers.
> 
> Historically we've taken into account recent versions of Visual Studio,
Xcode and Linux distribution long term support packages when deciding what CMake
version to require. While this does work, it has held us to old versions of
CMake for long periods of time.
> 
> During the roundtable we discussed that the amount of effort involved in
updating to a modern CMake is more or less the same regardless of what version
we choose. We believe this to be true because most OS package managers have
older versions, like the Ubuntu 18.04 LTS which has CMake 3.10.2 which was
released January 18, 2018. Additionally CMake.org provides binary and source
downloads, and building CMake from source has lower system requirements than
LLVM and is very simple.
> 
> With all of this in mind at the roundtable we decided to propose raising
the minimum required CMake version for all LLVM sub-projects to 3.15.0. Assuming
there are no objections to this proposal, I propose a cut-over date of December
2, 2019.
> 
Will users of the CMake export files generated by LLVM also need to use 3.15.0
or
is it possibly to have the export files require an older version?

The 10.0 branchpoint will be in early January, I think it would be better
to wait until after the branch.  Mainly because that doesn't give us a
lot of time to actually take advantage of the new features  in 3.15.0 before the
branch, so I don't see much benefit in switching over at that date.

I think we should also decide on a long-term plan for CMake requirements.
If the conclusion is that users should be able to handle using the latest
(3.15.0) version of CMake, maybe we should just plan to always up-date to the
latest version in trunk after each release branch is created.

-Tom
> Thanks,
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

+1 to bumping cmake version and cleaning build system somewhat.

On Tue, Oct 29, 2019 at 8:23 PM Tom Stellard via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> On 10/29/2019 10:10 AM, Chris Bieneman via llvm-dev wrote:
> > At the LLVM Developer Meeting on October 23rd we held a CMake
roundtable. One of the items we discussed was adopting a more regular timeline
for CMake upgrades. During the roundtable there was overwhelming support for
upgrading CMake, and support for treating CMake differently than how we treat
upgrading host compilers.
> >
> > Historically we've taken into account recent versions of Visual
Studio, Xcode and Linux distribution long term support packages when deciding
what CMake version to require. While this does work, it has held us to old
versions of CMake for long periods of time.
> >
> > During the roundtable we discussed that the amount of effort involved
in updating to a modern CMake is more or less the same regardless of what
version we choose. We believe this to be true because most OS package managers
have older versions, like the Ubuntu 18.04 LTS which has CMake 3.10.2 which was
released January 18, 2018. Additionally CMake.org provides binary and source
downloads, and building CMake from source has lower system requirements than
LLVM and is very simple.
> >
> > With all of this in mind at the roundtable we decided to propose
raising the minimum required CMake version for all LLVM sub-projects to 3.15.0.
Assuming there are no objections to this proposal, I propose a cut-over date of
December 2, 2019.
> >
>
> Will users of the CMake export files generated by LLVM also need to use
3.15.0 or
> is it possibly to have the export files require an older version?
>
> The 10.0 branchpoint will be in early January, I think it would be better
> to wait until after the branch.  Mainly because that doesn't give us a
> lot of time to actually take advantage of the new features  in 3.15.0
before the
> branch, so I don't see much benefit in switching over at that date.
>
> I think we should also decide on a long-term plan for CMake requirements.
> If the conclusion is that users should be able to handle using the latest
> (3.15.0) version of CMake, maybe we should just plan to always up-date to
the
> latest version in trunk after each release branch is created.
Would it please be possible to have some reasonably sane
dependency versioning policy? Even debian sid only has 3.13.4-1.
How about accepting/requiring version which is at least 1 year old?
That would incidentally, result in requiring cmake v3.13
> -Tom
Roman
> > Thanks,
> > -Chris
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

> On Oct 29, 2019, at 10:23 AM, Tom Stellard <tstellar at redhat.com>
wrote:
> 
> On 10/29/2019 10:10 AM, Chris Bieneman via llvm-dev wrote:
>> At the LLVM Developer Meeting on October 23rd we held a CMake
roundtable. One of the items we discussed was adopting a more regular timeline
for CMake upgrades. During the roundtable there was overwhelming support for
upgrading CMake, and support for treating CMake differently than how we treat
upgrading host compilers.
>> 
>> Historically we've taken into account recent versions of Visual
Studio, Xcode and Linux distribution long term support packages when deciding
what CMake version to require. While this does work, it has held us to old
versions of CMake for long periods of time.
>> 
>> During the roundtable we discussed that the amount of effort involved
in updating to a modern CMake is more or less the same regardless of what
version we choose. We believe this to be true because most OS package managers
have older versions, like the Ubuntu 18.04 LTS which has CMake 3.10.2 which was
released January 18, 2018. Additionally CMake.org provides binary and source
downloads, and building CMake from source has lower system requirements than
LLVM and is very simple.
>> 
>> With all of this in mind at the roundtable we decided to propose
raising the minimum required CMake version for all LLVM sub-projects to 3.15.0.
Assuming there are no objections to this proposal, I propose a cut-over date of
December 2, 2019.
>> 
> 
> Will users of the CMake export files generated by LLVM also need to use
3.15.0 or
> is it possibly to have the export files require an older version?
The export files don't contain cmake version requirements, so they should
work on older versions.
> 
> The 10.0 branchpoint will be in early January, I think it would be better
> to wait until after the branch.  Mainly because that doesn't give us a
> lot of time to actually take advantage of the new features  in 3.15.0
before the
> branch, so I don't see much benefit in switching over at that date.
No objection from me.
> 
> I think we should also decide on a long-term plan for CMake requirements.
> If the conclusion is that users should be able to handle using the latest
> (3.15.0) version of CMake, maybe we should just plan to always up-date to
the
> latest version in trunk after each release branch is created.
This also sounds reasonable to me.

-Chris
> 
> -Tom
> 
>> Thanks,
>> -Chris
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191029/bb37cd14/attachment.html>

On 10/29/19 1:10 PM, Chris Bieneman via llvm-dev wrote:
> At the LLVM Developer Meeting on October 23rd we held a CMake roundtable.
One of the items we discussed was adopting a more regular timeline for CMake
upgrades. During the roundtable there was overwhelming support for upgrading
CMake, and support for treating CMake differently than how we treat upgrading
host compilers.
> 
> Historically we've taken into account recent versions of Visual Studio,
Xcode and Linux distribution long term support packages when deciding what CMake
version to require. While this does work, it has held us to old versions of
CMake for long periods of time.
> 
> During the roundtable we discussed that the amount of effort involved in
updating to a modern CMake is more or less the same regardless of what version
we choose. We believe this to be true because most OS package managers have
older versions, like the Ubuntu 18.04 LTS which has CMake 3.10.2 which was
released January 18, 2018. Additionally CMake.org provides binary and source
downloads, and building CMake from source has lower system requirements than
LLVM and is very simple.
> 
> With all of this in mind at the roundtable we decided to propose raising
the minimum required CMake version for all LLVM sub-projects to 3.15.0. Assuming
there are no objections to this proposal, I propose a cut-over date of December
2, 2019.
Here's an argument for why to *lower* the minimum supported CMake version:

LLVM is a compiler backend.

LLVM is already the most difficult dependency to provide for most
projects that use it as a compiler backend. This is certainly true for
Zig, for example. This makes the requirements of building LLVM a
bottleneck in the bootstrapping process.

Any bump in minimum required CMake version increases the requirements of
the system to bootstrap a project which depends on LLVM.

For programming languages, a more minimal bootstrapping process is a
feature.

Therefore, bumping the minimum required CMake version of LLVM *removes a
feature* from any languages that depend on LLVM. This is O(N) cost where
N is the number of projects that depend on LLVM.

Zig has cmake_minimum_required(VERSION 2.8.5). LLVM's higher CMake
requirement is already the most demanding CMake version of all Zig's
dependencies. Making it any higher than it already is is strictly worse.

Why is newer CMake needed? What is LLVM's build process doing that is so
complicated that it needs bleeding edge CMake?

Regards,
Andrew



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191030/e9c66cb6/attachment.sig>

> On Oct 30, 2019, at 12:02 PM, Andrew Kelley via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> Here's an argument for why to *lower* the minimum supported CMake
version:
No offense, but that isn't going to happen.
> 
> LLVM is a compiler backend.
LLVM is *much* more than a compiler backend. Using LLVM as a compiler backend is
one use of LLVM. It is also a JIT infrastructure, an object file toolchain, the
underlying bits of a linker and a debugger. It is widely used in a lot of
situations.
> 
> LLVM is already the most difficult dependency to provide for most
> projects that use it as a compiler backend. This is certainly true for
> Zig, for example. This makes the requirements of building LLVM a
> bottleneck in the bootstrapping process.
> 
> Any bump in minimum required CMake version increases the requirements of
> the system to bootstrap a project which depends on LLVM.
Except that modern CMake's build requirements are a subset of LLVMs. Meaning
if you have everything you need to build LLVM except CMake, you have everything
you need to build a modern CMake.
> 
> For programming languages, a more minimal bootstrapping process is a
> feature.
> 
> Therefore, bumping the minimum required CMake version of LLVM *removes a
> feature* from any languages that depend on LLVM. This is O(N) cost where
> N is the number of projects that depend on LLVM.
> 
> Zig has cmake_minimum_required(VERSION 2.8.5). LLVM's higher CMake
> requirement is already the most demanding CMake version of all Zig's
> dependencies. Making it any higher than it already is is strictly worse.
> 
> Why is newer CMake needed? What is LLVM's build process doing that is
so
> complicated that it needs bleeding edge CMake?
CMake wasn't really designed to accurately represent all of the things that
LLVM does in its build process. Here is a list of a few things LLVM projects do
in CMake that we've had to implement hacks to work around in the past:

(1) LLVM builds a DSL compiler then uses that compiler to generate source files.
CMake historically couldn't represent those dependencies correctly, and
still can't in the Xcode generator
(2) The compiler-rt builtins libraries need to be configured and built with a
not fully functioning C toolchain, as Petr Hosek pointed out CMake 3.14 resolves
that by adding CMAKE_TRY_COMPILE_TARGET_TYPE
(3) MSVC support has been constantly evolving since 2.8, and various releases
have made huge strides in making it work
(4) LLDB generates an macOS framework target which was almost completely broken
before CMake 3.8

We've had lots of reasons in the past to update, and virtually none to stay
on old versions. If building CMake from source was a prohibitively difficult
problem, or CMake itself had wild requirements to bootstrap I'd probably
argue differently about this. That just isn't the case. Holding ourselves to
an old version of CMake is a maintenance burden on our community and IMO
provides little benefit to our users.

-Chris
> 
> Regards,
> Andrew
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191103/1778f20b/attachment.html>

Strong agree in general. A number of policy proposals were discussed
already. The thread still reads like a split between supporters and
opponents?

In my opinion the update process on build bots was not discussed enough.
If we choose a version and then see that it's gonna be a problem with
many build bots, this will be a blocker and potential time sink for
maintainers right? Should we do it first and then bump the CMake version?

Here's a proposal that gives build bot maintainers enough time:
(1) Once a new LLVM release version branches: start updating CMake to
latest available binary releases on all build bots
(2) Once the new LLVM version is released: bump minimum CMake
requirement on master to the lowest common denominator of the (updated)
build bots

What do you think?

On 29/10/2019 18:10, Chris Bieneman via llvm-dev wrote:
> At the LLVM Developer Meeting on October 23rd we held a CMake roundtable.
One of the items we discussed was adopting a more regular timeline for CMake
upgrades. During the roundtable there was overwhelming support for upgrading
CMake, and support for treating CMake differently than how we treat upgrading
host compilers.
>
> Historically we've taken into account recent versions of Visual Studio,
Xcode and Linux distribution long term support packages when deciding what CMake
version to require. While this does work, it has held us to old versions of
CMake for long periods of time.
>
> During the roundtable we discussed that the amount of effort involved in
updating to a modern CMake is more or less the same regardless of what version
we choose. We believe this to be true because most OS package managers have
older versions, like the Ubuntu 18.04 LTS which has CMake 3.10.2 which was
released January 18, 2018. Additionally CMake.org provides binary and source
downloads, and building CMake from source has lower system requirements than
LLVM and is very simple.
>
> With all of this in mind at the roundtable we decided to propose raising
the minimum required CMake version for all LLVM sub-projects to 3.15.0. Assuming
there are no objections to this proposal, I propose a cut-over date of December
2, 2019.
>
> Thanks,
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 
https://flowcrypt.com/pub/stefan.graenitz at gmail.com

> On Nov 8, 2019, at 6:10 AM, Stefan Gränitz via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Strong agree in general. A number of policy proposals were discussed
> already. The thread still reads like a split between supporters and
> opponents?
> 
> In my opinion the update process on build bots was not discussed enough.
> If we choose a version and then see that it's gonna be a problem with
> many build bots, this will be a blocker and potential time sink for
> maintainers right? Should we do it first and then bump the CMake version?
> 
> Here's a proposal that gives build bot maintainers enough time:
> (1) Once a new LLVM release version branches: start updating CMake to
> latest available binary releases on all build bots
> (2) Once the new LLVM version is released: bump minimum CMake
> requirement on master to the lowest common denominator of the (updated)
> build bots
> 
> What do you think?
I think that the timeline here is correct. Post-branch we should have a window
to update the bots (a couple weeks maybe to get everything squared). I don't
think we should update to a lowest common denominator though. I think we should
make a decision and tell the bots maintainers which version to update to. One of
the points of this thread is to get our project away from being held hostage on
old versions of CMake because package distributions don't update frequently
enough.
> 
> On 29/10/2019 18:10, Chris Bieneman via llvm-dev wrote:
>> At the LLVM Developer Meeting on October 23rd we held a CMake
roundtable. One of the items we discussed was adopting a more regular timeline
for CMake upgrades. During the roundtable there was overwhelming support for
upgrading CMake, and support for treating CMake differently than how we treat
upgrading host compilers.
>> 
>> Historically we've taken into account recent versions of Visual
Studio, Xcode and Linux distribution long term support packages when deciding
what CMake version to require. While this does work, it has held us to old
versions of CMake for long periods of time.
>> 
>> During the roundtable we discussed that the amount of effort involved
in updating to a modern CMake is more or less the same regardless of what
version we choose. We believe this to be true because most OS package managers
have older versions, like the Ubuntu 18.04 LTS which has CMake 3.10.2 which was
released January 18, 2018. Additionally CMake.org provides binary and source
downloads, and building CMake from source has lower system requirements than
LLVM and is very simple.
>> 
>> With all of this in mind at the roundtable we decided to propose
raising the minimum required CMake version for all LLVM sub-projects to 3.15.0.
Assuming there are no objections to this proposal, I propose a cut-over date of
December 2, 2019.
>> 
>> Thanks,
>> -Chris
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> -- 
> https://flowcrypt.com/pub/stefan.graenitz at gmail.com
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev