llvm dev - Mar 2020 - Upgrading LLVM's minimum required CMake version

We had this discussion a few months ago and it petered out, and it’s recently
been revived in the context of upgrading the CMake version specifically for
libc++ (at which point people suggested upgrading the CMake version used by all
of LLVM), so let’s try to move this forward.

Our current required minimum version is CMake 3.4.3, which was released on
January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3 on
May 31st 2016, which was just 4 months after its release.

Let’s look at the CMake versions available on various distros and operating
systems. I’m unfamiliar with many of these, so I apologize if I get something
wrong. (I’m using pkgs.org for most of this information.)
* RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
* RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
* RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org is screwy on this one,
because it doesn’t make sense that RHEL 7 should have a newer available version
than RHEL 8)
* Debian 9 (released June 17th 2017): 3.7.2
* Debian 10 (released July 6th 2019): 3.13.4
* Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
* Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
* FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in a
point release)
* FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in a
point release)
* NetBSD 8.1 (released May 31st 2019): 3.16.1
* NetBSD 9.0 (released February 14th 2020): 3.16.1
* OpenBSD: couldn’t find the version
* macOS: latest version is readily available through Homebrew
* Windows: You can install it yourself or use the one bundled with Visual
Studio. I don't know what versions are bundled with Visual Studio; some
searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019 has
3.15, though I have no confirmation of that.

Note that CMake provides prebuilt binaries for Linux, macOS, and Windows, and
it’s also straightforward to build from source (it has very conservative
compiler requirements). One suggestion that was brought up in the past was for
LLVM’s build system to just download a newer version of CMake if you attempted
to build it using one that was too old, but there was opposition [1]. There was
also a suggestion to have a script in LLVM to download and build CMake for you,
but there were mixed opinions on this too [2], particularly since many
developers might prefer downloading a binary release to building from source
themselves (though of course the script could also download binary releases if
applicable). I personally think downloading or building CMake yourself isn’t a
high barrier for anyone wanting to build LLVM (and in particular it’s *much*
more straightforward than building LLVM itself), but I can understand why people
would prefer to stick to versions available in distros.

Another suggestion that came up last time was to set a policy for upgrading
CMake versions on some regular basis. The opposition to this was that we should
upgrade CMake versions only when a newer version has a compelling enough feature
to justify upgrading, rather than always upgrading. I can see arguments for both
approaches, but we should definitely at least think about the benefits we can
get from upgrading versions. I've gone through the CMake release notes and
highlighted features which seemed potentially valuable for LLVM. Note that
I'm only highlighting features for which our minimum CMake version would
have to be bumped up in order for our build system to take advantage of. There
are other useful features in newer CMake versions, but you can take advantage of
them just by using a newer CMake yourself. For example, 3.9 loosens the
dependencies of object compilation, which should result in faster Ninja builds.

CMake 3.5 (released March 8th 2016):
* install(DIRECTORY) supports generator expressions

CMake 3.6 (released July 7th 2016):
* install() supports EXCLUDE_FROM_ALL
* list() supports FILTER to filter by regular expression
* Subninja support, which could theoretically be used for much faster runtimes
builds, although in practice we probably want to make ExternalProject support
this directly instead of trying to layer our own meta-build system on top
* CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static library
instead of an executable, which will greatly simplify the compiler-rt build

CMake 3.7 (released November 11th 2016):
* New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL

CMake 3.8 (released April 10th 2017):
* Compile features for C++17, which is required to build libc++ correctly
* Support for compile features for specific C++ features instead of only being
able to specify standard versions
* rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH variable
* Apple framework support for static libraries
* New swig_add_library command in the UseSWIG module
* New generator expression $<IF:cond,true-value,false-value>

CMake 3.9 (released July 18th 2017):
* install(TARGETS) and install(EXPORTS) support for object libraries, which will
simplify the compiler-rt build
* TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
* $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator expressions for Apple bundles

CMake 3.10 (released November 20th 2017):
* include_guard() command for proper guarding against double includes of CMake
scripts
* An interesting aside is that this is the first verion of CMake to require
C++11 to build, which should give a good sense of how conservative they are
about compiler requirements

CMake 3.11 (released March 28th 2018):
* add_library() and add_executable() can be called without sources as long as
target_sources() is used later
* target_compile_{definitions,features,options}, target_include_directories(),
target_sources(), and target_link_libraries() can set the corresponding
INTERFACE_* properties on imported targets
* COMPILE_DEFINITIONS supports generator expressions
* COMPILE_OPTIONS source file property added
* INCLUDE_DIRECTORIES source file property added
* Interface libraries support custom properites

CMake 3.12 (released July 17th 2018):
* add_compile_definitions() added to add compile definitions for targets (to
avoid the global pollution caused by add_definitions())
* cmake_minimum_required() supports a version range to indicate tested CMake
versions and set policies accordingly
* file(TOUCH) and file(TOUCH_NOCREATE) added
* list(JOIN), list(SUBLIST) and list(TRANSFORM) added
* string(JOIN) added
* SHELL: prefix support in target_compile_options to avoid errant deduplication
* target_link_libraries() supports object libraries and propagates usage
requirements
* EXPORT_PROPERTIES target property to control the target properties exported by
export() and install(EXPORT)
* FindLibXml2 provides imported targets
* New FindPython, FindPython2, and FindPython3 modules to ease location Python
and selecting a specific version
* Modernization of UseSWIG module
* New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
* Compile features support for C++20

CMake 3.13 (released November 20th 2018):
* cmake -E create_symlink supported on Windows
* target_link_directories() and target_link_options() commands to set link
options instead of awkwardly having to use target_link_libraries() for this
purpose
* UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation

CMake 3.14 (released March 14th 2019):
* file(CREATE_LINK) to create hard or symbolic links
* if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
* $<IN_LIST:...> generator expression correctly handles empty argument
* Fixes for object library linking propagation
* Link options to manage position independent executables added automatically

CMake 3.15 (released July 17th 2019):
* list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
* New message() types NOTICE, VERBOSE, DEBUG and TRACE
* string(REPEAT) added
* MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY variable
to select the runtime library type for MSVC
* $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
* $<COMPILE_LANG_AND_ID:...> generator expression added
* $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
* $<REMOVE_DUPLICATES:list> generator expression added
* New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
* $<TARGET_OBJECTS:...> generator expression supports executables and
static, shared, and module libraries

CMake 3.16 (released November 26th 2019):
* Support for generator expressions in BUILD_RPATH and INSTALL_RPATH

CMake 3.17 (released March 20th 2020):
* Ninja Multi-Config generator, which among other things would greatly simplify
LLVM_OPTIMIZED_TABLEGEN
* foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
* New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
* INSTALL_NAME_DIR supports generator expressions

Our build system is incredibly complex, and many of these features can be used
to clean it up and make it much more maintainable. I would personally like us to
at least bump up to CMake 3.12. I also do think it's worth establishing a
policy and process around upgrading CMake versions, since newer versions keep on
adding useful features (particularly better generator expression support), and
we want to be able to keep taking advantage of them.

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html 
[2] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html

> On Mar 26, 2020, at 16:07, Shoaib Meenai <smeenai at fb.com> wrote:
> 
> We had this discussion a few months ago and it petered out, and it’s
recently been revived in the context of upgrading the CMake version specifically
for libc++ (at which point people suggested upgrading the CMake version used by
all of LLVM), so let’s try to move this forward.
I was going to create a new thread to ask LLVM as a whole, but you did it faster
and better than I could have! Thanks a lot for all the research and context you
added below. This is great.
> 
> [...]
> 
> Our build system is incredibly complex, and many of these features can be
used to clean it up and make it much more maintainable. I would personally like
us to at least bump up to CMake 3.12. I also do think it's worth
establishing a policy and process around upgrading CMake versions, since newer
versions keep on adding useful features (particularly better generator
expression support), and we want to be able to keep taking advantage of them.

I'd like to express very strong support for this. I believe 3.12 is
reasonable, and it will give us a lot of features that can improve our life
significantly, while still being old enough that some distributions will support
it out of the box.

As far as establishing a policy goes, my preference would be to say that we
systematically bump the requirement every release to e.g. the CMake released 12
months ago. I understand some people would rather have a detailed analysis of
why we should upgrade every time, but I think it's unnecessary overhead
because:
1. CMake keeps adding new features that can simplify the build system, so
there's always "a reason" to upgrade
2. CMake is very easy to upgrade, so it makes sense to be more aggressive than
for e.g. bumping compiler requirements

Bumping systematically would have the benefit of lowering the effort to making
these upgrades and avoiding situations where upgrade efforts are stalled for
several years due to the effort of having to convince people over again. If we
bumped systematically, it would also ensure that users and bot owners are
trained to upgrade from time to time.

However, no matter what the policy ends up being, I think it's important to
at least bump it once soon, so I would be careful not to derail this effort in
search of a perfect policy.

Cheers,
Louis

On Thu, Mar 26, 2020 at 1:26 PM Louis Dionne via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
> > On Mar 26, 2020, at 16:07, Shoaib Meenai <smeenai at fb.com>
wrote:
> >
> > We had this discussion a few months ago and it petered out, and it’s
> recently been revived in the context of upgrading the CMake version
> specifically for libc++ (at which point people suggested upgrading the
> CMake version used by all of LLVM), so let’s try to move this forward.
>
> I was going to create a new thread to ask LLVM as a whole, but you did it
> faster and better than I could have! Thanks a lot for all the research and
> context you added below. This is great.
>
> >
> > [...]
> >
> > Our build system is incredibly complex, and many of these features can
> be used to clean it up and make it much more maintainable. I would
> personally like us to at least bump up to CMake 3.12. I also do think
it's
> worth establishing a policy and process around upgrading CMake versions,
> since newer versions keep on adding useful features (particularly better
> generator expression support), and we want to be able to keep taking
> advantage of them.
>
>
> I'd like to express very strong support for this. I believe 3.12 is
> reasonable, and it will give us a lot of features that can improve our life
> significantly, while still being old enough that some distributions will
> support it out of the box.
>
LLVM libc can live with 3.12. So, I lend my support for bumping to 3.12 at
least.

> As far as establishing a policy goes, my preference would be to say that
> we systematically bump the requirement every release to e.g. the CMake
> released 12 months ago. I understand some people would rather have a
> detailed analysis of why we should upgrade every time, but I think it's
> unnecessary overhead because:
> 1. CMake keeps adding new features that can simplify the build system, so
> there's always "a reason" to upgrade
> 2. CMake is very easy to upgrade, so it makes sense to be more aggressive
> than for e.g. bumping compiler requirements
>
> Bumping systematically would have the benefit of lowering the effort to
> making these upgrades and avoiding situations where upgrade efforts are
> stalled for several years due to the effort of having to convince people
> over again. If we bumped systematically, it would also ensure that users
> and bot owners are trained to upgrade from time to time.
>
> However, no matter what the policy ends up being, I think it's
important
> to at least bump it once soon, so I would be careful not to derail this
> effort in search of a perfect policy.
>
> Cheers,
> Louis
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200326/26327cf2/attachment.html>

On Thu, Mar 26, 2020 at 9:07 PM Shoaib Meenai via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> We had this discussion a few months ago and it petered out, and it’s
> recently been revived in the context of upgrading the CMake version
> specifically for libc++ (at which point people suggested upgrading the
> CMake version used by all of LLVM), so let’s try to move this forward.
>
> Our current required minimum version is CMake 3.4.3, which was released on
> January 25th 2016. It’s interesting to note that LLVM started requiring
> 3.4.3 on May 31st 2016, which was just 4 months after its release.
>
> Let’s look at the CMake versions available on various distros and
> operating systems. I’m unfamiliar with many of these, so I apologize if I
> get something wrong. (I’m using pkgs.org for most of this information.)
> * RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
> * RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
> * RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org is screwy on
> this one, because it doesn’t make sense that RHEL 7 should have a newer
> available version than RHEL 8)
> * Debian 9 (released June 17th 2017): 3.7.2
> * Debian 10 (released July 6th 2019): 3.13.4
> * Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
> * Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
> * FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in
> a point release)
> * FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in
> a point release)
> * NetBSD 8.1 (released May 31st 2019): 3.16.1
> * NetBSD 9.0 (released February 14th 2020): 3.16.1
> * OpenBSD: couldn’t find the version
> * macOS: latest version is readily available through Homebrew
> * Windows: You can install it yourself or use the one bundled with Visual
> Studio. I don't know what versions are bundled with Visual Studio; some
> searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019
> has 3.15, though I have no confirmation of that.
>
> Note that CMake provides prebuilt binaries for Linux, macOS, and Windows,
> and it’s also straightforward to build from source (it has very
> conservative compiler requirements). One suggestion that was brought up in
> the past was for LLVM’s build system to just download a newer version of
> CMake if you attempted to build it using one that was too old, but there
> was opposition [1]. There was also a suggestion to have a script in LLVM to
> download and build CMake for you, but there were mixed opinions on this too
> [2], particularly since many developers might prefer downloading a binary
> release to building from source themselves (though of course the script
> could also download binary releases if applicable). I personally think
> downloading or building CMake yourself isn’t a high barrier for anyone
> wanting to build LLVM (and in particular it’s *much* more straightforward
> than building LLVM itself), but I can understand why people would prefer to
> stick to versions available in distros.
>
> Another suggestion that came up last time was to set a policy for
> upgrading CMake versions on some regular basis. The opposition to this was
> that we should upgrade CMake versions only when a newer version has a
> compelling enough feature to justify upgrading, rather than always
> upgrading. I can see arguments for both approaches, but we should
> definitely at least think about the benefits we can get from upgrading
> versions. I've gone through the CMake release notes and highlighted
> features which seemed potentially valuable for LLVM. Note that I'm only
> highlighting features for which our minimum CMake version would have to be
> bumped up in order for our build system to take advantage of. There are
> other useful features in newer CMake versions, but you can take advantage
> of them just by using a newer CMake yourself. For example, 3.9 loosens the
> dependencies of object compilation, which should result in faster Ninja
> builds.
>
> CMake 3.5 (released March 8th 2016):
> * install(DIRECTORY) supports generator expressions
>
> CMake 3.6 (released July 7th 2016):
> * install() supports EXCLUDE_FROM_ALL
> * list() supports FILTER to filter by regular expression
> * Subninja support, which could theoretically be used for much faster
> runtimes builds, although in practice we probably want to make
> ExternalProject support this directly instead of trying to layer our own
> meta-build system on top
> * CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static
> library instead of an executable, which will greatly simplify the
> compiler-rt build
>
> CMake 3.7 (released November 11th 2016):
> * New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
> STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL
>
> CMake 3.8 (released April 10th 2017):
> * Compile features for C++17, which is required to build libc++ correctly
> * Support for compile features for specific C++ features instead of only
> being able to specify standard versions
> * rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH
> variable
> * Apple framework support for static libraries
> * New swig_add_library command in the UseSWIG module
> * New generator expression $<IF:cond,true-value,false-value>
>
> CMake 3.9 (released July 18th 2017):
> * install(TARGETS) and install(EXPORTS) support for object libraries,
> which will simplify the compiler-rt build
> * TARGET_OBJECTS generator expression support in add_custom_command and
> file(GENERATE)
> * $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator
> expressions for Apple bundles
>
> CMake 3.10 (released November 20th 2017):
> * include_guard() command for proper guarding against double includes of
> CMake scripts
> * An interesting aside is that this is the first verion of CMake to
> require C++11 to build, which should give a good sense of how conservative
> they are about compiler requirements
>
> CMake 3.11 (released March 28th 2018):
> * add_library() and add_executable() can be called without sources as long
> as target_sources() is used later
> * target_compile_{definitions,features,options},
> target_include_directories(), target_sources(), and target_link_libraries()
> can set the corresponding INTERFACE_* properties on imported targets
> * COMPILE_DEFINITIONS supports generator expressions
> * COMPILE_OPTIONS source file property added
> * INCLUDE_DIRECTORIES source file property added
> * Interface libraries support custom properites
>
> CMake 3.12 (released July 17th 2018):
> * add_compile_definitions() added to add compile definitions for targets
> (to avoid the global pollution caused by add_definitions())
> * cmake_minimum_required() supports a version range to indicate tested
> CMake versions and set policies accordingly
> * file(TOUCH) and file(TOUCH_NOCREATE) added
> * list(JOIN), list(SUBLIST) and list(TRANSFORM) added
> * string(JOIN) added
> * SHELL: prefix support in target_compile_options to avoid errant
> deduplication
> * target_link_libraries() supports object libraries and propagates usage
> requirements
> * EXPORT_PROPERTIES target property to control the target properties
> exported by export() and install(EXPORT)
> * FindLibXml2 provides imported targets
> * New FindPython, FindPython2, and FindPython3 modules to ease location
> Python and selecting a specific version
> * Modernization of UseSWIG module
> * New generator expressions $<GENEX_EVAL:...>,
> $<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and
> $<TARGET_NAME_IF_EXISTS:...>
> * Compile features support for C++20
>
> CMake 3.13 (released November 20th 2018):
> * cmake -E create_symlink supported on Windows
> * target_link_directories() and target_link_options() commands to set link
> options instead of awkwardly having to use target_link_libraries() for this
> purpose
> * UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation
>
> CMake 3.14 (released March 14th 2019):
> * file(CREATE_LINK) to create hard or symbolic links
> * if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
> * $<IN_LIST:...> generator expression correctly handles empty
argument
> * Fixes for object library linking propagation
> * Link options to manage position independent executables added
> automatically
>
> CMake 3.15 (released July 17th 2019):
> * list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
> * New message() types NOTICE, VERBOSE, DEBUG and TRACE
> * string(REPEAT) added
> * MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY
> variable to select the runtime library type for MSVC
> * $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>,
> and $<PLATFORM_ID:...> generator expressions support matching one
value
> from a list
> * $<COMPILE_LANG_AND_ID:...> generator expression added
> * $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
> * $<REMOVE_DUPLICATES:list> generator expression added
> * New $<TARGET_FILE*> generator expressions added:
> $<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
> $<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
> $<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>,
> $<TARGET_PDB_FILE_BASE_NAME:...>
> * $<TARGET_OBJECTS:...> generator expression supports executables and
> static, shared, and module libraries
>
> CMake 3.16 (released November 26th 2019):
> * Support for generator expressions in BUILD_RPATH and INSTALL_RPATH
>
> CMake 3.17 (released March 20th 2020):
> * Ninja Multi-Config generator, which among other things would greatly
> simplify LLVM_OPTIMIZED_TABLEGEN
> * foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
> * New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
> * INSTALL_NAME_DIR supports generator expressions
>
> Our build system is incredibly complex, and many of these features can be
> used to clean it up and make it much more maintainable. I would personally
> like us to at least bump up to CMake 3.12. I also do think it's worth
> establishing a policy and process around upgrading CMake versions, since
> newer versions keep on adding useful features (particularly better
> generator expression support), and we want to be able to keep taking
> advantage of them.
>
Thanks for the detailed analysis. Given this data, I believe the new CMake
target should be somewhere between 3.8.0 (for which there is a strong
motivation due to the libc++ requirement) and 3.10.2, which is the lower
bound of what is available on current LTS distros.

Personally, I don't think cleaning up the build system is sufficient
motivation for a version bump that would require a non-trivial fraction of
LLVM users to start using a custom CMake build. The build system already
works, so there doesn't seem to be any urgent need to improve it using new
CMake versions. If a new CMake version is needed to keep things working
going forward (as seems to be the case for libc++), that's an entirely
different matter, and a good motivator for bumping the version.

Regards,
Nikita
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200326/1046b2c1/attachment.html>

On Thu, Mar 26, 2020 at 11:48 PM Nikita Popov via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> On Thu, Mar 26, 2020 at 9:07 PM Shoaib Meenai via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>
>> We had this discussion a few months ago and it petered out, and it’s
recently been revived in the context of upgrading the CMake version specifically
for libc++ (at which point people suggested upgrading the CMake version used by
all of LLVM), so let’s try to move this forward.
>>
>> Our current required minimum version is CMake 3.4.3, which was released
on January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3
on May 31st 2016, which was just 4 months after its release.
>>
>> Let’s look at the CMake versions available on various distros and
operating systems. I’m unfamiliar with many of these, so I apologize if I get
something wrong. (I’m using pkgs.org for most of this information.)
>> * RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
>> * RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
>> * RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org is screwy on
this one, because it doesn’t make sense that RHEL 7 should have a newer
available version than RHEL 8)
>> * Debian 9 (released June 17th 2017): 3.7.2
>> * Debian 10 (released July 6th 2019): 3.13.4
>> * Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
>> * Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
>> * FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded
in a point release)
>> * FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded
in a point release)
>> * NetBSD 8.1 (released May 31st 2019): 3.16.1
>> * NetBSD 9.0 (released February 14th 2020): 3.16.1
>> * OpenBSD: couldn’t find the version
>> * macOS: latest version is readily available through Homebrew
>> * Windows: You can install it yourself or use the one bundled with
Visual Studio. I don't know what versions are bundled with Visual Studio;
some searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019
has 3.15, though I have no confirmation of that.
>>
>> Note that CMake provides prebuilt binaries for Linux, macOS, and
Windows, and it’s also straightforward to build from source (it has very
conservative compiler requirements). One suggestion that was brought up in the
past was for LLVM’s build system to just download a newer version of CMake if
you attempted to build it using one that was too old, but there was opposition
[1]. There was also a suggestion to have a script in LLVM to download and build
CMake for you, but there were mixed opinions on this too [2], particularly since
many developers might prefer downloading a binary release to building from
source themselves (though of course the script could also download binary
releases if applicable). I personally think downloading or building CMake
yourself isn’t a high barrier for anyone wanting to build LLVM (and in
particular it’s *much* more straightforward than building LLVM itself), but I
can understand why people would prefer to stick to versions available in
distros.
>>
>> Another suggestion that came up last time was to set a policy for
upgrading CMake versions on some regular basis. The opposition to this was that
we should upgrade CMake versions only when a newer version has a compelling
enough feature to justify upgrading, rather than always upgrading. I can see
arguments for both approaches, but we should definitely at least think about the
benefits we can get from upgrading versions. I've gone through the CMake
release notes and highlighted features which seemed potentially valuable for
LLVM. Note that I'm only highlighting features for which our minimum CMake
version would have to be bumped up in order for our build system to take
advantage of. There are other useful features in newer CMake versions, but you
can take advantage of them just by using a newer CMake yourself. For example,
3.9 loosens the dependencies of object compilation, which should result in
faster Ninja builds.
>>
>> CMake 3.5 (released March 8th 2016):
>> * install(DIRECTORY) supports generator expressions
>>
>> CMake 3.6 (released July 7th 2016):
>> * install() supports EXCLUDE_FROM_ALL
>> * list() supports FILTER to filter by regular expression
>> * Subninja support, which could theoretically be used for much faster
runtimes builds, although in practice we probably want to make ExternalProject
support this directly instead of trying to layer our own meta-build system on
top
>> * CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static
library instead of an executable, which will greatly simplify the compiler-rt
build
>>
>> CMake 3.7 (released November 11th 2016):
>> * New if() comparison operators LESS_EQUAL, GREATER_EQUAL,
STRLESS_EQUAL, STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL
>>
>> CMake 3.8 (released April 10th 2017):
>> * Compile features for C++17, which is required to build libc++
correctly
>> * Support for compile features for specific C++ features instead of
only being able to specify standard versions
>> * rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH
variable
>> * Apple framework support for static libraries
>> * New swig_add_library command in the UseSWIG module
>> * New generator expression $<IF:cond,true-value,false-value>
>>
>> CMake 3.9 (released July 18th 2017):
>> * install(TARGETS) and install(EXPORTS) support for object libraries,
which will simplify the compiler-rt build
>> * TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
>> * $<TARGET_BUNDLE_DIR:tgt> and
$<TARGET_BUNDLE_CONTENT_DIR:tgt> generator expressions for Apple bundles
>>
>> CMake 3.10 (released November 20th 2017):
>> * include_guard() command for proper guarding against double includes
of CMake scripts
>> * An interesting aside is that this is the first verion of CMake to
require C++11 to build, which should give a good sense of how conservative they
are about compiler requirements
>>
>> CMake 3.11 (released March 28th 2018):
>> * add_library() and add_executable() can be called without sources as
long as target_sources() is used later
>> * target_compile_{definitions,features,options},
target_include_directories(), target_sources(), and target_link_libraries() can
set the corresponding INTERFACE_* properties on imported targets
>> * COMPILE_DEFINITIONS supports generator expressions
>> * COMPILE_OPTIONS source file property added
>> * INCLUDE_DIRECTORIES source file property added
>> * Interface libraries support custom properites
>>
>> CMake 3.12 (released July 17th 2018):
>> * add_compile_definitions() added to add compile definitions for
targets (to avoid the global pollution caused by add_definitions())
>> * cmake_minimum_required() supports a version range to indicate tested
CMake versions and set policies accordingly
>> * file(TOUCH) and file(TOUCH_NOCREATE) added
>> * list(JOIN), list(SUBLIST) and list(TRANSFORM) added
>> * string(JOIN) added
>> * SHELL: prefix support in target_compile_options to avoid errant
deduplication
>> * target_link_libraries() supports object libraries and propagates
usage requirements
>> * EXPORT_PROPERTIES target property to control the target properties
exported by export() and install(EXPORT)
>> * FindLibXml2 provides imported targets
>> * New FindPython, FindPython2, and FindPython3 modules to ease location
Python and selecting a specific version
>> * Modernization of UseSWIG module
>> * New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
>> * Compile features support for C++20
>>
>> CMake 3.13 (released November 20th 2018):
>> * cmake -E create_symlink supported on Windows
>> * target_link_directories() and target_link_options() commands to set
link options instead of awkwardly having to use target_link_libraries() for this
purpose
>> * UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation
>>
>> CMake 3.14 (released March 14th 2019):
>> * file(CREATE_LINK) to create hard or symbolic links
>> * if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
>> * $<IN_LIST:...> generator expression correctly handles empty
argument
>> * Fixes for object library linking propagation
>> * Link options to manage position independent executables added
automatically
>>
>> CMake 3.15 (released July 17th 2019):
>> * list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
>> * New message() types NOTICE, VERBOSE, DEBUG and TRACE
>> * string(REPEAT) added
>> * MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY
variable to select the runtime library type for MSVC
>> * $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
>> * $<COMPILE_LANG_AND_ID:...> generator expression added
>> * $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
>> * $<REMOVE_DUPLICATES:list> generator expression added
>> * New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
>> * $<TARGET_OBJECTS:...> generator expression supports executables
and static, shared, and module libraries
>>
>> CMake 3.16 (released November 26th 2019):
>> * Support for generator expressions in BUILD_RPATH and INSTALL_RPATH
>>
>> CMake 3.17 (released March 20th 2020):
>> * Ninja Multi-Config generator, which among other things would greatly
simplify LLVM_OPTIMIZED_TABLEGEN
>> * foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
>> * New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
>> * INSTALL_NAME_DIR supports generator expressions
>>
>> Our build system is incredibly complex, and many of these features can
be used to clean it up and make it much more maintainable. I would personally
like us to at least bump up to CMake 3.12. I also do think it's worth
establishing a policy and process around upgrading CMake versions, since newer
versions keep on adding useful features (particularly better generator
expression support), and we want to be able to keep taking advantage of them.
>
>
> Thanks for the detailed analysis. Given this data, I believe the new CMake
target should be somewhere between 3.8.0 (for which there is a strong motivation
due to the libc++ requirement) and 3.10.2, which is the lower bound of what is
available on current LTS distros.
>
> Personally, I don't think cleaning up the build system is sufficient
motivation for a version bump that would require a non-trivial fraction of LLVM
users to start using a custom CMake build. The build system already works, so
there doesn't seem to be any urgent need to improve it using new CMake
versions. If a new CMake version is needed to keep things working going forward
(as seems to be the case for libc++), that's an entirely different matter,
and a good motivator for bumping the version.
+1
I think bumping it up to 3.10 may fine but i'm strongly against:
a) unconditionally bumping it just because there's a newer version
    available (including delayed updates)
b) considering "pull an executable from internet and run it" as
    the proper way to update cmake
c) not considering the versions available in debian stable/testing
    and latest ubuntu lts as soft blockers
d) trying to update cmake from within cmakefiles (or automatically
    fetching stuff from internet during build in general, for that matter)
> Regards,
> Nikita
Roman
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Ubuntu 20.04 LTS will be released soon, and I believe it’ll have CMake 3.16.3,
so that increases the LTS lower bound significantly.

I strongly disagree with the sentiment that the build system already works so
there’s no urgent need to improve it. I believe we should treat the build system
like code, and the same ideas around refactoring apply. Our build system is a
huge thorny mess; there’s tons of supported configurations, lots of dependencies
which rely on targets being processed in a certain order, and so on. Some of
that can be cleaned up without requiring a CMake version bump, but generator
expressions in particular are a very powerful mechanism which enable targets to
depend on each other without requiring a particular order for processing them
(which is very fragile and easy to break). In turn, build system cleanups reduce
bugs (and we get plenty of those), make it much easier to add features with
confidence, and make the build system much more understandable. I think a policy
of “we only upgrade CMake versions when it’s absolutely necessary to do so”
would be pretty harmful for the health of the build system.

From: Nikita Popov <nikita.ppv at gmail.com>
Date: Thursday, March 26, 2020 at 1:48 PM
To: Shoaib Meenai <smeenai at fb.com>
Cc: "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

On Thu, Mar 26, 2020 at 9:07 PM Shoaib Meenai via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
We had this discussion a few months ago and it petered out, and it’s recently
been revived in the context of upgrading the CMake version specifically for
libc++ (at which point people suggested upgrading the CMake version used by all
of LLVM), so let’s try to move this forward.

Our current required minimum version is CMake 3.4.3, which was released on
January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3 on
May 31st 2016, which was just 4 months after its release.

Let’s look at the CMake versions available on various distros and operating
systems. I’m unfamiliar with many of these, so I apologize if I get something
wrong. (I’m using
pkgs.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=HFfZBFuvCgx2NCbQTQcK9dd3xsqklGHL1FS9FciwJo8&s=MU0a1olX2nxFmGK5GGIN66nwK3i2q6QTS4XD0L7WGFg&e=>
for most of this information.)
* RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
* RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
* RHEL 8 (released May 7th 2019): 3.11.4 (maybe
pkgs.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=HFfZBFuvCgx2NCbQTQcK9dd3xsqklGHL1FS9FciwJo8&s=MU0a1olX2nxFmGK5GGIN66nwK3i2q6QTS4XD0L7WGFg&e=>
is screwy on this one, because it doesn’t make sense that RHEL 7 should have a
newer available version than RHEL 8)
* Debian 9 (released June 17th 2017): 3.7.2
* Debian 10 (released July 6th 2019): 3.13.4
* Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
* Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
* FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in a
point release)
* FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in a
point release)
* NetBSD 8.1 (released May 31st 2019): 3.16.1
* NetBSD 9.0 (released February 14th 2020): 3.16.1
* OpenBSD: couldn’t find the version
* macOS: latest version is readily available through Homebrew
* Windows: You can install it yourself or use the one bundled with Visual
Studio. I don't know what versions are bundled with Visual Studio; some
searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019 has
3.15, though I have no confirmation of that.

Note that CMake provides prebuilt binaries for Linux, macOS, and Windows, and
it’s also straightforward to build from source (it has very conservative
compiler requirements). One suggestion that was brought up in the past was for
LLVM’s build system to just download a newer version of CMake if you attempted
to build it using one that was too old, but there was opposition [1]. There was
also a suggestion to have a script in LLVM to download and build CMake for you,
but there were mixed opinions on this too [2], particularly since many
developers might prefer downloading a binary release to building from source
themselves (though of course the script could also download binary releases if
applicable). I personally think downloading or building CMake yourself isn’t a
high barrier for anyone wanting to build LLVM (and in particular it’s *much*
more straightforward than building LLVM itself), but I can understand why people
would prefer to stick to versions available in distros.

Another suggestion that came up last time was to set a policy for upgrading
CMake versions on some regular basis. The opposition to this was that we should
upgrade CMake versions only when a newer version has a compelling enough feature
to justify upgrading, rather than always upgrading. I can see arguments for both
approaches, but we should definitely at least think about the benefits we can
get from upgrading versions. I've gone through the CMake release notes and
highlighted features which seemed potentially valuable for LLVM. Note that
I'm only highlighting features for which our minimum CMake version would
have to be bumped up in order for our build system to take advantage of. There
are other useful features in newer CMake versions, but you can take advantage of
them just by using a newer CMake yourself. For example, 3.9 loosens the
dependencies of object compilation, which should result in faster Ninja builds.

CMake 3.5 (released March 8th 2016):
* install(DIRECTORY) supports generator expressions

CMake 3.6 (released July 7th 2016):
* install() supports EXCLUDE_FROM_ALL
* list() supports FILTER to filter by regular expression
* Subninja support, which could theoretically be used for much faster runtimes
builds, although in practice we probably want to make ExternalProject support
this directly instead of trying to layer our own meta-build system on top
* CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static library
instead of an executable, which will greatly simplify the compiler-rt build

CMake 3.7 (released November 11th 2016):
* New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL

CMake 3.8 (released April 10th 2017):
* Compile features for C++17, which is required to build libc++ correctly
* Support for compile features for specific C++ features instead of only being
able to specify standard versions
* rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH variable
* Apple framework support for static libraries
* New swig_add_library command in the UseSWIG module
* New generator expression $<IF:cond,true-value,false-value>

CMake 3.9 (released July 18th 2017):
* install(TARGETS) and install(EXPORTS) support for object libraries, which will
simplify the compiler-rt build
* TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
* $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator expressions for Apple bundles

CMake 3.10 (released November 20th 2017):
* include_guard() command for proper guarding against double includes of CMake
scripts
* An interesting aside is that this is the first verion of CMake to require
C++11 to build, which should give a good sense of how conservative they are
about compiler requirements

CMake 3.11 (released March 28th 2018):
* add_library() and add_executable() can be called without sources as long as
target_sources() is used later
* target_compile_{definitions,features,options}, target_include_directories(),
target_sources(), and target_link_libraries() can set the corresponding
INTERFACE_* properties on imported targets
* COMPILE_DEFINITIONS supports generator expressions
* COMPILE_OPTIONS source file property added
* INCLUDE_DIRECTORIES source file property added
* Interface libraries support custom properites

CMake 3.12 (released July 17th 2018):
* add_compile_definitions() added to add compile definitions for targets (to
avoid the global pollution caused by add_definitions())
* cmake_minimum_required() supports a version range to indicate tested CMake
versions and set policies accordingly
* file(TOUCH) and file(TOUCH_NOCREATE) added
* list(JOIN), list(SUBLIST) and list(TRANSFORM) added
* string(JOIN) added
* SHELL: prefix support in target_compile_options to avoid errant deduplication
* target_link_libraries() supports object libraries and propagates usage
requirements
* EXPORT_PROPERTIES target property to control the target properties exported by
export() and install(EXPORT)
* FindLibXml2 provides imported targets
* New FindPython, FindPython2, and FindPython3 modules to ease location Python
and selecting a specific version
* Modernization of UseSWIG module
* New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
* Compile features support for C++20

CMake 3.13 (released November 20th 2018):
* cmake -E create_symlink supported on Windows
* target_link_directories() and target_link_options() commands to set link
options instead of awkwardly having to use target_link_libraries() for this
purpose
* UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation

CMake 3.14 (released March 14th 2019):
* file(CREATE_LINK) to create hard or symbolic links
* if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
* $<IN_LIST:...> generator expression correctly handles empty argument
* Fixes for object library linking propagation
* Link options to manage position independent executables added automatically

CMake 3.15 (released July 17th 2019):
* list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
* New message() types NOTICE, VERBOSE, DEBUG and TRACE
* string(REPEAT) added
* MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY variable
to select the runtime library type for MSVC
* $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
* $<COMPILE_LANG_AND_ID:...> generator expression added
* $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
* $<REMOVE_DUPLICATES:list> generator expression added
* New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
* $<TARGET_OBJECTS:...> generator expression supports executables and
static, shared, and module libraries

CMake 3.16 (released November 26th 2019):
* Support for generator expressions in BUILD_RPATH and INSTALL_RPATH

CMake 3.17 (released March 20th 2020):
* Ninja Multi-Config generator, which among other things would greatly simplify
LLVM_OPTIMIZED_TABLEGEN
* foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
* New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
* INSTALL_NAME_DIR supports generator expressions

Our build system is incredibly complex, and many of these features can be used
to clean it up and make it much more maintainable. I would personally like us to
at least bump up to CMake 3.12. I also do think it's worth establishing a
policy and process around upgrading CMake versions, since newer versions keep on
adding useful features (particularly better generator expression support), and
we want to be able to keep taking advantage of them.

Thanks for the detailed analysis. Given this data, I believe the new CMake
target should be somewhere between 3.8.0 (for which there is a strong motivation
due to the libc++ requirement) and 3.10.2, which is the lower bound of what is
available on current LTS distros.

Personally, I don't think cleaning up the build system is sufficient
motivation for a version bump that would require a non-trivial fraction of LLVM
users to start using a custom CMake build. The build system already works, so
there doesn't seem to be any urgent need to improve it using new CMake
versions. If a new CMake version is needed to keep things working going forward
(as seems to be the case for libc++), that's an entirely different matter,
and a good motivator for bumping the version.

Regards,
Nikita
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200326/d0ad1886/attachment.html>

An important addition to CMake 3.16 that you didn't mention is the
addition of the target_precompile_headers() command.  This can greatly
speedup builds for users without internal build distribution
infrastructure.

On Thu, Mar 26, 2020 at 1:07 PM Shoaib Meenai via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> We had this discussion a few months ago and it petered out, and it’s
> recently been revived in the context of upgrading the CMake version
> specifically for libc++ (at which point people suggested upgrading the
> CMake version used by all of LLVM), so let’s try to move this forward.
>
> Our current required minimum version is CMake 3.4.3, which was released on
> January 25th 2016. It’s interesting to note that LLVM started requiring
> 3.4.3 on May 31st 2016, which was just 4 months after its release.
>
> Let’s look at the CMake versions available on various distros and
> operating systems. I’m unfamiliar with many of these, so I apologize if I
> get something wrong. (I’m using pkgs.org for most of this information.)
> * RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
> * RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
> * RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org is screwy on
> this one, because it doesn’t make sense that RHEL 7 should have a newer
> available version than RHEL 8)
> * Debian 9 (released June 17th 2017): 3.7.2
> * Debian 10 (released July 6th 2019): 3.13.4
> * Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
> * Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
> * FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in
> a point release)
> * FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in
> a point release)
> * NetBSD 8.1 (released May 31st 2019): 3.16.1
> * NetBSD 9.0 (released February 14th 2020): 3.16.1
> * OpenBSD: couldn’t find the version
> * macOS: latest version is readily available through Homebrew
> * Windows: You can install it yourself or use the one bundled with Visual
> Studio. I don't know what versions are bundled with Visual Studio; some
> searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019
> has 3.15, though I have no confirmation of that.
>
> Note that CMake provides prebuilt binaries for Linux, macOS, and Windows,
> and it’s also straightforward to build from source (it has very
> conservative compiler requirements). One suggestion that was brought up in
> the past was for LLVM’s build system to just download a newer version of
> CMake if you attempted to build it using one that was too old, but there
> was opposition [1]. There was also a suggestion to have a script in LLVM to
> download and build CMake for you, but there were mixed opinions on this too
> [2], particularly since many developers might prefer downloading a binary
> release to building from source themselves (though of course the script
> could also download binary releases if applicable). I personally think
> downloading or building CMake yourself isn’t a high barrier for anyone
> wanting to build LLVM (and in particular it’s *much* more straightforward
> than building LLVM itself), but I can understand why people would prefer to
> stick to versions available in distros.
>
> Another suggestion that came up last time was to set a policy for
> upgrading CMake versions on some regular basis. The opposition to this was
> that we should upgrade CMake versions only when a newer version has a
> compelling enough feature to justify upgrading, rather than always
> upgrading. I can see arguments for both approaches, but we should
> definitely at least think about the benefits we can get from upgrading
> versions. I've gone through the CMake release notes and highlighted
> features which seemed potentially valuable for LLVM. Note that I'm only
> highlighting features for which our minimum CMake version would have to be
> bumped up in order for our build system to take advantage of. There are
> other useful features in newer CMake versions, but you can take advantage
> of them just by using a newer CMake yourself. For example, 3.9 loosens the
> dependencies of object compilation, which should result in faster Ninja
> builds.
>
> CMake 3.5 (released March 8th 2016):
> * install(DIRECTORY) supports generator expressions
>
> CMake 3.6 (released July 7th 2016):
> * install() supports EXCLUDE_FROM_ALL
> * list() supports FILTER to filter by regular expression
> * Subninja support, which could theoretically be used for much faster
> runtimes builds, although in practice we probably want to make
> ExternalProject support this directly instead of trying to layer our own
> meta-build system on top
> * CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static
> library instead of an executable, which will greatly simplify the
> compiler-rt build
>
> CMake 3.7 (released November 11th 2016):
> * New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
> STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL
>
> CMake 3.8 (released April 10th 2017):
> * Compile features for C++17, which is required to build libc++ correctly
> * Support for compile features for specific C++ features instead of only
> being able to specify standard versions
> * rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH
> variable
> * Apple framework support for static libraries
> * New swig_add_library command in the UseSWIG module
> * New generator expression $<IF:cond,true-value,false-value>
>
> CMake 3.9 (released July 18th 2017):
> * install(TARGETS) and install(EXPORTS) support for object libraries,
> which will simplify the compiler-rt build
> * TARGET_OBJECTS generator expression support in add_custom_command and
> file(GENERATE)
> * $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator
> expressions for Apple bundles
>
> CMake 3.10 (released November 20th 2017):
> * include_guard() command for proper guarding against double includes of
> CMake scripts
> * An interesting aside is that this is the first verion of CMake to
> require C++11 to build, which should give a good sense of how conservative
> they are about compiler requirements
>
> CMake 3.11 (released March 28th 2018):
> * add_library() and add_executable() can be called without sources as long
> as target_sources() is used later
> * target_compile_{definitions,features,options},
> target_include_directories(), target_sources(), and target_link_libraries()
> can set the corresponding INTERFACE_* properties on imported targets
> * COMPILE_DEFINITIONS supports generator expressions
> * COMPILE_OPTIONS source file property added
> * INCLUDE_DIRECTORIES source file property added
> * Interface libraries support custom properites
>
> CMake 3.12 (released July 17th 2018):
> * add_compile_definitions() added to add compile definitions for targets
> (to avoid the global pollution caused by add_definitions())
> * cmake_minimum_required() supports a version range to indicate tested
> CMake versions and set policies accordingly
> * file(TOUCH) and file(TOUCH_NOCREATE) added
> * list(JOIN), list(SUBLIST) and list(TRANSFORM) added
> * string(JOIN) added
> * SHELL: prefix support in target_compile_options to avoid errant
> deduplication
> * target_link_libraries() supports object libraries and propagates usage
> requirements
> * EXPORT_PROPERTIES target property to control the target properties
> exported by export() and install(EXPORT)
> * FindLibXml2 provides imported targets
> * New FindPython, FindPython2, and FindPython3 modules to ease location
> Python and selecting a specific version
> * Modernization of UseSWIG module
> * New generator expressions $<GENEX_EVAL:...>,
> $<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and
> $<TARGET_NAME_IF_EXISTS:...>
> * Compile features support for C++20
>
> CMake 3.13 (released November 20th 2018):
> * cmake -E create_symlink supported on Windows
> * target_link_directories() and target_link_options() commands to set link
> options instead of awkwardly having to use target_link_libraries() for this
> purpose
> * UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation
>
> CMake 3.14 (released March 14th 2019):
> * file(CREATE_LINK) to create hard or symbolic links
> * if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
> * $<IN_LIST:...> generator expression correctly handles empty
argument
> * Fixes for object library linking propagation
> * Link options to manage position independent executables added
> automatically
>
> CMake 3.15 (released July 17th 2019):
> * list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
> * New message() types NOTICE, VERBOSE, DEBUG and TRACE
> * string(REPEAT) added
> * MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY
> variable to select the runtime library type for MSVC
> * $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>,
> and $<PLATFORM_ID:...> generator expressions support matching one
value
> from a list
> * $<COMPILE_LANG_AND_ID:...> generator expression added
> * $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
> * $<REMOVE_DUPLICATES:list> generator expression added
> * New $<TARGET_FILE*> generator expressions added:
> $<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
> $<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
> $<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>,
> $<TARGET_PDB_FILE_BASE_NAME:...>
> * $<TARGET_OBJECTS:...> generator expression supports executables and
> static, shared, and module libraries
>
> CMake 3.16 (released November 26th 2019):
> * Support for generator expressions in BUILD_RPATH and INSTALL_RPATH
>
> CMake 3.17 (released March 20th 2020):
> * Ninja Multi-Config generator, which among other things would greatly
> simplify LLVM_OPTIMIZED_TABLEGEN
> * foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
> * New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
> * INSTALL_NAME_DIR supports generator expressions
>
> Our build system is incredibly complex, and many of these features can be
> used to clean it up and make it much more maintainable. I would personally
> like us to at least bump up to CMake 3.12. I also do think it's worth
> establishing a policy and process around upgrading CMake versions, since
> newer versions keep on adding useful features (particularly better
> generator expression support), and we want to be able to keep taking
> advantage of them.
>
> [1] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html
> [2] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200326/9f57ee9a/attachment.html>

Good point! I didn’t mention it because I thought LLVM in general preferred
modules to precompiled headers, but it’s definitely a notable addition.

From: Zachary Turner <zturner at roblox.com>
Date: Thursday, March 26, 2020 at 6:10 PM
To: Shoaib Meenai <smeenai at fb.com>
Cc: "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Upgrading LLVM's minimum required CMake version

  An important addition to CMake 3.16 that you didn't mention is the
addition of the target_precompile_headers() command.  This can greatly speedup
builds for users without internal build distribution infrastructure.

On Thu, Mar 26, 2020 at 1:07 PM Shoaib Meenai via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
We had this discussion a few months ago and it petered out, and it’s recently
been revived in the context of upgrading the CMake version specifically for
libc++ (at which point people suggested upgrading the CMake version used by all
of LLVM), so let’s try to move this forward.

Our current required minimum version is CMake 3.4.3, which was released on
January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3 on
May 31st 2016, which was just 4 months after its release.

Let’s look at the CMake versions available on various distros and operating
systems. I’m unfamiliar with many of these, so I apologize if I get something
wrong. (I’m using
pkgs.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=MeoyZ40scZVzCrANlY15VCliBF69ZUZk693O25le_Gw&s=c2acpzPzDXRHK4PbD2_pOkW2q7jtDFF9GF-I0gv9SSc&e=>
for most of this information.)
* RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
* RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
* RHEL 8 (released May 7th 2019): 3.11.4 (maybe
pkgs.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=MeoyZ40scZVzCrANlY15VCliBF69ZUZk693O25le_Gw&s=c2acpzPzDXRHK4PbD2_pOkW2q7jtDFF9GF-I0gv9SSc&e=>
is screwy on this one, because it doesn’t make sense that RHEL 7 should have a
newer available version than RHEL 8)
* Debian 9 (released June 17th 2017): 3.7.2
* Debian 10 (released July 6th 2019): 3.13.4
* Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
* Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
* FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in a
point release)
* FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in a
point release)
* NetBSD 8.1 (released May 31st 2019): 3.16.1
* NetBSD 9.0 (released February 14th 2020): 3.16.1
* OpenBSD: couldn’t find the version
* macOS: latest version is readily available through Homebrew
* Windows: You can install it yourself or use the one bundled with Visual
Studio. I don't know what versions are bundled with Visual Studio; some
searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019 has
3.15, though I have no confirmation of that.

Note that CMake provides prebuilt binaries for Linux, macOS, and Windows, and
it’s also straightforward to build from source (it has very conservative
compiler requirements). One suggestion that was brought up in the past was for
LLVM’s build system to just download a newer version of CMake if you attempted
to build it using one that was too old, but there was opposition [1]. There was
also a suggestion to have a script in LLVM to download and build CMake for you,
but there were mixed opinions on this too [2], particularly since many
developers might prefer downloading a binary release to building from source
themselves (though of course the script could also download binary releases if
applicable). I personally think downloading or building CMake yourself isn’t a
high barrier for anyone wanting to build LLVM (and in particular it’s *much*
more straightforward than building LLVM itself), but I can understand why people
would prefer to stick to versions available in distros.

Another suggestion that came up last time was to set a policy for upgrading
CMake versions on some regular basis. The opposition to this was that we should
upgrade CMake versions only when a newer version has a compelling enough feature
to justify upgrading, rather than always upgrading. I can see arguments for both
approaches, but we should definitely at least think about the benefits we can
get from upgrading versions. I've gone through the CMake release notes and
highlighted features which seemed potentially valuable for LLVM. Note that
I'm only highlighting features for which our minimum CMake version would
have to be bumped up in order for our build system to take advantage of. There
are other useful features in newer CMake versions, but you can take advantage of
them just by using a newer CMake yourself. For example, 3.9 loosens the
dependencies of object compilation, which should result in faster Ninja builds.

CMake 3.5 (released March 8th 2016):
* install(DIRECTORY) supports generator expressions

CMake 3.6 (released July 7th 2016):
* install() supports EXCLUDE_FROM_ALL
* list() supports FILTER to filter by regular expression
* Subninja support, which could theoretically be used for much faster runtimes
builds, although in practice we probably want to make ExternalProject support
this directly instead of trying to layer our own meta-build system on top
* CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static library
instead of an executable, which will greatly simplify the compiler-rt build

CMake 3.7 (released November 11th 2016):
* New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL

CMake 3.8 (released April 10th 2017):
* Compile features for C++17, which is required to build libc++ correctly
* Support for compile features for specific C++ features instead of only being
able to specify standard versions
* rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH variable
* Apple framework support for static libraries
* New swig_add_library command in the UseSWIG module
* New generator expression $<IF:cond,true-value,false-value>

CMake 3.9 (released July 18th 2017):
* install(TARGETS) and install(EXPORTS) support for object libraries, which will
simplify the compiler-rt build
* TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
* $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator expressions for Apple bundles

CMake 3.10 (released November 20th 2017):
* include_guard() command for proper guarding against double includes of CMake
scripts
* An interesting aside is that this is the first verion of CMake to require
C++11 to build, which should give a good sense of how conservative they are
about compiler requirements

CMake 3.11 (released March 28th 2018):
* add_library() and add_executable() can be called without sources as long as
target_sources() is used later
* target_compile_{definitions,features,options}, target_include_directories(),
target_sources(), and target_link_libraries() can set the corresponding
INTERFACE_* properties on imported targets
* COMPILE_DEFINITIONS supports generator expressions
* COMPILE_OPTIONS source file property added
* INCLUDE_DIRECTORIES source file property added
* Interface libraries support custom properites

CMake 3.12 (released July 17th 2018):
* add_compile_definitions() added to add compile definitions for targets (to
avoid the global pollution caused by add_definitions())
* cmake_minimum_required() supports a version range to indicate tested CMake
versions and set policies accordingly
* file(TOUCH) and file(TOUCH_NOCREATE) added
* list(JOIN), list(SUBLIST) and list(TRANSFORM) added
* string(JOIN) added
* SHELL: prefix support in target_compile_options to avoid errant deduplication
* target_link_libraries() supports object libraries and propagates usage
requirements
* EXPORT_PROPERTIES target property to control the target properties exported by
export() and install(EXPORT)
* FindLibXml2 provides imported targets
* New FindPython, FindPython2, and FindPython3 modules to ease location Python
and selecting a specific version
* Modernization of UseSWIG module
* New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
* Compile features support for C++20

CMake 3.13 (released November 20th 2018):
* cmake -E create_symlink supported on Windows
* target_link_directories() and target_link_options() commands to set link
options instead of awkwardly having to use target_link_libraries() for this
purpose
* UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation

CMake 3.14 (released March 14th 2019):
* file(CREATE_LINK) to create hard or symbolic links
* if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
* $<IN_LIST:...> generator expression correctly handles empty argument
* Fixes for object library linking propagation
* Link options to manage position independent executables added automatically

CMake 3.15 (released July 17th 2019):
* list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
* New message() types NOTICE, VERBOSE, DEBUG and TRACE
* string(REPEAT) added
* MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY variable
to select the runtime library type for MSVC
* $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
* $<COMPILE_LANG_AND_ID:...> generator expression added
* $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
* $<REMOVE_DUPLICATES:list> generator expression added
* New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
* $<TARGET_OBJECTS:...> generator expression supports executables and
static, shared, and module libraries

CMake 3.16 (released November 26th 2019):
* Support for generator expressions in BUILD_RPATH and INSTALL_RPATH

CMake 3.17 (released March 20th 2020):
* Ninja Multi-Config generator, which among other things would greatly simplify
LLVM_OPTIMIZED_TABLEGEN
* foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
* New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
* INSTALL_NAME_DIR supports generator expressions

Our build system is incredibly complex, and many of these features can be used
to clean it up and make it much more maintainable. I would personally like us to
at least bump up to CMake 3.12. I also do think it's worth establishing a
policy and process around upgrading CMake versions, since newer versions keep on
adding useful features (particularly better generator expression support), and
we want to be able to keep taking advantage of them.

[1]
http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_pipermail_llvm-2Ddev_2019-2DNovember_136485.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=MeoyZ40scZVzCrANlY15VCliBF69ZUZk693O25le_Gw&s=scU27J-lb0Tbq5E_qtXWR1Yyv4XM0hiB0elRYCbDSE0&e=>
[2]
http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_pipermail_llvm-2Ddev_2019-2DNovember_136488.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=MeoyZ40scZVzCrANlY15VCliBF69ZUZk693O25le_Gw&s=_uk-1G0tibck9XlNqnHJTGOBYejMpEPs4H0krs4opLU&e=>

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=MeoyZ40scZVzCrANlY15VCliBF69ZUZk693O25le_Gw&s=7QnIAAbYYKe90PoYlzpFZ7oT_oBUpKUUYegR1Wngd5U&e=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200327/dd10fb36/attachment.html>

Thanks for this excellent summary of CMake release notes. This makes it
really easy for me to point out what we use in LLVM-libc and make a case
for the minimum version of 3.12.

CMake 3.9 (released July 18th 2017):
> * TARGET_OBJECTS generator expression support in add_custom_command and
> file(GENERATE)
>
This.

> CMake 3.11 (released March 28th 2018):
> * add_library() and add_executable() can be called without sources as long
> as target_sources() is used later
>
This.

> CMake 3.12 (released July 17th 2018):
> * target_link_libraries() supports object libraries and propagates usage
> requirements
>
This.

> CMake 3.13 (released November 20th 2018):
> * target_link_directories() and target_link_options() commands to set link
> options instead of awkwardly having to use target_link_libraries() for this
> purpose
>
We don't currently use this, but using this will allow us to eliminate some
of the awkwardness that is present currently. So, this is not a necessity,
but nice to have.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200326/9ace3ed2/attachment.html>

Okay, so we've had some discussion on this thread, and although some people
(including me) would like a more aggressive policy, I believe the following will
not get any objection (based on the thread). On April 23rd 2020, Ubuntu 20.04
LTS will ship with CMake 3.16.x. This will make the lower bound for LTS
distributions be 3.13.4, and so I suggest we upgrade to that. Here's a
proposed process:

1. Immediately add a CMake warning in <root>/llvm/CMakeLists.txt saying
that CMake 3.13.4 will be the new minimum version starting with LLVM 12.0.0, and
mentioning the versions used in various LTSes.
2. Immediately send a courtesy heads-up email to all build-bot owners telling
them about the upcoming change.
3. Right after we branch off the release branch for LLVM 11.0.0 (the next one),
make the minimum CMake version required be 3.13.4.
4. Iterate on (3) until all bots are migrated.
5. Send a message to the list saying the bump is complete. At that time,
projects are free to start using features from 3.13.4.

Unless someone else absolutely wants to bite the bullet, I volunteer to do the
above steps.

Thoughts?

Louis

> On Mar 26, 2020, at 16:07, Shoaib Meenai <smeenai at fb.com> wrote:
> 
> We had this discussion a few months ago and it petered out, and it’s
recently been revived in the context of upgrading the CMake version specifically
for libc++ (at which point people suggested upgrading the CMake version used by
all of LLVM), so let’s try to move this forward.
> 
> Our current required minimum version is CMake 3.4.3, which was released on
January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3 on
May 31st 2016, which was just 4 months after its release.
> 
> Let’s look at the CMake versions available on various distros and operating
systems. I’m unfamiliar with many of these, so I apologize if I get something
wrong. (I’m using pkgs.org for most of this information.)
> * RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
> * RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
> * RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org is screwy on this
one, because it doesn’t make sense that RHEL 7 should have a newer available
version than RHEL 8)
> * Debian 9 (released June 17th 2017): 3.7.2
> * Debian 10 (released July 6th 2019): 3.13.4
> * Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
> * Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
> * FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in a
point release)
> * FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in
a point release)
> * NetBSD 8.1 (released May 31st 2019): 3.16.1
> * NetBSD 9.0 (released February 14th 2020): 3.16.1
> * OpenBSD: couldn’t find the version
> * macOS: latest version is readily available through Homebrew
> * Windows: You can install it yourself or use the one bundled with Visual
Studio. I don't know what versions are bundled with Visual Studio; some
searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019 has
3.15, though I have no confirmation of that.
> 
> Note that CMake provides prebuilt binaries for Linux, macOS, and Windows,
and it’s also straightforward to build from source (it has very conservative
compiler requirements). One suggestion that was brought up in the past was for
LLVM’s build system to just download a newer version of CMake if you attempted
to build it using one that was too old, but there was opposition [1]. There was
also a suggestion to have a script in LLVM to download and build CMake for you,
but there were mixed opinions on this too [2], particularly since many
developers might prefer downloading a binary release to building from source
themselves (though of course the script could also download binary releases if
applicable). I personally think downloading or building CMake yourself isn’t a
high barrier for anyone wanting to build LLVM (and in particular it’s *much*
more straightforward than building LLVM itself), but I can understand why people
would prefer to stick to versions available in distros.
> 
> Another suggestion that came up last time was to set a policy for upgrading
CMake versions on some regular basis. The opposition to this was that we should
upgrade CMake versions only when a newer version has a compelling enough feature
to justify upgrading, rather than always upgrading. I can see arguments for both
approaches, but we should definitely at least think about the benefits we can
get from upgrading versions. I've gone through the CMake release notes and
highlighted features which seemed potentially valuable for LLVM. Note that
I'm only highlighting features for which our minimum CMake version would
have to be bumped up in order for our build system to take advantage of. There
are other useful features in newer CMake versions, but you can take advantage of
them just by using a newer CMake yourself. For example, 3.9 loosens the
dependencies of object compilation, which should result in faster Ninja builds.
> 
> CMake 3.5 (released March 8th 2016):
> * install(DIRECTORY) supports generator expressions
> 
> CMake 3.6 (released July 7th 2016):
> * install() supports EXCLUDE_FROM_ALL
> * list() supports FILTER to filter by regular expression
> * Subninja support, which could theoretically be used for much faster
runtimes builds, although in practice we probably want to make ExternalProject
support this directly instead of trying to layer our own meta-build system on
top
> * CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static
library instead of an executable, which will greatly simplify the compiler-rt
build
> 
> CMake 3.7 (released November 11th 2016):
> * New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL
> 
> CMake 3.8 (released April 10th 2017):
> * Compile features for C++17, which is required to build libc++ correctly
> * Support for compile features for specific C++ features instead of only
being able to specify standard versions
> * rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH
variable
> * Apple framework support for static libraries
> * New swig_add_library command in the UseSWIG module
> * New generator expression $<IF:cond,true-value,false-value>
> 
> CMake 3.9 (released July 18th 2017):
> * install(TARGETS) and install(EXPORTS) support for object libraries, which
will simplify the compiler-rt build
> * TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
> * $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator expressions for Apple bundles
> 
> CMake 3.10 (released November 20th 2017):
> * include_guard() command for proper guarding against double includes of
CMake scripts
> * An interesting aside is that this is the first verion of CMake to require
C++11 to build, which should give a good sense of how conservative they are
about compiler requirements
> 
> CMake 3.11 (released March 28th 2018):
> * add_library() and add_executable() can be called without sources as long
as target_sources() is used later
> * target_compile_{definitions,features,options},
target_include_directories(), target_sources(), and target_link_libraries() can
set the corresponding INTERFACE_* properties on imported targets
> * COMPILE_DEFINITIONS supports generator expressions
> * COMPILE_OPTIONS source file property added
> * INCLUDE_DIRECTORIES source file property added
> * Interface libraries support custom properites
> 
> CMake 3.12 (released July 17th 2018):
> * add_compile_definitions() added to add compile definitions for targets
(to avoid the global pollution caused by add_definitions())
> * cmake_minimum_required() supports a version range to indicate tested
CMake versions and set policies accordingly
> * file(TOUCH) and file(TOUCH_NOCREATE) added
> * list(JOIN), list(SUBLIST) and list(TRANSFORM) added
> * string(JOIN) added
> * SHELL: prefix support in target_compile_options to avoid errant
deduplication
> * target_link_libraries() supports object libraries and propagates usage
requirements
> * EXPORT_PROPERTIES target property to control the target properties
exported by export() and install(EXPORT)
> * FindLibXml2 provides imported targets
> * New FindPython, FindPython2, and FindPython3 modules to ease location
Python and selecting a specific version
> * Modernization of UseSWIG module
> * New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
> * Compile features support for C++20
> 
> CMake 3.13 (released November 20th 2018):
> * cmake -E create_symlink supported on Windows
> * target_link_directories() and target_link_options() commands to set link
options instead of awkwardly having to use target_link_libraries() for this
purpose
> * UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation
> 
> CMake 3.14 (released March 14th 2019):
> * file(CREATE_LINK) to create hard or symbolic links
> * if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
> * $<IN_LIST:...> generator expression correctly handles empty
argument
> * Fixes for object library linking propagation
> * Link options to manage position independent executables added
automatically
> 
> CMake 3.15 (released July 17th 2019):
> * list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
> * New message() types NOTICE, VERBOSE, DEBUG and TRACE
> * string(REPEAT) added
> * MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY
variable to select the runtime library type for MSVC
> * $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
> * $<COMPILE_LANG_AND_ID:...> generator expression added
> * $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
> * $<REMOVE_DUPLICATES:list> generator expression added
> * New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
> * $<TARGET_OBJECTS:...> generator expression supports executables and
static, shared, and module libraries
> 
> CMake 3.16 (released November 26th 2019):
> * Support for generator expressions in BUILD_RPATH and INSTALL_RPATH
> 
> CMake 3.17 (released March 20th 2020):
> * Ninja Multi-Config generator, which among other things would greatly
simplify LLVM_OPTIMIZED_TABLEGEN
> * foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
> * New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
> * INSTALL_NAME_DIR supports generator expressions
> 
> Our build system is incredibly complex, and many of these features can be
used to clean it up and make it much more maintainable. I would personally like
us to at least bump up to CMake 3.12. I also do think it's worth
establishing a policy and process around upgrading CMake versions, since newer
versions keep on adding useful features (particularly better generator
expression support), and we want to be able to keep taking advantage of them.
> 
> [1] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html 
> [2] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200402/663432af/attachment-0001.html>

Assuming this is a one-time version bump, this seems reasonable to me. Perhaps
this goes without saying, but the warning for point 1 should only happen if you
don’t have CMake >= 3.13.4 installed.

It sounded to me from your original message that you have an urgent need to
upgrade to 3.8. Were you planning on going ahead with that right away?

From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Louis
Dionne via llvm-dev
Sent: Thursday, April 2, 2020 7:20 AM
To: Shoaib Meenai <smeenai at fb.com>
Cc: llvm-dev at lists.llvm.org
Subject: [EXT] Re: [llvm-dev] Upgrading LLVM's minimum required CMake
version

Okay, so we've had some discussion on this thread, and although some people
(including me) would like a more aggressive policy, I believe the following will
not get any objection (based on the thread). On April 23rd 2020, Ubuntu 20.04
LTS will ship with CMake 3.16.x. This will make the lower bound for LTS
distributions be 3.13.4, and so I suggest we upgrade to that. Here's a
proposed process:

1. Immediately add a CMake warning in <root>/llvm/CMakeLists.txt saying
that CMake 3.13.4 will be the new minimum version starting with LLVM 12.0.0, and
mentioning the versions used in various LTSes.
2. Immediately send a courtesy heads-up email to all build-bot owners telling
them about the upcoming change.
3. Right after we branch off the release branch for LLVM 11.0.0 (the next one),
make the minimum CMake version required be 3.13.4.
4. Iterate on (3) until all bots are migrated.
5. Send a message to the list saying the bump is complete. At that time,
projects are free to start using features from 3.13.4.

Unless someone else absolutely wants to bite the bullet, I volunteer to do the
above steps.

Thoughts?

Louis


On Mar 26, 2020, at 16:07, Shoaib Meenai <smeenai at fb.com<mailto:smeenai
at fb.com>> wrote:

We had this discussion a few months ago and it petered out, and it’s recently
been revived in the context of upgrading the CMake version specifically for
libc++ (at which point people suggested upgrading the CMake version used by all
of LLVM), so let’s try to move this forward.

Our current required minimum version is CMake 3.4.3, which was released on
January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3 on
May 31st 2016, which was just 4 months after its release.

Let’s look at the CMake versions available on various distros and operating
systems. I’m unfamiliar with many of these, so I apologize if I get something
wrong. (I’m using pkgs.org<http://pkgs.org> for most of this information.)
* RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
* RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
* RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org<http://pkgs.org>
is screwy on this one, because it doesn’t make sense that RHEL 7 should have a
newer available version than RHEL 8)
* Debian 9 (released June 17th 2017): 3.7.2
* Debian 10 (released July 6th 2019): 3.13.4
* Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
* Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
* FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in a
point release)
* FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in a
point release)
* NetBSD 8.1 (released May 31st 2019): 3.16.1
* NetBSD 9.0 (released February 14th 2020): 3.16.1
* OpenBSD: couldn’t find the version
* macOS: latest version is readily available through Homebrew
* Windows: You can install it yourself or use the one bundled with Visual
Studio. I don't know what versions are bundled with Visual Studio; some
searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019 has
3.15, though I have no confirmation of that.

Note that CMake provides prebuilt binaries for Linux, macOS, and Windows, and
it’s also straightforward to build from source (it has very conservative
compiler requirements). One suggestion that was brought up in the past was for
LLVM’s build system to just download a newer version of CMake if you attempted
to build it using one that was too old, but there was opposition [1]. There was
also a suggestion to have a script in LLVM to download and build CMake for you,
but there were mixed opinions on this too [2], particularly since many
developers might prefer downloading a binary release to building from source
themselves (though of course the script could also download binary releases if
applicable). I personally think downloading or building CMake yourself isn’t a
high barrier for anyone wanting to build LLVM (and in particular it’s *much*
more straightforward than building LLVM itself), but I can understand why people
would prefer to stick to versions available in distros.

Another suggestion that came up last time was to set a policy for upgrading
CMake versions on some regular basis. The opposition to this was that we should
upgrade CMake versions only when a newer version has a compelling enough feature
to justify upgrading, rather than always upgrading. I can see arguments for both
approaches, but we should definitely at least think about the benefits we can
get from upgrading versions. I've gone through the CMake release notes and
highlighted features which seemed potentially valuable for LLVM. Note that
I'm only highlighting features for which our minimum CMake version would
have to be bumped up in order for our build system to take advantage of. There
are other useful features in newer CMake versions, but you can take advantage of
them just by using a newer CMake yourself. For example, 3.9 loosens the
dependencies of object compilation, which should result in faster Ninja builds.

CMake 3.5 (released March 8th 2016):
* install(DIRECTORY) supports generator expressions

CMake 3.6 (released July 7th 2016):
* install() supports EXCLUDE_FROM_ALL
* list() supports FILTER to filter by regular expression
* Subninja support, which could theoretically be used for much faster runtimes
builds, although in practice we probably want to make ExternalProject support
this directly instead of trying to layer our own meta-build system on top
* CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static library
instead of an executable, which will greatly simplify the compiler-rt build

CMake 3.7 (released November 11th 2016):
* New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL

CMake 3.8 (released April 10th 2017):
* Compile features for C++17, which is required to build libc++ correctly
* Support for compile features for specific C++ features instead of only being
able to specify standard versions
* rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH variable
* Apple framework support for static libraries
* New swig_add_library command in the UseSWIG module
* New generator expression $<IF:cond,true-value,false-value>

CMake 3.9 (released July 18th 2017):
* install(TARGETS) and install(EXPORTS) support for object libraries, which will
simplify the compiler-rt build
* TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
* $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator expressions for Apple bundles

CMake 3.10 (released November 20th 2017):
* include_guard() command for proper guarding against double includes of CMake
scripts
* An interesting aside is that this is the first verion of CMake to require
C++11 to build, which should give a good sense of how conservative they are
about compiler requirements

CMake 3.11 (released March 28th 2018):
* add_library() and add_executable() can be called without sources as long as
target_sources() is used later
* target_compile_{definitions,features,options}, target_include_directories(),
target_sources(), and target_link_libraries() can set the corresponding
INTERFACE_* properties on imported targets
* COMPILE_DEFINITIONS supports generator expressions
* COMPILE_OPTIONS source file property added
* INCLUDE_DIRECTORIES source file property added
* Interface libraries support custom properites

CMake 3.12 (released July 17th 2018):
* add_compile_definitions() added to add compile definitions for targets (to
avoid the global pollution caused by add_definitions())
* cmake_minimum_required() supports a version range to indicate tested CMake
versions and set policies accordingly
* file(TOUCH) and file(TOUCH_NOCREATE) added
* list(JOIN), list(SUBLIST) and list(TRANSFORM) added
* string(JOIN) added
* SHELL: prefix support in target_compile_options to avoid errant deduplication
* target_link_libraries() supports object libraries and propagates usage
requirements
* EXPORT_PROPERTIES target property to control the target properties exported by
export() and install(EXPORT)
* FindLibXml2 provides imported targets
* New FindPython, FindPython2, and FindPython3 modules to ease location Python
and selecting a specific version
* Modernization of UseSWIG module
* New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
* Compile features support for C++20

CMake 3.13 (released November 20th 2018):
* cmake -E create_symlink supported on Windows
* target_link_directories() and target_link_options() commands to set link
options instead of awkwardly having to use target_link_libraries() for this
purpose
* UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation

CMake 3.14 (released March 14th 2019):
* file(CREATE_LINK) to create hard or symbolic links
* if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
* $<IN_LIST:...> generator expression correctly handles empty argument
* Fixes for object library linking propagation
* Link options to manage position independent executables added automatically

CMake 3.15 (released July 17th 2019):
* list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
* New message() types NOTICE, VERBOSE, DEBUG and TRACE
* string(REPEAT) added
* MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY variable
to select the runtime library type for MSVC
* $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
* $<COMPILE_LANG_AND_ID:...> generator expression added
* $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
* $<REMOVE_DUPLICATES:list> generator expression added
* New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
* $<TARGET_OBJECTS:...> generator expression supports executables and
static, shared, and module libraries

CMake 3.16 (released November 26th 2019):
* Support for generator expressions in BUILD_RPATH and INSTALL_RPATH

CMake 3.17 (released March 20th 2020):
* Ninja Multi-Config generator, which among other things would greatly simplify
LLVM_OPTIMIZED_TABLEGEN
* foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
* New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
* INSTALL_NAME_DIR supports generator expressions

Our build system is incredibly complex, and many of these features can be used
to clean it up and make it much more maintainable. I would personally like us to
at least bump up to CMake 3.12. I also do think it's worth establishing a
policy and process around upgrading CMake versions, since newer versions keep on
adding useful features (particularly better generator expression support), and
we want to be able to keep taking advantage of them.

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html
[2] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200402/be490b62/attachment.html>

I’m in favor of all this. Thanks for volunteering! I’m happy to help out in
whatever way.

Some things it might be worth figuring out for future upgrades:
* If we want to limit ourselves to CMake versions supported by LTS releases of
distros, which distros should we consider, and how far back should we go (i.e.
is it just the latest LTS or the last two LTS versions)?
* For platforms like Ubuntu where CMake publishes its own packages (that you can
install via the platform’s package manager), do those count, or do we only
consider the CMake that comes in the OS packages?
* Do we have any limitations around how often/when we upgrade? You’re tying the
upgrade to after the branch, which is pretty standard, but e.g. if we wanted to
upgrade to 3.8.0 now and then upgrade to 3.13.4 after the branch, would people
be okay with that, or should we limit upgrades to just shortly after a branch?

From: <ldionne at apple.com> on behalf of Louis Dionne <ldionne at
apple.com>
Date: Thursday, April 2, 2020 at 7:20 AM
To: Shoaib Meenai <smeenai at fb.com>
Cc: "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>,
Chris Bieneman <beanz at apple.com>, Petr Hosek <phosek at
chromium.org>, Saleem Abdulrasool <compnerd at compnerd.org>,
"tstellar at redhat.com" <tstellar at redhat.com>
Subject: Re: Upgrading LLVM's minimum required CMake version

Okay, so we've had some discussion on this thread, and although some people
(including me) would like a more aggressive policy, I believe the following will
not get any objection (based on the thread). On April 23rd 2020, Ubuntu 20.04
LTS will ship with CMake 3.16.x. This will make the lower bound for LTS
distributions be 3.13.4, and so I suggest we upgrade to that. Here's a
proposed process:

1. Immediately add a CMake warning in <root>/llvm/CMakeLists.txt saying
that CMake 3.13.4 will be the new minimum version starting with LLVM 12.0.0, and
mentioning the versions used in various LTSes.
2. Immediately send a courtesy heads-up email to all build-bot owners telling
them about the upcoming change.
3. Right after we branch off the release branch for LLVM 11.0.0 (the next one),
make the minimum CMake version required be 3.13.4.
4. Iterate on (3) until all bots are migrated.
5. Send a message to the list saying the bump is complete. At that time,
projects are free to start using features from 3.13.4.

Unless someone else absolutely wants to bite the bullet, I volunteer to do the
above steps.

Thoughts?

Louis


On Mar 26, 2020, at 16:07, Shoaib Meenai <smeenai at fb.com<mailto:smeenai
at fb.com>> wrote:

We had this discussion a few months ago and it petered out, and it’s recently
been revived in the context of upgrading the CMake version specifically for
libc++ (at which point people suggested upgrading the CMake version used by all
of LLVM), so let’s try to move this forward.

Our current required minimum version is CMake 3.4.3, which was released on
January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3 on
May 31st 2016, which was just 4 months after its release.

Let’s look at the CMake versions available on various distros and operating
systems. I’m unfamiliar with many of these, so I apologize if I get something
wrong. (I’m using
pkgs.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=S6Qvuq5DqECZFfItkAJOL5xjTSp1psRWYtq_WOnXt_o&e=>
for most of this information.)
* RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
* RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
* RHEL 8 (released May 7th 2019): 3.11.4 (maybe
pkgs.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__pkgs.org&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=S6Qvuq5DqECZFfItkAJOL5xjTSp1psRWYtq_WOnXt_o&e=>
is screwy on this one, because it doesn’t make sense that RHEL 7 should have a
newer available version than RHEL 8)
* Debian 9 (released June 17th 2017): 3.7.2
* Debian 10 (released July 6th 2019): 3.13.4
* Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
* Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
* FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded in a
point release)
* FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded in a
point release)
* NetBSD 8.1 (released May 31st 2019): 3.16.1
* NetBSD 9.0 (released February 14th 2020): 3.16.1
* OpenBSD: couldn’t find the version
* macOS: latest version is readily available through Homebrew
* Windows: You can install it yourself or use the one bundled with Visual
Studio. I don't know what versions are bundled with Visual Studio; some
searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019 has
3.15, though I have no confirmation of that.

Note that CMake provides prebuilt binaries for Linux, macOS, and Windows, and
it’s also straightforward to build from source (it has very conservative
compiler requirements). One suggestion that was brought up in the past was for
LLVM’s build system to just download a newer version of CMake if you attempted
to build it using one that was too old, but there was opposition [1]. There was
also a suggestion to have a script in LLVM to download and build CMake for you,
but there were mixed opinions on this too [2], particularly since many
developers might prefer downloading a binary release to building from source
themselves (though of course the script could also download binary releases if
applicable). I personally think downloading or building CMake yourself isn’t a
high barrier for anyone wanting to build LLVM (and in particular it’s *much*
more straightforward than building LLVM itself), but I can understand why people
would prefer to stick to versions available in distros.

Another suggestion that came up last time was to set a policy for upgrading
CMake versions on some regular basis. The opposition to this was that we should
upgrade CMake versions only when a newer version has a compelling enough feature
to justify upgrading, rather than always upgrading. I can see arguments for both
approaches, but we should definitely at least think about the benefits we can
get from upgrading versions. I've gone through the CMake release notes and
highlighted features which seemed potentially valuable for LLVM. Note that
I'm only highlighting features for which our minimum CMake version would
have to be bumped up in order for our build system to take advantage of. There
are other useful features in newer CMake versions, but you can take advantage of
them just by using a newer CMake yourself. For example, 3.9 loosens the
dependencies of object compilation, which should result in faster Ninja builds.

CMake 3.5 (released March 8th 2016):
* install(DIRECTORY) supports generator expressions

CMake 3.6 (released July 7th 2016):
* install() supports EXCLUDE_FROM_ALL
* list() supports FILTER to filter by regular expression
* Subninja support, which could theoretically be used for much faster runtimes
builds, although in practice we probably want to make ExternalProject support
this directly instead of trying to layer our own meta-build system on top
* CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static library
instead of an executable, which will greatly simplify the compiler-rt build

CMake 3.7 (released November 11th 2016):
* New if() comparison operators LESS_EQUAL, GREATER_EQUAL, STRLESS_EQUAL,
STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL

CMake 3.8 (released April 10th 2017):
* Compile features for C++17, which is required to build libc++ correctly
* Support for compile features for specific C++ features instead of only being
able to specify standard versions
* rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH variable
* Apple framework support for static libraries
* New swig_add_library command in the UseSWIG module
* New generator expression $<IF:cond,true-value,false-value>

CMake 3.9 (released July 18th 2017):
* install(TARGETS) and install(EXPORTS) support for object libraries, which will
simplify the compiler-rt build
* TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
* $<TARGET_BUNDLE_DIR:tgt> and $<TARGET_BUNDLE_CONTENT_DIR:tgt>
generator expressions for Apple bundles

CMake 3.10 (released November 20th 2017):
* include_guard() command for proper guarding against double includes of CMake
scripts
* An interesting aside is that this is the first verion of CMake to require
C++11 to build, which should give a good sense of how conservative they are
about compiler requirements

CMake 3.11 (released March 28th 2018):
* add_library() and add_executable() can be called without sources as long as
target_sources() is used later
* target_compile_{definitions,features,options}, target_include_directories(),
target_sources(), and target_link_libraries() can set the corresponding
INTERFACE_* properties on imported targets
* COMPILE_DEFINITIONS supports generator expressions
* COMPILE_OPTIONS source file property added
* INCLUDE_DIRECTORIES source file property added
* Interface libraries support custom properites

CMake 3.12 (released July 17th 2018):
* add_compile_definitions() added to add compile definitions for targets (to
avoid the global pollution caused by add_definitions())
* cmake_minimum_required() supports a version range to indicate tested CMake
versions and set policies accordingly
* file(TOUCH) and file(TOUCH_NOCREATE) added
* list(JOIN), list(SUBLIST) and list(TRANSFORM) added
* string(JOIN) added
* SHELL: prefix support in target_compile_options to avoid errant deduplication
* target_link_libraries() supports object libraries and propagates usage
requirements
* EXPORT_PROPERTIES target property to control the target properties exported by
export() and install(EXPORT)
* FindLibXml2 provides imported targets
* New FindPython, FindPython2, and FindPython3 modules to ease location Python
and selecting a specific version
* Modernization of UseSWIG module
* New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
* Compile features support for C++20

CMake 3.13 (released November 20th 2018):
* cmake -E create_symlink supported on Windows
* target_link_directories() and target_link_options() commands to set link
options instead of awkwardly having to use target_link_libraries() for this
purpose
* UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation

CMake 3.14 (released March 14th 2019):
* file(CREATE_LINK) to create hard or symbolic links
* if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
* $<IN_LIST:...> generator expression correctly handles empty argument
* Fixes for object library linking propagation
* Link options to manage position independent executables added automatically

CMake 3.15 (released July 17th 2019):
* list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
* New message() types NOTICE, VERBOSE, DEBUG and TRACE
* string(REPEAT) added
* MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY variable
to select the runtime library type for MSVC
* $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
* $<COMPILE_LANG_AND_ID:...> generator expression added
* $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
* $<REMOVE_DUPLICATES:list> generator expression added
* New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
* $<TARGET_OBJECTS:...> generator expression supports executables and
static, shared, and module libraries

CMake 3.16 (released November 26th 2019):
* Support for generator expressions in BUILD_RPATH and INSTALL_RPATH

CMake 3.17 (released March 20th 2020):
* Ninja Multi-Config generator, which among other things would greatly simplify
LLVM_OPTIMIZED_TABLEGEN
* foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
* New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
* INSTALL_NAME_DIR supports generator expressions

Our build system is incredibly complex, and many of these features can be used
to clean it up and make it much more maintainable. I would personally like us to
at least bump up to CMake 3.12. I also do think it's worth establishing a
policy and process around upgrading CMake versions, since newer versions keep on
adding useful features (particularly better generator expression support), and
we want to be able to keep taking advantage of them.

[1]
http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_pipermail_llvm-2Ddev_2019-2DNovember_136485.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=HtGj57-MndDqyK71vXRwheQXms3WKx9rT-8WAVyTB3c&e=>
[2]
http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_pipermail_llvm-2Ddev_2019-2DNovember_136488.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=atRWn0u_rLa9ITqRSW-W8QvRJN244hhQmDWillcW3gE&s=hHrZYrGk0WStJ3TjjIsXg2NMvVUP-f4woTFtaFlYkG8&e=>

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200402/e86f2d85/attachment.html>

> On Apr 2, 2020, at 10:19, Louis Dionne via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Okay, so we've had some discussion on this thread, and although some
people (including me) would like a more aggressive policy, I believe the
following will not get any objection (based on the thread). On April 23rd 2020,
Ubuntu 20.04 LTS will ship with CMake 3.16.x. This will make the lower bound for
LTS distributions be 3.13.4, and so I suggest we upgrade to that. Here's a
proposed process:
> 
> 1. Immediately add a CMake warning in <root>/llvm/CMakeLists.txt
saying that CMake 3.13.4 will be the new minimum version starting with LLVM
12.0.0, and mentioning the versions used in various LTSes.
> 2. Immediately send a courtesy heads-up email to all build-bot owners
telling them about the upcoming change.
> 3. Right after we branch off the release branch for LLVM 11.0.0 (the next
one), make the minimum CMake version required be 3.13.4.
> 4. Iterate on (3) until all bots are migrated.
> 5. Send a message to the list saying the bump is complete. At that time,
projects are free to start using features from 3.13.4.
> 
> Unless someone else absolutely wants to bite the bullet, I volunteer to do
the above steps.
Ok, so the thread has kept going, but AFAICT all the discussion is about a
policy for "automatic" upgrades. So I've went ahead and did the
first two steps of the above algorithm:

(1) Review at https://reviews.llvm.org/D77740
(2) Email sent to the owners of all build slaves listed on
http://lab.llvm.org:8011/buildslaves

Just to reiterate, this means that as soon as we branch LLVM 11.0.0 (the NEXT
release), CMake 3.13.4 will be the minimum required version.

Cheers,
Louis
>> On Mar 26, 2020, at 16:07, Shoaib Meenai <smeenai at fb.com
<mailto:smeenai at fb.com>> wrote:
>> 
>> We had this discussion a few months ago and it petered out, and it’s
recently been revived in the context of upgrading the CMake version specifically
for libc++ (at which point people suggested upgrading the CMake version used by
all of LLVM), so let’s try to move this forward.
>> 
>> Our current required minimum version is CMake 3.4.3, which was released
on January 25th 2016. It’s interesting to note that LLVM started requiring 3.4.3
on May 31st 2016, which was just 4 months after its release.
>> 
>> Let’s look at the CMake versions available on various distros and
operating systems. I’m unfamiliar with many of these, so I apologize if I get
something wrong. (I’m using pkgs.org <http://pkgs.org/> for most of this
information.)
>> * RHEL 6 (released Nov 10th 2010) : 3.6.1 (via EPEL)
>> * RHEL 7 (released June 10th 2014): 3.14.7 (via EPEL)
>> * RHEL 8 (released May 7th 2019): 3.11.4 (maybe pkgs.org
<http://pkgs.org/> is screwy on this one, because it doesn’t make sense
that RHEL 7 should have a newer available version than RHEL 8)
>> * Debian 9 (released June 17th 2017): 3.7.2
>> * Debian 10 (released July 6th 2019): 3.13.4
>> * Ubuntu 16.04 LTS (released April 21st 2016): 3.5.1
>> * Ubuntu 18.04 LTS (released April 26th 2018): 3.10.2
>> * FreeBSD 11 (released October 10th 2016): 3.15.5 (presumably upgraded
in a point release)
>> * FreeBSD 12 (released December 11th 2018): 3.15.5 (presumably upgraded
in a point release)
>> * NetBSD 8.1 (released May 31st 2019): 3.16.1
>> * NetBSD 9.0 (released February 14th 2020): 3.16.1
>> * OpenBSD: couldn’t find the version
>> * macOS: latest version is readily available through Homebrew
>> * Windows: You can install it yourself or use the one bundled with
Visual Studio. I don't know what versions are bundled with Visual Studio;
some searching suggests Visual Studio 2017 has CMake 3.12 and Visual Studio 2019
has 3.15, though I have no confirmation of that.
>> 
>> Note that CMake provides prebuilt binaries for Linux, macOS, and
Windows, and it’s also straightforward to build from source (it has very
conservative compiler requirements). One suggestion that was brought up in the
past was for LLVM’s build system to just download a newer version of CMake if
you attempted to build it using one that was too old, but there was opposition
[1]. There was also a suggestion to have a script in LLVM to download and build
CMake for you, but there were mixed opinions on this too [2], particularly since
many developers might prefer downloading a binary release to building from
source themselves (though of course the script could also download binary
releases if applicable). I personally think downloading or building CMake
yourself isn’t a high barrier for anyone wanting to build LLVM (and in
particular it’s *much* more straightforward than building LLVM itself), but I
can understand why people would prefer to stick to versions available in
distros.
>> 
>> Another suggestion that came up last time was to set a policy for
upgrading CMake versions on some regular basis. The opposition to this was that
we should upgrade CMake versions only when a newer version has a compelling
enough feature to justify upgrading, rather than always upgrading. I can see
arguments for both approaches, but we should definitely at least think about the
benefits we can get from upgrading versions. I've gone through the CMake
release notes and highlighted features which seemed potentially valuable for
LLVM. Note that I'm only highlighting features for which our minimum CMake
version would have to be bumped up in order for our build system to take
advantage of. There are other useful features in newer CMake versions, but you
can take advantage of them just by using a newer CMake yourself. For example,
3.9 loosens the dependencies of object compilation, which should result in
faster Ninja builds.
>> 
>> CMake 3.5 (released March 8th 2016):
>> * install(DIRECTORY) supports generator expressions
>> 
>> CMake 3.6 (released July 7th 2016):
>> * install() supports EXCLUDE_FROM_ALL
>> * list() supports FILTER to filter by regular expression
>> * Subninja support, which could theoretically be used for much faster
runtimes builds, although in practice we probably want to make ExternalProject
support this directly instead of trying to layer our own meta-build system on
top
>> * CMAKE_TRY_COMPILE_TARGET_TYPE to tell try_compile to build a static
library instead of an executable, which will greatly simplify the compiler-rt
build
>> 
>> CMake 3.7 (released November 11th 2016):
>> * New if() comparison operators LESS_EQUAL, GREATER_EQUAL,
STRLESS_EQUAL, STRGREATER_EQUAL, VERSION_LESS_EQUAL, and VERSION_GREATER_EQUAL
>> 
>> CMake 3.8 (released April 10th 2017):
>> * Compile features for C++17, which is required to build libc++
correctly
>> * Support for compile features for specific C++ features instead of
only being able to specify standard versions
>> * rpath support via BUILD_RPATH target property and CMAKE_BUILD_RPATH
variable
>> * Apple framework support for static libraries
>> * New swig_add_library command in the UseSWIG module
>> * New generator expression $<IF:cond,true-value,false-value>
>> 
>> CMake 3.9 (released July 18th 2017):
>> * install(TARGETS) and install(EXPORTS) support for object libraries,
which will simplify the compiler-rt build
>> * TARGET_OBJECTS generator expression support in add_custom_command and
file(GENERATE)
>> * $<TARGET_BUNDLE_DIR:tgt> and
$<TARGET_BUNDLE_CONTENT_DIR:tgt> generator expressions for Apple bundles
>> 
>> CMake 3.10 (released November 20th 2017):
>> * include_guard() command for proper guarding against double includes
of CMake scripts
>> * An interesting aside is that this is the first verion of CMake to
require C++11 to build, which should give a good sense of how conservative they
are about compiler requirements
>> 
>> CMake 3.11 (released March 28th 2018):
>> * add_library() and add_executable() can be called without sources as
long as target_sources() is used later
>> * target_compile_{definitions,features,options},
target_include_directories(), target_sources(), and target_link_libraries() can
set the corresponding INTERFACE_* properties on imported targets
>> * COMPILE_DEFINITIONS supports generator expressions
>> * COMPILE_OPTIONS source file property added
>> * INCLUDE_DIRECTORIES source file property added
>> * Interface libraries support custom properites
>> 
>> CMake 3.12 (released July 17th 2018):
>> * add_compile_definitions() added to add compile definitions for
targets (to avoid the global pollution caused by add_definitions())
>> * cmake_minimum_required() supports a version range to indicate tested
CMake versions and set policies accordingly
>> * file(TOUCH) and file(TOUCH_NOCREATE) added
>> * list(JOIN), list(SUBLIST) and list(TRANSFORM) added
>> * string(JOIN) added
>> * SHELL: prefix support in target_compile_options to avoid errant
deduplication
>> * target_link_libraries() supports object libraries and propagates
usage requirements
>> * EXPORT_PROPERTIES target property to control the target properties
exported by export() and install(EXPORT)
>> * FindLibXml2 provides imported targets
>> * New FindPython, FindPython2, and FindPython3 modules to ease location
Python and selecting a specific version
>> * Modernization of UseSWIG module
>> * New generator expressions $<GENEX_EVAL:...>,
$<TARGET_GENEX_EVAL:target,...>, $<IN_LIST:...>,
$<TARGET_EXISTS:...> and $<TARGET_NAME_IF_EXISTS:...>
>> * Compile features support for C++20
>> 
>> CMake 3.13 (released November 20th 2018):
>> * cmake -E create_symlink supported on Windows
>> * target_link_directories() and target_link_options() commands to set
link options instead of awkwardly having to use target_link_libraries() for this
purpose
>> * UseSWIG can manage INCLUDE_DIRECTORIES for SWIG compilation
>> 
>> CMake 3.14 (released March 14th 2019):
>> * file(CREATE_LINK) to create hard or symbolic links
>> * if(DEFINED CACHE{VAR}) for checking if a cache variable is defined
>> * $<IN_LIST:...> generator expression correctly handles empty
argument
>> * Fixes for object library linking propagation
>> * Link options to manage position independent executables added
automatically
>> 
>> CMake 3.15 (released July 17th 2019):
>> * list(PREPEND), list(POP_FRONT) and list(POP_BACK) added
>> * New message() types NOTICE, VERBOSE, DEBUG and TRACE
>> * string(REPEAT) added
>> * MSVC_RUNTIME_LIBRARY target property and CMAKE_MSVC_RUNTIME_LIBRARY
variable to select the runtime library type for MSVC
>> * $<C_COMPILER_ID:...>, $<CXX_COMPILER_ID:...>,
$<COMPILE_LANGUAGE:...>, and $<PLATFORM_ID:...> generator
expressions support matching one value from a list
>> * $<COMPILE_LANG_AND_ID:...> generator expression added
>> * $<FILTER:list,INCLUDE|EXCLUDE,regex> generator expression added
>> * $<REMOVE_DUPLICATES:list> generator expression added
>> * New $<TARGET_FILE*> generator expressions added:
$<TARGET_FILE_PREFIX:...>, $<TARGET_FILE_BASE_NAME:...>,
$<TARGET_FILE_SUFFIX:...>, $<TARGET_LINKER_FILE_PREFIX:...>,
$<TARGET_LINKER_FILE_BASE_NAME:...>,
$<TARGET_LINKER_FILE_SUFFIX:...>, $<TARGET_PDB_FILE_BASE_NAME:...>
>> * $<TARGET_OBJECTS:...> generator expression supports executables
and static, shared, and module libraries
>> 
>> CMake 3.16 (released November 26th 2019):
>> * Support for generator expressions in BUILD_RPATH and INSTALL_RPATH
>> 
>> CMake 3.17 (released March 20th 2020):
>> * Ninja Multi-Config generator, which among other things would greatly
simplify LLVM_OPTIMIZED_TABLEGEN
>> * foreach(ZIP_LISTS) added to iterate multiple lists simultaneously
>> * New message() keywords CHECK_START, CHECK_PASS, and CHECK_FAIL
>> * INSTALL_NAME_DIR supports generator expressions
>> 
>> Our build system is incredibly complex, and many of these features can
be used to clean it up and make it much more maintainable. I would personally
like us to at least bump up to CMake 3.12. I also do think it's worth
establishing a policy and process around upgrading CMake versions, since newer
versions keep on adding useful features (particularly better generator
expression support), and we want to be able to keep taking advantage of them.
>> 
>> [1] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html
<http://lists.llvm.org/pipermail/llvm-dev/2019-November/136485.html>
>> [2] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html
<http://lists.llvm.org/pipermail/llvm-dev/2019-November/136488.html>
>> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200408/9897189d/attachment.html>