llvm dev - Oct 2021 - Upcoming change with how libc++, libc++abi and libunwind are being built

Hi folks,

This message is both a heads up and a request for comments on an upcoming
change we want to make to the way we build libc++/libc++abi/libunwind.

For a long time now, those three projects have supported various ways to be
built:

1. The Project build, where one would use <root>/llvm as the root of the
CMake invocation and pass LLVM_ENABLE_PROJECTS.
2. The Standalone build, where one would perform one CMake invocation per
project being built, and fill in various CMake variables to tell each
project where to find bits of the other projects.
3. The "Runtimes" or "Bootstrapping" build, where one would
use <root>/llvm
as the root of the CMake invocation and pass LLVM_ENABLE_RUNTIMES. This
would result in Clang being built, and then the runtimes being built with
that fresh Clang.
4. The "Unified Standalone" build, where one would use
<root>/libcxx/utils/ci/runtimes as the root of the CMake invocation and
pass LLVM_ENABLE_PROJECTS. This was introduced to work around the fact that
the Project build doesn't work on several embedded platforms, and I think
it is only used at Apple.

Through the years, this has gotten rather confusing and difficult to
support. We currently support all of those on a best-effort basis and we do
have build bots checking each of them, however the complexity is often
getting in the way of providing proper support and improving the build
system of those projects. We've been trying to migrate to a simpler way to
build the runtimes that works for all use cases (notably embedded
platforms) for a long time, and we recently got to a point where this is
possible (thanks mainly to @phosek and @mstorsjo). To summarize what we're
aiming for, we'll now have two ways of building libc++/lib++abi/libunwind:

1. The "Default" build, where the CMake invocation is rooted at
<root>/runtimes and one uses LLVM_ENABLE_RUNTIMES.
2. The "Bootstrapping" build, where the CMake invocation is rooted at
<root>/llvm and one uses LLVM_ENABLE_RUNTIMES (this is the current
"Bootstrapping" build unchanged).

If you have been using the "Project", "Standalone" or
"Unified Standalone"
builds, you should be moving to the new "Default" build. This should
be
really easy to achieve, basically you can just change LLVM_ENABLE_PROJECTS
to LLVM_ENABLE_RUNTIMES, perform a single invocation of CMake instead of
multiple invocations, and pass all the LIBCXX_FOO, LIBCXXABI_FOO and
LIBUNWIND_FOO options to that single CMake invocation. See the release note
in the PR linked below for more details.

We are aiming to deprecate all the old ways to build as soon as possible,
and to keep supporting them until after the LLVM 14 release, at which point
we would like to remove support for those and get started with making
improvements. The review at https://reviews.llvm.org/D111356 enshrines this
deprecation in the documentation. For technical discussion on this proposed
change, please comment on that review to keep everything in one place.

Cheers,
Louis

Questions you might be asking yourself:

Q: Why doesn't the Project build work for embedded platforms?
A: A lot of CMake code in <root>/llvm expects that we're building for
a
target with the usual capabilities for a build host, but that isn't the
case for several embedded targets. While it doesn't make sense to build
LLVM or Clang for those embedded targets (which would never be used as
build hosts), it makes sense to build the runtimes for those targets if you
are cross-compiling.

Q: Currently, we use the Standalone build and are not required to have all
of the Monorepo available. Does this change our ability to do that?
A: Not in the mid term. Building any of the runtimes already requires all
the runtimes to be co-located. The only thing you could get away with
previously was not having the part in <root>/llvm. With the
"Default"
build, this will be possible as soon as we move a few CMake helpers from
<root>/llvm/cmake to <root>/cmake. Then, you'll only have to
have
<root>/runtimes, <root>/cmake and the projects you are building.

Q: What's the difference between the "Runtimes build" and the
"Bootstrapping build"?
A: None. We're trying to move away from the term "Runtimes build"
because
it's waaaay too confusing, and doesn't describe the fact that we're
really
building the runtimes with a just-built Clang.

Q: What's up with naming the new default build the "Default
build"?
A: The name sucks, but we had to pick one and the "Runtimes build" was
already taken for the build that does a bootstrap. Suggestions are welcome
-- basically what this performs is a normal build of the runtimes using
your system compiler, like you used to do with LLVM_ENABLE_PROJECTS (but
it's better).

Q: What are the benefits of this migration beyond simplifying the code for
maintainers?
A: Reducing the number of ways we can build the runtimes means that all the
workflows for building the runtimes will become simpler for users
(especially vendors). Furthermore, the new "Default" build is as
simple as
the old Project build, but it is faster to configure and works on embedded
platforms. In the near future, it should also be possible to perform the
equivalent of an old "Project build" without having all of the
monorepo
available.

Q: Does this affect other "runtimes" projects like compiler-rt?
A: Not for the time being. We will probably attempt to move those over in
the future as well, however for the time being I can only confirm that this
works fine for libc++, libc++abi and libunwind, so that's where my
deprecation efforts are focused.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211011/6f8bce3a/attachment.html>

What should the buildbots do after this change? Should they all use
the bootstrap build? I am worried that this would reduces the test
coverage since it would only build the runtimes with the top-of-tree
clang. As by [1], at least libc++ supports also compilation with gcc
and AppleClang.

Name suggestion for the "default" build: "single target"
build. Since
only the bootstrap build supports multiple targets with
LLVM_RUNTIME_TARGETS.

[1] https://libcxx.llvm.org/

Am Mo., 11. Okt. 2021 um 11:18 Uhr schrieb Louis Dionne via llvm-dev
<llvm-dev at lists.llvm.org>:
>
> Hi folks,
>
> This message is both a heads up and a request for comments on an upcoming
change we want to make to the way we build libc++/libc++abi/libunwind.
>
> For a long time now, those three projects have supported various ways to be
built:
>
> 1. The Project build, where one would use <root>/llvm as the root of
the CMake invocation and pass LLVM_ENABLE_PROJECTS.
> 2. The Standalone build, where one would perform one CMake invocation per
project being built, and fill in various CMake variables to tell each project
where to find bits of the other projects.
> 3. The "Runtimes" or "Bootstrapping" build, where one
would use <root>/llvm as the root of the CMake invocation and pass
LLVM_ENABLE_RUNTIMES. This would result in Clang being built, and then the
runtimes being built with that fresh Clang.
> 4. The "Unified Standalone" build, where one would use
<root>/libcxx/utils/ci/runtimes as the root of the CMake invocation and
pass LLVM_ENABLE_PROJECTS. This was introduced to work around the fact that the
Project build doesn't work on several embedded platforms, and I think it is
only used at Apple.
>
> Through the years, this has gotten rather confusing and difficult to
support. We currently support all of those on a best-effort basis and we do have
build bots checking each of them, however the complexity is often getting in the
way of providing proper support and improving the build system of those
projects. We've been trying to migrate to a simpler way to build the
runtimes that works for all use cases (notably embedded platforms) for a long
time, and we recently got to a point where this is possible (thanks mainly to
@phosek and @mstorsjo). To summarize what we're aiming for, we'll now
have two ways of building libc++/lib++abi/libunwind:
>
> 1. The "Default" build, where the CMake invocation is rooted at
<root>/runtimes and one uses LLVM_ENABLE_RUNTIMES.
> 2. The "Bootstrapping" build, where the CMake invocation is
rooted at <root>/llvm and one uses LLVM_ENABLE_RUNTIMES (this is the
current "Bootstrapping" build unchanged).
>
> If you have been using the "Project", "Standalone" or
"Unified Standalone" builds, you should be moving to the new
"Default" build. This should be really easy to achieve, basically you
can just change LLVM_ENABLE_PROJECTS to LLVM_ENABLE_RUNTIMES, perform a single
invocation of CMake instead of multiple invocations, and pass all the
LIBCXX_FOO, LIBCXXABI_FOO and LIBUNWIND_FOO options to that single CMake
invocation. See the release note in the PR linked below for more details.
>
> We are aiming to deprecate all the old ways to build as soon as possible,
and to keep supporting them until after the LLVM 14 release, at which point we
would like to remove support for those and get started with making improvements.
The review at https://reviews.llvm.org/D111356 enshrines this deprecation in the
documentation. For technical discussion on this proposed change, please comment
on that review to keep everything in one place.
>
> Cheers,
> Louis
>
> Questions you might be asking yourself:
>
> Q: Why doesn't the Project build work for embedded platforms?
> A: A lot of CMake code in <root>/llvm expects that we're building
for a target with the usual capabilities for a build host, but that isn't
the case for several embedded targets. While it doesn't make sense to build
LLVM or Clang for those embedded targets (which would never be used as build
hosts), it makes sense to build the runtimes for those targets if you are
cross-compiling.
>
> Q: Currently, we use the Standalone build and are not required to have all
of the Monorepo available. Does this change our ability to do that?
> A: Not in the mid term. Building any of the runtimes already requires all
the runtimes to be co-located. The only thing you could get away with previously
was not having the part in <root>/llvm. With the "Default"
build, this will be possible as soon as we move a few CMake helpers from
<root>/llvm/cmake to <root>/cmake. Then, you'll only have to
have <root>/runtimes, <root>/cmake and the projects you are
building.
>
> Q: What's the difference between the "Runtimes build" and the
"Bootstrapping build"?
> A: None. We're trying to move away from the term "Runtimes
build" because it's waaaay too confusing, and doesn't describe the
fact that we're really building the runtimes with a just-built Clang.
>
> Q: What's up with naming the new default build the "Default
build"?
> A: The name sucks, but we had to pick one and the "Runtimes
build" was already taken for the build that does a bootstrap. Suggestions
are welcome -- basically what this performs is a normal build of the runtimes
using your system compiler, like you used to do with LLVM_ENABLE_PROJECTS (but
it's better).
>
> Q: What are the benefits of this migration beyond simplifying the code for
maintainers?
> A: Reducing the number of ways we can build the runtimes means that all the
workflows for building the runtimes will become simpler for users (especially
vendors). Furthermore, the new "Default" build is as simple as the old
Project build, but it is faster to configure and works on embedded platforms. In
the near future, it should also be possible to perform the equivalent of an old
"Project build" without having all of the monorepo available.
>
> Q: Does this affect other "runtimes" projects like compiler-rt?
> A: Not for the time being. We will probably attempt to move those over in
the future as well, however for the time being I can only confirm that this
works fine for libc++, libc++abi and libunwind, so that's where my
deprecation efforts are focused.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi Louis,

This sounds like a huge simplification, thank you!

With this change, will we still be able to use LLVM_ENABLE_PROJECTS to 
build libc++, libc++abi and libunwind? IIUC, you are disabling this 
option entirely, right? Will CMake warn us about the 
incorrect/deprecated usage? Should the LLVM documentation be updated [1]?

Thank you,
-Andrzej

[1] https://llvm.org/docs/CMake.html

On 11/10/2021 17:17, Louis Dionne via llvm-dev wrote:
> Hi folks,
> 
> This message is both a heads up and a request for comments on an 
> upcoming change we want to make to the way we build 
> libc++/libc++abi/libunwind.
> 
> For a long time now, those three projects have supported various ways to 
> be built:
> 
> 1. The Project build, where one would use <root>/llvm as the root of
the
> CMake invocation and pass LLVM_ENABLE_PROJECTS.
> 2. The Standalone build, where one would perform one CMake invocation 
> per project being built, and fill in various CMake variables to tell 
> each project where to find bits of the other projects.
> 3. The "Runtimes" or "Bootstrapping" build, where one
would use
> <root>/llvm as the root of the CMake invocation and pass 
> LLVM_ENABLE_RUNTIMES. This would result in Clang being built, and then 
> the runtimes being built with that fresh Clang.
> 4. The "Unified Standalone" build, where one would use 
> <root>/libcxx/utils/ci/runtimes as the root of the CMake invocation
and
> pass LLVM_ENABLE_PROJECTS. This was introduced to work around the fact 
> that the Project build doesn't work on several embedded platforms, and
I
> think it is only used at Apple.
> 
> Through the years, this has gotten rather confusing and difficult to 
> support. We currently support all of those on a best-effort basis and we 
> do have build bots checking each of them, however the complexity is 
> often getting in the way of providing proper support and improving the 
> build system of those projects. We've been trying to migrate to a 
> simpler way to build the runtimes that works for all use cases (notably 
> embedded platforms) for a long time, and we recently got to a point 
> where this is possible (thanks mainly to @phosek and @mstorsjo). To 
> summarize what we're aiming for, we'll now have two ways of
building
> libc++/lib++abi/libunwind:
> 
> 1. The "Default" build, where the CMake invocation is rooted at 
> <root>/runtimes and one uses LLVM_ENABLE_RUNTIMES.
> 2. The "Bootstrapping" build, where the CMake invocation is
rooted at
> <root>/llvm and one uses LLVM_ENABLE_RUNTIMES (this is the current 
> "Bootstrapping" build unchanged).
> 
> If you have been using the "Project", "Standalone" or
"Unified
> Standalone" builds, you should be moving to the new
"Default" build.
> This should be really easy to achieve, basically you can just change 
> LLVM_ENABLE_PROJECTS to LLVM_ENABLE_RUNTIMES, perform a single 
> invocation of CMake instead of multiple invocations, and pass all the 
> LIBCXX_FOO, LIBCXXABI_FOO and LIBUNWIND_FOO options to that single CMake 
> invocation. See the release note in the PR linked below for more details.
> 
> We are aiming to deprecate all the old ways to build as soon as 
> possible, and to keep supporting them until after the LLVM 14 release, 
> at which point we would like to remove support for those and get started 
> with making improvements. The review at https://reviews.llvm.org/D111356 
> <https://reviews.llvm.org/D111356> enshrines this deprecation in the 
> documentation. For technical discussion on this proposed change, please 
> comment on that review to keep everything in one place.
> 
> Cheers,
> Louis
> 
> Questions you might be asking yourself:
> 
> Q: Why doesn't the Project build work for embedded platforms?
> A: A lot of CMake code in <root>/llvm expects that we're building
for a
> target with the usual capabilities for a build host, but that isn't the
> case for several embedded targets. While it doesn't make sense to build
> LLVM or Clang for those embedded targets (which would never be used as 
> build hosts), it makes sense to build the runtimes for those targets if 
> you are cross-compiling.
> 
> Q: Currently, we use the Standalone build and are not required to have 
> all of the Monorepo available. Does this change our ability to do that?
> A: Not in the mid term. Building any of the runtimes already requires 
> all the runtimes to be co-located. The only thing you could get away 
> with previously was not having the part in <root>/llvm. With the 
> "Default" build, this will be possible as soon as we move a few
CMake
> helpers from <root>/llvm/cmake to <root>/cmake. Then,
you'll only have
> to have <root>/runtimes, <root>/cmake and the projects you are
building.
> 
> Q: What's the difference between the "Runtimes build" and the
> "Bootstrapping build"?
> A: None. We're trying to move away from the term "Runtimes
build"
> because it's waaaay too confusing, and doesn't describe the fact
that
> we're really building the runtimes with a just-built Clang.
> 
> Q: What's up with naming the new default build the "Default
build"?
> A: The name sucks, but we had to pick one and the "Runtimes
build" was
> already taken for the build that does a bootstrap. Suggestions are 
> welcome -- basically what this performs is a normal build of the 
> runtimes using your system compiler, like you used to do with 
> LLVM_ENABLE_PROJECTS (but it's better).
> 
> Q: What are the benefits of this migration beyond simplifying the code 
> for maintainers?
> A: Reducing the number of ways we can build the runtimes means that all 
> the workflows for building the runtimes will become simpler for users 
> (especially vendors). Furthermore, the new "Default" build is as
simple
> as the old Project build, but it is faster to configure and works on 
> embedded platforms. In the near future, it should also be possible to 
> perform the equivalent of an old "Project build" without having
all of
> the monorepo available.
> 
> Q: Does this affect other "runtimes" projects like compiler-rt?
> A: Not for the time being. We will probably attempt to move those over 
> in the future as well, however for the time being I can only confirm 
> that this works fine for libc++, libc++abi and libunwind, so that's 
> where my deprecation efforts are focused.
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>