> On Mar 3, 2021, at 14:17, Mehdi AMINI <joker.eph at gmail.com> wrote:
>
>
>
> On Wed, Mar 3, 2021 at 10:32 AM David Blaikie <dblaikie at gmail.com
<mailto:dblaikie at gmail.com>> wrote:
> On Wed, Mar 3, 2021 at 9:31 AM Mehdi AMINI via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hi,
>
> It seems to me that this would require one extra stage of bootstrap in CI
for many buildbots.
> For example, today I have a Linux bot with a clang-8 host compiler and
libstdc++. The goal is to ensure that MLIR (but it is applicable to any project)
builds with clang and libc++ at the top of the main branch.
> So the setup is:
> - stage1: build clang/libc++ with host clang-8/libstdc++
> - stage2: build test "anything" using stage1 (`ninja check-all`
in the monorepo for example, but applicable to any other external project)
>
> With this proposal, the setup would be:
>
> - stage1: build just clang with host clang-8/libstdc++
> - stage2: build clang/libc++ with stage1 clang and host libstdc++
> - stage3: build test "anything" using stage2 (`ninja check-all`
in the monorepo for example, but applicable to any other external project)
>
> Would it be possible to change the build system so that libc++ can be built
like compiler-rt, using the just-built clang? That would then avoid the need for
the extra stage? (though it would bottleneck the usual build a bit - not being
able to start the libc++ build until after clang build)
>
> That's a good point:
> - stage1: build just clang with host clang-8/libstdc++
> - stage1.5: build libc++ with stage1 clang
> - stage 2: assemble toolchain with clang from stage1 and libc++ from stage2
> - stage3: build test "anything" using stage2 (`ninja check-all`
in the monorepo for example, but applicable to any other external project)
>
> Since this "stage 2" is the new "stage1", I believe
that this should be made completely straightforward to achieve. Ideally it
should boil down to a single standard CMake invocation to produce this
configuration.
I think the Runtimes build is exactly what you’re looking for. With the runtimes
build, you say:
$ cmake -S "${MONOREPO_ROOT}/llvm" -B "${BUILD_DIR}” \
-DLLVM_ENABLE_PROJECTS="clang” \
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi” \
-DLLVM_RUNTIME_TARGETS="x86_64-unknown-linux-gnu”
And then you can just do:
$ make -C $BUILD_DIR cxx
That will bootstrap Clang and then build libc++ with the just-built Clang. I
don’t know whether you consider that to be one or two stages, but it happens
automatically in that single CMake invocation. And since building libc++ is
basically trivial, this takes approximately the same time as building Clang
only.
>
>
> & again, this isn't so much a proposal of change, but one of
documenting the current state of things - which reveals the current situations
are sort of unsupported? (though it also goes against the claim that they're
untested) - so I'll be curious to hear from the libc++ folks about this for
sure.
>
> Right: I'm absolutely not convinced by the "we're documenting
the current state of things" actually.
> In particular my take in general on what we call "supported" is a
policy that "we revert if we break a supported configuration" and
"we accept patches to fix a supported configuration". So the change
here is that libc++ would not accept to revert when they break an older
toolchain, and we wouldn't accept patches to libc++ to fix it.
> We don't necessarily have buildbots for every configuration that we
claim LLVM is supporting, yet this is the policy, and I'm quite wary of
defining the "current state of things" based exclusively on the
current public buildbots setup.
To be clear, what we do today to “fix” older compilers is usually to mark
failing tests in the test suite with XFAIL or UNSUPPORTED annotations. We don’t
actually provide a good level of support for those compilers. There’s also other
things that we simply can’t fix, like the fact that a libc++ built with a
compiler that doesn’t know about char8_t (for example) won’t produce the RTTI
for char8_t in the dylib, and hence will produce a dylib where some random uses
of char8_t will break down. This is just an example, but my point is that it’s
far better to clarify the support policy to something that *we know* will work,
and that we can commit to supporting. There's a small upfront cost for
people running build bots right now, but once things are setup it’ll just be
better for everyone.
>
>
>
> The only way to avoid adding a stage in the bootstrap is to keep updating
the bots with a very recent host clang (I'm not convinced that increasing
the cost of maintenance for CI / infra is good in general).
>
> We should aim for a better balance: it is possible that clang-5 is too old
(I don't know?), but there are people (like me, and possibly others) who are
testing HEAD with older compiler (clang-8 here) and it does not seem broken at
the moment (or the recent years), I feel there should be a strong motivation to
break it.
Libc++ on Clang 8 doesn’t look broken because it builds. And it builds because
you’ve been pinging us on Phabricator when we break you with a change, and we
add a “workaround” that makes it build. But there’s no guarantee about the
“quality" of the libc++ that you get in that case though. That’s exactly
what we want to avoid - you get something that “kinda works”, yet we still have
to insert random workarounds in the code. It’s a lose/lose situation.
> Could we find something more intermediate here? Like a time-based support
(2 years?) or something based on the latest Ubuntu release or something like
that. That would at least keep the cost of upgrading bots a bit more controlled
(and avoid a costly extra stage of bootstrap).
As I said above, I don’t think there’s any extra stage of bootstrap. The only
difference is that you build your libc++ using the Clang you just built, instead
of against the system compiler. In both cases you need to build both Clang and
libc++ anyway.
Furthermore, we specifically support the last released Clang. If you were in a
situation where you didn’t want to build Clang but wanted to build libc++, you’d
just have to download a sufficiently recent Clang release and use that.
Louis
>
> On Tue, Mar 2, 2021 at 7:10 AM Louis Dionne via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>
>
> > On Mar 1, 2021, at 15:41, Joerg Sonnenberger via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> >
> > On Mon, Mar 01, 2021 at 12:40:36PM -0500, Louis Dionne via llvm-dev
wrote:
> >> However, for a library like libc++, things are a bit different.
> >
> > So how does this prevent the libstdc++ mess that you need to lock step
> > the RTL with the compiler and more importantly, get constantly screwed
> > over when you need to upgrade or downgrade the compiler in a complex
> > environment like an actual Operating System?
>
> Could you please elaborate on what issue you’re thinking about here? As
someone who ships libc++ as part of an operating system and SDK (which isn’t
necessarily in perfect lockstep with the compiler), I don’t see any issues. The
guarantee that you can still use a ~6 months old Clang is specifically intended
to allow for that use case, i.e. shipping libc++ as part of an OS instead of a
toolchain.
>
>
> > I consider this proposal a major step backwards...
>
> To be clear, we only want to make official the level of support that we
already provide in reality. As I explained in my original email, if you’ve been
relying on libc++ working on much older compilers, I would suggest that you stop
doing so because nobody is testing that and we don’t really support it, despite
what the documentation says. So IMO this can’t be a step backwards, since we
already don’t support these compilers, we just pretend that we do.
>
> Louis
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210303/def47573/attachment-0001.html>