llvm dev - Nov 2021 - Proposal: Make the VE target official

Hi Renato,

On Mon, 2021-11-08 at 10:09 +0000, Renato Golin wrote:
> On Wed, 3 Nov 2021 at 12:52, Simon Moll <Simon.Moll at emea.nec.com>
> wrote:
> > We propose the switch from 'experimental' backend status to
> > 'official',
> > 
> 
> 
> Hi Simon,
> 
> The VE community seems to have adopted all requirements laid out in
> the new targets' document[1], so I guess it's just a matter of
> ironing out the transition.
> 
> > * The staging builder (clang-ve-ninja) [2].  This builder builds
> > and
> >   checks LLVM+Clang and compiler-rt. The functional compiler-rt
> > tests
> >   run C code on a VE device.
> > 
> 
> 
> This bot seems red for a long time. One of the ideas that reflect a
> target to be stable is that its buildbot is "mostly" green
throughout
> the warmup period (and hopefully just before moving out of
> experimental).
If you look at the build logs of clang-ve-ninja, you will see that the
"check all" tests for LLVM+Clang have been passing for a while.
What's failing is compiler-rt and we have a patch for that.
> 
> > * D113093 [3] has the changes to make all compiler-rt tests pass on
> > VE.
> >   Remaining failures are due to denormal support, alignment
> >   requirements and subtle differences in syscalls.
> > 
> 
> 
> Will this make the bots green? It's ok to disable tests that fail for
> known reasons (or are irrelevant), but we shouldn't disable tests
> just to make the bots green to promote the target.
Yes, the compiler-rt tests are failing for well understood reasons
(documented in the patch - check-all on LLVM+Clang is green). The patch
will make compiler-rt pass on VE by accounting for those (no denorm
support, syscall differences).
We explicitly include compiler-rt testing (even though it is failing)
to have LLVM-compiled code running on the VE in CI.. this is not
something we'd do for slick optics.
> 
> > * The downstream reference implementation (LLVM-VE [4]).  This
> > includes
> >   compiler-rt,libcxxabi,libcxx and openmp target offloading and
> > will
> > be
> >   the basis for upstreaming.
> > 
> 
> 
> I looked at the branch and it seems to have a single merged patch
> that is not only huge, but has a lot of style changes to unrelated
> code (ex. clang-tools-extra).
> 
> It would be much simpler if you had a branch with the changes that
> you want to upstream into separated patches rebase onto LLVM's main
> branch.
The github repo is for reference only. If you look at our upstream
patch history, you will see that we submit small patches with tests and
follow the review protocol.
> 
> cheers,
> --renato
> 
Thanks,
Simon

On Mon, 8 Nov 2021 at 14:56, Simon Moll <Simon.Moll at emea.nec.com>
wrote:
> If you look at the build logs of clang-ve-ninja, you will see that the
> "check all" tests for LLVM+Clang have been passing for a while.
> What's failing is compiler-rt and we have a patch for that.
>
Right, what we mean by "green bots" is that there should be no
conditions
for the bot to be considered a success.

Buildbots *must* not only test all known functionality that is expected to
work, but they also must not be "red".

This is something that perhaps isn't clear on the new target section of the
documents but it's the modus operandi for a long time.

If the bot is red, or turns red easily, it can't be relied upon to convey
success in target testing, because you can't expect non-NEC developers to
know what's good and what's not, or what should pass and what
shouldn't.

It's the responsibility of the bot owner (and ultimately, the target's
community), to make sure the bots accurately reflect the quality of the
target.

Therefore, a (perhaps undocumented) item on the checklist before moving out
of experimental is: the bots must test the target and they must be green
and stable (weeks without crashing for spurious reasons).

In VE's case, looking at the earlier builds and seeing that "clang
check"
passes them all, should be enough to assert history, but before the target
is built by everyone else, the bot must be green.


Yes, the compiler-rt tests are failing for well understood
reasons
> (documented in the patch - check-all on LLVM+Clang is green). The patch
> will make compiler-rt pass on VE by accounting for those (no denorm
> support, syscall differences).
> We explicitly include compiler-rt testing (even though it is failing)
> to have LLVM-compiled code running on the VE in CI.. this is not
> something we'd do for slick optics.
>
Right, I've done the same thing when turning on the Arm back-end. I built
enough buildbots that shown that the target was working on the basic level,
then disabled the compiler-rt and test-suite that were not passing with
specific bugzilla items for each one, and then with time, I fixed all of
them and then all Arm bots were green.

In your case, no other bot (should? will?) build compiler-rt for VE, so
this shouldn't hit other bots, which will start building VE once it builds
by default.

But your buildbot will still be the *only* bot that build VE proper and
uses hardware, so it will be the representative of the VE target.

If it continues red, and it later on problems start to appear in the LIT
tests, then other developers will look at your bot, red for ages, and will
likely infer that no one cares, and disable the broken test.

Overall, it's much easier if the main bot is green and all the disabled
tests have bugzilla entries showing that you are working on it.

The github repo is for reference only. If you look at our
upstream
> patch history, you will see that we submit small patches with tests and
> follow the review protocol.

I know, that's not what I meant.

My point is that it's really hard to use that branch for reference because
of all of the other non-VE stuff that is there too, bundled in a single
merge commit.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211108/922d474e/attachment.html>