llvm dev - Nov 2021 - Proposal: Make the VE target official

On Wed, 10 Nov 2021 at 14:10, Simon Moll <Simon.Moll at emea.nec.com>
wrote:
> We should be able to bring that down. clang-ve-ninja currently builds
> everything from scratch (and it's all static - i'd love to have
working
> shared component libraries for faster/incremental builds).
>
> We are also considering a second, faster builder that only builds and
> tests LLVM+Clang. That would be the canary for any issues with the VE
> backend.
>
IIUC, your builder is an x86_64 machine cross testing on a VE target.

If the canary builder doesn't run anything on VE hardware, then after VE is
official, it won't be different than any other x86_64 builder
building/testing VE.


Once D113093 is in, clang-ve-ninja is expected to be
green.
> We can call that the stable state - everything that's tested is
> supposed to work and any red-ness implies breakage.
>
Excellent! Make sure once green that the bot be moved to the production
server. Check with Galina to make sure you're on the right place.


This may be a good moment to mention that the compiler-rt
patch
> disables tests that will never work on VE - there is no fp denormal
> support, for example.
>
Ah, right, so then permanently disabled is the right thing to do, no need
for bugzilla entries.


We add a new staging buildbot that builds everything
clang-ve-ninja
> does plus the yet-unsupported features that we are currently working
> on.
>
If you do that, make sure you add it to the staging server (or a private
one). This bot cannot notify people of breakages (email, IRC, nothing).


Initially, that bot will be 'red' while the official one has to be
kept
> 'green'. Once we are confident about the feature/runtime - both
bots
> are 'green' - we will make the official bot test that feature,
thereby
> declaring the feature official. The staging bot will turn to new
> experimental features.
>
Once it's official, and green, move the bot to the production (noisy)
server and create a new one for new features in the staging one.

Once the target is out of experimental, every buildbot that doesn't
restrict the targets it builds (most bots) will build VE and run its LIT
tests.

Because you're the VE code owner, if your target breaks other people's
bots, developers will (hopefully) notify you, too.

So your strategy would have at the very least three (classes of) bots:

1. Fast "canary" bot, building only Clang and LLVM and running LIT
tests
(maybe you'll want to add some cross-VE testing here, too)

2. Slow complete bot, building everything that is supposed to be supported.
This can be multiple bots with different configurations or one huge build.

3. A staging (silent) build with new stuff that the team is working on.
This can also be different bots, totally up to the team, and doesn't even
need to be public.

I did a similar division for Arm and we've been doing it since for Arm 32
and 64. Even if we use the same server for multiple bots, it's easier to
debug breakages when we build less stuff.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211110/daba333a/attachment.html>

On Wed, 2021-11-10 at 17:52 +0000, Renato Golin wrote:
> On Wed, 10 Nov 2021 at 14:10, Simon Moll <Simon.Moll at emea.nec.com>
> wrote:
> > We should be able to bring that down. clang-ve-ninja currently
> > builds
> > everything from scratch (and it's all static - i'd love to
have
> > working
> > shared component libraries for faster/incremental builds).
> > 
> > We are also considering a second, faster builder that only builds
> > and
> > tests LLVM+Clang. That would be the canary for any issues with the
> > VE
> > backend.
> > 
> 
>  
> IIUC, your builder is an x86_64 machine cross testing on a VE
> target. 
> 
> If the canary builder doesn't run anything on VE hardware, then after
> VE is official, it won't be different than any other x86_64 builder
> building/testing VE.
> 
> 
> > Once D113093 is in, clang-ve-ninja is expected to be green.
> > We can call that the stable state - everything that's tested is
> > supposed to work and any red-ness implies breakage.
> > 
> 
> 
> Excellent! Make sure once green that the bot be moved to the
> production server. Check with Galina to make sure you're on the right
> place.
> 
> 
> > This may be a good moment to mention that the compiler-rt patch
> > disables tests that will never work on VE - there is no fp denormal
> > support, for example.
> > 
> 
> 
> Ah, right, so then permanently disabled is the right thing to do, no
> need for bugzilla entries.
> 
> 
> > We add a new staging buildbot that builds everything clang-ve-ninja
> > does plus the yet-unsupported features that we are currently
> > working
> > on.
> > 
In essense, we keep the current staging clang-ve-ninja as is and attach
a clone to the main buildbot. The clone is then the official bot and
the staging one is the one for work-in-progress features.
> 
> 
> If you do that, make sure you add it to the staging server (or a
> private one). This bot cannot notify people of breakages (email, IRC,
> nothing).
> 
> 
> > Initially, that bot will be 'red' while the official one has
to be
> > kept
> > 'green'. Once we are confident about the feature/runtime -
both
> > bots
> > are 'green' - we will make the official bot test that feature,
> > thereby
> > declaring the feature official. The staging bot will turn to new
> > experimental features.
> > 
> 
> 
> Once it's official, and green, move the bot to the production (noisy)
> server and create a new one for new features in the staging one.
> 
> Once the target is out of experimental, every buildbot that doesn't
> restrict the targets it builds (most bots) will build VE and run its
> LIT tests.
> 
> Because you're the VE code owner, if your target breaks other
> people's bots, developers will (hopefully) notify you, too.
> 
> So your strategy would have at the very least three (classes of)
> bots:
> 
> 1. Fast "canary" bot, building only Clang and LLVM and running
LIT
> tests (maybe you'll want to add some cross-VE testing here, too)
> 
> 2. Slow complete bot, building everything that is supposed to be
> supported. This can be multiple bots with different configurations or
> one huge build.
> 
> 3. A staging (silent) build with new stuff that the team is working
> on. This can also be different bots, totally up to the team, and
> doesn't even need to be public.
That's the plan. Given that other/fast bots will build the VE target
and run its lit tests, we may not need the canary though.
> 
> I did a similar division for Arm and we've been doing it since for
> Arm 32 and 64. Even if we use the same server for multiple bots, it's
> easier to debug breakages when we build less stuff.
libunwind, compiler-rt, libcxxabi, libcxx really dependend on each
other. As long as build times are reasonable, we will keep it to full
stack testing.

- Simon
> 
> cheers,
> --renato