[llvm-dev] [cfe-dev] RFC: End-to-end testing

Renato Golin via llvm-dev
Thu, 10 Oct 2019 10:39:08 -0000

On Thu, 10 Oct 2019 at 10:56, Florian Hahn <florian_hahn at apple.com>
wrote:
> Have you considered alternatives to checking the assembly for ensuring
vectorization or other transformations? For example, instead of checking the
assembly, we could check LLVM’s statistics or optimization remarks. If you want
to ensure a loop got vectorized, you could check the loop-vectorize remarks,
which should give you the position of the loop in the source and
vectorization/interleave factor used. There are few other things that could go
wrong later on that would prevent vector instruction selection, but I think it
should be sufficient to guard against most cases where we loose vectorization
and should be much more robust to unrelated changes. If there are additional
properties you want to ensure, they potentially could be added to the remark as
well.

We used to have lots of them, at least in the initial implementation
of the loop vectoriser (I know, many years ago).

The thread has enough points not to repeat here, but I guess the main
point is to make sure we don't duplicate tests, increasing CI cost
(especially on slower hardware).

I'd recommend trying to move any e2e tests into the test-suite and
make it easier to run, and leave specific tests only in the repo (to
guarantee independence of components).

The last thing we want is to create direct paths from front-ends to
back-ends and make LLVM IR transformation less flexible.

cheers,
--renato