[llvm-dev] RFC Adding Fortran tests to LLVM Test Suite

Romero, Nichols A. via llvm-dev
Wed, 09 Dec 2020 17:06:19 -0000

Renato, See replies below.

From: Renato Golin <rengolin at gmail.com>
Date: Sunday, December 6, 2020 at 6:22 AM
To: Nichols Romero <naromero at anl.gov>
Cc: Johannes Doerfert <johannesdoerfert at gmail.com>, via llvm-dev
<llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] RFC Adding Fortran tests to LLVM Test Suite

On Sat, 5 Dec 2020 at 06:15, Romero, Nichols A. <naromero at
anl.gov<mailto:naromero at anl.gov>> wrote:
As I was planning this out in my head, I do think in the first differential we
would add the CMake plumbing and a simple opens source program or two (e.g.
hand-coded GEMM or something along those lines). SPEC would come in the next
batch.

That sounds like a good initial plan. Do you also plan to add more OSS
applications, benchmarks and tests right after SPEC?

Yes, I am working on identifying some OSS applications and benchmarks.


The main reason for the test-suite is not benchmark, but end-to-end regression
testing on multiple architectures. We can only claim to support languages and
targets if we can compile entire applications, run them and get the expected
results.

So having a set of language tests and some real world applications on the
test-suite will be required for us to claim Fortran support of any kind.

Agreed.

Those are also your best friends in making sure all the work you've done on
the front-end doesn't regress on any target, from release to release, or
during normal development.

That's why I'm pushing to have those tests and applications as soon as
possible. It's for Flang's own benefit.

On a related note, are there any buildbots running the llvm-test-suite or are
folks just running llvm-test-suite manually?

Plenty, on both testing and benchmark modes. It's also part of the release
process on all architectures.

Are they in some public dashboard like this one?
http://lab.llvm.org:8011/#/console


You should add support for both testing and benchmark modes, so that the
benchmarks (not SPEC) get regression tested as well as be able to report and
compare performance numbers.

The goal is to have an analogous set-up to what is done now for C/C++. I am just
getting up to speed now on it.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201209/3bf6c160/attachment-0001.html>