[LLVMdev] make check issue with llvm-cov

Daniel Sanders
Fri, 21 Feb 2014 10:58:35 -0000

> > And in the test file there is a line:
> > XFAIL: powerpc64, s390x, mips, sparc
>
> This is a crude attempt at "XFAIL: big-endian". The mips entry
here is just
> wrong if the system is little-endian - the test passes on little-endian
machines
> and fails on big-endian. This is obviously a problem.

'XFAIL: mips' counts as an XFAIL for all mips targets because lit is
checking for substrings (using the 'in' operator) of the target triple
and 'mips' is a substring of 'mipsel-...'. I haven't tested
this but I think that we need 'XFAIL: mips-, mips64-' to distinguish
little/big-endian correctly for MIPS.
The relevant code is the isExpectedToFail() function in utils/lit/lit/Test.py.

> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at
cs.uiuc.edu]
> On Behalf Of Justin Bogner
> Sent: 21 February 2014 08:21
> To: Reed Kotler
> Cc: LLVMdev at cs.uiuc.edu
> Subject: Re: [LLVMdev] make check issue with llvm-cov
>
> reed kotler <rkotler at mips.com> writes:
> > On a native mips linux machine, we are failing a make check test which
> > seems like it is supposed to already xfail.
>
> I think you're mistaken. The test below looks like it's passing.
>
> > For one, the file test.c does not even exist (it is called test.cpp)
>
> This should probably be test.o, rather than test.c or test.cpp, but the
> interface to llvm-cov is compatible with gcov, which completely ignores the
> file given as an argument other than to base the .gcno and .gcno file names
> on it.
>
> > And in the test file there is a line:
> > XFAIL: powerpc64, s390x, mips, sparc
>
> This is a crude attempt at "XFAIL: big-endian". The mips entry
here is just
> wrong if the system is little-endian - the test passes on little-endian
machines
> and fails on big-endian. This is obviously a problem.
>
> > The script in make check thinks that it passed but clearly it does
not.
> > llvm-cov is returning 0 in this case.
>
> I'm not sure which case you're referring to. This test file really
has too many
> tests in it, which makes it confusing to debug failures.
> Someone (me?) should probably split this into several tests that invoke
llvm-
> cov once each. The use of diff should be replaced with FileCheck as well,
so
> that error messages are clearer.
>
> In any case, all of the llvm-cov calls except the read/checksum fail
invocations
> should (and do) return zero on little-endian. I think the only problem here
is
> that the XFAIL line is wrong.