> > And in the test file there is a line: > > XFAIL: powerpc64, s390x, mips, sparc > > This is a crude attempt at "XFAIL: big-endian". The mips entry here is just > wrong if the system is little-endian - the test passes on little-endian machines > and fails on big-endian. This is obviously a problem.'XFAIL: mips' counts as an XFAIL for all mips targets because lit is checking for substrings (using the 'in' operator) of the target triple and 'mips' is a substring of 'mipsel-...'. I haven't tested this but I think that we need 'XFAIL: mips-, mips64-' to distinguish little/big-endian correctly for MIPS. The relevant code is the isExpectedToFail() function in utils/lit/lit/Test.py.> -----Original Message----- > From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] > On Behalf Of Justin Bogner > Sent: 21 February 2014 08:21 > To: Reed Kotler > Cc: LLVMdev at cs.uiuc.edu > Subject: Re: [LLVMdev] make check issue with llvm-cov > > reed kotler <rkotler at mips.com> writes: > > On a native mips linux machine, we are failing a make check test which > > seems like it is supposed to already xfail. > > I think you're mistaken. The test below looks like it's passing. > > > For one, the file test.c does not even exist (it is called test.cpp) > > This should probably be test.o, rather than test.c or test.cpp, but the > interface to llvm-cov is compatible with gcov, which completely ignores the > file given as an argument other than to base the .gcno and .gcno file names > on it. > > > And in the test file there is a line: > > XFAIL: powerpc64, s390x, mips, sparc > > This is a crude attempt at "XFAIL: big-endian". The mips entry here is just > wrong if the system is little-endian - the test passes on little-endian machines > and fails on big-endian. This is obviously a problem. > > > The script in make check thinks that it passed but clearly it does not. > > llvm-cov is returning 0 in this case. > > I'm not sure which case you're referring to. This test file really has too many > tests in it, which makes it confusing to debug failures. > Someone (me?) should probably split this into several tests that invoke llvm- > cov once each. The use of diff should be replaced with FileCheck as well, so > that error messages are clearer. > > In any case, all of the llvm-cov calls except the read/checksum fail invocations > should (and do) return zero on little-endian. I think the only problem here is > that the XFAIL line is wrong. > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
That test cannot work at all right now because there is no test.c and that is being called out for (there is test.cpp). ________________________________________ From: Daniel Sanders Sent: Friday, February 21, 2014 2:58 AM To: Justin Bogner; Reed Kotler Cc: LLVMdev at cs.uiuc.edu Subject: RE: [LLVMdev] make check issue with llvm-cov> > And in the test file there is a line: > > XFAIL: powerpc64, s390x, mips, sparc > > This is a crude attempt at "XFAIL: big-endian". The mips entry here is just > wrong if the system is little-endian - the test passes on little-endian machines > and fails on big-endian. This is obviously a problem.'XFAIL: mips' counts as an XFAIL for all mips targets because lit is checking for substrings (using the 'in' operator) of the target triple and 'mips' is a substring of 'mipsel-...'. I haven't tested this but I think that we need 'XFAIL: mips-, mips64-' to distinguish little/big-endian correctly for MIPS. The relevant code is the isExpectedToFail() function in utils/lit/lit/Test.py.> -----Original Message----- > From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] > On Behalf Of Justin Bogner > Sent: 21 February 2014 08:21 > To: Reed Kotler > Cc: LLVMdev at cs.uiuc.edu > Subject: Re: [LLVMdev] make check issue with llvm-cov > > reed kotler <rkotler at mips.com> writes: > > On a native mips linux machine, we are failing a make check test which > > seems like it is supposed to already xfail. > > I think you're mistaken. The test below looks like it's passing. > > > For one, the file test.c does not even exist (it is called test.cpp) > > This should probably be test.o, rather than test.c or test.cpp, but the > interface to llvm-cov is compatible with gcov, which completely ignores the > file given as an argument other than to base the .gcno and .gcno file names > on it. > > > And in the test file there is a line: > > XFAIL: powerpc64, s390x, mips, sparc > > This is a crude attempt at "XFAIL: big-endian". The mips entry here is just > wrong if the system is little-endian - the test passes on little-endian machines > and fails on big-endian. This is obviously a problem. > > > The script in make check thinks that it passed but clearly it does not. > > llvm-cov is returning 0 in this case. > > I'm not sure which case you're referring to. This test file really has too many > tests in it, which makes it confusing to debug failures. > Someone (me?) should probably split this into several tests that invoke llvm- > cov once each. The use of diff should be replaced with FileCheck as well, so > that error messages are clearer. > > In any case, all of the llvm-cov calls except the read/checksum fail invocations > should (and do) return zero on little-endian. I think the only problem here is > that the XFAIL line is wrong. > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On 02/21/2014 02:58 AM, Daniel Sanders wrote:>>> And in the test file there is a line: >>> XFAIL: powerpc64, s390x, mips, sparc >> This is a crude attempt at "XFAIL: big-endian". The mips entry here is just >> wrong if the system is little-endian - the test passes on little-endian machines >> and fails on big-endian. This is obviously a problem. > 'XFAIL: mips' counts as an XFAIL for all mips targets because lit is checking for substrings (using the 'in' operator) of the target triple and 'mips' is a substring of 'mipsel-...'. I haven't tested this but I think that we need 'XFAIL: mips-, mips64-' to distinguish little/big-endian correctly for MIPS. > The relevant code is the isExpectedToFail() function in utils/lit/lit/Test.py.In this case what seems to be happening is the llvm-cov is NOT failing and it should. That causes an unexpected pass which is what is making "make check" fail.>> -----Original Message----- >> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] >> On Behalf Of Justin Bogner >> Sent: 21 February 2014 08:21 >> To: Reed Kotler >> Cc: LLVMdev at cs.uiuc.edu >> Subject: Re: [LLVMdev] make check issue with llvm-cov >> >> reed kotler <rkotler at mips.com> writes: >>> On a native mips linux machine, we are failing a make check test which >>> seems like it is supposed to already xfail. >> I think you're mistaken. The test below looks like it's passing. >> >>> For one, the file test.c does not even exist (it is called test.cpp) >> This should probably be test.o, rather than test.c or test.cpp, but the >> interface to llvm-cov is compatible with gcov, which completely ignores the >> file given as an argument other than to base the .gcno and .gcno file names >> on it. >> >>> And in the test file there is a line: >>> XFAIL: powerpc64, s390x, mips, sparc >> This is a crude attempt at "XFAIL: big-endian". The mips entry here is just >> wrong if the system is little-endian - the test passes on little-endian machines >> and fails on big-endian. This is obviously a problem. >> >>> The script in make check thinks that it passed but clearly it does not. >>> llvm-cov is returning 0 in this case. >> I'm not sure which case you're referring to. This test file really has too many >> tests in it, which makes it confusing to debug failures. >> Someone (me?) should probably split this into several tests that invoke llvm- >> cov once each. The use of diff should be replaced with FileCheck as well, so >> that error messages are clearer. >> >> In any case, all of the llvm-cov calls except the read/checksum fail invocations >> should (and do) return zero on little-endian. I think the only problem here is >> that the XFAIL line is wrong. >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
If you can help get it working on big-endian systems, we should be able to remove the XFAIL. That seems like the cleanest way out of this. Yuchen sent a patch to llvm-commits on 12/19/13. (I can resend it to you if you don’t have that.) Can you try that out on a BE mips system? On Feb 21, 2014, at 7:11 AM, Reed Kotler <Reed.Kotler at imgtec.com> wrote:> On 02/21/2014 02:58 AM, Daniel Sanders wrote: >>>> And in the test file there is a line: >>>> XFAIL: powerpc64, s390x, mips, sparc >>> This is a crude attempt at "XFAIL: big-endian". The mips entry here is just >>> wrong if the system is little-endian - the test passes on little-endian machines >>> and fails on big-endian. This is obviously a problem. >> 'XFAIL: mips' counts as an XFAIL for all mips targets because lit is checking for substrings (using the 'in' operator) of the target triple and 'mips' is a substring of 'mipsel-...'. I haven't tested this but I think that we need 'XFAIL: mips-, mips64-' to distinguish little/big-endian correctly for MIPS. >> The relevant code is the isExpectedToFail() function in utils/lit/lit/Test.py. > In this case what seems to be happening is the llvm-cov is NOT failing and it should. > That causes an unexpected pass which is what is making "make check" fail. > >>> -----Original Message----- >>> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] >>> On Behalf Of Justin Bogner >>> Sent: 21 February 2014 08:21 >>> To: Reed Kotler >>> Cc: LLVMdev at cs.uiuc.edu >>> Subject: Re: [LLVMdev] make check issue with llvm-cov >>> >>> reed kotler <rkotler at mips.com> writes: >>>> On a native mips linux machine, we are failing a make check test which >>>> seems like it is supposed to already xfail. >>> I think you're mistaken. The test below looks like it's passing. >>> >>>> For one, the file test.c does not even exist (it is called test.cpp) >>> This should probably be test.o, rather than test.c or test.cpp, but the >>> interface to llvm-cov is compatible with gcov, which completely ignores the >>> file given as an argument other than to base the .gcno and .gcno file names >>> on it. >>> >>>> And in the test file there is a line: >>>> XFAIL: powerpc64, s390x, mips, sparc >>> This is a crude attempt at "XFAIL: big-endian". The mips entry here is just >>> wrong if the system is little-endian - the test passes on little-endian machines >>> and fails on big-endian. This is obviously a problem. >>> >>>> The script in make check thinks that it passed but clearly it does not. >>>> llvm-cov is returning 0 in this case. >>> I'm not sure which case you're referring to. This test file really has too many >>> tests in it, which makes it confusing to debug failures. >>> Someone (me?) should probably split this into several tests that invoke llvm- >>> cov once each. The use of diff should be replaced with FileCheck as well, so >>> that error messages are clearer. >>> >>> In any case, all of the llvm-cov calls except the read/checksum fail invocations >>> should (and do) return zero on little-endian. I think the only problem here is >>> that the XFAIL line is wrong. >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev