search for: xfail

..., The other day as I was digging through lldb’s test suite I noticed they support something kinda neat. In their python test harness, the attribute they use to denote expected failures supports a parameter for specifying the bug number. This got me thinking. I believe that any test that is marked XFAIL is a bug, and we can use LIT to enforce that. So I wrote a patch (https://reviews.llvm.org/D25035 <https://reviews.llvm.org/D25035>) to add a feature to LIT which would support mapping XFAILs to PRs, and a flag to turn XFAILS without PRs into failures. My proposal is to add this feature to L...

Hi, I am currently trying to set up new performance and regression testers for Polly and LLVM and would like to XFAIL two test cases. I am using the LNTBuilder instead of the NightlyTestBuilder out of the assumption that the LNTBuilder is the more modern solution. However, when trying to xfail test cases I realized the xfail=[] parameter of getLNTFactor is ignored. Previously this was not an issue, as LNT would...

...gt; <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > > I cannot think of any situation where a universally failing test > > should be in-tree unless it is a bug that someone is expecting to fix. > > It seems moderately common to mark something XFAIL temporarily to get > the bots green while then going ahead to fix the issue. Your proposal > would add extra overhead to that flow by requiring a PR as well. This > has value when it turns out that fix can't happen in the short term for > any reason. I don't have a feel for h...

On 28 September 2016 at 10:08, Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org> wrote: > I cannot think of any situation where a universally failing test > should be in-tree unless it is a bug that someone is expecting to fix. It seems moderately common to mark something XFAIL temporarily to get the bots green while then going ahead to fix the issue. Your proposal would add extra overhead to that flow by requiring a PR as well. This has value when it turns out that fix can't happen in the short term for any reason. I don't have a feel for how common that is,...

This may be an unpopular opinion (and I don’t have the full context on those specific issues), but I believe that these are an abuse of XFAIL, and should probably be written in terms of REQUIRES instead of XFAIL. I believe XFAIL tests actually execute, and are just marked as expected failure. If a test is not expected to ever succeed, we shouldn’t bother running it, which is what the REQUIRES directives are for. -Chris > On Sep 28,...

> > And in the test file there is a line: > > XFAIL: powerpc64, s390x, mips, sparc > > This is a crude attempt at "XFAIL: big-endian". The mips entry here is just > wrong if the system is little-endian - the test passes on little-endian machines > and fails on big-endian. This is obviously a problem. 'XFAIL: mips' cou...

> -----Original Message----- > From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of > Krzysztof Parzyszek via llvm-dev > Sent: Monday, October 03, 2016 10:40 AM > To: llvm-dev at lists.llvm.org > Subject: Re: [llvm-dev] [RFC] Require PRs for XFAILing tests > > On 10/3/2016 12:21 PM, Robinson, Paul via llvm-dev wrote: > > As David Blaikie mentioned, our bug hygiene is not really that good. > > Adding a process to create more PRs is not going to change that. Of course not. The point of my remark is that linking an XFAIL t...

If you can help get it working on big-endian systems, we should be able to remove the XFAIL. That seems like the cleanest way out of this. Yuchen sent a patch to llvm-commits on 12/19/13. (I can resend it to you if you don’t have that.) Can you try that out on a BE mips system? On Feb 21, 2014, at 7:11 AM, Reed Kotler <Reed.Kotler at imgtec.com> wrote: > On 02/21/2014 02:58 AM,...

...e: RUN lines are basically sequences of commands that are put into a bash script and executed. Here is an example: RUN: llvm-as < %s | llvm-dis | llvm-as) Pros of Dejagnu: 1) Adding a test is very quick and easy. In most cases it requires just adding a RUN line to the file. No gui needed. 2) XFAILing a test is also quick and easy. You just add a XFAIL line to a single test with a * for all platforms, or specify part of a target triplet to match. No gui needed. 3) Because you add the RUN and XFAIL lines to each individual test, it's all located in one file. This makes it very easy un...

Hi all, I'm investigating the following test case that reports as an unexpected pass on powerpc64-unknown-linux-gnu. Clang : CodeGenCXX/member-alignment.cpp This test case is marked as XFAIL for arm and powerpc. However, the test passes fine for powerpc64-unknown-linux-gnu. There are two tests of this form: void t::bar(void) { // CHECK: _ZN1t3barEv{{.*}} align 2 } For these member functions, the "align 2" clause is output as it's suppose...

> -----Original Message----- > From: Alex Bradbury [mailto:asb at asbradbury.org] > Sent: Saturday, October 01, 2016 1:06 PM > To: Robinson, Paul > Cc: Renato Golin; Chris Bieneman; llvm-dev at lists.llvm.org > Subject: Re: [llvm-dev] [RFC] Require PRs for XFAILing tests > > On 28 September 2016 at 19:58, Robinson, Paul via llvm-dev > <llvm-dev at lists.llvm.org> wrote: > > On 28 September 2016 at 10:08, Chris Bieneman via llvm-dev > > <llvm-dev at lists.llvm.org> wrote: > >> I cannot think of any situation where...

From http://llvm.org/docs/TestingGuide.html Some tests are known to fail. Some are bugs that we have not fixed yet; others are features that we haven't added yet (or may never add). In DejaGNU, the result for such tests will be XFAIL (eXpected FAILure). In this way, you can tell the difference between an expected and unexpected failure. The tests in the test suite have no such feature at this time. If the test passes, only warnings and other miscellaneous output will be generated. If a test fails, a large <program> F...

...************************** * dom0: SLES10 GM * dom0 boots fine * xend starts without problem --- Linux HVM domain status: --- * Xm-test completes. * Able to boot multiple HVM domains ISSUES: * 22 fails on xm-test.dd SUMMARY: Xm-test execution summary: PASS: 49 FAIL: 22 XPASS: 0 XFAIL: 3 Details: FAIL: 01_create_basic_pos Unknown reason FAIL: 11_create_concurrent_pos Unknown reason FAIL: 12_create_concurrent_stress_pos Unknown reason FAIL: 13_create_multinic_pos Unknown reason FAIL: 14_create_blockroot_pos Unknown rea...

..., Patrick Alexander Simmons wrote: > From http://llvm.org/docs/TestingGuide.html > > Some tests are known to fail. Some are bugs that we have not fixed > yet; > others are features that we haven't added yet (or may never add). In > DejaGNU, the result for such tests will be XFAIL (eXpected FAILure). > In > this way, you can tell the difference between an expected and > unexpected > failure. > > The tests in the test suite have no such feature at this time. If the > test passes, only warnings and other miscellaneous output will be > generated. If...

...er Simmons wrote: > >> From http://llvm.org/docs/TestingGuide.html >> >> Some tests are known to fail. Some are bugs that we have not fixed yet; >> others are features that we haven't added yet (or may never add). In >> DejaGNU, the result for such tests will be XFAIL (eXpected FAILure). In >> this way, you can tell the difference between an expected and unexpected >> failure. >> >> The tests in the test suite have no such feature at this time. If the >> test passes, only warnings and other miscellaneous output will be >> gene...

...s of commands that are put into a > bash script and executed. Here is an example: RUN: llvm-as < %s | llvm-dis > | llvm-as) > > Pros of Dejagnu: > 1) Adding a test is very quick and easy. In most cases it requires > just adding a RUN line to the file. No gui needed. > 2) XFAILing a test is also quick and easy. You just add a XFAIL > line to a single test with a * for all platforms, or specify part > of a target triplet to match. No gui needed. > 3) Because you add the RUN and XFAIL lines to each individual > test, it's all located in one file. This ma...

Hi, Currently, no tests that use lli without "-force-interpreter" are expected to pass when executing on an AArch64 model. However, they will pass if built and run on (say) X86, just setting the default target triple. So XFAIL gets unexpected passes on a compiler merely targetting AArch64 and leaving the tests as they are gives unexpected failures when they're run on a model. Does anyone know of a good solution to this? I believe only four tests are affected (test/ExecutionEngine/* has a more global solution and isn...

The attached patch eliminates http://llvm.org/bugs/show_bug.cgi?id=15817 by removing the remaining "; XFAIL:*" added in http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20130415/171812.html. The Isl/CodeGen/scevcodegen-1.ll testcase in polly appears as an XPASS in current llvm/polly 3.3 and trunk svn for both x86_64-apple-darwin* and x86_64 Fedora 15 when built against isl 0.11.1/2 and cl...

...******* * dom0: SLES10 GM * dom0 boots fine * xend starts without problem --- Linux HVM domain status: --- * Xm-test completes. * Able to boot multiple HVM domains ISSUES: * 10 fails on xm-test. Network failures SUMMARY: Xm-test execution summary: PASS: 61 FAIL: 10 XPASS: 0 XFAIL: 3 Details: FAIL: 16_create_smallmem_neg Starting a console with 16 MB passed, expected test to fail XFAIL: 02_network_local_ping_pos ping loopback failed for size 65507. ping eth0 failed for size 1 48 64 512 1440 1500 1505 4096 4192 32767 65507. FAIL: 03_network_lo...

Hi folks, The llvm-arm-linux buildbot, although old, is up and running and the only failures I can see were XFAIL, but still being run on ARM: http://lab.llvm.org:8011/builders/llvm-arm-linux/builds/2158 Any ideas why they're still being run on that buildbot? If we can clear those, we can get it passing again. cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... U...