similar to: [LLVMdev] QMTest vs. Dejagnu

Tanya Lattner wrote: > I've finished adding the -rundejagnu option to the nightly tester script, > which was the last step to fully support Dejagnu. I think now is the > appropriate time to discuss keeping QMTest or switching to Dejagnu. A lot > of work went into using QMTest, so I think we should make this decision > carefully and before the 1.4 release. > > Here are the

On Sunday 28 November 2004 00:24, Tanya Lattner wrote: Just some comments from a QMTest user... Note however, that even with them, dejagnu looks better. > Cons of QMTest: > 1) You have to use the gui to add directories. I think you're wrong. Manually creating a directory would work, as QMTest does not place anything special in directories. > 2) You have to use the gui to XFAIL

> > Cons of QMTest: > > 1) You have to use the gui to add directories. > > I think you're wrong. Manually creating a directory would work, as QMTest does > not place anything special in directories. I may have worded this poorly, but I think you have to use the gui to add new directories or tests, or specific tests. Otherwise, it does not know what to do with those

In an effort to simplify how tests are added to the LLVM testsuite, I've added support for Dejagnu. This only applies to the Feature and Regression tests. If this experiment goes well, we may switch from using QMTest to Dejagnu. I'm asking for people (especially those running nightly testers) to give Dejagnu a try. You will need to install Dejagnu (http://www.gnu.org/software/dejagnu/)

Talin wrote: > I've been using gtest (http://code.google.com/p/googletest/) for all of > my frontend unit tests and I'm very happy with it. It does all of that > automatic test discovery stuff pretty well. I haven't tried the XML test > report generation stuff, but it does have that capability. > Ok. > I don't know much about DejaGNU, and from what little I

I attempt install and use qmtest for run LLVM testsuit at FreeBSD, but without success. I have installed qm-2.2 (latest and single version accessable from http://www.codesourcery.com/qmtest/download.html in sources) When I start qmtest for LLVM ("make qmtest" in <llvm-obj-dir>/test directory - right?) error printed: ---8X---------------

On 28 September 2016 at 10:08, Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org> wrote: > I cannot think of any situation where a universally failing test > should be in-tree unless it is a bug that someone is expecting to fix. It seems moderately common to mark something XFAIL temporarily to get the bots green while then going ahead to fix the issue. Your proposal would add

From http://llvm.org/docs/TestingGuide.html Some tests are known to fail. Some are bugs that we have not fixed yet; others are features that we haven't added yet (or may never add). In DejaGNU, the result for such tests will be XFAIL (eXpected FAILure). In this way, you can tell the difference between an expected and unexpected failure. The tests in the test suite have no such feature at

Boy does THAT look familiar .. The problem is that you need an older version of QMTest, version 2.0.3 to be exact. The configuration of QMTest that LLVM uses won't work with 2.2 Reid. On Sun, 2004-06-20 at 10:53, Vladimir Merzliakov wrote: > I attempt install and use qmtest for run LLVM testsuit at FreeBSD, but > without success. > > I have installed qm-2.2 (latest and single

Hi, We were wondering if *all* of the dejagnu tests should pass (in the sense that no result should be unexpected), because we have a few failed tests on our build. I will mail the summary once my current build is done.

> On Sep 29, 2016, at 7:52 AM, David Blaikie <dblaikie at gmail.com> wrote: > > > > On Wed, Sep 28, 2016 at 11:58 AM Robinson, Paul via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > On 28 September 2016 at 10:08, Chris Bieneman via llvm-dev > <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>

Does anyone have any objections to removing support for QMtest from the LLVM regression testing framework? The LLVM Dejagnu framework has all of the functionality provided by QMtest and more. -Chris -- http://nondot.org/sabre/ http://llvm.org/

I'm re-thinking my penchant for automake. automake is great for many standard applications that just need to get portable makefiles up and running quickly. However, it turns out that LLVM is "different enough" from a standard application that automake might not be the best choice. Here's some of the problems I've run into: 1. There's no way to tell automake to build

On Jul 22, 2010, at 2:44 PMPDT, Patrick Alexander Simmons wrote: > From http://llvm.org/docs/TestingGuide.html > > Some tests are known to fail. Some are bugs that we have not fixed > yet; > others are features that we haven't added yet (or may never add). In > DejaGNU, the result for such tests will be XFAIL (eXpected FAILure). > In > this way, you can tell the

Hello LLVM-Dev, The other day as I was digging through lldb’s test suite I noticed they support something kinda neat. In their python test harness, the attribute they use to denote expected failures supports a parameter for specifying the bug number. This got me thinking. I believe that any test that is marked XFAIL is a bug, and we can use LIT to enforce that. So I wrote a patch

This may be an unpopular opinion (and I don’t have the full context on those specific issues), but I believe that these are an abuse of XFAIL, and should probably be written in terms of REQUIRES instead of XFAIL. I believe XFAIL tests actually execute, and are just marked as expected failure. If a test is not expected to ever succeed, we shouldn’t bother running it, which is what the REQUIRES

One more update. The Makefile.am for analyze was wrong. It wasn't linking in the some of the passes. The new size is 56951088 which is in line with the other executables. Also, I have now completed a run of projects/llvm-test/MultiSource with the tools generated by automake. The only errors were for: TEST (llc) 'sgefa' FAILED! TEST (jit) 'sgefa' FAILED! TEST (jit)

On Sun, 2004-08-29 at 16:01, Jeff Cohen wrote: > Fair enough... The following tests under MultiSource fail: > > Benchmarks/Olden/power > Benchmarks/OptimizerEval > Benchmarks/Ptrdist/ks > Benchmarks/MallocBench/perl > Applications/sgefa > > However, they also fail in the exact same way without my change. > OptimizerEval appears to be non-deterministic; it produces

On 10/18/2012 01:39 PM, David Blaikie wrote: > On Thu, Oct 18, 2012 at 11:19 AM, Rick Foos<rfoos at codeaurora.org> wrote: >> On 10/18/2012 10:36 AM, David Blaikie wrote: >>> On Thu, Oct 18, 2012 at 12:48 AM, Renato Golin<rengolin at systemcall.org> >>> wrote: >>>> On 18 October 2012 05:11, Rick Foos<rfoos at codeaurora.org> wrote:

On Wed, Feb 25, 2009 at 12:02:18PM -0800, Julien Lerouge wrote: > For 2), I think the NewNightlyTest.pl script would require some small > changes, to invoke the jcheck target instead of check, pass the desired > -j flag, and also parse the log. I'll take a look. In any case, the > changes would only apply to the DejaGNU tests (llvm-test already works > fine with -jX). Here is