similar to: [RFC] Changing `lit` default output

(+ the real cfe-dev) On Tue, May 31, 2016 at 4:48 PM, Daniel Dunbar <daniel at zuster.org> wrote: > Hi all, > > I have some patches up in: > http://reviews.llvm.org/D20721 > to make `lit` output more readable when commands fail. I'd like to get a > little more feedback on this before landing, since it impacts how test > failures appear when using the internal

> On May 31, 2016, at 4:48 PM, Daniel Dunbar via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hi all, > > I have some patches up in: > http://reviews.llvm.org/D20721 <http://reviews.llvm.org/D20721> > to make `lit` output more readable when commands fail. I'd like to get a little more feedback on this before landing, since it impacts how test failures

As long as the individual commands are still copy-pastable into both cmd and bash, I'm happy. I think practice this means the output should look more like this: some test: // RUN: %clang_cc1 %s -verify output: $ "C:\\src\\llvm\\build\\bin\\clang.exe" -cc1 "C:\\....\\test.cpp" -verify Essentially every program used in the LLVM test suite understands forward slashes, so we

This part I didn't change, but I'm not completely sure I understood your message -- is there a change you wanted here? - Daniel On Tue, May 31, 2016 at 5:01 PM, Reid Kleckner <rnk at google.com> wrote: > As long as the individual commands are still copy-pastable into both cmd > and bash, I'm happy. I think practice this means the output should look > more like this:

On Sat, Jul 8, 2017 at 10:07 AM Daniel Dunbar <daniel_dunbar at apple.com> wrote: > > > On Jul 8, 2017, at 7:32 AM, David Blaikie <dblaikie at gmail.com> wrote: > > Ideally/the better integration with Buildbot would be to have these > outputs referenced as "associated files" (it's been a while since I played > with buildbot - I remember finding this

On 08/12/2013 16:43, Daniel Dunbar wrote: > > With one-to-one mapping, it becomes possible to use simple tools > like grep to validate common mistakes like %clang / %clang_cc1 > mixups, a missing -o flag and so on. > > Right now there's no obvious way to do those checks and we've > ended up without an easy way to lint for broken tests as a result.

Ideally/the better integration with Buildbot would be to have these outputs referenced as "associated files" (it's been a while since I played with buildbot - I remember finding this and considering how it could be done, but not getting all the way through) so they'd come back as actual files on the build master, linked from the results page that you could click on to

Hi, I am observing a deadlock with llvm-lit on windows 7. When I attached a debugger, the communicate() call is blocked. In file utils/lit/lit/TestRunner.py > # FIXME: There is probably still deadlock potential here. Yawn. > procData = [None] * len(procs) > procData[-1] = procs[-1].communicate() I am invoking python directly on windows to run the unit tests.

Ok, that makes sense. I don't see this as a good enough argument to remove backslash support though. For problems like the clang_cc1 substitution mistakes, it would be much better to just improve the substitution support so that those cause immediate test failures. I would definitely support a move to make lit's substitution machinery more strict. For problems like your first search, I

On 08/12/2013 13:12, Chandler Carruth wrote: > >> * Removing trailing \ will introduce the neat property that >> one RUN line corresponds precisely to one command that's >> executed. This is good for humans and will enable >> simplifications in the test runner. >> >> FWIW, I've never really had a

I'd like to propose deprecating and shortly thereafter removing the lit test runner feature that concatenates RUN lines ending in a trailing \ *Rationale:** * * Trailing \ has a special meaning in various language standards that we support nowadays. In the C preprocessor, for example, it's handled _before_ comments. Various compilers handle this differently and it

Hi, A feature I've wanted in lit for a while is a having a timeout per test. Attached are patches that implement this idea. I'm e-mailing llvm-dev rather than llvm-commits because I want to gather more feedback on my initial implementation and hopefully some answers to some unresolved issues with my implementation. Currently in lit you can set a global timeout for all of the tests but

On Sun, Dec 8, 2013 at 2:11 AM, Alp Toker <alp at nuanti.com> wrote: > I'd like to propose deprecating and shortly thereafter removing the lit > test runner feature that concatenates RUN lines ending in a trailing \ > I'm really opposed to this. Especially for the Clang test suite where run lines are often *very* long and hard to organize, read, and edit without this

On Sun, Dec 8, 2013 at 6:04 AM, Alp Toker <alp at nuanti.com> wrote: > > On 08/12/2013 13:12, Chandler Carruth wrote: > >> >> * Removing trailing \ will introduce the neat property that >>> >>> one RUN line corresponds precisely to one command that's >>> executed. This is good for humans and will enable

On 08/12/2013 11:22, Chandler Carruth wrote: > On Sun, Dec 8, 2013 at 2:11 AM, Alp Toker <alp at nuanti.com > <mailto:alp at nuanti.com>> wrote: > > I'd like to propose deprecating and shortly thereafter removing > the lit test runner feature that concatenates RUN lines ending in > a trailing \ > > > I'm really opposed to this. Especially

Hi Matthias, Thank you for your answer. But can you answer for some more questions? First of all, now LNT uses make-style of running tests and parse results from result csv file. Are there any plans to go to cmake? As I understood lit will run and collect all metrics, but there is no opportunity to make any settings for choosing what metrics I would like to collect. Test reports files allow to

Hi all, quite a few tests use the pattern "2>&1 | FileCheck %s". AFAIK how stdout and stderr are merged into a single character stream is undefined and depends e.g. on whether stdout is buffered. I think we are often saved by the fact that standard output is written only at the end of the program and stderr is unbuffered, i.e. always written before stdout. A lot of tests disable

> On Jul 7, 2017, at 2:19 PM, Reid Kleckner <rnk at google.com> wrote: > > On Fri, Jul 7, 2017 at 1:20 PM, George Karpenkov via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > Thus, I propose modifying FileCheck default behavior to dump all swallowed output on stderr when the test has failed. > Would there be any objections to

Hi all, In the past there was some interest in having lit supporting use with Python3, which had become the default on some Linux distributions. I've been working on this recently and as of r188398, the LLVM/Clang tests all run and pass on my system with Python 3.3. There are still some quality of implementation issues to iron out, and there may be problems with lit-based tests in other

On Sun, Dec 8, 2013 at 4:24 AM, Alp Toker <alp at nuanti.com> wrote: > > On 08/12/2013 11:22, Chandler Carruth wrote: > > On Sun, Dec 8, 2013 at 2:11 AM, Alp Toker <alp at nuanti.com> wrote: > >> I'd like to propose deprecating and shortly thereafter removing the lit >> test runner feature that concatenates RUN lines ending in a trailing \ >>