similar to: [LLVMdev] Getting FileCheck's colored output through lit (& possibly ninja)

I'm just wondering if anyone's using Ninja (which buffers output from subcommands, such as lit/FileCheck/etc) & has thought about/found a way to force FileCheck to use colour in spite of Ninja's buffering. Thanks, - David

Nope. An easy solution (both here and for clang) is for ninja to set an environment variable (`NINJA_ISATTY`?), and add some magic to LLVM's "isatty" to return true if it sees it. (for matters of correctness, it would actually be best if ninja sets the environment variable only when *it's* stdout/stderr isatty (also taking into account this environment variable), so that

I was curious how widespread bad usage of FileCheck prefixes is in the LLVM/Clang test suite. Running the attached script yields over 200 tests (~1.5%) which are essentially broken because they contain checks that are never checked. Most widespread is the usage of the "CHECK" prefix while all FileCheck invocations have an explicit check prefix. Then there are quite a few prefixes that

We currently don't use pipefail when running test under make check. This has the undesirable property that it is really easy for tests to bitrot. For example, something like llc %s | FileCheck %s will still pass if llc crashes after printing what FileCheck was looking for. It is also easy to break the tests when refactoring. I have fixed tests that were doing %clang_cc1 -a-driver-options

On further thought, I assume this issue actually hits everyone - it's not just Ninja getting in the way, but lit as well. (which also means we'd probably need to fix both Ninja and lit to resolve this issue - but everyone should at least be a little invested in fixing lit). On Sun, Jan 6, 2013 at 12:18 AM, Sean Silva <silvas at purdue.edu> wrote: > Nope. An easy solution (both

On 23.01.2013 00:45, Eli Bendersky wrote: > I do mean semantic extraction which provides a representation that's > meaningful to a user and hence can be effectively compared in a test. > But really, I gave up arguing on this topic a few messages (and heated > IRC discussions) ago. RFC retracted. I actually hit the same predicament for COFF. The few tests that are there usually use

On 10/12/2013 18:03, Jim Grosbach wrote: >> That causes dissonance between what the compiler sees and what lit.py >> sees for no particularly good reason. One of the nice properties of >> lit tests is that they're also valid compiler inputs, so trailing >> slash is a bit unfortunate. >> > > How does the backslash break this in any way? The backslash is

Hello, FileCheck allows us to define match variables and use them on subsequent lines. This is quite useful, but could be even more useful if it was possible to use the variable later on the same line it matched. For example, I would want to write this: ; CHECK: bic [[REG:r[0-9]+]], [[REG]], #3 But I currently can't because [[REG]] will only match a REG variable defined on a _previous_

If I understand correctly, the desire is to change the current behavior in a kind of subtle way. Is there some way you could instrument trunk's FileCheck to die if the old behavior is encountered, and use that to definitively find all tests which rely on the current behavior and migrate them (possibly to an interim solution) in preparation for the change in semantics? How much is the old

On 2019-01-24 20:17, David Greene via llvm-dev wrote: > alexp via llvm-dev <llvm-dev at lists.llvm.org> writes: > >> I have a project which uses llvm, but is (at least not yet) a >> contribution / not in the source tree of llvm. >> Is it a good idea (e.g. instead of using boost test framework) to use >> the llvm testsuite related tools in this case? > >

Hi all, quite a few tests use the pattern "2>&1 | FileCheck %s". AFAIK how stdout and stderr are merged into a single character stream is undefined and depends e.g. on whether stdout is buffered. I think we are often saved by the fact that standard output is written only at the end of the program and stderr is unbuffered, i.e. always written before stdout. A lot of tests disable

When writing MachineFunction unit tests, I find it quite tedious to verify correctness in C++. I would like to use FileCheck in UnitTests because FileCheck is extremely convenient/robust to verify correctness. In order to do so, I moved most of FileCheck’s implementation into a header (Support/FileCheck.h) and updated FileCheck.cpp to use this. I ran into this while writing some target agnostic

Hello all, I am writing test cases which are dedicated to be executed by llvm-lit. Most of my test cases have the same logic but different types. For example: // RUN: %clang_cc1 -fsyntax-only func (int a) { a = 3; } // RUN: %clang_cc1 -fsyntax-only func (char a) { a = 3; } // RUN: %clang_cc1 -fsyntax-only func (unsigned a) { a = 3; } Now I put them in three different test cases but it

On Sep 10, 2016, at 3:33 AM, Steve Canon <scanon at apple.com> wrote: >>> >>> Pretty much. In particular, imagine a user trying to debug an unexpected floating point result caused by conversion of a*b + c into fma(a, b, c). >> >> I think that’s unavoidable, because of the way the optimization levels work. Even fma contraction is on by default (something I’d

Hi, I was reading the FileCheck tool documentation [1] this morning and a two parts of the CHECK-DAG: directive don't quite seem right to me. 1. ``` With captured variables, CHECK-DAG: is able to match valid topological orderings of a DAG with edges from the definition of a variable to its use. ``` This is a little confusing. It is not explained what "captured variables" are. My

Hi, this patch fixes a problem with leading/trailing whitespace matching for FileCheck --strict-whitespace --match-full-lines. Consider a text file: ... $ cat DUMP bla1 bla2 bla3 bla4 bla5 ... with some leading and trailing spaces, made more visible like this: ... $ sed 's/ /_/g' DUMP bla1 bla2 _bla3 bla4_ _bla5_ ... and a FileCheck file CHECK to match DUMP: ... $ cat CHECK //

Hi, I was wondering if there is a way with llvm-lit and FileCheck to identify if a specific test should run or not based on the processor architecture. For example, I have a test that can run only in an Intel KNL, so if I run the tests in a Sandybridge I want to ignore that test. Any idea? Thanks! Simone ----------------------------------------------------------------------------------- This

+jyknight, who added --match-full-lines On 12/14/16 5:37 AM, Tom de Vries wrote: > Hi, > > this patch fixes a problem with leading/trailing whitespace matching > for FileCheck --strict-whitespace --match-full-lines. > > The resulting regexp string is '^bla3$' instead of '^ bla3$'. > > The patch fixes this, and makes the behavior match the documentation.

Hi Chris, On Thu, Jun 18, 2020 at 1:37 PM Chris Tetreault via llvm-dev < llvm-dev at lists.llvm.org> wrote: > The thing I use normally only shows the first N lines by default (I don’t > know off hand what N is). Honestly, I don’t feel very strongly about the > specific order, but it’s not useful when somebody proposes something on the > list, and nobody voices any dissent

Hello, I am not sold on FileCheck's new behaviour. For failing tests in verbose mode, it first dump the actual error messages, followed by the annotated input file to FileCheck. The result is I can't immediately see error messages if the input is more than just a few lines long, so I have to scroll all the way up to see the errors, then down again, etc. I do see some advantages of