similar to: [LLVMdev] FileCheck + Ninja coloured output

Nope. An easy solution (both here and for clang) is for ninja to set an environment variable (`NINJA_ISATTY`?), and add some magic to LLVM's "isatty" to return true if it sees it. (for matters of correctness, it would actually be best if ninja sets the environment variable only when *it's* stdout/stderr isatty (also taking into account this environment variable), so that

On further thought, I assume this issue actually hits everyone - it's not just Ninja getting in the way, but lit as well. (which also means we'd probably need to fix both Ninja and lit to resolve this issue - but everyone should at least be a little invested in fixing lit). On Sun, Jan 6, 2013 at 12:18 AM, Sean Silva <silvas at purdue.edu> wrote: > Nope. An easy solution (both

So I manage to get clang's colored output through ninja simply by force (CMAKE_CXX_FLAGS=-fcolor-diagnostics), which isn't ideal (if I were to pipe ninja's output to a file it'd still have color escapes, etc) but it works. But I haven't found a similar solution for FileCheck & I'm wondering has anyone already solved this problem for themselves - if so, how? if not,

Hi All, FileCheck is a great tool for writing tests. However, it can be also a great tool for hiding the error cause from you, especially if a very common idiom RUN: ./myprog | FileCheck %s is used. If you need to know the output of “myprog” to figure out why your test is failing — then well, good luck, you would need to manually copy’n paste that line, run it, and then note the output.

I see many tests that are prefixed with "requires: shell". A quick search over the codebase shows 101 tests that contain this directive. This basically means there are 101 tests that say "I don't support running on Windows". I would like to get this number to 0. Ironically, many of these tests can be made to run on Windows simply by removing the requires: shell line.

llvm-lit is already put into <output-tree>/bin by CMake. There is a CMake target called `llvm-lit` that will generate the <output-dir>/bin/llvm-lit script. If you built only a few specific llvm targets such as FileCheck, etc then you may need to run this lit target manually. If you just run "ninja" for example, or "ninja check-llvm", it's automatically

Johannes, This patch seems to be causing test failures when I just do "ninja check", without running "ninja" or "ninja all" first. $ CC=clang CXX=clang++ cmake -G Ninja ~/git/llvm-project/llvm -DLLVM_USE_LINKER=lld -DLLVM_PARALLEL_LINK_JOBS=4 -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 && ninja check [...] FAIL: LLVM ::

This seems to be happening to me, too. I think this happens for me on "ninja check-llvm" in a clean build. Let me know if you need more info. On Tue, Dec 10, 2019 at 2:22 PM Doerfert, Johannes via llvm-dev < llvm-dev at lists.llvm.org> wrote: > Hi Jay, > > I am not sure why this happens. > Can you give me some information so I can reproduce it: > Is this happens

Sorry if I wasn't clear about my use case. In my daily dev work, I do many local "ninja check"s, or "llvm-lit" on a subdirectory as a quick(er) smoke test if I am making changes in that area (e.g. "llvm-lit ../llvm/test/CodeGen"). Nothing wrong here, as indeed nothing changed here. But in case of a test failure, I want to run just that test: bin/llvm-lit

Jay, Hiroshi, did [0] resolve the problem you've seen. [0] https://github.com/llvm/llvm-project/commit/d23c61490c282a7a8f29aaa5c021cbfdaf87fb6f On 12/10, Doerfert, Johannes wrote: > I think the problem is that only clang has a dependence on the new > library and when you check LLVM without building clang it does not exist > yet. I will commit a LLVM patch today that will cause LLVM

On Thu, Nov 5, 2020 at 11:36 AM David Blaikie via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > > > On Thu, Nov 5, 2020 at 10:46 AM Mircea Trofin <mtrofin at google.com> wrote: >> >> >> >> On Thu, Nov 5, 2020 at 10:40 AM David Blaikie <dblaikie at gmail.com> wrote: >>> >>> >>> >>> On Thu, Nov 5, 2020 at

> I don't know how you proceed to debug FileCheck failures, but for me most of the time I'll have to figure out which "RUN" line fail and try to execute it manually and then remove the FileCheck pipe to get the raw input and then painfully tried to match the FileCheck error to the actual input. Yeah, not very different from what you described here. If I 'm creating or

I have been working for extending test coverage for years. Nowadays, I have several cross-testing (target != host). See http://bb.pgr.jp/console Each of them (test-*-linux) is doing; - Assume a preceding builder passes with warming ccache. - All compilation units will hit ccache whenever the tree is built before lit. - Almost all compilation units will hit ccache except for Host.cpp when

On Thu, Jan 17, 2013 at 8:43 AM, Chris Lattner <clattner at apple.com> wrote: > On Jan 16, 2013, at 1:19 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >>> I agree that a command line option would be better. But in that case >>> all tests should be updated. It is not an issue for me -- it is >>> mostly mechanical. So should I change tests to use

On Jan 16, 2013, at 1:19 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >> I agree that a command line option would be better. But in that case >> all tests should be updated. It is not an issue for me -- it is >> mostly mechanical. So should I change tests to use %FileCheck? > > Here's a third attempt. Thanks in advance for driving this forward.

Hi all, I've got https://reviews.llvm.org/D37756 up for review currently which simplifies a lot of the logic in our lit config files. For most people, this will be completely transparent and "just work", but it breaks one workflow that people should be aware of. Problem: If you run lit by putting llvm-config in your path and then running <source-root>/utils/lit/lit.py

On Wed, Jan 16, 2013 at 9:24 PM, Chris Lattner <clattner at apple.com> wrote: > > On Jan 16, 2013, at 10:32 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: > >> Hello, >> >> When someone breaks a FileCheck-based test on some buildbot, sometimes >> it may not be obvious *why* did it fail. If the failure can not be >> reproduced locally, it can

Hello LLVM community, For testing IR passes, LLVM currently has two kinds of tests: 1. regression tests (in llvm/test); .ll files invoking opt, and matching its text output using FileCheck. 2. unittests (in llvm/unittests); Google tests containing the IR as a string, constructing a pass pipeline, and inspecting the output using code. I propose to add an additional kind of test, which I call

On Wed, Jan 16, 2013 at 9:33 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote: > On Wed, Jan 16, 2013 at 9:24 PM, Chris Lattner <clattner at apple.com> wrote: >> >> On Jan 16, 2013, at 10:32 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >> >>> Hello, >>> >>> When someone breaks a FileCheck-based test on some buildbot,

On Thu, Jan 17, 2013 at 8:36 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: > We have to options: > (a) replace 'FileCheck' with '%FileCheck' in all tests, and teach > 'lit' to replace '%FileCheck' with 'FileCheck --dump-input-on-error'; > > (b) teach 'lit' to replace a plain 'FileCheck'. > > The first approach