similar to: [LLVMdev] [RFC] Switching make check to use 'set -o pipefail'

Hi Rafael, I think you saw my other e-mail, but just in case you haven't, do you have any thoughts about making this an option that could be easily disabled on the command line without maintaing a patch to lit? I think it would help out-of-tree target maintainers to transition, since I'm sure there will be a lot of similarly broken tests to fix. Stephen On Thu, Jul 4, 2013 at 8:56 PM,

> Hmm, I don't know LLVM's Makefile system well enough to know the > easiest way to implement an option; if it's non-trivial then maybe > it's not worth it. That is my impression at least. These errors are somewhat easy to introduce, but also easy to fix. > I also don't know the workflow of most people doing out-of-tree work, > so I'm not sure how much

On Thu, Jul 4, 2013 at 8:56 PM, Rafael Espíndola <rafael.espindola at gmail.com> wrote: > We currently don't use pipefail when running test under make check. > This has the undesirable property that it is really easy for tests to > bitrot. Hi Rafael, Did this discussion ever get a conclusion? I support enabling pipefail. Fallout for out of tree users should be easy to fix.

> Hi Rafael, > > Did this discussion ever get a conclusion? I support enabling > pipefail. Fallout for out of tree users should be easy to fix. As we > learned from LLVM tests, almost all tests that start to fail actually > indicate a real problem that was hidden. So far I got some positive feedback, but no strong LGTM from someone in the area :-( > Dmitri > Cheers,

> I don't think it is all that many since it was less than one day of > work for the in tree ones. But if there is the desire for such an > option I can try to add it. What should I use? An environment > variable? Hmm, I don't know LLVM's Makefile system well enough to know the easiest way to implement an option; if it's non-trivial then maybe it's not worth it. I

On 5 July 2013 00:15, Stephen Lin <swlin at post.harvard.edu> wrote: > Hi Rafael, > > I think you saw my other e-mail, but just in case you haven't, do you > have any thoughts about making this an option that could be easily > disabled on the command line without maintaing a patch to lit? I think > it would help out-of-tree target maintainers to transition, since

On Wed, Jul 17, 2013 at 9:48 PM, Rafael Espíndola < rafael.espindola at gmail.com> wrote: > > Hi Rafael, > > > > Did this discussion ever get a conclusion? I support enabling > > pipefail. Fallout for out of tree users should be easy to fix. As we > > learned from LLVM tests, almost all tests that start to fail actually > > indicate a real problem that

On Wed, Jul 17, 2013 at 6:48 PM, Rafael Espíndola < rafael.espindola at gmail.com> wrote: > > Hi Rafael, > > > > Did this discussion ever get a conclusion? I support enabling > > pipefail. Fallout for out of tree users should be easy to fix. As we > > learned from LLVM tests, almost all tests that start to fail actually > > indicate a real problem that

> Ok, here is a strong LGTM. =] > > Please make the change, and do the following things to aid out-of-tree > maintainers: > > 1) Add a flag to lit and an option to configure/make (I don't care about > CMake here as that is much less frequently used for out-of-tree work) to > disable pipefail. > I have just fixed the last failures on windows. I have also added

I just committed a change (r187261) to use pipefail when running the regression tests. This means that tests that use pipes like opt .... | FileCheck will now fail if opt fails. This would have avoid some test bitrot in the past. For example, we had tests doing opt -S ... | not grep ... and they were still passing even after the the test itself was not even passing the verifier. If you have

So I manage to get clang's colored output through ninja simply by force (CMAKE_CXX_FLAGS=-fcolor-diagnostics), which isn't ideal (if I were to pipe ninja's output to a file it'd still have color escapes, etc) but it works. But I haven't found a similar solution for FileCheck & I'm wondering has anyone already solved this problem for themselves - if so, how? if not,

On 10/12/2013 18:03, Jim Grosbach wrote: >> That causes dissonance between what the compiler sees and what lit.py >> sees for no particularly good reason. One of the nice properties of >> lit tests is that they're also valid compiler inputs, so trailing >> slash is a bit unfortunate. >> > > How does the backslash break this in any way? The backslash is

> On 2 Jul 2018, at 15:13, Matthias Braun via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > I had similar gripes with unit testing machine function stuff. I personally would have preferred to create more tests based on a tool like llc rather than pushing more on the unit test side. Anyway I tried to push https://reviews.llvm.org/D48850 <https://reviews.llvm.org/D48850> in

Hello all, I am writing test cases which are dedicated to be executed by llvm-lit. Most of my test cases have the same logic but different types. For example: // RUN: %clang_cc1 -fsyntax-only func (int a) { a = 3; } // RUN: %clang_cc1 -fsyntax-only func (char a) { a = 3; } // RUN: %clang_cc1 -fsyntax-only func (unsigned a) { a = 3; } Now I put them in three different test cases but it

On Thu, Jan 17, 2013 at 8:36 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: > We have to options: > (a) replace 'FileCheck' with '%FileCheck' in all tests, and teach > 'lit' to replace '%FileCheck' with 'FileCheck --dump-input-on-error'; > > (b) teach 'lit' to replace a plain 'FileCheck'. > > The first approach

On Thu, Jan 17, 2013 at 1:19 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: > On Wed, Jan 16, 2013 at 9:33 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >> On Wed, Jan 16, 2013 at 9:24 PM, Chris Lattner <clattner at apple.com> wrote: >>> >>> On Jan 16, 2013, at 10:32 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >>>

On Wed, Jan 16, 2013 at 9:33 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote: > On Wed, Jan 16, 2013 at 9:24 PM, Chris Lattner <clattner at apple.com> wrote: >> >> On Jan 16, 2013, at 10:32 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >> >>> Hello, >>> >>> When someone breaks a FileCheck-based test on some buildbot,

On Thu, Jan 17, 2013 at 8:43 AM, Chris Lattner <clattner at apple.com> wrote: > On Jan 16, 2013, at 1:19 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >>> I agree that a command line option would be better. But in that case >>> all tests should be updated. It is not an issue for me -- it is >>> mostly mechanical. So should I change tests to use

On Wed, Jan 16, 2013 at 9:24 PM, Chris Lattner <clattner at apple.com> wrote: > > On Jan 16, 2013, at 10:32 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: > >> Hello, >> >> When someone breaks a FileCheck-based test on some buildbot, sometimes >> it may not be obvious *why* did it fail. If the failure can not be >> reproduced locally, it can

On Thu, Jan 17, 2013 at 1:23 AM, Evgeniy Stepanov <eugeni.stepanov at gmail.com> wrote: > On Thu, Jan 17, 2013 at 1:19 AM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >> On Wed, Jan 16, 2013 at 9:33 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote: >>> On Wed, Jan 16, 2013 at 9:24 PM, Chris Lattner <clattner at apple.com> wrote: >>>>