I was attempting to write a test that involves grepping though the stderr produced by opt -analyze, but found that my test was passing even before I fixed the bug I was writing the test for! I found that this one-line sure-fail test: ; RUN: echo hi |& false passes. I also tried 2>&1, because I've seen some tests do that, though that doesn't appear to work in this context either. It didn't seem to create a file nameed &1 though, as the LLVM test-writing documentation says. After some investigation, I believe the problem is due to the regexp on this line in test/lib/llvm.exp: } elseif {[regexp {RUN: *([^&]+)(&&)?} $line match oneline suffix]} { I don't understand what this is trying to do. What is intended by the special handling of && here? I found only one test that uses && on a RUN line, test/CodeGen/X86/2007-07-03-GR64ToVR64.ll, and it's not clear to me what it's doing there. Dan -- Dan Gohman, Cray Inc.
On Nov 20, 2007, at 8:22 AM, Dan Gohman wrote:> I was attempting to write a test that involves grepping though the > stderr produced by opt -analyze, but found that my test was passing > even before I fixed the bug I was writing the test for! > > I found that this one-line sure-fail test: > > ; RUN: echo hi |& false > > passes. I also tried 2>&1, because I've seen some tests do that, > though that doesn't appear to work in this context either. It didn't > seem to create a file nameed &1 though, as the LLVM test-writing > documentation says. > > After some investigation, I believe the problem is due to the regexp > on this line in test/lib/llvm.exp: > > } elseif {[regexp {RUN: *([^&]+)(&&)?} $line match oneline > suffix]} { > > I don't understand what this is trying to do. What is intended by > the special handling of && here? > > I found only one test that uses && on a RUN line, > test/CodeGen/X86/2007-07-03-GR64ToVR64.ll, and it's not clear to > me what it's doing there. >I think that it's a hold-over to how things used to be done. IIRC, you had to have the && at the end of the RUN line if you had another RUN line that needed to be executed. That's no longer the case, of course. If you want to catch stderr, there is one test doing that: Analysis/ScalarEvolution/2007-07-15-NegativeStride.ll: ; RUN: llvm-as < %s | opt -analyze -scalar-evolution 2>&1 | grep "Loop bb: 100 iterations" -bw
On Tue, Nov 20, 2007 at 07:01:25PM -0800, Bill Wendling wrote:> I think that it's a hold-over to how things used to be done. IIRC, > you had to have the && at the end of the RUN line if you had another > RUN line that needed to be executed. That's no longer the case, of > course.Thanks. I'll remove the && from the one test that still has it then. Then we can look at removing the associated code from llvm.exp and fixing the bug. This will take a while though, at it exposes a large number of failures that are currently hidden.> If you want to catch stderr, there is one test doing that: > > Analysis/ScalarEvolution/2007-07-15-NegativeStride.ll: > > ; RUN: llvm-as < %s | opt -analyze -scalar-evolution 2>&1 | grep > "Loop bb: 100 iterations"2>&1 doesn't evoke the desired behavior; the RUN line is unfortunately interpreted by TCL rather than a regular shell. The llvm.exp bug has allowed this and a variety of other common bourne-shell-isms to slip by unnoticed. Dan -- Dan Gohman, Cray Inc.
Reasonably Related Threads
- [LLVMdev] Problem with regression tests using stderr
- [LLVMdev] Problem with regression tests using stderr
- [LLVMdev] Problem with regression tests using stderr
- [LLVMdev] Problem with regression tests using stderr
- [LLVMdev] [PATCH] CMake add_version_info_from_vcs SVN_REPOSITORY