Jason Kim <jasonwkim at google.com> writes:>>> There is definitely something to this. If I take a random failing >>> testcase and run the test in isolation in the shell, it works. So >>> what, if anything, does lit/FileCheck/etc. do that might run >>> interference if there is another copy of lit/FileCheck/etc. running >>> at the same time? I tried strace -etrace=file but nothing obvious >>> pops out. > > Perhaps its this? - http://llvm.org/bugs/show_bug.cgi?id=8199Good catch! I think that is probably it. I have old installations of llvm-gcc/opt/llc/etc. that I use to tell the new build where llvm-gcc is. It would explain why results for Debug/Release differ because I do separate installations for each build flavor and branch. This may also explain why I've sometimes seen tests working in the past where others see failures. Wow, this probably explains a lot. For now, I think if I tweak the way I do the build to always build without pointing to llvm-gcc first, build and test LLVM then build llvm-gcc and re-build LLVM, it should work. It will take much longer, though. :( -Dave
greened at obbligato.org (David A. Greene) writes:> For now, I think if I tweak the way I do the build to always build > without pointing to llvm-gcc first, build and test LLVM then build > llvm-gcc and re-build LLVM, it should work. It will take much longer, > though. :(I updated the bug explaining what I'm seeing. I think the correct fix is to use absolute paths to tools being tested rather than relying on PATH. -Dave
greened at obbligato.org (David A. Greene) writes:> greened at obbligato.org (David A. Greene) writes: > >> For now, I think if I tweak the way I do the build to always build >> without pointing to llvm-gcc first, build and test LLVM then build >> llvm-gcc and re-build LLVM, it should work. It will take much longer, >> though. :( > > I updated the bug explaining what I'm seeing. I think the correct fix > is to use absolute paths to tools being tested rather than relying on > PATH.So I tried adding this to llvm.exp::substitute: [...] global llvmdsymutil valgrind grep gas bugpoint_topts llvmtoolsdir [...] # Replace references to llvm tools to tools in OBJDIR. regsub -all {bugpoint } $new_line "$llvmtoolsdir/bugpoint " new_line regsub -all {llc } $new_line "$llvmtoolsdir/llc " new_line regsub -all {lli } $new_line "$llvmtoolsdir/lli " new_line regsub -all {llvm-ar } $new_line "$llvmtoolsdir/llvm-ar " new_line regsub -all {llvm-as } $new_line "$llvmtoolsdir/llvm-as " new_line regsub -all {llvm-bcanalyzer } $new_line "$llvmtoolsdir/llvm-bcanalyzer " new_line regsub -all {llvm-dis } $new_line "$llvmtoolsdir/llvm-dis " new_line regsub -all {llvm-extract } $new_line "$llvmtoolsdir/llvm-extract " new_line regsub -all {llvm-ld } $new_line "$llvmtoolsdir/llvm-ld " new_line regsub -all {llvm-link } $new_line "$llvmtoolsdir/llvm-link " new_line regsub -all {llvm-nm } $new_line "$llvmtoolsdir/llvm-nm " new_line regsub -all {llvm-prof } $new_line "$llvmtoolsdir/llvm-prof " new_line regsub -all {llvm-ranlib } $new_line "$llvmtoolsdir/llvm-ranlib " new_line regsub -all {([^a-zA-Z_-])opt } $new_line "$llvmtoolsdir/\\1opt " new_line regsub -all {opt } $new_line "$llvmtoolsdir/opt " new_line regsub -all {tblgen } $new_line "$llvmtoolsdir/tblgen " new_line It appears to work for clang tests: [x86_64-off-opt]: FAIL: Clang :: CodeCompletion/ordinary-name.c (432 of 8411) [x86_64-off-opt]: ******************** TEST 'Clang :: CodeCompletion/ordinary-name.c' FAILED ******************** [x86_64-off-opt]: Script: [x86_64-off-opt]: -- [x86_64-off-opt]: /ptmp/dag/build.llvm.trunk.official.opt/x86_64-unknown-linux-gnu/Release+Asserts/bin/clang -cc1 -isystem /ptmp/dag/llvm-project.official/llvm/trunk/tools/clang/test/CodeCompletion/Inputs -fsyntax-only -code-completion-at=/ptmp/dag/llvm-project.official/llvm/trunk/tools/clang/test/CodeCompletion/ordinary-name.c:6:9 /ptmp/dag/llvm-project.official/llvm/trunk/tools/clang/test/CodeCompletion/ordinary-name.c -o - | FileCheck -check-prefix=CHECK-CC1 /ptmp/dag/llvm-project.official/llvm/trunk/tools/clang/test/CodeCompletion/ordinary-name.c [x86_64-off-opt]: /ptmp/dag/build.llvm.trunk.official.opt/x86_64-unknown-linux-gnu/Release+Asserts/bin/clang -cc1 -isystem /ptmp/dag/llvm-project.official/llvm/trunk/tools/clang/test/CodeCompletion/Inputs -fsyntax-only -code-completion-at=/ptmp/dag/llvm-project.official/llvm/trunk/tools/clang/test/CodeCompletion/ordinary-name.c:1:11 /ptmp/dag/llvm-project.official/llvm/trunk/tools/clang/test/CodeCompletion/ordinary-name.c [x86_64-off-opt]: -- [x86_64-off-opt]: Exit Code: 134 but it does not appear to do anything for LLVM tests: [x86_64-off-opt]: FAIL: LLVM :: Analysis/BasicAA/2010-09-15-GEP-SignedArithmetic.ll (2731 of 8411) [x86_64-off-opt]: ******************** TEST 'LLVM :: Analysis/BasicAA/2010-09-15-GEP-SignedArithmetic.ll' FAILED ******************** [x86_64-off-opt]: Script: [x86_64-off-opt]: -- [x86_64-off-opt]: opt < /ptmp/dag/llvm-project.official/llvm/trunk/test/Analysis/BasicAA/2010-09-15-GEP-SignedArithmetic.ll -basicaa -aa-eval -print-all-alias-modref-info -disable-output |& grep {1 may alias} [x86_64-off-opt]: -- [x86_64-off-opt]: Exit Code: 1 [x86_64-off-opt]: [x86_64-off-opt]: ******************** I cannot figure out where in lit the test script gets printed out. Any hints on how to make absolute paths stick? -Dave