search for: litconfig

Hi, A feature I've wanted in lit for a while is a having a timeout per test. Attached are patches that implement this idea. I'm e-mailing llvm-dev rather than llvm-commits because I want to gather more feedback on my initial implementation and hopefully some answers to some unresolved issues with my implementation. Currently in lit you can set a global timeout for all of the tests but

...******** Testing Time: 176.39s ******************** Failing Tests (2): LLVM :: Transforms/SRETPromotion/basictest.ll LLVM-Unit :: support/debug/SupportTests.exe/CastingTest.cast Expected Passes : 8103 Expected Failures : 75 Unsupported Tests : 553 Unexpected Failures: 2 lit.py: LitConfig.py:99: warning: Unable to find 'bash.exe'. lit.py: LitConfig.py:99: warning: Unable to find 'bash.exe'. lit.py: lit.cfg:143: note: using clang: 'E:/llvm/build/cmake-vs8/bin/Debug/clang.EXE' 2 warning(s) in tests. -------------- next part -------------- Running Clang and LLVM...

...t; > lit.main() > File "M:\Development\Source\LLVM\utils\lit\lit\main.py", line 515, in main > testSuiteCache, localConfigCache)[1]) > File "M:\Development\Source\LLVM\utils\lit\lit\main.py", line 230, in getTests > ts,path_in_suite = getTestSuite(path, litConfig, testSuiteCache) > File "M:\Development\Source\LLVM\utils\lit\lit\main.py", line 199, in getTestSuite > ts, relative = search(item) > File "M:\Development\Source\LLVM\utils\lit\lit\main.py", line 183, in search > cache[path] = res = search1(path) > File &...

...r http://llvm.org/docs/TestingGuide.html: Under 'Requirements', might be worth mentioning that running the basic llvm/clang regression tests (check-all) requires a version of bash that supports pipefail, i.e. bash 3.0+. On older system where /bin/sh is too old, one simply needs to hack LitConfig.py to point to /somewhere/else/bash that meets this requirement. (This method has been tested on powerpc-darwin8.) Thanks! Fang -- David Fang http://www.csl.cornell.edu/~fang/

...the following test results to share. > Summary below, full log at: > http://www.csl.cornell.edu/~fang/sw/llvm/r146586-powerpc-darwin8-results.txt > > The only edits required were those I posted to llvm-commits yesterday (re: > "some missing clang libs").  And I also edited LitConfig.py to point to > /sw/bin/bash (4.2) because /bin/bash is missing support for pipefail. > > Some of the tests hung indefinitely on 'lli', so I had to kill them. > > My question is: which of the following test failures should be addressed first > as top priority?  Are there...

I'm not sure what LLVM_ENABLE_THREADS impacts - whether it is LLVM executables themselves or LLVM-generated code - but it seems to be on by default on the Windows CMake build (I'm building LLVM+Clang at trunk head using CMake/Visual Studio 2013 Win64). But 3 unit tests in IR, which are compiled only if define LLVM_ENABLE_THREADS is set, fail if LLVM_ENABLE_THEADS=ON:

...(g++-4.0.1), and have the following test results to share. Summary below, full log at: http://www.csl.cornell.edu/~fang/sw/llvm/r146586-powerpc-darwin8-results.txt The only edits required were those I posted to llvm-commits yesterday (re: "some missing clang libs"). And I also edited LitConfig.py to point to /sw/bin/bash (4.2) because /bin/bash is missing support for pipefail. Some of the tests hung indefinitely on 'lli', so I had to kill them. My question is: which of the following test failures should be addressed first as top priority? Are there any low-hanging fruit that...

...t;> +++ test-suite/trunk/litsupport/test.py Thu Feb 25 05:06:15 >>>>>>> 2016 >>>>>>> @@ -56,7 +56,7 @@ class TestSuiteTest(FileBasedTest): >>>>>>> res = testscript.parse(test.getSourcePath()) >>>>>>> if litConfig.noExecute: >>>>>>> return lit.Test.Result(Test.PASS) >>>>>>> - runscript, verifyscript = res >>>>>>> + runscript, verifyscript, metricscripts = res >>>>>>> >>>>>>> #...

...suite/trunk/litsupport/test.py Thu Feb 25 05:06:15 >>>>>>>> 2016 >>>>>>>> @@ -56,7 +56,7 @@ class TestSuiteTest(FileBasedTest): >>>>>>>> res = testscript.parse(test.getSourcePath()) >>>>>>>> if litConfig.noExecute: >>>>>>>> return lit.Test.Result(Test.PASS) >>>>>>>> - runscript, verifyscript = res >>>>>>>> + runscript, verifyscript, metricscripts = res >>>>>>>> >>>>&gt...

...upport/test.py Thu Feb 25 05:06:15 >>>>>>>>> 2016 >>>>>>>>> @@ -56,7 +56,7 @@ class TestSuiteTest(FileBasedTest): >>>>>>>>> res = testscript.parse(test.getSourcePath()) >>>>>>>>> if litConfig.noExecute: >>>>>>>>> return lit.Test.Result(Test.PASS) >>>>>>>>> - runscript, verifyscript = res >>>>>>>>> + runscript, verifyscript, metricscripts = res >>>>>>>>> &gt...

...results to share. >> Summary below, full log at: >> http://www.csl.cornell.edu/~fang/sw/llvm/r146586-powerpc-darwin8-results.txt >> >> The only edits required were those I posted to llvm-commits yesterday (re: >> "some missing clang libs").  And I also edited LitConfig.py to point to >> /sw/bin/bash (4.2) because /bin/bash is missing support for pipefail. >> >> Some of the tests hung indefinitely on 'lli', so I had to kill them. >> >> My question is: which of the following test failures should be addressed first >> as t...

...t;>> Summary below, full log at: >>> http://www.csl.cornell.edu/~fang/sw/llvm/r146586-powerpc-darwin8-results.txt >>> >>> The only edits required were those I posted to llvm-commits yesterday (re: >>> "some missing clang libs").  And I also edited LitConfig.py to point to >>> /sw/bin/bash (4.2) because /bin/bash is missing support for pipefail. >>> >>> Some of the tests hung indefinitely on 'lli', so I had to kill them. >>> >>> My question is: which of the following test failures should be addressed...

Hi, > Cool, I hope this succeeds. I tried implementing per-test timeouts before, and couldn't get it to work in all cases. The review eventually fizzled out, and I abandoned it. > > Here's that old review: http://reviews.llvm.org/D6584 Perhaps you can cannibalize testcases from it. Thanks for that. I'll take a look. > >> >> I'm e-mailing llvm-dev rather

Dear testers, Start your engines; 6.0.0-rc1 was just tagged. I know there are still open blockers and it's early in the process in a way, but I'd like to find out where we are. Please run the test script, let me know the results, and upload binaries. Thanks, Hans