search for: execution_time

Hi Folks, Looks like our LNT ARM buildbot with the vectorizer is running and producing good results. There are only 11 failures: FAIL: MultiSource/Applications/Burg/burg.execution_time (1 of 1104) FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (2 of 1104) FAIL: MultiSource/Applications/lemon/lemon.execution_time (3 of 1104) FAIL: MultiSource/Applications/sqlite3/sqlite3.execution_time (4 of 1104) FAIL: MultiSource/Benchmarks/McCat/12-IOtest/iotest.execution_time (5...

...could see where we are degrading things? When you say good results, I take it you mean successfully completing the test, not execution time of the resulting binary? Or did you do an analysis of performance, too? > There are only 11 failures: > > FAIL: MultiSource/Applications/Burg/burg.execution_time (1 of 1104) > FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (2 of 1104) > FAIL: MultiSource/Applications/lemon/lemon.execution_time (3 of 1104) > FAIL: MultiSource/Applications/sqlite3/sqlite3.execution_time (4 of 1104) > FAIL: MultiSource/Benchmarks/McCat/12-IOtest/iote...

... especially as we did not spend  >too much time optimising this. Yes, we should look into the compile-execution performance trade-off.  I have summarized some benchmarks (compile-time overhead is more than 200%) as follows: SingleSource/Benchmarks/Shootout/nestedloop, compile_time(+6355.56%), execution_time(-99.21%) SingleSource/Benchmarks/Polybench/stencils/seidel-2d/seidel-2d, compile_time(+1275.00%), execution_time (0%) SingleSource/Benchmarks/Shootout-C++/nestedloop, compile_time(+1155.56%), execution_time(-99.23%) MultiSource/Benchmarks/ASC_Sequoia/AMGmk/AMGmk, compile_time(+491.80%), executio...

Hi Renato, I've investigated a few of these for AArch64 recently, and some of the results will be applicable in the 32-bit world too. > MultiSource/Benchmarks/McCat/12-IOtest/iotest.execution_time This is because of disagreement between ABIs over whether "char" is signed. ARM says no, x86 says yes. > MultiSource/Benchmarks/MiBench/automotive-bitcount/automotive-bitcount.execution_time This is also failing on x86, I think (http://lab.llvm.org:8011/builders/clang-x86_64-debian-...

There is almost certainly a bug in lnt or the makefiles. I changed the body of Burg main to the following: + printf("Hello World\n"); + return 0; I re-ran the test-suite again and got the following errors: --- Tested: 986 tests -- FAIL: MultiSource/Applications/Burg/burg.execution_time (494 of 986) FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (495 of 986) FAIL: MultiSource/Applications/lemon/lemon.execution_time (496 of 986) FAIL: MultiSource/Benchmarks/MiBench/automotive-bitcount/automotive-bitcount.exec ution_time (497 of 986) FAIL: MultiSource/Benchmarks/MiB...

...Importing 'tmpItDGOn.json' Import succeeded. Processing Times ---------------- Load : 0.00s Import : 0.22s Report : 0.48s Total : 0.48s Imported Data ------------- Added Machines: 1 Added Runs : 1 Added Tests : 493 --- Tested: 986 tests -- FAIL: MultiSource/Applications/Burg/burg.execution_time (494 of 986) FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (495 of 986) FAIL: MultiSource/Applications/lemon/lemon.execution_time (496 of 986) FAIL: MultiSource/Benchmarks/MiBench/automotive-bitcount/automotive- bitcount.execution_time (497 of 986) FAIL: MultiSource/Benchmarks/MiBen...

On 08/10/2013 06:59 PM, Star Tan wrote: > Hi all, > > I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981). Results can be viewed on: http://188.40.87.11:8000. Hi Star Tan, thanks for the update. > There are mainly five new tests and each test is run with 10 samples: > clang (run id = 27): clang -O3 > pollyBasic (run id =

Hi, I figured out how to resolve the failures. I noticed that Mountain Lion includes Bison 2.3 while Ubuntu 12.04 includes Bison 2.5. I installed Bison 2.3 from source in Ubuntu and the failures went away. I'm a little concerned that the bison version fixed all the failures I was seeing. To my knowledge the only failing test that depended on bison was Burg. It almost looks like one failure

...or the makefiles. > > I changed the body of Burg main to the following: > > + printf("Hello World\n"); > + return 0; > > > I re-ran the test-suite again and got the following errors: > > --- Tested: 986 tests -- > FAIL: MultiSource/Applications/Burg/burg.execution_time (494 of 986) > FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (495 of 986) > FAIL: MultiSource/Applications/lemon/lemon.execution_time (496 of 986) > FAIL: > MultiSource/Benchmarks/MiBench/automotive-bitcount/automotive-bitcount.exec > ution_time (497 of 986) > FAIL...

Hi all, I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981).  Results can be viewed on: http://188.40.87.11:8000. There are mainly five new tests and each test is run with 10 samples: clang (run id = 27):  clang -O3 pollyBasic (run id = 28):  clang -O3 -load LLVMPolly.so pollyNoGen (run id = 29):  pollycc -O3 -mllvm -polly-optimizer=none

...mbine. Benchmark Results ================= These are performance differences reported by LNT when running llvm-test-suite, spec2000, and spec2006 at -O3 with and without the patch linked above (using trunk llvm from a week or so ago). AArch64 results on ARM Cortex-A72: Performance Regressions - execution_time Change SingleSource/Benchmarks/Shootout/Shootout-ary3 9.48% MultiSource/Benchmarks/TSVC/Packing-flt/Packing-flt 3.79% SingleSource/Benchmarks/CoyoteBench/huffbench 1.40% Performance Improvements - exec...

Hi, I get the following failures when I run the test-suite on linux (Ubuntu 12.04) using LNT (lnt runtest nt ...): (all are execution failures) MultiSource/Applications/Burg MultiSource/Applications/ClamAV MultiSource/Applications/lemon MultiSource/Applications/obsequi MultiSource/Benchmarks/MiBench/automotive-bitcount

...added -m32: lnt runtest nt --sandbox SANDBOX --cflag=-m32 --cc <path-to-my-clang> --test-suite /data/repo/test-suite -j 8 I get three failures: --- Tested: 2465 tests -- FAIL: MultiSource/Applications/ClamAV/clamscan.compile_time (1 of 2465) FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (494 of 2465) FAIL: MultiSource/Benchmarks/DOE-ProxyApps-C/XSBench/XSBench.execution_time (495 of 2465) Is this known/expected or do I do something stupid? Thanks, Mikael

The vectorizer buildbot currently shows 4 miscompiles: http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/1727/steps/lnt.nightly-test/logs/tests.FAIL MultiSource/Applications/oggenc/oggenc.execution_time MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000.execution_time MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset.execution_time MultiSource/Benchmarks/Olden/power/power.execution_time I had looked at the first one (oggenc) a couple of weeks ago, and because there is only one sign...

...ance differences reported by LNT when running > llvm-test-suite, > spec2000, and spec2006 at -O3 with and without the patch linked > above (using > trunk llvm from a week or so ago). > > AArch64 results on ARM Cortex-A72: > > Performance Regressions - execution_time           Change > SingleSource/Benchmarks/Shootout/Shootout-ary3                9.48% > MultiSource/Benchmarks/TSVC/Packing-flt/Packing-flt               >   3.79% > SingleSource/Benchmarks/CoyoteBench/huffbench                 1.40% > > Performance Improvemen...

...reported by LNT when running llvm-test-suite, >>> spec2000, and spec2006 at -O3 with and without the patch linked above (using >>> trunk llvm from a week or so ago). >>> >>> AArch64 results on ARM Cortex-A72: >>> >>> Performance Regressions - execution_time Change >>> SingleSource/Benchmarks/Shootout/Shootout-ary3 9.48% >>> MultiSource/Benchmarks/TSVC/Packing-flt/Packing-flt 3.79% >>> SingleSource/Benchmarks/CoyoteBench/huffbench...

...he item 3. Add a nav-link in the sidebar for the graph button, so that it is easy to navigate to the bottom of the page in order to graph 4. Add drop down boxes for Runs and Compare to (in the sidebar) that pops-down upon hover, rather than links to 4 neighboring runs. 5. Make the execution_time and compilation_time results table as sortable and a checkbox to select all the items in the table. a. Clicking the workload url in the table will open a new window instead of a tab, for simpler navigation. b. Next Steps: Add a pop-up to display the graph upon hovering over the workloa...

...g=-m32 --cc <path-to-my-clang> --test-suite /data/repo/test-suite -j 8 >> >> I get three failures: >> >> --- Tested: 2465 tests -- >> FAIL: MultiSource/Applications/ClamAV/clamscan.compile_time (1 of 2465) >> FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (494 of 2465) >> FAIL: MultiSource/Benchmarks/DOE-ProxyApps-C/XSBench/XSBench.execution_time (495 of 2465) >> >> Is this known/expected or do I do something stupid? >> >> Thanks, >> Mikael >> _______________________________________________ >> LLVM D...

...cc <path-to-my-clang> > --test-suite /data/repo/test-suite -j 8 > > > > I get three failures: > > > > --- Tested: 2465 tests -- > > FAIL: MultiSource/Applications/ClamAV/clamscan.compile_time (1 of 2465) > > FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (494 of > 2465) > > FAIL: MultiSource/Benchmarks/DOE-ProxyApps-C/XSBench/XSBench.execution_time > (495 of 2465) > > > > Is this known/expected or do I do something stupid? > > > > Thanks, > > Mikael > > _______________________________________________...

Hi Daniel, I'm trying to add LivermoreLoops test to the benchmark suite (tar ball attached), but I'm getting the error below: --- Tested: 2 tests -- FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.compile_time (1 of 2) FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.execution_time (2 of 2) When I use the option to only run this test: --only-test SingleSource/Benchmarks/LivermoreLoops I can see the binary and if I execute, it runs (still missing bits and bobs, but ignore that for now). I've seen some files with extension "reference_output", does that have any...