similar to: [LLVMdev] New benchmark in test-suite

Hey Renato, You are right, the failure on compile_time indicates that the test isn't even building. As provided, the tests don't actually define the cpuida() or calculateMHz() functions so that seems expected to me. The compile failures end up getting buried in the logs, but they will either be in the test.log file in the top-level sandbox directory, or inside the

On 6 November 2012 22:34, Daniel Dunbar <daniel at zuster.org> wrote: > You are right, the failure on compile_time indicates that the test isn't > even building. As provided, the tests don't actually define the cpuida() or > calculateMHz() functions so that seems expected to me. I defined both functions as NOPs. I got what it was. The original makefile had a "-o

At 2013-08-12 01:18:30,"Tobias Grosser" <tobias at grosser.es> wrote: >On 08/10/2013 06:59 PM, Star Tan wrote: >> Hi all, >> >> I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981).  Results can be viewed on: http://188.40.87.11:8000. > >Hi Star Tan, > >thanks for the update. >

On 08/10/2013 06:59 PM, Star Tan wrote: > Hi all, > > I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981). Results can be viewed on: http://188.40.87.11:8000. Hi Star Tan, thanks for the update. > There are mainly five new tests and each test is run with 10 samples: > clang (run id = 27): clang -O3 > pollyBasic (run id =

Hi all, I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981).  Results can be viewed on: http://188.40.87.11:8000. There are mainly five new tests and each test is run with 10 samples: clang (run id = 27):  clang -O3 pollyBasic (run id = 28):  clang -O3 -load LLVMPolly.so pollyNoGen (run id = 29):  pollycc -O3 -mllvm -polly-optimizer=none

Hi, I'm trying to run the benchmark suite: http://llvm.org/docs/TestingGuide.html#test-suite-quickstart I'm doing it the lnt way, as described at: http://llvm.org/docs/lnt/quickstart.html I don't know what to expect but the results seems to be quite noisy and unstable. E.g I've done two runs on two different commits that only differ by a space in CODE_OWNERS.txt on my 12

On 16 November 2012 21:43, Nadav Rotem <nrotem at apple.com> wrote: > Once Michael Ilseman commits the fast math patch we will be able to implement floating point reductions. That's great news! Attached is the whole benchmark, divided into 24 kernels and running on LNT with FP comparison and timings. Unpack the file onto SingleSource/Benchmarks and change the Makefile to add

> On Feb 27, 2017, at 1:36 AM, Kristof Beyls via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hi Mikael, > > Some noisiness in benchmark results is expected, but the numbers you see seem to be higher than I'd expect. > A number of tricks people use to get lower noise results are (with the lnt runtest nt command line options to enable it between brackets): > *

Two other things: 1) I get massively more stable execution times on 16.04 than on 14.04 on both x86 and ARM because 16.04 does far fewer gratuitous moves from one core to another, even without explicit pinning. 2) turn off ASLR: "echo 0 > /proc/sys/kernel/randomize_va_space". As well as getting stable addresses for debugging repeatability, it also stabilizes execution time

I'm getting three failures. TEST-FAIL: exec /home/rkotler/llvmpb3/build/projects/test-suite/SingleSource/UnitTests/Vector/SSE/sse.expandfft TEST-RESULT-exec-time: user 0.3200 TEST-RESULT-exec-real-time: real 0.3172 TEST-FAIL: exec /home/rkotler/llvmpb3/build/projects/test-suite/SingleSource/UnitTests/Vector/SSE/sse.stepfft TEST-RESULT-exec-time: user 0.4000

Ok, I got the benchmark to work on test-suite, but it's not printing details for each run (or execution wouldn't work). I had to comment out the printf lines, but nothing more than that. I'm not sure how individual timings would have to be extracted, but the program produces output via text file, which can be used for comparison. Also, it does check the results and does report if they

On 8 January 2013 04:49, Duncan Sands <baldrick at free.fr> wrote: > While this should be investigated, > I'm tempted to just move everything over to LNT instead... > That's the latent bugs that David mentioned. I agree we should have LNT and LNT+LTO and possibly other configurations in the future. Regarding your buildbots, gcc12 is easy to replace by LNT, because the

I forgot to mention that you can also run "make TEST=simple report" which will generate a nice report. Do you know why these tests fail ? You can step into the test directory and run 'make TEST=simple' from there. It will save you some time. On Dec 12, 2012, at 4:04 PM, reed kotler <rkotler at mips.com> wrote: > I'm getting three failures. > > TEST-FAIL:

The first one failed on a diff: ******************** TEST (simple) 'sse.expandfft' FAILED! ******************** Execution Context Diff: /home/rkotler/llvmpb3/build/projects/test-suite/tools/fpcmp: Compared: 1.139094e-07 and 1.159249e-07 abs. diff = 2.015500e-09 rel.diff = 1.738626e-02 Out of tolerance: rel/abs: 1.600000e-02/0.000000e+00 ******************** TEST (simple)

On Tue, Jan 8, 2013 at 1:40 AM, Renato Golin <renato.golin at linaro.org> wrote: > On 8 January 2013 04:49, Duncan Sands <baldrick at free.fr> wrote: >> >> While this should be investigated, >> I'm tempted to just move everything over to LNT instead... > > > That's the latent bugs that David mentioned. I agree we should have LNT and > LNT+LTO

when I create the report, there are no failures in it. so maybe these are being filtered for known failures. On 12/12/2012 05:03 PM, reed kotler wrote: > The first one failed on a diff: > ******************** TEST (simple) 'sse.expandfft' FAILED! > ******************** > Execution Context Diff: > /home/rkotler/llvmpb3/build/projects/test-suite/tools/fpcmp: Compared: >

Hi Renato! Thanks for working on this! It's really important to have more array-ish benchmarks. On Nov 16, 2012, at 12:28 PM, Renato Golin <rengolin at systemcall.org> wrote: > Daniel, Nadav, Hal, > > So, after some painstakingly boring re-formatting, I've split the 24 > kernels into 24 files (and left a horrible header file with code in > it, which I'll

I use the 'make TEST=simple' as a pre-commit test. I think that everybody should run these tests before committing to LLVM. On Dec 12, 2012, at 5:06 PM, reed kotler <rkotler at mips.com> wrote: > when I create the report, there are no failures in it. so maybe these are being filtered for known failures. > > On 12/12/2012 05:03 PM, reed kotler wrote: >> The first

Daniel, Nadav, Hal, So, after some painstakingly boring re-formatting, I've split the 24 kernels into 24 files (and left a horrible header file with code in it, which I'll clean up later). Since we're taking times in the benchmark tool, and we're trying to assert the quality of the FP approximation by the vectorization, I'll try to come up with a reasonable watermark for each

That would be great! On Nov 5, 2012, at 2:11 PM, Renato Golin <rengolin at systemcall.org> wrote: > On 5 November 2012 17:41, Nadav Rotem <nrotem at apple.com> wrote: >> 1. We do not allow reductions on floating point types. We should allow them when unsafe-math is used. >> 2. All of the arrays are located in a struct. At the moment we don't detect that these