similar to: Noisy benchmark results?

Two other things: 1) I get massively more stable execution times on 16.04 than on 14.04 on both x86 and ARM because 16.04 does far fewer gratuitous moves from one core to another, even without explicit pinning. 2) turn off ASLR: "echo 0 > /proc/sys/kernel/randomize_va_space". As well as getting stable addresses for debugging repeatability, it also stabilizes execution time

> On Feb 27, 2017, at 1:36 AM, Kristof Beyls via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hi Mikael, > > Some noisiness in benchmark results is expected, but the numbers you see seem to be higher than I'd expect. > A number of tricks people use to get lower noise results are (with the lnt runtest nt command line options to enable it between brackets): > *

Hi, I get the following failures when I run the test-suite on linux (Ubuntu 12.04) using LNT (lnt runtest nt ...): (all are execution failures) MultiSource/Applications/Burg MultiSource/Applications/ClamAV MultiSource/Applications/lemon MultiSource/Applications/obsequi MultiSource/Benchmarks/MiBench/automotive-bitcount

Hi, I figured out how to resolve the failures. I noticed that Mountain Lion includes Bison 2.3 while Ubuntu 12.04 includes Bison 2.5. I installed Bison 2.3 from source in Ubuntu and the failures went away. I'm a little concerned that the bison version fixed all the failures I was seeing. To my knowledge the only failing test that depended on bison was Burg. It almost looks like one failure

There is almost certainly a bug in lnt or the makefiles. I changed the body of Burg main to the following: + printf("Hello World\n"); + return 0; I re-ran the test-suite again and got the following errors: --- Tested: 986 tests -- FAIL: MultiSource/Applications/Burg/burg.execution_time (494 of 986) FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (495 of 986) FAIL:

On 08/10/2013 06:59 PM, Star Tan wrote: > Hi all, > > I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981). Results can be viewed on: http://188.40.87.11:8000. Hi Star Tan, thanks for the update. > There are mainly five new tests and each test is run with 10 samples: > clang (run id = 27): clang -O3 > pollyBasic (run id =

Hi all, I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981).  Results can be viewed on: http://188.40.87.11:8000. There are mainly five new tests and each test is run with 10 samples: clang (run id = 27):  clang -O3 pollyBasic (run id = 28):  clang -O3 -load LLVMPolly.so pollyNoGen (run id = 29):  pollycc -O3 -mllvm -polly-optimizer=none

At 2013-08-12 01:18:30,"Tobias Grosser" <tobias at grosser.es> wrote: >On 08/10/2013 06:59 PM, Star Tan wrote: >> Hi all, >> >> I have evaluated Polly's performance on LLVM test-suite with latest LLVM (r188054) and Polly (r187981).  Results can be viewed on: http://188.40.87.11:8000. > >Hi Star Tan, > >thanks for the update. >

Hi Folks, Looks like our LNT ARM buildbot with the vectorizer is running and producing good results. There are only 11 failures: FAIL: MultiSource/Applications/Burg/burg.execution_time (1 of 1104) FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (2 of 1104) FAIL: MultiSource/Applications/lemon/lemon.execution_time (3 of 1104) FAIL:

While porting my backends to llvm-3.2, I found a few places where the optimizers could have performed better. I believe the mainstream targets can also benefits from my tweaks. But before upstreaming my changes, I would like to quantify their merits on other applications --- not just my domain specific codes. In a word, it seemed the right time for me to start using LNT :) I followed the

tl;dr in low data situations we don’t look at past information, and that increases the false positive regression rate. We should look at the possibly incorrect recent past runs to fix that. Motivation: LNT’s current regression detection system has false positive rate that is too high to make it useful. With test suites as large as the llvm “test-suite” a single report will show hundreds of

On 22 Apr 2016, at 11:14, Mehdi Amini <mehdi.amini at apple.com<mailto:mehdi.amini at apple.com>> wrote: On Apr 22, 2016, at 12:45 AM, Kristof Beyls via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: On 21 Apr 2016, at 17:44, Sergey Yakoushkin <sergey.yakoushkin at gmail.com<mailto:sergey.yakoushkin at gmail.com>> wrote: Hi

Hi, I wrote a pass in LLVM and would like to test it on the programs in LLVM test-suite. To be specific, I would like to get the stats upon running my pass on these programs. What are the commands to run my pass on the test suite? -- Rekha -------------- next part -------------- An HTML attachment was scrubbed... URL:

Of course it'll be running as now. But user will need have installed MongoDB. Installation on linux with support of .deb packages is quite easy. sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927 echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/3.2 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list sudo apt-get update sudo apt-get

From: Elena Lepilkina Sent: Monday, April 25, 2016 2:33 PM To: 'James Molloy' <james at jamesmolloy.co.uk>; Kristof Beyls <Kristof.Beyls at arm.com>; Mehdi Amini <mehdi.amini at apple.com> Cc: nd <nd at arm.com>; Matthias Braun <matze at braunis.de> Subject: RE: [llvm-dev] RFC: LNT/Test-suite support for custom metrics and test parameterization Hi everyone,

Hi, Chris. Thank you for your answer about compile tests. As I understood during looking through code of compile tests they don’t use test suite at all. Am I right? There is lack of information and examples of running compile tests in LNT documentation. We understood that there are two groups of users: users using servers and collecting a lot of data and SQLite users, but these users as I think

Summary ======= I'm planning on adjusting SimplifyCFG so that it doesn't turn two-entry phi nodes into selects until later in the pass pipeline, to give passes which can understand phis but not selects more opportunity to optimize. The thing I'm trying to do which made me think of doing this is described below, but from the benchmarking I've done it looks like this is overall a

Hi Elena, Thanks for pushing forward with this. I like the idea of using a NoSQL solution. My primary reservation is about adding the new NoSQL stuff as an extra backend. I think the current database backend and its use of SQLAlchemy is extremely complex and is in fact the most complex part of LNT. Adding something more (as opposed to *replacing* it) would just make this worse and make it more

On Sun, Jan 20, 2013 at 1:26 PM, Redmond, Paul <paul.redmond at intel.com> wrote: > There is almost certainly a bug in lnt or the makefiles. > > I changed the body of Burg main to the following: > > + printf("Hello World\n"); > + return 0; > > > I re-ran the test-suite again and got the following errors: > > --- Tested: 986 tests -- > FAIL:

Hi, I'm trying to setup the test-suite and am getting short of running it... Following the steps here: http://llvm.org/docs/lnt/quickstart.html I got as far as getting lnt installed, but when I run the "lnt runtest nt" it fails: compilers.py:81: error: unable to determine compiler version: 'clang++': 'clang version 3.2 ' Seems my string doesn't have a