similar to: Benchmarks for LLVM-generated Binaries

> On Sep 1, 2016, at 8:14 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > On 1 September 2016 at 07:45, Dean Michael Berris via llvm-dev > <llvm-dev at lists.llvm.org> wrote: >> I've lately been wondering where benchmarks for LLVM-generated binaries are hosted, and whether they're tracked over time. > > Hi Dean, > > Do you

Hi Eric, Yeah, I know about Externals and SPEC specifically. But as far as I understand, you have to have kind of description of the tests in test-suite even if you don’t provide the source codes - that’s what I would like to avoid. I.e. you have to have CMakeLists.txt and other files in place all the time, open to everyone. Now, imagine I have a small testsuite, which probably is not very

> On 2 Sep 2016, at 01:14, Renato Golin <renato.golin at linaro.org> wrote: > > On 1 September 2016 at 07:45, Dean Michael Berris via llvm-dev > <llvm-dev at lists.llvm.org> wrote: >> I've lately been wondering where benchmarks for LLVM-generated binaries are hosted, and whether they're tracked over time. > > Hi Dean, > > Do you mean Perf? >

tl;dr in low data situations we don’t look at past information, and that increases the false positive regression rate. We should look at the possibly incorrect recent past runs to fix that. Motivation: LNT’s current regression detection system has false positive rate that is too high to make it useful. With test suites as large as the llvm “test-suite” a single report will show hundreds of

On 21 Apr 2016, at 17:44, Sergey Yakoushkin <sergey.yakoushkin at gmail.com<mailto:sergey.yakoushkin at gmail.com>> wrote: Hi Kristof, The way we use LNT, we would run different configuration (e.g. -O3 vs -Os) as different "machines" in LNT's model. O2/O3 is indeed bad example. We're also using different machines for Os/O3 - such parameters apply to all

Hi, The new LNT server instance http://lnt.llvm.org seems to fail in many cases. Any entrance to a 'Run page' (e.g. http://lnt.llvm.org/db_default/v4/nts/62475) and lately also many perf bots result submissions (e.g. http://lab.llvm.org:8014/builders/clang-native-arm-lnt-perf/builds/2262/steps/test-suite/logs/stdio ) fails with: "500 Internal Server Error". Any ideas? Thanks,

> On Jan 24, 2017, at 7:18 AM, Sanjay Patel <spatel at rotateright.com> wrote: > > > > On Mon, Jan 23, 2017 at 10:53 PM, Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: > >> On Jan 23, 2017, at 3:48 PM, Sanjay Patel via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >>

> On Jan 23, 2017, at 3:48 PM, Sanjay Patel via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > All targets are likely affected in some way by the icmp+shl fold introduced with r292492. It's a basic pattern that occurs in lots of code. Did you see any perf wins on your targets with this commit? > > Sadly, it is also likely that many (all?) targets are negatively

I've set up a public LNT server to show the result of perf stat. There is a huge improvement compared with timeit tool. http://parkas16.inria.fr:8000/ Patch is updated to pin the process to a single core, the readings are even more accurate. It's hard coded to run everything on core 0, so don't run parallel testing with it for now. The tool now depends on Linux perf and schedtool.

The run page problem were triggered by one of my commits (sorry) and should be mitigated now, see the thread at http://lists.llvm.org/pipermail/llvm-dev/2017-July/115971.html <http://lists.llvm.org/pipermail/llvm-dev/2017-July/115971.html> I don't know about the submission problems, could they just an occasional network problem or are they a common phenomenon? Chris did some

Is O3-vectorize redundant now that the loop vectorizer is enabled by default? On 2013-01-28, at 12:25 PM, David Blaikie wrote: > On Mon, Jan 28, 2013 at 8:57 AM, Redmond, Paul <paul.redmond at intel.com> wrote: >> Is there a reason why existing buildbots are not generating LNT results? > > Those running LNT should be/are: > >

Hi, I would like to announce a new set of LNT -O3 performance testers. In a discussion titled "Question about results reliability in LNT infrustructure" Anton suggested that one way to get statistically reliable test results from the LNT infrastructure is to use a larger sample size (5-10) as well as a more robust statistical test (Wilcoxon/Mann-Whitney). Another requirement to

Did something change with exception handling recently? A bunch of lit bots are showing slower compile times for many tests. Ciao, Duncan. On 20/06/12 07:53, llvm-testresults at cs.uiuc.edu wrote: > > lab-mini-03__O0-g__clang_DEV__x86_64 test results > <http://llvm.org/perf/db_default/v4/nts/1283?compare_to=1278&baseline=999> > > Run Order Start Time Duration >

Nothing that I'm aware of has changed with EH. Is it possible to bisect the problem? -bw On Jun 20, 2012, at 12:38 AM, Duncan Sands <baldrick at free.fr> wrote: > Did something change with exception handling recently? A bunch of lit bots are > showing slower compile times for many tests. > > Ciao, Duncan. > > On 20/06/12 07:53, llvm-testresults at cs.uiuc.edu

Hi, Does the LLVM project perform any kind of tracking for commit-by-commit compile-time changes? It looks like LNT only tracks run-time performance (and to be honest I wasn't able to make heads or tails of the results even for that -- the interface was pretty unintuitive to me.) While it is "normal" that each new LLVM release regresses compile-time by 2-3%, LLVM 10 seems to be

Hi Bill, > Nothing that I'm aware of has changed with EH. Is it possible to bisect the problem? I don't see any relevant LLVM changes, so I guess clang C++ compilation slowed down due to some clang changes. I'm not going to investigate this. Ciao, Duncan. > > -bw > > On Jun 20, 2012, at 12:38 AM, Duncan Sands <baldrick at free.fr> wrote: > >> Did

Are these 225 compile time regressions real? It sure looks bad! Ciao, Duncan. On 01/12/11 09:39, llvm-testresults at cs.uiuc.edu wrote: > > bwilson__llvm-gcc_PROD__i386 nightly tester results > > URL http://llvm.org/perf/db_default/simple/nts/380/ > Nickname bwilson__llvm-gcc_PROD__i386:4 > Name curlew.apple.com > > Run ID Order Start Time End Time > Current 380

Is there a reason why existing buildbots are not generating LNT results? On 2013-01-28, at 11:37 AM, David Blaikie wrote: They're just build bots running LNT - check the build bot configuration code in the zorg llvm project repository. You'll probably need to do some work to get a machine quiet enough to have reliable/useful performance results, though On Jan 28, 2013 8:33 AM,

Hi, I have been following the development of the /zorg/trunk/lnt project for a while and am wondering if there is some regular LLVM performance testing using LNT that can be accessed online? Are there any plans to create an officially used web service for this like e.g the llvm buildbots? Thanks a lot Tobi

Right - you usually won't see a normal distribution in the noise of test results. You'll see results clustered around the lower bound with a long tail of slower and slower results. Depending on how many samples you do it might be appropriate to take the mean of the best 3, for example - but the general approach of taking the fastest N does have some basis in any case. Not necessarily the