similar to: LNT debuginfo-statistics not running?

The debug info statistics bot is triggered by this job: http://green.lab.llvm.org/green/job/clang-stage2-Rthinlto/ which unfortunately hasn't been green in a very long time (>1mo). Alex/Azhar, do you know what's blocking that job? -- adrian > On Nov 20, 2019, at 9:46 AM, David Blaikie <dblaikie at gmail.com> wrote: > > +usual debug info folks (but I think in this case

Hi, The new LNT server instance http://lnt.llvm.org seems to fail in many cases. Any entrance to a 'Run page' (e.g. http://lnt.llvm.org/db_default/v4/nts/62475) and lately also many perf bots result submissions (e.g. http://lab.llvm.org:8014/builders/clang-native-arm-lnt-perf/builds/2262/steps/test-suite/logs/stdio ) fails with: "500 Internal Server Error". Any ideas? Thanks,

Hi, I have been following the development of the /zorg/trunk/lnt project for a while and am wondering if there is some regular LLVM performance testing using LNT that can be accessed online? Are there any plans to create an officially used web service for this like e.g the llvm buildbots? Thanks a lot Tobi

The run page problem were triggered by one of my commits (sorry) and should be mitigated now, see the thread at http://lists.llvm.org/pipermail/llvm-dev/2017-July/115971.html <http://lists.llvm.org/pipermail/llvm-dev/2017-July/115971.html> I don't know about the submission problems, could they just an occasional network problem or are they a common phenomenon? Chris did some

Hi, I am currently investigating how to ensure that LNT only shows relevant performance regressions for the -O3 performance tests I am running. One question that came up here is why the default aggregate function for LNT is 'min' instead of 'mean'. This looks a little surprising from the statistical point, but also from looking at my test results picking 'min' seems

Right - you usually won't see a normal distribution in the noise of test results. You'll see results clustered around the lower bound with a long tail of slower and slower results. Depending on how many samples you do it might be appropriate to take the mean of the best 3, for example - but the general approach of taking the fastest N does have some basis in any case. Not necessarily the

On Fri, Apr 19, 2013 at 1:13 PM, Renato Golin <renato.golin at linaro.org>wrote: > On 19 April 2013 17:48, Török Edwin <edwin at etorok.net> wrote: > >> Otherwise what might seem like a 20% improvement >> could very well be just a 0.2% improvement in practice. >> > > This is (maybe to a lesser extent) what happens with most of our > benchmarks, and

On Dec 6, 2010, at 10:40 AM, Tobias Grosser wrote: > Hi, > > I have been following the development of the /zorg/trunk/lnt project for > a while and am wondering if there is some regular LLVM performance > testing using LNT that can be accessed online? Are there any plans to > create an officially used web service for this like e.g the llvm buildbots? I have a nightly tester

On 12/06/2010 03:33 PM, Bob Wilson wrote: > > On Dec 6, 2010, at 10:40 AM, Tobias Grosser wrote: > >> Hi, >> >> I have been following the development of the /zorg/trunk/lnt project for >> a while and am wondering if there is some regular LLVM performance >> testing using LNT that can be accessed online? Are there any plans to >> create an officially

> On Jan 23, 2017, at 3:48 PM, Sanjay Patel via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > All targets are likely affected in some way by the icmp+shl fold introduced with r292492. It's a basic pattern that occurs in lots of code. Did you see any perf wins on your targets with this commit? > > Sadly, it is also likely that many (all?) targets are negatively

On 19 February 2013 15:16, Arnold Schwaighofer <aschwaighofer at apple.com>wrote: > Do you have a base run with vectorization turned off? So we could see > where we are degrading things? > I wanted to, but after a few failed attempts, I couldn't pass the option to clang to disable vectorization. I don't want to make Galina reconfig the master every time, so I set up a

> On Jan 24, 2017, at 7:18 AM, Sanjay Patel <spatel at rotateright.com> wrote: > > > > On Mon, Jan 23, 2017 at 10:53 PM, Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: > >> On Jan 23, 2017, at 3:48 PM, Sanjay Patel via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >>

Hi, I've lately been wondering where benchmarks for LLVM-generated binaries are hosted, and whether they're tracked over time. I'm asking because I'm thinking of where to put some benchmarks I've written using the open source Google benchmarking library [0] to test certain costs of XRay-instrumented binaries, the XRay runtime, and other related measurements (effect of

On 21 Apr 2016, at 17:44, Sergey Yakoushkin <sergey.yakoushkin at gmail.com<mailto:sergey.yakoushkin at gmail.com>> wrote: Hi Kristof, The way we use LNT, we would run different configuration (e.g. -O3 vs -Os) as different "machines" in LNT's model. O2/O3 is indeed bad example. We're also using different machines for Os/O3 - such parameters apply to all

tl;dr in low data situations we don’t look at past information, and that increases the false positive regression rate. We should look at the possibly incorrect recent past runs to fix that. Motivation: LNT’s current regression detection system has false positive rate that is too high to make it useful. With test suites as large as the llvm “test-suite” a single report will show hundreds of

> On Sep 1, 2016, at 8:14 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > On 1 September 2016 at 07:45, Dean Michael Berris via llvm-dev > <llvm-dev at lists.llvm.org> wrote: >> I've lately been wondering where benchmarks for LLVM-generated binaries are hosted, and whether they're tracked over time. > > Hi Dean, > > Do you

On Thu, Jan 16, 2014 at 5:32 PM, Tobias Grosser <tobias at grosser.es> wrote: > On 01/17/2014 02:17 AM, David Blaikie wrote: > >> Right - you usually won't see a normal distribution in the noise of test >> results. You'll see results clustered around the lower bound with a long >> tail of slower and slower results. Depending on how many samples you do it

Hi Eric, Yeah, I know about Externals and SPEC specifically. But as far as I understand, you have to have kind of description of the tests in test-suite even if you don’t provide the source codes - that’s what I would like to avoid. I.e. you have to have CMakeLists.txt and other files in place all the time, open to everyone. Now, imagine I have a small testsuite, which probably is not very

Hi, I would like to announce a new set of LNT -O3 performance testers. In a discussion titled "Question about results reliability in LNT infrustructure" Anton suggested that one way to get statistically reliable test results from the LNT infrastructure is to use a larger sample size (5-10) as well as a more robust statistical test (Wilcoxon/Mann-Whitney). Another requirement to

Hi, Does the LLVM project perform any kind of tracking for commit-by-commit compile-time changes? It looks like LNT only tracks run-time performance (and to be honest I wasn't able to make heads or tails of the results even for that -- the interface was pretty unintuitive to me.) While it is "normal" that each new LLVM release regresses compile-time by 2-3%, LLVM 10 seems to be