similar to: [RFC] LNT Feature: tracking performance changes

I agree. I plan to implement that next! > On Sep 21, 2015, at 4:25 PM, Hal Finkel <hfinkel at anl.gov> wrote: > > However, I think that unless we're able to e-mail commit authors who cause regressions like we do for buildbot failures, as well as others who are architecture performance experts on the relevant systems, this won't see the use it needs. -------------- next

tl;dr in low data situations we don’t look at past information, and that increases the false positive regression rate. We should look at the possibly incorrect recent past runs to fix that. Motivation: LNT’s current regression detection system has false positive rate that is too high to make it useful. With test suites as large as the llvm “test-suite” a single report will show hundreds of

On 2 August 2014 00:40, Renato Golin <renato.golin at linaro.org> wrote: > If memory serves me well (it doesn't), these are the list of things we > agreed on making, and their progress: > > 1. Performance-specific test-suite: a group of specific benchmarks > that should be tracked with the LNT infrastructure. Hal proposed to > look at this, but other people helped

Hi, Yesterday, I’ve landed a description for how reported bugs should be flowing through the various stages of a bug’s life (triage, fixing, closing, …) at http://llvm.org/docs/BugLifeCycle.html. Thanks for the many many people who provided ideas and feedback for this! With there now being a description of what is expected during bug triaging

Hi Zach, Thanks for elaborating. I like your proposal. I agree it still groups per area of expertise. And it makes the set of components we have easier to manage. Before making changes though I hope to hear opinions from others on this. What do others think? Thanks, Kristof On 9 Nov 2018, at 18:05, Zachary Turner <zturner at google.com<mailto:zturner at google.com>> wrote: To

Hi Zach, Thanks for putting the data in a spreadsheet - that’s easier to navigate. And thanks for re-raising the question whether we have the right components in bugzilla. As I think this could be an area for lots of different opinions, without any near-perfect solution, it has the potential to be a discussion that drags on for a long time. I thought half of all bugs not getting triaged was a

I had considered a libraries/Backends:Other as well that would be separate from libraries/Other On Fri, Nov 9, 2018 at 11:20 AM Derek Schuff <dschuff at google.com> wrote: > I wonder if backends are a special case to the heuristic of "let's not > make a bug component for code components that are too small". LLVM is > factored to cleanly separate backend code, to the

Hi Bill, The PPC LNT buildbot is randomly failing on LNT because of timeouts. http://lab.llvm.org:8011/builders/clang-ppc64le-linux-lnt I've checked all recent LNT failures and they were spotted on some processes (lua, sqlite3, tramp3d, expandfft, etc), so I doubt this is in any was a real time out due to code-gen issues. I imagine they're all at the edge already and the standard

Hi, I was looking at LNT's "compile" tests, listed here: $ lnt-sandbox/bin/lnt showtests Available tests: compile - Single file compile-time performance testing nt - LLVM test-suite compile and execution tests When trying to run it, I get Python exceptions in the code trying to parse the output of 'ifconfig' in order to infer my MAC address (this is on 64-bit

Hi, working on the LNT documentation I am wondering about a few things: - LNT documentation recommends to do `easy_install lnt==0.4.2dev` (in the install box on the left side of the title page). The last available version of LNT on pypy is version 0.4.0 from 2012. Are there plans to keep this up to date or should we better remove the entry and not mention pypy packages anymore? - Building the

Hi, I noticed that easy_install of lnt 0.4.1dev doesn't work: # easy_install lnt==0.4.1dev Searching for lnt==0.4.1dev Reading https://pypi.python.org/simple/lnt/ No local packages or download links found for lnt==0.4.1dev error: Could not find suitable distribution for Requirement.parse('lnt==0.4.1dev') -Gabriel -------------- next part -------------- An HTML attachment was

The run page problem were triggered by one of my commits (sorry) and should be mitigated now, see the thread at http://lists.llvm.org/pipermail/llvm-dev/2017-July/115971.html <http://lists.llvm.org/pipermail/llvm-dev/2017-July/115971.html> I don't know about the submission problems, could they just an occasional network problem or are they a common phenomenon? Chris did some

Dear all, Following the Benchmarking BOF from 2013 US dev meeting, I’d like to propose some improvements to the LNT performance tracking software. The most significant issue with current implementation is that the report is filled with extremely noisy values. Hence it is hard to notice performance improvements or regressions. After investigation of LNT and the LLVM test suite, I propose

Folks, I'm scratching my head to understand what's wrong with this bot: http://lab.llvm.org:8011/builders/clang-native-arm-lnt/builds/7783/steps/lnt.nightly-test/logs/stdio There were no recent commits to lnt or the test-suite and I've cleaned the lnt.venv on the bot, but it's still failing. I also haven't updated the packages or anything on the bot. Can any LNT specialist

I followed the lnt quickstart <http://llvm.org/docs/lnt/quickstart.html> directions but got this diagnostic when doing the setup: bash-3.2$ ~/mysandbox/bin/python ~/lnt/setup.py develop /Users/dcallahan/mysandbox/lib/python2.7/site-packages/setuptools/dist.py:284: UserWarning: The version spec\ ified requires normalization, consider using '0.4.1.dev0' instead of

I'd be fine pulling the pypi (sic) entry, shall I make it so? - Daniel On Tue, Jul 18, 2017 at 11:06 AM Chris Matthews <chris.matthews at apple.com> wrote: > Yep, we should pull that package down, or start to update it. > > The docs builds work for me. Is it possible that is not in your venv or > something like that? > > > On Jul 18, 2017, at 10:48 AM, Matthias

On 06/23/2013 11:12 PM, Star Tan wrote: > Hi all, > > > When we compare two testings, each of which is run with three samples, how would LNT show whether the comparison is reliable or not? > > > I have seen that the function get_value_status in reporting/analysis.py uses a very simple algorithm to infer data status. For example, if abs(self.delta) <= (self.stddev *

Hi David, That's weird, I have setup LNT in multiple different distros and have never seen this. Looks like no one ever tested on the system you're running. Can you share a bit more of your environment? Also, you can check the setup.py to see if it does any stripping of package names, which could go wrong in the wrong environment. cheers, --renato On 29 January 2015 at 20:13, David

Hi, I have recently moved the clang-native-arm-lnt-perf bot from the nt producer to the test-suite producer. It seems to be working fine but it doesn't manage to submit the results to http://lnt.llvm.org/submitRun. If you scroll down to the bottom of [1], you can see this error message: 2017-08-28 07:06:32: submitting result to 'http://lnt.llvm.org/submitRun' error: lnt server:

I updated the docs to show a pip/svn install like this: pip install svn+http://llvm.org/svn/llvm-project/lnt/trunk <svn+http://llvm.org/svn/llvm-project/lnt/trunk> > On Jul 18, 2017, at 11:24 AM, Kristof Beyls <kristof.beyls at arm.com> wrote: > > >> On 18 Jul 2017, at 20:21, Daniel Dunbar <daniel at zuster.org <mailto:daniel at zuster.org>> wrote: