similar to: Separate test harness for LLVM test-suite

If this is mainly about size then I would rather propose to split the "ABI-Testsuite" folder into a separate repository. With 1.5G it takes up more space than the rest of the repository combined yet it is coneptually independent of the rest of the testsuite and most people are not running it. - Matthias > On Jul 9, 2018, at 9:11 AM, Michael Kruse via llvm-dev <llvm-dev at

Hi Matthias, Thank you for your answer. But can you answer for some more questions? First of all, now LNT uses make-style of running tests and parse results from result csv file. Are there any plans to go to cmake? As I understood lit will run and collect all metrics, but there is no opportunity to make any settings for choosing what metrics I would like to collect. Test reports files allow to

Hi Elena, > First of all, now LNT uses make-style of running tests and parse results from result csv file. Are there any plans to go to cmake? There are two test drivers in LNT. "lnt runtests nt" uses the old Makefile-based system and "lnt runtests test-suite" uses the new cmake-based system. It sounds like you are using the former and should switch to the latter.

Hi Elena, This is great, I would love to see extensible support for arbitrary metrics. On Tue, Apr 19, 2016 at 12:39 AM, Kristof Beyls <Kristof.Beyls at arm.com> wrote: > Hi Elena, > > Many thanks for working on this! > > May I first suggest to convert the google document to email content? That > may make it a little bit easier for more people to review. It also makes

> On May 25, 2016, at 1:54 AM, Elena Lepilkina via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hi Matthias, > > Thank you for your answer. > But can you answer for some more questions? > First of all, now LNT uses make-style of running tests and parse results from result csv file. Are there any plans to go to cmake? As James already said "lnt runtest

Greetings everyone, We would like to improve LNT. The following RFC describes two LNT enhancements: * Custom (extensible) metrics * Test parameterization The main idea is in document https://docs.google.com/document/d/1zWWfu_iBQhFaHo73mhqqcL6Z82thHNAoCxaY7BveSf4/edit?usp=sharing. Thanks, Elena. -------------- next part -------------- An HTML attachment was scrubbed... URL:

On 21 Apr 2016, at 15:00, Elena Lepilkina <Elena.Lepilkina at synopsys.com<mailto:Elena.Lepilkina at synopsys.com>> wrote: Hi Kristof and Daniel, Thanks for your answers. Unfortunately I haven’t tried scaling up to a large data set before. Today I tried and results are quite bad. So database scheme should be rebuild. Now I thought about creating one sample table for each test-suite,

> On May 12, 2016, at 11:21 PM, Elena Lepilkina via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hi all, > > As we understood great changes will be done in LNT, so we are waiting to new LNT version and stopped our work in LNT. > > One more question about using test-suite separately with cmake. Cmake can only build all tests and generate lit tests. After that we

Hi all, As we understood great changes will be done in LNT, so we are waiting to new LNT version and stopped our work in LNT. One more question about using test-suite separately with cmake. Cmake can only build all tests and generate lit tests. After that we can run LIT and get report which is not equal with report (simple) got with make. Cmake test-suite version has no features to run custom

I understood your modules and I see as them can be used in LNT. But there are some question about old features. 1. With Makefiles we can write pass and collect some statistics. There was an example of branch pass. Metrics can be collected by @-$(LOPT) -load dcc888$(SHLIBEXT) -branch-counter -stats \ -time-passes -disable-output $< 2>>$@ in makefile. In report file we write how

Of course it'll be running as now. But user will need have installed MongoDB. Installation on linux with support of .deb packages is quite easy. sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927 echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/3.2 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list sudo apt-get update sudo apt-get

Hi Elena, Thanks for pushing forward with this. I like the idea of using a NoSQL solution. My primary reservation is about adding the new NoSQL stuff as an extra backend. I think the current database backend and its use of SQLAlchemy is extremely complex and is in fact the most complex part of LNT. Adding something more (as opposed to *replacing* it) would just make this worse and make it more

> On May 26, 2016, at 7:08 AM, Elena Lepilkina via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > I understood your modules and I see as them can be used in LNT. But there are some question about old features. > 1. With Makefiles we can write pass and collect some statistics. There was an example of branch pass. Metrics can be collected by > @-$(LOPT) -load

On 21 Apr 2016, at 17:44, Sergey Yakoushkin <sergey.yakoushkin at gmail.com<mailto:sergey.yakoushkin at gmail.com>> wrote: Hi Kristof, The way we use LNT, we would run different configuration (e.g. -O3 vs -Os) as different "machines" in LNT's model. O2/O3 is indeed bad example. We're also using different machines for Os/O3 - such parameters apply to all

Hi, Chris. Thank you for your answer about compile tests. As I understood during looking through code of compile tests they don’t use test suite at all. Am I right? There is lack of information and examples of running compile tests in LNT documentation. We understood that there are two groups of users: users using servers and collecting a lot of data and SQLite users, but these users as I think

From: Elena Lepilkina Sent: Monday, April 25, 2016 2:33 PM To: 'James Molloy' <james at jamesmolloy.co.uk>; Kristof Beyls <Kristof.Beyls at arm.com>; Mehdi Amini <mehdi.amini at apple.com> Cc: nd <nd at arm.com>; Matthias Braun <matze at braunis.de> Subject: RE: [llvm-dev] RFC: LNT/Test-suite support for custom metrics and test parameterization Hi everyone,

It's equivalent to {{\b}}whatever{{\b}}. I amn't sure if assertion \b is supported. \s will not match with start and of line, but it should be matched. Elena. -----Original Message----- From: Jonathan Roelofs [mailto:jonathan at codesourcery.com] Sent: Tuesday, May 24, 2016 5:14 PM To: Elena Lepilkina <Elena.Lepilkina at synopsys.com>; llvm-dev <llvm-dev at lists.llvm.org>

Ok, when I change syntax I will load new patch. Thanks, Elena. -----Original Message----- From: vsk at apple.com [mailto:vsk at apple.com] Sent: Thursday, September 01, 2016 9:38 PM To: Elena Lepilkina <Elena.Lepilkina at synopsys.com> Cc: Mehdi Amini <mehdi.amini at apple.com>; llvm-dev at lists.llvm.org Subject: Re: [llvm-dev] RFC: FileCheck Enhancements It is necessary to be

Hi, CHECK-WORD - If you want find some string in file, but you want to be sure, that this string should be a separate word. There are examples in file. Prefixes which can be described as regular expressions should be turning on with option -regex-prefixes . By default, you can't use it. Thanks for your comments. -----Original Message----- From: Jonathan Roelofs [mailto:jonathan at

On 5/24/16 8:10 AM, Elena Lepilkina wrote: > Hi, > > CHECK-WORD - If you want find some string in file, but you want to be sure, that this string should be a separate word. Is this functionally equivalent to doing: // CHECK: {{\s}}whatever{{\s}} Or is there some other subtlety about it? Jon > > There are examples in file. > Prefixes which can be described as regular