Hi all, Some discussions and comments were made in reviews. Much time has already passed since last comment and uploading changed patches. I made small summary report about features here, because there are some doubts about syntax of some features and changes in patches and it'll be great to know more opinions. 1. FileCheck Enhancement - CHECK-WORD (https://reviews.llvm.org/D22353) I replace special directives by flag --check-word, which turns on mode for each directive in file. It's obvious that this mode can be replaced using \b assert, but current regexp library doesn't have support of this assert and I have no answer to question about possibility of change current library. There was the discussion about that such mode can be made default, but there were doubts about necessity of a lot of work for changing existing tests. And I made experiment which proves that a lot of old tests will be failed with such mode on. Expected Passes : 15810 Expected Failures : 125 Unsupported Tests : 195 Unexpected Passes : 4 Unexpected Failures: 1128 2. FileCheck Enhancement - pattern templates ( https://reviews.llvm.org/D22403) There are some doubts about syntax of templates. I agree that use of \#, \:, \= is quite different from usual characters in FileCheck and was chosen because of same approach for escaping in regexp. Adrian Prantl suggested to use double-brackets "[[" to escape. Old syntax: \#(template_name) - use of template 'template_name'. It can occur in CHECK-PATTERN line, when description of one template includes other templates described before. (Without quote, I don't know how escape # here) \:(Variable_name)- template variable with name 'variable_name' \:(variable_name)\=(value) - current value of template variable(it's needed when you use template with variables). Suggested new syntax: [[#template_name]] - use of template 'template_name'. It can occur in CHECK-PATTERN line, when description of one template includes other templates described before. (Without quote, I don't know how escape # here) [[:Variable_name]] - template variable with name 'variable_name' [[:variable_name=value]] - current value of template variable(it's needed when you use template with variables). It'll be great to hear more opinions and suggestions about syntax. May be someone has really good ideas. Then I'll be able to change it. 3. FileCheck Enhancement - repeats in regular expressions (https://reviews.llvm.org/D22454), FileCheck Enhancement - Including files (https://reviews.llvm.org/D22500), FileCheck Enhancement - Expressions repeat for CHECK and CHECK-NEXT(https://reviews.llvm.org/D22501), FileCheck Enhancement - CHECK-LABEL-DAG(https://reviews.llvm.org/D22502), FileCheck Enhancement - prefixes-regular expressions (https://reviews.llvm.org/D22503) There were no comments about these enhancements at all. Your opinions are very important. I hope that some of these changes will be useful for FileCheck users, so I need your opinions to get opportunity for review to be resumed. Thanks, Elena. -----Original Message----- From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Elena Lepilkina via llvm-dev Sent: Wednesday, July 20, 2016 4:52 PM To: vsk at apple.com Cc: llvm-dev at lists.llvm.org Subject: Re: [llvm-dev] RFC: FileCheck Enhancements List of last patches: 1. FileCheck Enhancement - CHECK-WORD (llvm-commits was added later as diff update) - https://reviews.llvm.org/D22353 2. FileCheck Enhancement - pattern templates(llvm-commits was added later as diff update) - https://reviews.llvm.org/D22403 3. FileCheck Enhancement - repeats in regular expressions (new review with llvm-commits) - https://reviews.llvm.org/D22454 4. FileCheck Enhancement - Including files (new review with llvm-commits) - https://reviews.llvm.org/D22500 5. FileCheck Enhancement - Expressions repeat for CHECK and CHECK-NEXT (new review with llvm-commits) - https://reviews.llvm.org/D22501 6. FileCheck Enhancement - CHECK-LABEL-DAG (new review with llvm-commits) - https://reviews.llvm.org/D22502 7. FileCheck Enhancement - prefixes-regular expressions (new review with llvm-commits) - https://reviews.llvm.org/D22503 Thanks, Elena. -----Original Message----- From: vsk at apple.com [mailto:vsk at apple.com] Sent: Tuesday, July 19, 2016 8:42 PM To: Elena Lepilkina <Elena.Lepilkina at synopsys.com> Cc: Dean Michael Berris <dean.berris at gmail.com>; Mehdi Amini <mehdi.amini at apple.com>; llvm-dev at lists.llvm.org Subject: Re: [llvm-dev] RFC: FileCheck Enhancements Hi Elena,> On Jul 19, 2016, at 6:36 AM, Elena Lepilkina via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hi all, > > I made new patches for most of changes with llvm-commits subscriber. But two patches were updated, because there are a lot of comments (patch for CHECK-WORD and patch for templates pattern). Will it be ok?IMO it's fine to keep some of the original reviews if you don't want to discard/recreate their state. Please list the most up-to-date set of Phab URL's here, with a little note next to the ones which did not initially CC llvm-commits. Thanks again for working on this! vedant> > Thanks, Elena. > > -----Original Message----- > From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of > Dean Michael Berris via llvm-dev > Sent: Tuesday, July 19, 2016 6:53 AM > To: Mehdi Amini <mehdi.amini at apple.com> > Cc: via llvm-dev <llvm-dev at lists.llvm.org> > Subject: Re: [llvm-dev] RFC: FileCheck Enhancements > > >> On 19 Jul 2016, at 04:18, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> We had a long thread about that a few weeks (months?) ago: the conclusion (as I remember) was roughly a guideline to “always start a new revision to have a proper mailing-list thread starting with context (i.e. patch description)” >> (and my dissident minority opinion that it is only worth it if there >> hasn’t been significant round of reviews going on on the existing >> revision) >> > > Pardon me for missing that discussion, this may have already been asked before: but is it possible to make arcanist default subscribe the correct commits mailing list in the process? This should make it at least harder to forget. > > Cheers > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev_______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> On Aug 24, 2016, at 2:04 AM, Elena Lepilkina <Elena.Lepilkina at synopsys.com> wrote: > > Hi all, > > Some discussions and comments were made in reviews. Much time has already passed since last comment and uploading changed patches. I made small summary report about features here, because there are some doubts about syntax of some features and changes in patches and it'll be great to know more opinions. > > 1. FileCheck Enhancement - CHECK-WORD (https://reviews.llvm.org/D22353) > I replace special directives by flag --check-word, which turns on mode for each directive in file. It's obvious that this mode can be replaced using \b assert, but current regexp library doesn't have support of this assert and I have no answer to question about possibility of change current library. > There was the discussion about that such mode can be made default, but there were doubts about necessity of a lot of work for changing existing tests. > And I made experiment which proves that a lot of old tests will be failed with such mode on. > Expected Passes : 15810 > Expected Failures : 125 > Unsupported Tests : 195 > Unexpected Passes : 4 > Unexpected Failures: 1128I would rather not introduce churn in our tests by turning on --check-word by default. I'm also not convinced that turning on --check-word at the test level is the right move: having a CHECK-WORD directive is more flexible, and not a serious inconvenience (as compared to writing "CHECK").> > 2. FileCheck Enhancement - pattern templates ( https://reviews.llvm.org/D22403) > There are some doubts about syntax of templates. I agree that use of \#, \:, \= is quite different from usual characters in FileCheck and was chosen because of same approach for escaping in regexp. Adrian Prantl suggested to use double-brackets "[[" to escape. > Old syntax: > \#(template_name) - use of template 'template_name'. It can occur in CHECK-PATTERN line, when description of one template includes other templates described before. (Without quote, I don't know how escape # here) > \:(Variable_name)- template variable with name 'variable_name' > \:(variable_name)\=(value) - current value of template variable(it's needed when you use template with variables). > Suggested new syntax: > [[#template_name]] - use of template 'template_name'. It can occur in CHECK-PATTERN line, when description of one template includes other templates described before. (Without quote, I don't know how escape # here) > [[:Variable_name]] - template variable with name 'variable_name' > [[:variable_name=value]] - current value of template variable(it's needed when you use template with variables). > It'll be great to hear more opinions and suggestions about syntax. May be someone has really good ideas. Then I'll be able to change it.First, I want to recap the FileCheck workflow Elena is proposing: 1. Define patterns using the CHECK-DEFINE-PATTERN directive. Defined patterns have a name and may optionally have parameters. 2. Use defined patterns in the usual CHECK* directives. This is similar to how FileCheck patterns work already. The difference is that the patterns are defined using a dedicated directive, *not* when the pattern is first encountered. E.g, here is what you can do today: // RUN: echo "%r1 %r2" | FileCheck %s // CHECK: %[[register:[a-z]+]]1 // CHECK-SAME: %[[register]]2 With the proposed changes, we'll be able to write something like: // RUN: echo "%cmp %cmp" | FileCheck %s // CHECK-DEFINE-PATERN: register(n): {{[a-z]+}}n // CHECK: %[[register("1")]] // CHECK-SAME: %[[register("2")]] I saw "something like" because we haven't decided on the syntax for defining and using patterns (that's what this thread is for). Briefly, here's the syntax I'd like to use: // Defining patterns. CHECK-DEFINE-PATERN: <Name>(<Ident>, ...)?: <Pattern> Where <Pattern> is a list of <PatternElement>, and a <PatternElement> is either a regex ('{{' POSIX_REGEX '}}') or an argument identifier (IDENT). // Using patterns. CHECK: [[<Name>(<Argument>, ...)?]] Fleshing this out some more, here is my candidate grammar (see the end of this email for the current grammar): ACTION <- CHECK ':' MATCH '\n' ; ACTION <- CHECK-DEFINE-PATTERN ':' IDENT PARAMLIST? ':' PATTERN_ELEMENT* '\n' ; PARAMLIST <- '(' IDENT (',' IDENT)* ')' ; PATTERN_ELEMENT <- IDENT ; PATTERN_ELEMENT <- REGEX ; MATCH <- ; MATCH <- TEXT MATCH ; MATCH <- REGEX MATCH ; MATCH <- VAR MATCH ; REGEX <- '{{' POSIX_REGEX '}}' ; VAR <- '[[' IDENT ':' POSIX_REGEX ']]' ; VAR <- '[[' IDENT ARGLIST? ']]' ; ARGLIST <- '(' ARG (',' ARG)* ')' ; ARG <- "([^"]|\\")*" ;> 3. FileCheck Enhancement - repeats in regular expressions (https://reviews.llvm.org/D22454), FileCheck Enhancement - Including files (https://reviews.llvm.org/D22500), FileCheck Enhancement - Expressions repeat for CHECK and CHECK-NEXT(https://reviews.llvm.org/D22501), FileCheck Enhancement - CHECK-LABEL-DAG(https://reviews.llvm.org/D22502), FileCheck Enhancement - prefixes-regular expressions (https://reviews.llvm.org/D22503) > There were no comments about these enhancements at all. Your opinions are very important.I personally am waiting for some version of D22403 to land in-tree before starting on the other reviews. This would help me gauge what others in the community are thinking and what they need.> > I hope that some of these changes will be useful for FileCheck users, so I need your opinions to get opportunity for review to be resumed.thanks, vedant Original FileCheck grammar (shamelessly copied from the grammar Adrian posted to D22403): ACTION <- CHECK ':' MATCH '\n' ; MATCH <- ; MATCH <- TEXT MATCH ; MATCH <- REGEX MATCH ; MATCH <- VAR MATCH ; REGEX <- '{{' POSIX_REGEX '}}' ; VAR <- '[[' IDENT ':' POSIX_REGEX ']]' ; VAR <- '[[' IDENT ']]' ;> > -----Original Message----- > From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Elena Lepilkina via llvm-dev > Sent: Wednesday, July 20, 2016 4:52 PM > To: vsk at apple.com > Cc: llvm-dev at lists.llvm.org > Subject: Re: [llvm-dev] RFC: FileCheck Enhancements > > List of last patches: > > 1. FileCheck Enhancement - CHECK-WORD (llvm-commits was added later as diff update) - https://reviews.llvm.org/D22353 2. FileCheck Enhancement - pattern templates(llvm-commits was added later as diff update) - https://reviews.llvm.org/D22403 3. FileCheck Enhancement - repeats in regular expressions (new review with llvm-commits) - https://reviews.llvm.org/D22454 4. FileCheck Enhancement - Including files (new review with llvm-commits) - https://reviews.llvm.org/D22500 > 5. FileCheck Enhancement - Expressions repeat for CHECK and CHECK-NEXT (new review with llvm-commits) - https://reviews.llvm.org/D22501 > 6. FileCheck Enhancement - CHECK-LABEL-DAG (new review with llvm-commits) - https://reviews.llvm.org/D22502 7. FileCheck Enhancement - prefixes-regular expressions (new review with llvm-commits) - https://reviews.llvm.org/D22503 > > Thanks, > Elena. > > -----Original Message----- > From: vsk at apple.com [mailto:vsk at apple.com] > Sent: Tuesday, July 19, 2016 8:42 PM > To: Elena Lepilkina <Elena.Lepilkina at synopsys.com> > Cc: Dean Michael Berris <dean.berris at gmail.com>; Mehdi Amini <mehdi.amini at apple.com>; llvm-dev at lists.llvm.org > Subject: Re: [llvm-dev] RFC: FileCheck Enhancements > > Hi Elena, > > >> On Jul 19, 2016, at 6:36 AM, Elena Lepilkina via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> Hi all, >> >> I made new patches for most of changes with llvm-commits subscriber. But two patches were updated, because there are a lot of comments (patch for CHECK-WORD and patch for templates pattern). Will it be ok? > > IMO it's fine to keep some of the original reviews if you don't want to discard/recreate their state. > > Please list the most up-to-date set of Phab URL's here, with a little note next to the ones which did not initially CC llvm-commits. > > Thanks again for working on this! > > vedant > >> >> Thanks, Elena. >> >> -----Original Message----- >> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of >> Dean Michael Berris via llvm-dev >> Sent: Tuesday, July 19, 2016 6:53 AM >> To: Mehdi Amini <mehdi.amini at apple.com> >> Cc: via llvm-dev <llvm-dev at lists.llvm.org> >> Subject: Re: [llvm-dev] RFC: FileCheck Enhancements >> >> >>> On 19 Jul 2016, at 04:18, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: >>> >>> We had a long thread about that a few weeks (months?) ago: the conclusion (as I remember) was roughly a guideline to “always start a new revision to have a proper mailing-list thread starting with context (i.e. patch description)” >>> (and my dissident minority opinion that it is only worth it if there >>> hasn’t been significant round of reviews going on on the existing >>> revision) >>> >> >> Pardon me for missing that discussion, this may have already been asked before: but is it possible to make arcanist default subscribe the correct commits mailing list in the process? This should make it at least harder to forget. >> >> Cheers >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Hi Vedant,
Thank you for your detailed answer. There was such idea with brackets for
template parameters after I made patch. I think this is a good idea. Question is
only if empty brackets are necessary when pattern is without parameters.
I have doubts about syntax of pattern usage. I think that '?' at the end
will be inconspicuous. Moreover this is not similar to current syntax in
FileCheck. Metacharacters are usually used at the beginning, aren't they?
Why not then describe usage as ?[[<pattern>]] ?
One advantages of current syntax is that if pattern is used inside regular
expression there is no need to interrupt regular expression and then start it
again. But with new syntax I will need to write something like this
{{regex_part1}}[[<pattern>?]]{{regex_part2}}.
Thanks,
Elena.
-----Original Message-----
From: vsk at apple.com [mailto:vsk at apple.com]
Sent: Thursday, August 25, 2016 2:47 AM
To: Elena Lepilkina <Elena.Lepilkina at synopsys.com>
Cc: llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] RFC: FileCheck Enhancements
> On Aug 24, 2016, at 2:04 AM, Elena Lepilkina <Elena.Lepilkina at
synopsys.com> wrote:
>
> Hi all,
>
> Some discussions and comments were made in reviews. Much time has already
passed since last comment and uploading changed patches. I made small summary
report about features here, because there are some doubts about syntax of some
features and changes in patches and it'll be great to know more opinions.
>
> 1. FileCheck Enhancement - CHECK-WORD
> (https://reviews.llvm.org/D22353) I replace special directives by flag
--check-word, which turns on mode for each directive in file. It's obvious
that this mode can be replaced using \b assert, but current regexp library
doesn't have support of this assert and I have no answer to question about
possibility of change current library.
> There was the discussion about that such mode can be made default, but
there were doubts about necessity of a lot of work for changing existing tests.
> And I made experiment which proves that a lot of old tests will be failed
with such mode on.
> Expected Passes : 15810
> Expected Failures : 125
> Unsupported Tests : 195
> Unexpected Passes : 4
> Unexpected Failures: 1128
I would rather not introduce churn in our tests by turning on --check-word by
default. I'm also not convinced that turning on --check-word at the test
level is the right move: having a CHECK-WORD directive is more flexible, and not
a serious inconvenience (as compared to writing "CHECK").
>
> 2. FileCheck Enhancement - pattern templates (
> https://reviews.llvm.org/D22403) There are some doubts about syntax of
templates. I agree that use of \#, \:, \= is quite different from usual
characters in FileCheck and was chosen because of same approach for escaping in
regexp. Adrian Prantl suggested to use double-brackets "[[" to escape.
> Old syntax:
> \#(template_name) - use of template 'template_name'. It can occur
in
> CHECK-PATTERN line, when description of one template includes other
> templates described before. (Without quote, I don't know how escape #
> here)
> \:(Variable_name)- template variable with name 'variable_name'
> \:(variable_name)\=(value) - current value of template variable(it's
needed when you use template with variables).
> Suggested new syntax:
> [[#template_name]] - use of template 'template_name'. It can occur
in
> CHECK-PATTERN line, when description of one template includes other
templates described before. (Without quote, I don't know how escape # here)
[[:Variable_name]] - template variable with name 'variable_name'
> [[:variable_name=value]] - current value of template variable(it's
needed when you use template with variables).
> It'll be great to hear more opinions and suggestions about syntax. May
be someone has really good ideas. Then I'll be able to change it.
First, I want to recap the FileCheck workflow Elena is proposing:
1. Define patterns using the CHECK-DEFINE-PATTERN directive. Defined patterns
have a name and may optionally have parameters.
2. Use defined patterns in the usual CHECK* directives.
This is similar to how FileCheck patterns work already. The difference is that
the patterns are defined using a dedicated directive, *not* when the pattern is
first encountered. E.g, here is what you can do today:
// RUN: echo "%r1 %r2" | FileCheck %s
// CHECK: %[[register:[a-z]+]]1
// CHECK-SAME: %[[register]]2
With the proposed changes, we'll be able to write something like:
// RUN: echo "%cmp %cmp" | FileCheck %s
// CHECK-DEFINE-PATERN: register(n): {{[a-z]+}}n
// CHECK: %[[register("1")]]
// CHECK-SAME: %[[register("2")]]
I saw "something like" because we haven't decided on the syntax
for defining and using patterns (that's what this thread is for). Briefly,
here's the syntax I'd like to use:
// Defining patterns.
CHECK-DEFINE-PATERN: <Name>(<Ident>, ...)?: <Pattern>
Where <Pattern> is a list of <PatternElement>, and a
<PatternElement> is
either a regex ('{{' POSIX_REGEX '}}') or an argument
identifier (IDENT).
// Using patterns.
CHECK: [[<Name>(<Argument>, ...)?]]
Fleshing this out some more, here is my candidate grammar (see the end of this
email for the current grammar):
ACTION <- CHECK ':' MATCH '\n' ;
ACTION <- CHECK-DEFINE-PATTERN ':' IDENT PARAMLIST? ':'
PATTERN_ELEMENT* '\n' ;
PARAMLIST <- '(' IDENT (',' IDENT)* ')' ;
PATTERN_ELEMENT <- IDENT ;
PATTERN_ELEMENT <- REGEX ;
MATCH <- ;
MATCH <- TEXT MATCH ;
MATCH <- REGEX MATCH ;
MATCH <- VAR MATCH ;
REGEX <- '{{' POSIX_REGEX '}}' ;
VAR <- '[[' IDENT ':' POSIX_REGEX ']]' ;
VAR <- '[[' IDENT ARGLIST? ']]' ;
ARGLIST <- '(' ARG (',' ARG)* ')' ;
ARG <- "([^"]|\\")*" ;
> 3. FileCheck Enhancement - repeats in regular expressions
> (https://reviews.llvm.org/D22454), FileCheck Enhancement - Including files
(https://reviews.llvm.org/D22500), FileCheck Enhancement - Expressions repeat
for CHECK and CHECK-NEXT(https://reviews.llvm.org/D22501), FileCheck Enhancement
- CHECK-LABEL-DAG(https://reviews.llvm.org/D22502), FileCheck Enhancement -
prefixes-regular expressions (https://reviews.llvm.org/D22503) There were no
comments about these enhancements at all. Your opinions are very important.
I personally am waiting for some version of D22403 to land in-tree before
starting on the other reviews. This would help me gauge what others in the
community are thinking and what they need.
>
> I hope that some of these changes will be useful for FileCheck users, so I
need your opinions to get opportunity for review to be resumed.
thanks,
vedant
Original FileCheck grammar (shamelessly copied from the grammar Adrian posted to
D22403):
ACTION <- CHECK ':' MATCH '\n' ;
MATCH <- ;
MATCH <- TEXT MATCH ;
MATCH <- REGEX MATCH ;
MATCH <- VAR MATCH ;
REGEX <- '{{' POSIX_REGEX '}}' ;
VAR <- '[[' IDENT ':' POSIX_REGEX ']]' ;
VAR <- '[[' IDENT ']]' ;
>
> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of
> Elena Lepilkina via llvm-dev
> Sent: Wednesday, July 20, 2016 4:52 PM
> To: vsk at apple.com
> Cc: llvm-dev at lists.llvm.org
> Subject: Re: [llvm-dev] RFC: FileCheck Enhancements
>
> List of last patches:
>
> 1. FileCheck Enhancement - CHECK-WORD (llvm-commits was added later as diff
update) - https://reviews.llvm.org/D22353 2. FileCheck Enhancement - pattern
templates(llvm-commits was added later as diff update) -
https://reviews.llvm.org/D22403 3. FileCheck Enhancement - repeats in regular
expressions (new review with llvm-commits) - https://reviews.llvm.org/D22454 4.
FileCheck Enhancement - Including files (new review with llvm-commits) -
https://reviews.llvm.org/D22500
> 5. FileCheck Enhancement - Expressions repeat for CHECK and CHECK-NEXT (new
review with llvm-commits) - https://reviews.llvm.org/D22501
> 6. FileCheck Enhancement - CHECK-LABEL-DAG (new review with
> llvm-commits) - https://reviews.llvm.org/D22502 7. FileCheck
> Enhancement - prefixes-regular expressions (new review with
> llvm-commits) - https://reviews.llvm.org/D22503
>
> Thanks,
> Elena.
>
> -----Original Message-----
> From: vsk at apple.com [mailto:vsk at apple.com]
> Sent: Tuesday, July 19, 2016 8:42 PM
> To: Elena Lepilkina <Elena.Lepilkina at synopsys.com>
> Cc: Dean Michael Berris <dean.berris at gmail.com>; Mehdi Amini
> <mehdi.amini at apple.com>; llvm-dev at lists.llvm.org
> Subject: Re: [llvm-dev] RFC: FileCheck Enhancements
>
> Hi Elena,
>
>
>> On Jul 19, 2016, at 6:36 AM, Elena Lepilkina via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>
>> Hi all,
>>
>> I made new patches for most of changes with llvm-commits subscriber.
But two patches were updated, because there are a lot of comments (patch for
CHECK-WORD and patch for templates pattern). Will it be ok?
>
> IMO it's fine to keep some of the original reviews if you don't
want to discard/recreate their state.
>
> Please list the most up-to-date set of Phab URL's here, with a little
note next to the ones which did not initially CC llvm-commits.
>
> Thanks again for working on this!
>
> vedant
>
>>
>> Thanks, Elena.
>>
>> -----Original Message-----
>> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of
>> Dean Michael Berris via llvm-dev
>> Sent: Tuesday, July 19, 2016 6:53 AM
>> To: Mehdi Amini <mehdi.amini at apple.com>
>> Cc: via llvm-dev <llvm-dev at lists.llvm.org>
>> Subject: Re: [llvm-dev] RFC: FileCheck Enhancements
>>
>>
>>> On 19 Jul 2016, at 04:18, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>>
>>> We had a long thread about that a few weeks (months?) ago: the
conclusion (as I remember) was roughly a guideline to “always start a new
revision to have a proper mailing-list thread starting with context (i.e. patch
description)”
>>> (and my dissident minority opinion that it is only worth it if
there
>>> hasn’t been significant round of reviews going on on the existing
>>> revision)
>>>
>>
>> Pardon me for missing that discussion, this may have already been asked
before: but is it possible to make arcanist default subscribe the correct
commits mailing list in the process? This should make it at least harder to
forget.
>>
>> Cheers
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> On Aug 24, 2016, at 4:46 PM, Vedant Kumar via llvm-dev <llvm-dev at lists.llvm.org> wrote: > >> >> On Aug 24, 2016, at 2:04 AM, Elena Lepilkina <Elena.Lepilkina at synopsys.com> wrote: >> >> Hi all, >> >> Some discussions and comments were made in reviews. Much time has already passed since last comment and uploading changed patches. I made small summary report about features here, because there are some doubts about syntax of some features and changes in patches and it'll be great to know more opinions. >> >> 1. FileCheck Enhancement - CHECK-WORD (https://reviews.llvm.org/D22353) >> I replace special directives by flag --check-word, which turns on mode for each directive in file. It's obvious that this mode can be replaced using \b assert, but current regexp library doesn't have support of this assert and I have no answer to question about possibility of change current library. >> There was the discussion about that such mode can be made default, but there were doubts about necessity of a lot of work for changing existing tests. >> And I made experiment which proves that a lot of old tests will be failed with such mode on. >> Expected Passes : 15810 >> Expected Failures : 125 >> Unsupported Tests : 195 >> Unexpected Passes : 4 >> Unexpected Failures: 1128 > > I would rather not introduce churn in our tests by turning on --check-word by > default. I'm also not convinced that turning on --check-word at the test level > is the right move: having a CHECK-WORD directive is more flexible, and not a > serious inconvenience (as compared to writing "CHECK"). > > >> >> 2. FileCheck Enhancement - pattern templates ( https://reviews.llvm.org/D22403 <https://reviews.llvm.org/D22403>) >> There are some doubts about syntax of templates. I agree that use of \#, \:, \= is quite different from usual characters in FileCheck and was chosen because of same approach for escaping in regexp. Adrian Prantl suggested to use double-brackets "[[" to escape. >> Old syntax: >> \#(template_name) - use of template 'template_name'. It can occur in CHECK-PATTERN line, when description of one template includes other templates described before. (Without quote, I don't know how escape # here) >> \:(Variable_name)- template variable with name 'variable_name' >> \:(variable_name)\=(value) - current value of template variable(it's needed when you use template with variables). >> Suggested new syntax: >> [[#template_name]] - use of template 'template_name'. It can occur in CHECK-PATTERN line, when description of one template includes other templates described before. (Without quote, I don't know how escape # here) >> [[:Variable_name]] - template variable with name 'variable_name' >> [[:variable_name=value]] - current value of template variable(it's needed when you use template with variables). >> It'll be great to hear more opinions and suggestions about syntax. May be someone has really good ideas. Then I'll be able to change it. > > First, I want to recap the FileCheck workflow Elena is proposing: > > 1. Define patterns using the CHECK-DEFINE-PATTERN directive. Defined patterns > have a name and may optionally have parameters. > > 2. Use defined patterns in the usual CHECK* directives. > > This is similar to how FileCheck patterns work already. The difference is > that the patterns are defined using a dedicated directive, *not* when the > pattern is first encountered. E.g, here is what you can do today: > > // RUN: echo "%r1 %r2" | FileCheck %s > // CHECK: %[[register:[a-z]+]]1 > // CHECK-SAME: %[[register]]2 > > With the proposed changes, we'll be able to write something like: > > // RUN: echo "%cmp %cmp" | FileCheck %s > // CHECK-DEFINE-PATERN: register(n): {{[a-z]+}}n > // CHECK: %[[register("1")]] > // CHECK-SAME: %[[register("2")]]At first I thought that `register(n)` was some sort of macro, but if it is suppose to be equivalent to the example above of what we do “today”, then using “register(“1”)” is supposed to “capture” the ‘r’ part of the register on the first match. So you cannot reuse “register()” later to capture another expression. For instance: // RUN: FileCheck %s // CHECK-DEFINE-PATERN: register(n): {{[a-z]+}}n // CHECK: %[[register("1")]] // CHECK-SAME: %[[register("2")]] // CHECK: %[[register("1")]] // CHECK-SAME: %[[register("2")]] %r1 %r2 %reg1 %reg2 #will fail here. If true, I find this confusing, if not, I missed something in your example. — Mehdi> > I saw "something like" because we haven't decided on the syntax for defining > and using patterns (that's what this thread is for). Briefly, here's the syntax > I'd like to use: > > // Defining patterns. > CHECK-DEFINE-PATERN: <Name>(<Ident>, ...)?: <Pattern> > > Where <Pattern> is a list of <PatternElement>, and a <PatternElement> is > either a regex ('{{' POSIX_REGEX '}}') or an argument identifier (IDENT). > > // Using patterns. > CHECK: [[<Name>(<Argument>, ...)?]] > > Fleshing this out some more, here is my candidate grammar (see the end of this > email for the current grammar): > > ACTION <- CHECK ':' MATCH '\n' ; > ACTION <- CHECK-DEFINE-PATTERN ':' IDENT PARAMLIST? ':' PATTERN_ELEMENT* '\n' ; > PARAMLIST <- '(' IDENT (',' IDENT)* ')' ; > PATTERN_ELEMENT <- IDENT ; > PATTERN_ELEMENT <- REGEX ; > MATCH <- ; > MATCH <- TEXT MATCH ; > MATCH <- REGEX MATCH ; > MATCH <- VAR MATCH ; > REGEX <- '{{' POSIX_REGEX '}}' ; > VAR <- '[[' IDENT ':' POSIX_REGEX ']]' ; > VAR <- '[[' IDENT ARGLIST? ']]' ; > ARGLIST <- '(' ARG (',' ARG)* ')' ; > ARG <- "([^"]|\\")*" ; > > >> 3. FileCheck Enhancement - repeats in regular expressions (https://reviews.llvm.org/D22454 <https://reviews.llvm.org/D22454>), FileCheck Enhancement - Including files (https://reviews.llvm.org/D22500 <https://reviews.llvm.org/D22500>), FileCheck Enhancement - Expressions repeat for CHECK and CHECK-NEXT(https://reviews.llvm.org/D22501 <https://reviews.llvm.org/D22501>), FileCheck Enhancement - CHECK-LABEL-DAG(https://reviews.llvm.org/D22502 <https://reviews.llvm.org/D22502>), FileCheck Enhancement - prefixes-regular expressions (https://reviews.llvm.org/D22503 <https://reviews.llvm.org/D22503>) >> There were no comments about these enhancements at all. Your opinions are very important. > > I personally am waiting for some version of D22403 to land in-tree before > starting on the other reviews. This would help me gauge what others in the > community are thinking and what they need. > >> >> I hope that some of these changes will be useful for FileCheck users, so I need your opinions to get opportunity for review to be resumed. > > thanks, > vedant > > Original FileCheck grammar (shamelessly copied from the grammar Adrian posted > to D22403): > > ACTION <- CHECK ':' MATCH '\n' ; > MATCH <- ; > MATCH <- TEXT MATCH ; > MATCH <- REGEX MATCH ; > MATCH <- VAR MATCH ; > REGEX <- '{{' POSIX_REGEX '}}' ; > VAR <- '[[' IDENT ':' POSIX_REGEX ']]' ; > VAR <- '[[' IDENT ']]' ; > > >> >> -----Original Message----- >> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org <mailto:llvm-dev-bounces at lists.llvm.org>] On Behalf Of Elena Lepilkina via llvm-dev >> Sent: Wednesday, July 20, 2016 4:52 PM >> To: vsk at apple.com <mailto:vsk at apple.com> >> Cc: llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >> Subject: Re: [llvm-dev] RFC: FileCheck Enhancements >> >> List of last patches: >> >> 1. FileCheck Enhancement - CHECK-WORD (llvm-commits was added later as diff update) - https://reviews.llvm.org/D22353 <https://reviews.llvm.org/D22353> 2. FileCheck Enhancement - pattern templates(llvm-commits was added later as diff update) - https://reviews.llvm.org/D22403 <https://reviews.llvm.org/D22403> 3. FileCheck Enhancement - repeats in regular expressions (new review with llvm-commits) - https://reviews.llvm.org/D22454 <https://reviews.llvm.org/D22454> 4. FileCheck Enhancement - Including files (new review with llvm-commits) - https://reviews.llvm.org/D22500 <https://reviews.llvm.org/D22500> >> 5. FileCheck Enhancement - Expressions repeat for CHECK and CHECK-NEXT (new review with llvm-commits) - https://reviews.llvm.org/D22501 <https://reviews.llvm.org/D22501> >> 6. FileCheck Enhancement - CHECK-LABEL-DAG (new review with llvm-commits) - https://reviews.llvm.org/D22502 <https://reviews.llvm.org/D22502> 7. FileCheck Enhancement - prefixes-regular expressions (new review with llvm-commits) - https://reviews.llvm.org/D22503 <https://reviews.llvm.org/D22503> >> >> Thanks, >> Elena. >> >> -----Original Message----- >> From: vsk at apple.com [mailto:vsk at apple.com] >> Sent: Tuesday, July 19, 2016 8:42 PM >> To: Elena Lepilkina <Elena.Lepilkina at synopsys.com> >> Cc: Dean Michael Berris <dean.berris at gmail.com>; Mehdi Amini <mehdi.amini at apple.com>; llvm-dev at lists.llvm.org >> Subject: Re: [llvm-dev] RFC: FileCheck Enhancements >> >> Hi Elena, >> >> >>> On Jul 19, 2016, at 6:36 AM, Elena Lepilkina via llvm-dev <llvm-dev at lists.llvm.org> wrote: >>> >>> Hi all, >>> >>> I made new patches for most of changes with llvm-commits subscriber. But two patches were updated, because there are a lot of comments (patch for CHECK-WORD and patch for templates pattern). Will it be ok? >> >> IMO it's fine to keep some of the original reviews if you don't want to discard/recreate their state. >> >> Please list the most up-to-date set of Phab URL's here, with a little note next to the ones which did not initially CC llvm-commits. >> >> Thanks again for working on this! >> >> vedant >> >>> >>> Thanks, Elena. >>> >>> -----Original Message----- >>> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of >>> Dean Michael Berris via llvm-dev >>> Sent: Tuesday, July 19, 2016 6:53 AM >>> To: Mehdi Amini <mehdi.amini at apple.com> >>> Cc: via llvm-dev <llvm-dev at lists.llvm.org> >>> Subject: Re: [llvm-dev] RFC: FileCheck Enhancements >>> >>> >>>> On 19 Jul 2016, at 04:18, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: >>>> >>>> We had a long thread about that a few weeks (months?) ago: the conclusion (as I remember) was roughly a guideline to “always start a new revision to have a proper mailing-list thread starting with context (i.e. patch description)” >>>> (and my dissident minority opinion that it is only worth it if there >>>> hasn’t been significant round of reviews going on on the existing >>>> revision) >>>> >>> >>> Pardon me for missing that discussion, this may have already been asked before: but is it possible to make arcanist default subscribe the correct commits mailing list in the process? This should make it at least harder to forget. >>> >>> Cheers >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160831/4aa2d0e5/attachment-0001.html>
On Sep 13, 2016, at 11:16 AM, Vedant Kumar <vsk at apple.com> wrote:>> Instead of doing this, has anyone considered baking “modes” into Filecheck to support the important clients (e.g. LLVM IR, MC, clang, etc)? This would mean that a test could just add a “--mode=llvmir” flag to filecheck and get a bunch of baked in patterns, potentially with magic syntax. Something like this would avoid having to redundantly enter "%[0-9]+” a kajillion times all over the place. > > I hadn't considered adding modes to FileCheck. That seems like it could work > pretty well. > > Even if that's the route we're going to take, I'd prefer that the FileCheck > modes simply expose pre-defined patterns instead of introducing magic syntax. > It would be a shame to have to hack FileCheck to use check patterns for clients > that aren't LLVM IR, clang, etc.Why? This is a tradeoff: the more extensible you make filecheck (by introducing ever more complicated DSLs into it) the more esoteric it gets, and the less likely that someone will be able to actually UNDERSTAND a random filecheck invocation that they come across. The advantage of adding a few hard coded modes to filecheck is that we can have specific standardized solutions to important use cases that the LLVM project has. FileCheck lives in service of LLVM and its downstream projects, it shouldn’t aspire to be some general tool that needs a DSL. If a downstream project outside of the LLVM project (e.g. the Rust compiler) wanted its own mode, I think it would be fine to add it, and having a gaggle of modes seems preferable to me than building in a complex DSL for describing named patterns. -Chris
> On Sep 13, 2016, at 4:50 PM, Chris Lattner <clattner at apple.com> wrote: > > On Sep 13, 2016, at 11:16 AM, Vedant Kumar <vsk at apple.com> wrote: >>> Instead of doing this, has anyone considered baking “modes” into Filecheck to support the important clients (e.g. LLVM IR, MC, clang, etc)? This would mean that a test could just add a “--mode=llvmir” flag to filecheck and get a bunch of baked in patterns, potentially with magic syntax. Something like this would avoid having to redundantly enter "%[0-9]+” a kajillion times all over the place. >> >> I hadn't considered adding modes to FileCheck. That seems like it could work >> pretty well. >> >> Even if that's the route we're going to take, I'd prefer that the FileCheck >> modes simply expose pre-defined patterns instead of introducing magic syntax. >> It would be a shame to have to hack FileCheck to use check patterns for clients >> that aren't LLVM IR, clang, etc. > > Why? > > This is a tradeoff: the more extensible you make filecheck (by introducing ever more complicated DSLs into it) the more esoteric it gets, and the less likely that someone will be able to actually UNDERSTAND a random filecheck invocation that they come across. > > The advantage of adding a few hard coded modes to filecheck is that we can have specific standardized solutions to important use cases that the LLVM project has. FileCheck lives in service of LLVM and its downstream projects, it shouldn’t aspire to be some general tool that needs a DSL. If a downstream project outside of the LLVM project (e.g. the Rust compiler) wanted its own mode, I think it would be fine to add it, and having a gaggle of modes seems preferable to me than building in a complex DSL for describing named patterns. > > -ChrisLet me start by saying that adding a few hard-coded patterns to FileCheck could work well for many of llvm's tests. I'd be perfectly fine with that approach. My thinking was that if it isn't easy to teach FileCheck about new patterns, it's less likely that people will do it. I'm not interested in making FileCheck a general-purpose testing tool, but I would like to use custom patterns to simplify in-tree tests that aren't in *.ll or *.s files. I don't think that all extensions to the FileCheck syntax would cause serious maintainability issues. Being able to define patterns in some tests could actually make them more readable. However, if there isn't a way to extend FileCheck without creating serious readability issues, I agree that we shouldn't do it. vedant
On Sep 13, 2016, at 8:14 PM, Vedant Kumar <vsk at apple.com> wrote:> > Let me start by saying that adding a few hard-coded patterns to FileCheck could > work well for many of llvm's tests. I'd be perfectly fine with that approach. > > My thinking was that if it isn't easy to teach FileCheck about new patterns, > it's less likely that people will do it. I'm not interested in making FileCheck > a general-purpose testing tool, but I would like to use custom patterns to > simplify in-tree tests that aren't in *.ll or *.s files. > > I don't think that all extensions to the FileCheck syntax would cause serious > maintainability issues. Being able to define patterns in some tests could > actually make them more readable. However, if there isn't a way to extend > FileCheck without creating serious readability issues, I agree that we > shouldn't do it.Ok, it sounds like you agree that modes would be a good way to solve the most common case. How about we start with that. After that lands and gets adopted, we can see how big the middle zone served by custom pattern is, and what we are leaving on the table by not supporting it. -Chris
> On Sep 14, 2016, at 3:10 PM, Chris Lattner via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > On Sep 13, 2016, at 8:14 PM, Vedant Kumar <vsk at apple.com> wrote: >> >> Let me start by saying that adding a few hard-coded patterns to FileCheck could >> work well for many of llvm's tests. I'd be perfectly fine with that approach. >> >> My thinking was that if it isn't easy to teach FileCheck about new patterns, >> it's less likely that people will do it. I'm not interested in making FileCheck >> a general-purpose testing tool, but I would like to use custom patterns to >> simplify in-tree tests that aren't in *.ll or *.s files. >> >> I don't think that all extensions to the FileCheck syntax would cause serious >> maintainability issues. Being able to define patterns in some tests could >> actually make them more readable. However, if there isn't a way to extend >> FileCheck without creating serious readability issues, I agree that we >> shouldn't do it. > > Ok, it sounds like you agree that modes would be a good way to solve the most common case. How about we start with that. After that lands and gets adopted, we can see how big the middle zone served by custom pattern is, and what we are leaving on the table by not supporting it.How you would define the “modes” without a DSL? Hardcoding them inside FileCheck itself? It seems that it would make the bar higher to add new pattern, and lock-us with this particular hard-coded solution (i.e. moving to a DSL solution after the fact would be harder). On the opposite, I would expect that having the “DSL” support would make it easier to create/define these “modes”, even if we don’t use the “advanced DSL" in the tests themselves (we may even be able to forbid the use of these patterns outside of “mode file” if it is a strong concern). I tend to think that with a low entry bar to create a “mode file”, it may be easier to have for example a “mode file” that defines patterns for some target-specific register format, and include it only in the tests for this particular target. — Mehdi
> On Sep 14, 2016, at 3:23 PM, Mehdi Amini <mehdi.amini at apple.com> wrote: > > >> On Sep 14, 2016, at 3:10 PM, Chris Lattner via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> On Sep 13, 2016, at 8:14 PM, Vedant Kumar <vsk at apple.com> wrote: >>> >>> Let me start by saying that adding a few hard-coded patterns to FileCheck could >>> work well for many of llvm's tests. I'd be perfectly fine with that approach. >>> >>> My thinking was that if it isn't easy to teach FileCheck about new patterns, >>> it's less likely that people will do it. I'm not interested in making FileCheck >>> a general-purpose testing tool, but I would like to use custom patterns to >>> simplify in-tree tests that aren't in *.ll or *.s files. >>> >>> I don't think that all extensions to the FileCheck syntax would cause serious >>> maintainability issues. Being able to define patterns in some tests could >>> actually make them more readable. However, if there isn't a way to extend >>> FileCheck without creating serious readability issues, I agree that we >>> shouldn't do it. >> >> Ok, it sounds like you agree that modes would be a good way to solve the most common case. How about we start with that. After that lands and gets adopted, we can see how big the middle zone served by custom pattern is, and what we are leaving on the table by not supporting it. > > How you would define the “modes” without a DSL? Hardcoding them inside FileCheck itself?Yes.> It seems that it would make the bar higher to add new pattern,How so? It is as easy to change filecheck as it is to change a testcase, at least in the llvm repo.> and lock-us with this particular hard-coded solution (i.e. moving to a DSL solution after the fact would be harder).I don’t see how. The uses of filecheck in practice are very stratified based on what is being tested (e.g. llc output vs .ll files), I think that < 12 modes would cover the vast majority of the cases we care about.> On the opposite, I would expect that having the “DSL” support would make it easier to create/define these “modes”, even if we don’t use the “advanced DSL" in the tests themselves (we may even be able to forbid the use of these patterns outside of “mode file” if it is a strong concern).Right, but this is exactly what I *don’t* want. I don’t want it to be “too easy” to create micro DSLs, I want the bar to be somewhat higher so people use fewer better modes, rather than geeking out and getting crazy with a custom DSL that is only used in one or two tests. Most people encounter a random variety of tests based on when they break them in their development. We don’t want them to have to spend an excessive amount of time to understand “how the test works” before they are able to dig into “what they did that broke the test”. Fewer but better modes is good this way. -Chris
Excuse me, but as I understood mode will have predefined named patterns set. Then each mode will have a lot of patterns. Do you think that it's better for users to remember a lot of pattern names? When new user reads test with usage of such pattern he needs to find its meaning in documentation. There isn't very easy for new user understand tests. Moreover, there is already hardcoded LINE variable. Suggested syntax of using patterns is very similar to using LINE. If patterns will be hardcoded then it's difficult to see difference between LINE and pattern. Moreover, FileCheck is quite complicated now. Understanding tests for new user is difficult process. I don't like reading FileCheck source code if I don't know how this pattern works exactly. I understood that you are afraid of chaos in tests. But I can create terrible test for reading even with current syntax. All FileCheck users are programmers, and if they want they will add patterns in FileCheck source code. I think nobody will use patterns without necessity if there is such syntax in FileCheck. I'm not against modes with predefined patterns, but I think that patterns can do some test easily to read by removing repeats of big regular expressions. Thanks, Elena. -----Original Message----- From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Chris Lattner via llvm-dev Sent: Thursday, September 15, 2016 1:38 AM To: Mehdi Amini <mehdi.amini at apple.com> Cc: llvm-dev at lists.llvm.org Subject: Re: [llvm-dev] RFC: FileCheck Enhancements> On Sep 14, 2016, at 3:23 PM, Mehdi Amini <mehdi.amini at apple.com> wrote: > > >> On Sep 14, 2016, at 3:10 PM, Chris Lattner via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> On Sep 13, 2016, at 8:14 PM, Vedant Kumar <vsk at apple.com> wrote: >>> >>> Let me start by saying that adding a few hard-coded patterns to >>> FileCheck could work well for many of llvm's tests. I'd be perfectly fine with that approach. >>> >>> My thinking was that if it isn't easy to teach FileCheck about new >>> patterns, it's less likely that people will do it. I'm not >>> interested in making FileCheck a general-purpose testing tool, but I >>> would like to use custom patterns to simplify in-tree tests that aren't in *.ll or *.s files. >>> >>> I don't think that all extensions to the FileCheck syntax would >>> cause serious maintainability issues. Being able to define patterns >>> in some tests could actually make them more readable. However, if >>> there isn't a way to extend FileCheck without creating serious >>> readability issues, I agree that we shouldn't do it. >> >> Ok, it sounds like you agree that modes would be a good way to solve the most common case. How about we start with that. After that lands and gets adopted, we can see how big the middle zone served by custom pattern is, and what we are leaving on the table by not supporting it. > > How you would define the “modes” without a DSL? Hardcoding them inside FileCheck itself?Yes.> It seems that it would make the bar higher to add new pattern,How so? It is as easy to change filecheck as it is to change a testcase, at least in the llvm repo.> and lock-us with this particular hard-coded solution (i.e. moving to a DSL solution after the fact would be harder).I don’t see how. The uses of filecheck in practice are very stratified based on what is being tested (e.g. llc output vs .ll files), I think that < 12 modes would cover the vast majority of the cases we care about.> On the opposite, I would expect that having the “DSL” support would make it easier to create/define these “modes”, even if we don’t use the “advanced DSL" in the tests themselves (we may even be able to forbid the use of these patterns outside of “mode file” if it is a strong concern).Right, but this is exactly what I *don’t* want. I don’t want it to be “too easy” to create micro DSLs, I want the bar to be somewhat higher so people use fewer better modes, rather than geeking out and getting crazy with a custom DSL that is only used in one or two tests. Most people encounter a random variety of tests based on when they break them in their development. We don’t want them to have to spend an excessive amount of time to understand “how the test works” before they are able to dig into “what they did that broke the test”. Fewer but better modes is good this way. -Chris _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev