llvm dev - Jan 2013 - [LLVMdev] [llvm-commits] [PATCH] A "very verbose" mode for FileCheck

On Thu, Jan 17, 2013 at 8:36 AM, Dmitri Gribenko <gribozavr at gmail.com>
wrote:
> We have to options:
> (a) replace 'FileCheck' with '%FileCheck' in all tests, and
teach
> 'lit' to replace '%FileCheck' with 'FileCheck
--dump-input-on-error';
>
> (b) teach 'lit' to replace a plain 'FileCheck'.
>
> The first approach seems cleaner to developers who read and write
> tests (it suggests that they are invoking some "macro" -- but
does
> that matter?)  The second approach is much easier to implement since
> tests will be unchanged.
IMO the biggest issue with (a) is that developers will continue to use
`FileCheck` instead of `%FileCheck`. So IMO (a) should only be
implemented if simultaneously there is a change that makes just plain
`FileCheck` an error.

Another approach would be to have a script or symlink
`FileCheck-dump-input-on-error` which lives in build/bin/ alongside
FileCheck, and which just invokes FileCheck in a way that causes it to
go into the "dump input on error" mode (e.g. it checks argv[0], or a
magic environment variable, or whatever). On the build bots,
FileCheck-dump-input-on-error and regular FileCheck could be swapped
by a configure option. That's kind of ugly, but some variation of it
may be more palatable.

-- Sean Silva

On Thu, Jan 17, 2013 at 7:51 PM, Sean Silva <silvas at purdue.edu>
wrote:
> On Thu, Jan 17, 2013 at 8:36 AM, Dmitri Gribenko <gribozavr at
gmail.com> wrote:
>> We have to options:
>> (a) replace 'FileCheck' with '%FileCheck' in all tests,
and teach
>> 'lit' to replace '%FileCheck' with 'FileCheck
--dump-input-on-error';
>>
>> (b) teach 'lit' to replace a plain 'FileCheck'.
>>
>> The first approach seems cleaner to developers who read and write
>> tests (it suggests that they are invoking some "macro" -- but
does
>> that matter?)  The second approach is much easier to implement since
>> tests will be unchanged.
>
> IMO the biggest issue with (a) is that developers will continue to use
> `FileCheck` instead of `%FileCheck`. So IMO (a) should only be
> implemented if simultaneously there is a change that makes just plain
> `FileCheck` an error.
I think that within a month this knowledge will be propagated to all developers.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at
gmail.com>*/

On Jan 17, 2013, at 9:57 AM, Dmitri Gribenko <gribozavr at gmail.com>
wrote:
> On Thu, Jan 17, 2013 at 7:51 PM, Sean Silva <silvas at purdue.edu>
wrote:
>> On Thu, Jan 17, 2013 at 8:36 AM, Dmitri Gribenko <gribozavr at
gmail.com> wrote:
>>> We have to options:
>>> (a) replace 'FileCheck' with '%FileCheck' in all
tests, and teach
>>> 'lit' to replace '%FileCheck' with 'FileCheck
--dump-input-on-error';
>>> 
>>> (b) teach 'lit' to replace a plain 'FileCheck'.
>>> 
>>> The first approach seems cleaner to developers who read and write
>>> tests (it suggests that they are invoking some "macro" --
but does
>>> that matter?)  The second approach is much easier to implement
since
>>> tests will be unchanged.
>> 
>> IMO the biggest issue with (a) is that developers will continue to use
>> `FileCheck` instead of `%FileCheck`. So IMO (a) should only be
>> implemented if simultaneously there is a change that makes just plain
>> `FileCheck` an error.
> 
> I think that within a month this knowledge will be propagated to all
developers.
I'd like to think so, too, but we still get patches that write tests using
'grep' instead of FileCheck.

-Jim
> 
> Dmitri
> 
> -- 
> main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
> (j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at
gmail.com>*/
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev