llvm dev - Mar 2014 - [LLVMdev] RFC - Adding an optimization report facility?

On Thu, Mar 6, 2014 at 2:44 PM, Tobias Grosser <tobias at grosser.es>
wrote:

as others already pointed out, I committed the first piece of such
an
> infrastructure in LLVM commit 202474 and clang commit 202475. This is
> mostly the backend and printing infrastructure.
>

Thanks, Tobias.  I've browsed the two patches and I think they're going
to
be exactly what I need. IIUC, the patches add two new LLVM instructions
remark and note. These are inserted in the IL by the passes and the
compiler emits them as diagnostic if the right -W flag is enabled?

One question I have from the Clang patch. If I compile with -Weverything,
will this enable all warnings *and* remarks? In this context, I only want
to enable all remarks.  In fact, I want to enable a family of remarks: the
optimization remarks.

There will be other modifiers to these remarks as well:


   1. Report *missed* optimizations, instead of the successful ones.
   2. Increase verbosity of the report. This would be done using note
   nodes, I expect. But we may want varying degrees of verbosity.
   3. Group families of optimizations. For example, i want to report on all
   loop-related optimizations.
   4. IIRC, GCC's -fopt-info will also allow you to collect the reports
   into a separate text file. Not sure how useful I find this feature myself.


Thanks. Diego.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140306/3e9fa1ce/attachment.html>

On Thu, Mar 6, 2014 at 12:22 PM, Diego Novillo <dnovillo at google.com>
wrote:
> On Thu, Mar 6, 2014 at 2:44 PM, Tobias Grosser <tobias at grosser.es>
wrote:
>
> as others already pointed out, I committed the first piece of such an
>> infrastructure in LLVM commit 202474 and clang commit 202475. This is
>> mostly the backend and printing infrastructure.
>>
>
>
> Thanks, Tobias.  I've browsed the two patches and I think they're
going to
> be exactly what I need. IIUC, the patches add two new LLVM instructions
> remark and note. These are inserted in the IL by the passes and the
> compiler emits them as diagnostic if the right -W flag is enabled?
>
> One question I have from the Clang patch. If I compile with -Weverything,
> will this enable all warnings *and* remarks? In this context, I only want
> to enable all remarks.  In fact, I want to enable a family of remarks: the
> optimization remarks.
>
> There will be other modifiers to these remarks as well:
>
>
>    1. Report *missed* optimizations, instead of the successful ones.
>    2. Increase verbosity of the report. This would be done using note
>    nodes, I expect. But we may want varying degrees of verbosity.
>    3. Group families of optimizations. For example, i want to report on
>    all loop-related optimizations.
>    4. IIRC, GCC's -fopt-info will also allow you to collect the reports
>    into a separate text file. Not sure how useful I find this feature
myself.
>
>
Re. 4, to name a few that can be handy sometimes: 1) not contaminate
stderr; 2) per TU report; 3) Per opt-group report etc.

Also in GCC's original design, per-pass report filtering was supported, but
that was considered too developer oriented.

David
>
>
>
> Thanks. Diego.
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140306/41609d18/attachment.html>

On Thu, Mar 6, 2014 at 4:29 PM, Xinliang David Li <xinliangli at
gmail.com>wrote:
>
>
>
> On Thu, Mar 6, 2014 at 12:22 PM, Diego Novillo <dnovillo at
google.com>wrote:
>
>> On Thu, Mar 6, 2014 at 2:44 PM, Tobias Grosser <tobias at
grosser.es> wrote:
>>
>> as others already pointed out, I committed the first piece of such an
>>> infrastructure in LLVM commit 202474 and clang commit 202475. This
is
>>> mostly the backend and printing infrastructure.
>>>
>>
>>
>> Thanks, Tobias.  I've browsed the two patches and I think
they're going
>> to be exactly what I need. IIUC, the patches add two new LLVM
instructions
>> remark and note. These are inserted in the IL by the passes and the
>> compiler emits them as diagnostic if the right -W flag is enabled?
>>
>> One question I have from the Clang patch. If I compile with
-Weverything,
>> will this enable all warnings *and* remarks? In this context, I only
want
>> to enable all remarks.  In fact, I want to enable a family of remarks:
the
>> optimization remarks.
>>
>> There will be other modifiers to these remarks as well:
>>
>>
>>    1. Report *missed* optimizations, instead of the successful ones.
>>    2. Increase verbosity of the report. This would be done using note
>>    nodes, I expect. But we may want varying degrees of verbosity.
>>    3. Group families of optimizations. For example, i want to report on
>>    all loop-related optimizations.
>>    4. IIRC, GCC's -fopt-info will also allow you to collect the
reports
>>    into a separate text file. Not sure how useful I find this feature
myself.
>>
>>
> Re. 4, to name a few that can be handy sometimes: 1) not contaminate
> stderr; 2) per TU report; 3) Per opt-group report etc.
>
Ah, thanks.  The stderr and per-tu report thing is handy.  Your #3 seems
similar to my #3 (group families of optimizations).

Diego.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140306/6a62201c/attachment.html>

On 03/06/2014 09:22 PM, Diego Novillo wrote:
> On Thu, Mar 6, 2014 at 2:44 PM, Tobias Grosser <tobias at grosser.es>
wrote:
>
> as others already pointed out, I committed the first piece of such an
>> infrastructure in LLVM commit 202474 and clang commit 202475. This is
>> mostly the backend and printing infrastructure.
>>
>
>
> Thanks, Tobias.  I've browsed the two patches and I think they're
going to
> be exactly what I need. IIUC, the patches add two new LLVM instructions
> remark and note. These are inserted in the IL by the passes and the
> compiler emits them as diagnostic if the right -W flag is enabled?
Close.

The patch adds one new LLVM diagnostic called 'remark'. 'note'
has
already been available to add extended information to warnings, but it 
could not appear independently. This still holds, but 'notes' can now be
used to add additional information to remarks, too.

At the moment, remarks can similar to warnings be in a diagnostic group 
which is enabled with a '-Wgroupname' flag.
> One question I have from the Clang patch. If I compile with -Weverything,
> will this enable all warnings *and* remarks?
Yes, currently this enables all diagnostics, which means all warnings 
and remarks.
> In this context, I only want
> to enable all remarks.
I can see that. How to do this is really open for discussion. I could 
see us establishing two new flags '-Wall-remarks' for remarks and 
'-Wall-warnings' for warnings. Or, what I prefer, we could establish a 
new flag hierarchy e.g. with '-Reverything'.
> In fact, I want to enable a family of remarks: the
> optimization remarks.
>
> There will be other modifiers to these remarks as well:
>
>     1. Report *missed* optimizations, instead of the successful ones.
>     2. Increase verbosity of the report. This would be done using note
>     nodes, I expect. But we may want varying degrees of verbosity.
>     3. Group families of optimizations. For example, i want to report on
all
>     loop-related optimizations.
>     4. IIRC, GCC's -fopt-info will also allow you to collect the
reports
>     into a separate text file. Not sure how useful I find this feature
myself.
5. Enable/Disable remarks with pragmas / per function

Yes, there are a lot of things to do here. The diagnostic infrastructure
already provides great features, e.g.:

   - Enabling/Disabling diagnostics with pragmas and per function
   - Nested diagnostic groups
   - Enabling a diagnostic group, but disabling individual diagnostics
     or this group.
   - Good integration in tools through libclang
   - ...

However, the command line interface might need to be improved before we 
start adding a large number of remarks. Some goals I see:

1) We want to reuse as much of the diagnostic infrastructure as possible
2) We do not want to increase the complexity of the warning flags too much
3) We still want to have a similar user interface as the warning flags 
provide
4) People raised interest in upgrading remarks to warnings or errors
    (I am personally not too sure about this one, but it already works
     with -Werror=groupname)

Several people (including Chris) warned to build up an additional 
complex diagnostic system on the side, but as Chandler pointed out, we 
can most likely reuse most of the existing infrastructure but could 
still adjust the user interface if needed.

I personally believe the best approach is to start small and let the 
design be driven by the intended use cases. If you are interested, I 
would propose to add a single diagnostic for the loop inliner using just 
the existing '-W' infrastructure e.g. with '-Winline-remarkers'.
This both increases test coverage of the exiting features and also
will help to get an idea of the needed features. At the same time, you 
could help reviewing my first patch on modularizing the clang diagnostic 
infrastructure. Again, something that may help to get an idea of where 
to head at this side.

Cheers,
Tobias

On Thu, Mar 6, 2014 at 6:43 PM, Tobias Grosser <tobias at grosser.es>
wrote:
>
>  In this context, I only want
>> to enable all remarks.
>>
>
> I can see that. How to do this is really open for discussion. I could see
> us establishing two new flags '-Wall-remarks' for remarks and
> '-Wall-warnings' for warnings. Or, what I prefer, we could
establish a new
> flag hierarchy e.g. with '-Reverything'.
>
I quite like the notion of a new flag hierarchy.  Using -W seems like
semantic overload.  -W already implies diagnostics about dubious constructs
in the code. These notices are pure reporting on optimization activities.


5. Enable/Disable remarks with pragmas / per function
>
> Yes, there are a lot of things to do here. The diagnostic infrastructure
> already provides great features, e.g.:
>
>   - Enabling/Disabling diagnostics with pragmas and per function
>   - Nested diagnostic groups
>   - Enabling a diagnostic group, but disabling individual diagnostics
>     or this group.
>   - Good integration in tools through libclang
>   - ...
>
> However, the command line interface might need to be improved before we
> start adding a large number of remarks. Some goals I see:
>
> 1) We want to reuse as much of the diagnostic infrastructure as possible
> 2) We do not want to increase the complexity of the warning flags too much
> 3) We still want to have a similar user interface as the warning flags
> provide
> 4) People raised interest in upgrading remarks to warnings or errors
>    (I am personally not too sure about this one, but it already works
>     with -Werror=groupname)
>
Yes to all.  Not so sure about #4.  Convert the reports into errors?  As
in, fail the build if an inline didn't happen?

>
> Several people (including Chris) warned to build up an additional complex
> diagnostic system on the side, but as Chandler pointed out, we can most
> likely reuse most of the existing infrastructure but could still adjust the
> user interface if needed.
>
Yes, the one additional ability we'll likely want is to direct these
reports to files. Though that could be added later.

>
> I personally believe the best approach is to start small and let the
> design be driven by the intended use cases. If you are interested, I would
> propose to add a single diagnostic for the loop inliner using just the
> existing '-W' infrastructure e.g. with
'-Winline-remarkers'.
>
Agreed. I prefer an evolutionary approach.

This both increases test coverage of the exiting features and
also
> will help to get an idea of the needed features. At the same time, you
> could help reviewing my first patch on modularizing the clang diagnostic
> infrastructure. Again, something that may help to get an idea of where to
> head at this side.
>
Sure. You mean r202475?


Diego.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140307/f2e6aecb/attachment.html>