llvm dev - May 2016 - Filter optimization remarks by the hotness of the code region

> On May 11, 2016, at 3:37 AM, Hal Finkel <hfinkel at anl.gov> wrote:
> 
> ----- Original Message -----
>> From: "Adam Nemet" <anemet at apple.com>
>> To: "Hal Finkel" <hfinkel at anl.gov>
>> Cc: "llvm-dev (llvm-dev at lists.llvm.org)" <llvm-dev at
lists.llvm.org>
>> Sent: Wednesday, May 11, 2016 1:15:42 AM
>> Subject: Re: Filter optimization remarks by the hotness of the code
region
>> 
>> Hi Hal,
>> 
>>> On May 10, 2016, at 5:39 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
>>> 
>>> Hi Adam,
>>> 
>>> I think would be a really useful feature to have. I don't think
>>> that the backend should be responsible for filtering, but should
>>> pass the relative hotness information to the frontend. Given that
>>> these diagnostics are not just going to be used for -Rpass and
>>> friends, but also for generating reports by other tools (see the
>>> discussion around D19678, for example), I think it is important to
>>> allow the frontend to filter.
>> 
>> I am not sure I follow, can you please elaborate.  Are you saying
>> that for example in the listing use case we don’t want the filtered
>> diagnostics?  In other words it should be up to the remark handler
>> to decide whether it wants filtered or unfiltered remarks?
> 
> We might or might not want them. The user might want to select different
ratios and filters.
> 
>>> The frontend, or other tool, might also want to collect information
>>> from different compilation jobs to provide the user with an
>>> overall ranking.
>> 
>> Strictly speaking about PGO, the different compilation jobs get the
>> same PGO with the aggregated profile, thus the hotness calculated
>> should be global.  I am not sure why an extra aggregation step is
>> necessary.
> 
> I agree. However, I think that the frontend might employ a combination of
factors in deciding what information to present. We might, for example, have
pick different hotness thresholds for different kinds of remarks.
> 
> Especially since we're likely going with a design for the optimization
reports where the frontend just creates some YAML files with the diagnostic
information, and then a separate tool processes the files to produce reports, I
think that we should give those tools the maximum about of practical
flexibility. Such a tool might provide the user with non-trivial filtering
options.
I think this all makes sense.  So I guess the steps are:

1. YAML support for the existing remarks
2. Add optional relative hotness to the opt-remark API
3. Exposed relative hotness in the YAML output

Are you working on 1 or should I get started?

There was another issue that came up while discussing this with John McCall.  We
could have a pretty large number of such remarks.  I imagine that if the
post-processing is performed by the external tool, it may be useful to
effectively enable all optimization remarks (i.e.
-Rpass/-Rpass-missed/-Rpass-analysis=loop-vectorize/inline/etc.).  For large
programs this may not be feasible and we may need to locally filter the remarks
in LLVM.  This however seems like a somewhat orthogonal issue that we could
probably postpone for now.

Thanks for your input!
Adam
> 
> Thanks again,
> Hal
> 
>> 
>>> The default diagnostic, furthermore, does not provide enough
>>> information to identify a code "region". I think that the
pass
>>> generating the diagnostic needs to provide the information,
>>> however, we could certainly create some utility functions that
>>> take a pointer to the BFI analysis and a Value* that can do the
>>> right thing in most simple cases.
>> 
>> Good point.
>> 
>> Thanks for your feedback.
>> 
>> Adam
>> 
>>> 
>>> Thanks again,
>>> Hal
>>> 
>>> ----- Original Message -----
>>>> From: "Adam Nemet" <anemet at apple.com>
>>>> To: "llvm-dev (llvm-dev at lists.llvm.org)"
<llvm-dev at lists.llvm.org>
>>>> Cc: "Hal Finkel" <hfinkel at anl.gov>
>>>> Sent: Wednesday, May 4, 2016 1:12:17 PM
>>>> Subject: Filter optimization remarks by the hotness of the code
>>>> region
>>>> 
>>>> This idea came up a few times recently [1][2] so I’d like start
>>>> prototyping it.  To summarize, we can emit optimization remarks
>>>> using the -Rpass* options.  These are currently emitted by
>>>> optimizations like vectorization[3], unrolling, inlining and
since
>>>> last week loop distribution.
>>>> 
>>>> For large programs however this can amount to a lot of
diagnostics
>>>> output to sift through.  Filtering this by the hotness of the
>>>> region
>>>> can help to focus the user on performance opportunities that
are
>>>> likely to pay off.
>>>> 
>>>> The approach I am thinking of taking is to install a wrapper as
>>>> the
>>>> diagnostics handler that will only forward to the original
handler
>>>> if the region of code is considered hot.  This will be
installed
>>>> by
>>>> a new pass that will use BlockFrequencyInfo to determine the
top N
>>>> hot regions.
>>>> 
>>>> This is at very early stage right now.  I would appreciate any
>>>> feedback.
>>>> 
>>>> Thanks,
>>>> Adam
>>>> 
>>>> [1]
>>>> http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
>>>> [2]
http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
>>>> [3]
>>>>
http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
>>> 
>>> --
>>> Hal Finkel
>>> Assistant Computational Scientist
>>> Leadership Computing Facility
>>> Argonne National Laboratory
>> 
>> 
> 
> -- 
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160511/c97289d6/attachment.html>

----- Original Message -----
> From: "Adam Nemet" <anemet at apple.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: "llvm-dev (llvm-dev at lists.llvm.org)" <llvm-dev at
lists.llvm.org>,
> "John McCall" <rjmccall at apple.com>
> Sent: Wednesday, May 11, 2016 12:45:32 PM
> Subject: Re: Filter optimization remarks by the hotness of the code
> region
> > On May 11, 2016, at 3:37 AM, Hal Finkel < hfinkel at anl.gov >
wrote:
> 
> > ----- Original Message -----
> 
> > > From: "Adam Nemet" < anemet at apple.com >
> > 
> 
> > > To: "Hal Finkel" < hfinkel at anl.gov >
> > 
> 
> > > Cc: "llvm-dev ( llvm-dev at lists.llvm.org )" <
> > > llvm-dev at lists.llvm.org
> > > >
> > 
> 
> > > Sent: Wednesday, May 11, 2016 1:15:42 AM
> > 
> 
> > > Subject: Re: Filter optimization remarks by the hotness of the
> > > code
> > > region
> > 
> 
> > > Hi Hal,
> > 
> 
> > > > On May 10, 2016, at 5:39 PM, Hal Finkel < hfinkel at
anl.gov >
> > > > wrote:
> > > 
> > 
> 
> > > > Hi Adam,
> > > 
> > 
> 
> > > > I think would be a really useful feature to have. I
don't think
> > > 
> > 
> 
> > > > that the backend should be responsible for filtering, but
> > > > should
> > > 
> > 
> 
> > > > pass the relative hotness information to the frontend. Given
> > > > that
> > > 
> > 
> 
> > > > these diagnostics are not just going to be used for -Rpass
and
> > > 
> > 
> 
> > > > friends, but also for generating reports by other tools (see
> > > > the
> > > 
> > 
> 
> > > > discussion around D19678, for example), I think it is
important
> > > > to
> > > 
> > 
> 
> > > > allow the frontend to filter.
> > > 
> > 
> 
> > > I am not sure I follow, can you please elaborate. Are you saying
> > 
> 
> > > that for example in the listing use case we don’t want the
> > > filtered
> > 
> 
> > > diagnostics? In other words it should be up to the remark handler
> > 
> 
> > > to decide whether it wants filtered or unfiltered remarks?
> > 
> 
> > We might or might not want them. The user might want to select
> > different ratios and filters.
> 
> > > > The frontend, or other tool, might also want to collect
> > > > information
> > > 
> > 
> 
> > > > from different compilation jobs to provide the user with an
> > > 
> > 
> 
> > > > overall ranking.
> > > 
> > 
> 
> > > Strictly speaking about PGO, the different compilation jobs get
> > > the
> > 
> 
> > > same PGO with the aggregated profile, thus the hotness calculated
> > 
> 
> > > should be global. I am not sure why an extra aggregation step is
> > 
> 
> > > necessary.
> > 
> 
> > I agree. However, I think that the frontend might employ a
> > combination of factors in deciding what information to present. We
> > might, for example, have pick different hotness thresholds for
> > different kinds of remarks.
> 
> > Especially since we're likely going with a design for the
> > optimization reports where the frontend just creates some YAML
> > files
> > with the diagnostic information, and then a separate tool processes
> > the files to produce reports, I think that we should give those
> > tools the maximum about of practical flexibility. Such a tool might
> > provide the user with non-trivial filtering options.
> 
> I think this all makes sense. So I guess the steps are:
> 1. YAML support for the existing remarks
> 2. Add optional relative hotness to the opt-remark API
> 3. Exposed relative hotness in the YAML output
> Are you working on 1 or should I get started?
This is on my TODO list, but I've not started yet. Next few days seem
unlikely too.

One thing we should now thing about is where this YAML encoding happens. It
could be in the backend, or it could be in the remark handler in the frontend.
If they'll be no real programmatic interaction in the frontend with the
remark content (other than serializing it into YAML), it might make sense to
make the YAML-serialization capability a property of the remark itself handled
by its implementation in the backend.
> There was another issue that came up while discussing this with John
> McCall. We could have a pretty large number of such remarks. I
> imagine that if the post-processing is performed by the external
> tool, it may be useful to effectively enable all optimization
> remarks (i.e.
> -Rpass/-Rpass-missed/-Rpass-analysis=loop-vectorize/inline/etc.).
> For large programs this may not be feasible and we may need to
> locally filter the remarks in LLVM. This however seems like a
> somewhat orthogonal issue that we could probably postpone for now.
I've also thought about this, and this is certainly something we'll need
to keep an eye on. It is specifically why I thought at first that we'd
prefer to put the report-generation functionality into Clang. I'm happy,
however, to postpone this unless and until it proves to be a problem.
There's certainly a benefit to separating the functionality like this.

Thanks again, 
Hal 
> Thanks for your input!
> Adam
> > Thanks again,
> 
> > Hal
> 
> > > > The default diagnostic, furthermore, does not provide enough
> > > 
> > 
> 
> > > > information to identify a code "region". I think
that the pass
> > > 
> > 
> 
> > > > generating the diagnostic needs to provide the information,
> > > 
> > 
> 
> > > > however, we could certainly create some utility functions
that
> > > 
> > 
> 
> > > > take a pointer to the BFI analysis and a Value* that can do
the
> > > 
> > 
> 
> > > > right thing in most simple cases.
> > > 
> > 
> 
> > > Good point.
> > 
> 
> > > Thanks for your feedback.
> > 
> 
> > > Adam
> > 
> 
> > > > Thanks again,
> > > 
> > 
> 
> > > > Hal
> > > 
> > 
> 
> > > > ----- Original Message -----
> > > 
> > 
> 
> > > > > From: "Adam Nemet" < anemet at apple.com
>
> > > > 
> > > 
> > 
> 
> > > > > To: "llvm-dev ( llvm-dev at lists.llvm.org )"
<
> > > > > llvm-dev at lists.llvm.org
> > > > > >
> > > > 
> > > 
> > 
> 
> > > > > Cc: "Hal Finkel" < hfinkel at anl.gov >
> > > > 
> > > 
> > 
> 
> > > > > Sent: Wednesday, May 4, 2016 1:12:17 PM
> > > > 
> > > 
> > 
> 
> > > > > Subject: Filter optimization remarks by the hotness of
the
> > > > > code
> > > > 
> > > 
> > 
> 
> > > > > region
> > > > 
> > > 
> > 
> 
> > > > > This idea came up a few times recently [1][2] so I’d
like
> > > > > start
> > > > 
> > > 
> > 
> 
> > > > > prototyping it. To summarize, we can emit optimization
> > > > > remarks
> > > > 
> > > 
> > 
> 
> > > > > using the -Rpass* options. These are currently emitted
by
> > > > 
> > > 
> > 
> 
> > > > > optimizations like vectorization[3], unrolling,
inlining and
> > > > > since
> > > > 
> > > 
> > 
> 
> > > > > last week loop distribution.
> > > > 
> > > 
> > 
> 
> > > > > For large programs however this can amount to a lot of
> > > > > diagnostics
> > > > 
> > > 
> > 
> 
> > > > > output to sift through. Filtering this by the hotness
of the
> > > > 
> > > 
> > 
> 
> > > > > region
> > > > 
> > > 
> > 
> 
> > > > > can help to focus the user on performance opportunities
that
> > > > > are
> > > > 
> > > 
> > 
> 
> > > > > likely to pay off.
> > > > 
> > > 
> > 
> 
> > > > > The approach I am thinking of taking is to install a
wrapper
> > > > > as
> > > > 
> > > 
> > 
> 
> > > > > the
> > > > 
> > > 
> > 
> 
> > > > > diagnostics handler that will only forward to the
original
> > > > > handler
> > > > 
> > > 
> > 
> 
> > > > > if the region of code is considered hot. This will be
> > > > > installed
> > > > 
> > > 
> > 
> 
> > > > > by
> > > > 
> > > 
> > 
> 
> > > > > a new pass that will use BlockFrequencyInfo to
determine the
> > > > > top
> > > > > N
> > > > 
> > > 
> > 
> 
> > > > > hot regions.
> > > > 
> > > 
> > 
> 
> > > > > This is at very early stage right now. I would
appreciate any
> > > > 
> > > 
> > 
> 
> > > > > feedback.
> > > > 
> > > 
> > 
> 
> > > > > Thanks,
> > > > 
> > > 
> > 
> 
> > > > > Adam
> > > > 
> > > 
> > 
> 
> > > > > [1]
> > > > 
> > > 
> > 
> 
> > > > >
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
> > > > 
> > > 
> > 
> 
> > > > > [2]
> > > > >
http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
> > > > 
> > > 
> > 
> 
> > > > > [3]
> > > > 
> > > 
> > 
> 
> > > > >
http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
> > > > 
> > > 
> > 
> 
> > > > --
> > > 
> > 
> 
> > > > Hal Finkel
> > > 
> > 
> 
> > > > Assistant Computational Scientist
> > > 
> > 
> 
> > > > Leadership Computing Facility
> > > 
> > 
> 
> > > > Argonne National Laboratory
> > > 
> > 
> 
> > --
> 
> > Hal Finkel
> 
> > Assistant Computational Scientist
> 
> > Leadership Computing Facility
> 
> > Argonne National Laboratory
> 
-- 

Hal Finkel 
Assistant Computational Scientist 
Leadership Computing Facility 
Argonne National Laboratory 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160512/94a88caa/attachment.html>

> On May 12, 2016, at 7:17 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> 
> From: "Adam Nemet" <anemet at apple.com <mailto:anemet at
apple.com>>
> To: "Hal Finkel" <hfinkel at anl.gov <mailto:hfinkel at
anl.gov>>
> Cc: "llvm-dev (llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>)" <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>>, "John McCall" <rjmccall at apple.com
<mailto:rjmccall at apple.com>>
> Sent: Wednesday, May 11, 2016 12:45:32 PM
> Subject: Re: Filter optimization remarks by the hotness of the code region
> 
> 
> On May 11, 2016, at 3:37 AM, Hal Finkel <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>> wrote:
> 
> 
> From: "Adam Nemet" <anemet at apple.com <mailto:anemet at
apple.com>>
> To: "Hal Finkel" <hfinkel at anl.gov <mailto:hfinkel at
anl.gov>>
> Cc: "llvm-dev (llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>)" <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>>
> Sent: Wednesday, May 11, 2016 1:15:42 AM
> Subject: Re: Filter optimization remarks by the hotness of the code region
> 
> Hi Hal,
> 
> On May 10, 2016, at 5:39 PM, Hal Finkel <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>> wrote:
> 
> Hi Adam,
> 
> I think would be a really useful feature to have. I don't think
> that the backend should be responsible for filtering, but should
> pass the relative hotness information to the frontend. Given that
> these diagnostics are not just going to be used for -Rpass and
> friends, but also for generating reports by other tools (see the
> discussion around D19678, for example), I think it is important to
> allow the frontend to filter.
> 
> I am not sure I follow, can you please elaborate.  Are you saying
> that for example in the listing use case we don’t want the filtered
> diagnostics?  In other words it should be up to the remark handler
> to decide whether it wants filtered or unfiltered remarks?
> 
> We might or might not want them. The user might want to select different
ratios and filters.
> 
> The frontend, or other tool, might also want to collect information
> from different compilation jobs to provide the user with an
> overall ranking.
> 
> Strictly speaking about PGO, the different compilation jobs get the
> same PGO with the aggregated profile, thus the hotness calculated
> should be global.  I am not sure why an extra aggregation step is
> necessary.
> 
> I agree. However, I think that the frontend might employ a combination of
factors in deciding what information to present. We might, for example, have
pick different hotness thresholds for different kinds of remarks.
> 
> Especially since we're likely going with a design for the optimization
reports where the frontend just creates some YAML files with the diagnostic
information, and then a separate tool processes the files to produce reports, I
think that we should give those tools the maximum about of practical
flexibility. Such a tool might provide the user with non-trivial filtering
options.
> 
> I think this all makes sense.  So I guess the steps are:
> 
> 1. YAML support for the existing remarks
> 2. Add optional relative hotness to the opt-remark API
> 3. Exposed relative hotness in the YAML output
> 
> Are you working on 1 or should I get started?
> This is on my TODO list, but I've not started yet. Next few days seem
unlikely too.
> 
> One thing we should now thing about is where this YAML encoding happens. It
could be in the backend, or it could be in the remark handler in the frontend.
If they'll be no real programmatic interaction in the frontend with the
remark content (other than serializing it into YAML), it might make sense to
make the YAML-serialization capability a property of the remark itself handled
by its implementation in the backend.
I was under the impression that your notion of source locations is opaquely
encoded and so something in the frontend would need to interpret them.  If
that's not the case, then it doesn't matter much, I think, as long as
the format supports certain kinds of query.  For example, a tool should be able
to start with a source location and find the interesting remarks about it +
surrounding code.  As long as the format allows that query to occur without
needing pass-specific knowledge of a remark's schema, it's probably not
all that important who outputs the data.
> There was another issue that came up while discussing this with John
McCall.  We could have a pretty large number of such remarks.  I imagine that if
the post-processing is performed by the external tool, it may be useful to
effectively enable all optimization remarks (i.e.
-Rpass/-Rpass-missed/-Rpass-analysis=loop-vectorize/inline/etc.).  For large
programs this may not be feasible and we may need to locally filter the remarks
in LLVM.  This however seems like a somewhat orthogonal issue that we could
probably postpone for now.
> I've also thought about this, and this is certainly something we'll
need to keep an eye on. It is specifically why I thought at first that we'd
prefer to put the report-generation functionality into Clang. I'm happy,
however, to postpone this unless and until it proves to be a problem.
There's certainly a benefit to separating the functionality like this.
Right, I think we can handle this as optional configuration data for the
pass's remark generation: the default is to generate all remarks (if this is
enabled at all), but we can evolve ways to filter that, e.g. opting specific
places into more comprehensive remarks.

John.
> 
> Thanks again,
> Hal
> 
> Thanks for your input!
> Adam
> 
> 
> Thanks again,
> Hal
> 
> 
> The default diagnostic, furthermore, does not provide enough
> information to identify a code "region". I think that the pass
> generating the diagnostic needs to provide the information,
> however, we could certainly create some utility functions that
> take a pointer to the BFI analysis and a Value* that can do the
> right thing in most simple cases.
> 
> Good point.
> 
> Thanks for your feedback.
> 
> Adam
> 
> 
> Thanks again,
> Hal
> 
> ----- Original Message -----
> From: "Adam Nemet" <anemet at apple.com <mailto:anemet at
apple.com>>
> To: "llvm-dev (llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>)" <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>>
> Cc: "Hal Finkel" <hfinkel at anl.gov <mailto:hfinkel at
anl.gov>>
> Sent: Wednesday, May 4, 2016 1:12:17 PM
> Subject: Filter optimization remarks by the hotness of the code
> region
> 
> This idea came up a few times recently [1][2] so I’d like start
> prototyping it.  To summarize, we can emit optimization remarks
> using the -Rpass* options.  These are currently emitted by
> optimizations like vectorization[3], unrolling, inlining and since
> last week loop distribution.
> 
> For large programs however this can amount to a lot of diagnostics
> output to sift through.  Filtering this by the hotness of the
> region
> can help to focus the user on performance opportunities that are
> likely to pay off.
> 
> The approach I am thinking of taking is to install a wrapper as
> the
> diagnostics handler that will only forward to the original handler
> if the region of code is considered hot.  This will be installed
> by
> a new pass that will use BlockFrequencyInfo to determine the top N
> hot regions.
> 
> This is at very early stage right now.  I would appreciate any
> feedback.
> 
> Thanks,
> Adam
> 
> [1]
> http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
<http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html>
> [2] http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
<http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html>
> [3]
> http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
<http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html>
> 
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> 
> 
> 
> -- 
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> 
> 
> 
> 
> -- 
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160512/b2220ff4/attachment.html>

> On May 11, 2016, at 10:45 AM, Adam Nemet via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
>> 
>> On May 11, 2016, at 3:37 AM, Hal Finkel <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>> wrote:
>> 
>> ----- Original Message -----
>>> From: "Adam Nemet" <anemet at apple.com
<mailto:anemet at apple.com>>
>>> To: "Hal Finkel" <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>>
>>> Cc: "llvm-dev (llvm-dev at lists.llvm.org <mailto:llvm-dev
at lists.llvm.org>)" <llvm-dev at lists.llvm.org <mailto:llvm-dev
at lists.llvm.org>>
>>> Sent: Wednesday, May 11, 2016 1:15:42 AM
>>> Subject: Re: Filter optimization remarks by the hotness of the code
region
>>> 
>>> Hi Hal,
>>> 
>>>> On May 10, 2016, at 5:39 PM, Hal Finkel <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>> wrote:
>>>> 
>>>> Hi Adam,
>>>> 
>>>> I think would be a really useful feature to have. I don't
think
>>>> that the backend should be responsible for filtering, but
should
>>>> pass the relative hotness information to the frontend. Given
that
>>>> these diagnostics are not just going to be used for -Rpass and
>>>> friends, but also for generating reports by other tools (see
the
>>>> discussion around D19678, for example), I think it is important
to
>>>> allow the frontend to filter.
>>> 
>>> I am not sure I follow, can you please elaborate.  Are you saying
>>> that for example in the listing use case we don’t want the filtered
>>> diagnostics?  In other words it should be up to the remark handler
>>> to decide whether it wants filtered or unfiltered remarks?
>> 
>> We might or might not want them. The user might want to select
different ratios and filters.
>> 
>>>> The frontend, or other tool, might also want to collect
information
>>>> from different compilation jobs to provide the user with an
>>>> overall ranking.
>>> 
>>> Strictly speaking about PGO, the different compilation jobs get the
>>> same PGO with the aggregated profile, thus the hotness calculated
>>> should be global.  I am not sure why an extra aggregation step is
>>> necessary.
>> 
>> I agree. However, I think that the frontend might employ a combination
of factors in deciding what information to present. We might, for example, have
pick different hotness thresholds for different kinds of remarks.
>> 
>> Especially since we're likely going with a design for the
optimization reports where the frontend just creates some YAML files with the
diagnostic information, and then a separate tool processes the files to produce
reports, I think that we should give those tools the maximum about of practical
flexibility. Such a tool might provide the user with non-trivial filtering
options.
> 
> I think this all makes sense.  So I guess the steps are:
> 
> 1. YAML support for the existing remarks
> 2. Add optional relative hotness to the opt-remark API
I submitted for the first patch to implement this:
http://reviews.llvm.org/D21771

Adam
> 3. Exposed relative hotness in the YAML output
> 
> Are you working on 1 or should I get started?
> 
> There was another issue that came up while discussing this with John
McCall.  We could have a pretty large number of such remarks.  I imagine that if
the post-processing is performed by the external tool, it may be useful to
effectively enable all optimization remarks (i.e.
-Rpass/-Rpass-missed/-Rpass-analysis=loop-vectorize/inline/etc.).  For large
programs this may not be feasible and we may need to locally filter the remarks
in LLVM.  This however seems like a somewhat orthogonal issue that we could
probably postpone for now.
> 
> Thanks for your input!
> Adam
> 
>> 
>> Thanks again,
>> Hal
>> 
>>> 
>>>> The default diagnostic, furthermore, does not provide enough
>>>> information to identify a code "region". I think that
the pass
>>>> generating the diagnostic needs to provide the information,
>>>> however, we could certainly create some utility functions that
>>>> take a pointer to the BFI analysis and a Value* that can do the
>>>> right thing in most simple cases.
>>> 
>>> Good point.
>>> 
>>> Thanks for your feedback.
>>> 
>>> Adam
>>> 
>>>> 
>>>> Thanks again,
>>>> Hal
>>>> 
>>>> ----- Original Message -----
>>>>> From: "Adam Nemet" <anemet at apple.com
<mailto:anemet at apple.com>>
>>>>> To: "llvm-dev (llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>)" <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
>>>>> Cc: "Hal Finkel" <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>>
>>>>> Sent: Wednesday, May 4, 2016 1:12:17 PM
>>>>> Subject: Filter optimization remarks by the hotness of the
code
>>>>> region
>>>>> 
>>>>> This idea came up a few times recently [1][2] so I’d like
start
>>>>> prototyping it.  To summarize, we can emit optimization
remarks
>>>>> using the -Rpass* options.  These are currently emitted by
>>>>> optimizations like vectorization[3], unrolling, inlining
and since
>>>>> last week loop distribution.
>>>>> 
>>>>> For large programs however this can amount to a lot of
diagnostics
>>>>> output to sift through.  Filtering this by the hotness of
the
>>>>> region
>>>>> can help to focus the user on performance opportunities
that are
>>>>> likely to pay off.
>>>>> 
>>>>> The approach I am thinking of taking is to install a
wrapper as
>>>>> the
>>>>> diagnostics handler that will only forward to the original
handler
>>>>> if the region of code is considered hot.  This will be
installed
>>>>> by
>>>>> a new pass that will use BlockFrequencyInfo to determine
the top N
>>>>> hot regions.
>>>>> 
>>>>> This is at very early stage right now.  I would appreciate
any
>>>>> feedback.
>>>>> 
>>>>> Thanks,
>>>>> Adam
>>>>> 
>>>>> [1]
>>>>>
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
<http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html>
>>>>> [2]
http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
>>>>> [3]
>>>>>
http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
>>>> 
>>>> --
>>>> Hal Finkel
>>>> Assistant Computational Scientist
>>>> Leadership Computing Facility
>>>> Argonne National Laboratory
>>> 
>>> 
>> 
>> -- 
>> Hal Finkel
>> Assistant Computational Scientist
>> Leadership Computing Facility
>> Argonne National Laboratory
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160627/f68d4b6d/attachment.html>

> On Jun 27, 2016, at 3:30 PM, Adam Nemet via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
>> 
>> On May 11, 2016, at 10:45 AM, Adam Nemet via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>>> 
>>> On May 11, 2016, at 3:37 AM, Hal Finkel <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>> wrote:
>>> 
>>> ----- Original Message -----
>>>> From: "Adam Nemet" <anemet at apple.com
<mailto:anemet at apple.com>>
>>>> To: "Hal Finkel" <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>>
>>>> Cc: "llvm-dev (llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>)" <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
>>>> Sent: Wednesday, May 11, 2016 1:15:42 AM
>>>> Subject: Re: Filter optimization remarks by the hotness of the
code region
>>>> 
>>>> Hi Hal,
>>>> 
>>>>> On May 10, 2016, at 5:39 PM, Hal Finkel <hfinkel at
anl.gov <mailto:hfinkel at anl.gov>> wrote:
>>>>> 
>>>>> Hi Adam,
>>>>> 
>>>>> I think would be a really useful feature to have. I
don't think
>>>>> that the backend should be responsible for filtering, but
should
>>>>> pass the relative hotness information to the frontend.
Given that
>>>>> these diagnostics are not just going to be used for -Rpass
and
>>>>> friends, but also for generating reports by other tools
(see the
>>>>> discussion around D19678, for example), I think it is
important to
>>>>> allow the frontend to filter.
>>>> 
>>>> I am not sure I follow, can you please elaborate.  Are you
saying
>>>> that for example in the listing use case we don’t want the
filtered
>>>> diagnostics?  In other words it should be up to the remark
handler
>>>> to decide whether it wants filtered or unfiltered remarks?
>>> 
>>> We might or might not want them. The user might want to select
different ratios and filters.
>>> 
>>>>> The frontend, or other tool, might also want to collect
information
>>>>> from different compilation jobs to provide the user with an
>>>>> overall ranking.
>>>> 
>>>> Strictly speaking about PGO, the different compilation jobs get
the
>>>> same PGO with the aggregated profile, thus the hotness
calculated
>>>> should be global.  I am not sure why an extra aggregation step
is
>>>> necessary.
>>> 
>>> I agree. However, I think that the frontend might employ a
combination of factors in deciding what information to present. We might, for
example, have pick different hotness thresholds for different kinds of remarks.
>>> 
>>> Especially since we're likely going with a design for the
optimization reports where the frontend just creates some YAML files with the
diagnostic information, and then a separate tool processes the files to produce
reports, I think that we should give those tools the maximum about of practical
flexibility. Such a tool might provide the user with non-trivial filtering
options.
>> 
>> I think this all makes sense.  So I guess the steps are:
>> 
>> 1. YAML support for the existing remarks
>> 2. Add optional relative hotness to the opt-remark API
> 
> I submitted for the first patch to implement this:
http://reviews.llvm.org/D21771 <http://reviews.llvm.org/D21771>
> 
> Adam
> 
>> 3. Exposed relative hotness in the YAML output
The RFC patch implementing 1 and 3 is at https://reviews.llvm.org/D24587
<https://reviews.llvm.org/D24587>.

Adam
>> 
>> Are you working on 1 or should I get started?
>> 
>> There was another issue that came up while discussing this with John
McCall.  We could have a pretty large number of such remarks.  I imagine that if
the post-processing is performed by the external tool, it may be useful to
effectively enable all optimization remarks (i.e.
-Rpass/-Rpass-missed/-Rpass-analysis=loop-vectorize/inline/etc.).  For large
programs this may not be feasible and we may need to locally filter the remarks
in LLVM.  This however seems like a somewhat orthogonal issue that we could
probably postpone for now.
>> 
>> Thanks for your input!
>> Adam
>> 
>>> 
>>> Thanks again,
>>> Hal
>>> 
>>>> 
>>>>> The default diagnostic, furthermore, does not provide
enough
>>>>> information to identify a code "region". I think
that the pass
>>>>> generating the diagnostic needs to provide the information,
>>>>> however, we could certainly create some utility functions
that
>>>>> take a pointer to the BFI analysis and a Value* that can do
the
>>>>> right thing in most simple cases.
>>>> 
>>>> Good point.
>>>> 
>>>> Thanks for your feedback.
>>>> 
>>>> Adam
>>>> 
>>>>> 
>>>>> Thanks again,
>>>>> Hal
>>>>> 
>>>>> ----- Original Message -----
>>>>>> From: "Adam Nemet" <anemet at apple.com
<mailto:anemet at apple.com>>
>>>>>> To: "llvm-dev (llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>)" <llvm-dev at lists.llvm.org
<mailto:llvm-dev at lists.llvm.org>>
>>>>>> Cc: "Hal Finkel" <hfinkel at anl.gov
<mailto:hfinkel at anl.gov>>
>>>>>> Sent: Wednesday, May 4, 2016 1:12:17 PM
>>>>>> Subject: Filter optimization remarks by the hotness of
the code
>>>>>> region
>>>>>> 
>>>>>> This idea came up a few times recently [1][2] so I’d
like start
>>>>>> prototyping it.  To summarize, we can emit optimization
remarks
>>>>>> using the -Rpass* options.  These are currently emitted
by
>>>>>> optimizations like vectorization[3], unrolling,
inlining and since
>>>>>> last week loop distribution.
>>>>>> 
>>>>>> For large programs however this can amount to a lot of
diagnostics
>>>>>> output to sift through.  Filtering this by the hotness
of the
>>>>>> region
>>>>>> can help to focus the user on performance opportunities
that are
>>>>>> likely to pay off.
>>>>>> 
>>>>>> The approach I am thinking of taking is to install a
wrapper as
>>>>>> the
>>>>>> diagnostics handler that will only forward to the
original handler
>>>>>> if the region of code is considered hot.  This will be
installed
>>>>>> by
>>>>>> a new pass that will use BlockFrequencyInfo to
determine the top N
>>>>>> hot regions.
>>>>>> 
>>>>>> This is at very early stage right now.  I would
appreciate any
>>>>>> feedback.
>>>>>> 
>>>>>> Thanks,
>>>>>> Adam
>>>>>> 
>>>>>> [1]
>>>>>>
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
<http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html>
>>>>>> [2]
http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
<http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html>
>>>>>> [3]
>>>>>>
http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
<http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html>
>>>>> 
>>>>> --
>>>>> Hal Finkel
>>>>> Assistant Computational Scientist
>>>>> Leadership Computing Facility
>>>>> Argonne National Laboratory
>>>> 
>>>> 
>>> 
>>> -- 
>>> Hal Finkel
>>> Assistant Computational Scientist
>>> Leadership Computing Facility
>>> Argonne National Laboratory
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160914/a5acb3aa/attachment-0001.html>