llvm dev - May 2016 - Filter optimization remarks by the hotness of the code region

This idea came up a few times recently [1][2] so I’d like start prototyping it. 
To summarize, we can emit optimization remarks using the -Rpass* options.  These
are currently emitted by optimizations like vectorization[3], unrolling,
inlining and since last week loop distribution.

For large programs however this can amount to a lot of diagnostics output to
sift through.  Filtering this by the hotness of the region can help to focus the
user on performance opportunities that are likely to pay off.

The approach I am thinking of taking is to install a wrapper as the diagnostics
handler that will only forward to the original handler if the region of code is
considered hot.  This will be installed by a new pass that will use
BlockFrequencyInfo to determine the top N hot regions.

This is at very early stage right now.  I would appreciate any feedback.

Thanks,
Adam

[1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
[2] http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
[3] http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html

I think it is a good idea, and it reminds me a discussion about Polly at the
last llvm-dev meeting, where we considered limiting compile-time impact by
running polly only the code that is deemed to be "hot".
There could be the same kind of logic for things like LoopVersioningLICMPass, or
specific optimizations like maybe the vectorization:  if the remark is not
relevant because the user should not care about this loop, why does the
optimizer care in the first place?

-- 
Mehdi

> On May 4, 2016, at 11:12 AM, Adam Nemet via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> This idea came up a few times recently [1][2] so I’d like start prototyping
it.  To summarize, we can emit optimization remarks using the -Rpass* options. 
These are currently emitted by optimizations like vectorization[3], unrolling,
inlining and since last week loop distribution.
> 
> For large programs however this can amount to a lot of diagnostics output
to sift through.  Filtering this by the hotness of the region can help to focus
the user on performance opportunities that are likely to pay off.
> 
> The approach I am thinking of taking is to install a wrapper as the
diagnostics handler that will only forward to the original handler if the region
of code is considered hot.  This will be installed by a new pass that will use
BlockFrequencyInfo to determine the top N hot regions.
> 
> This is at very early stage right now.  I would appreciate any feedback.
> 
> Thanks,
> Adam
> 
> [1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
> [2] http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
> [3] http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Sure.  The vectorizer has LoopVectorizeWithBlockFrequency which was meant to
adapt the aggressiveness of the vectorizer to the hotness of the code.  I think
it got turned off by default because at the time BFI didn’t really provide a
good measure of hotness.  This should probably be looked at again especially if
as a result we could turn on vectorization *with versioning* for -Os.

Also just to be clear the main use case is to apply
-Rpass-misssed/-Rpass-analysis with PGO and see why we miss in hot regions.

Adam
> On May 4, 2016, at 4:54 PM, Mehdi Amini <mehdi.amini at apple.com>
wrote:
> 
> I think it is a good idea, and it reminds me a discussion about Polly at
the last llvm-dev meeting, where we considered limiting compile-time impact by
running polly only the code that is deemed to be "hot".
> There could be the same kind of logic for things like
LoopVersioningLICMPass, or specific optimizations like maybe the vectorization: 
if the remark is not relevant because the user should not care about this loop,
why does the optimizer care in the first place?
> 
> -- 
> Mehdi
> 
> 
>> On May 4, 2016, at 11:12 AM, Adam Nemet via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>> 
>> This idea came up a few times recently [1][2] so I’d like start
prototyping it.  To summarize, we can emit optimization remarks using the
-Rpass* options.  These are currently emitted by optimizations like
vectorization[3], unrolling, inlining and since last week loop distribution.
>> 
>> For large programs however this can amount to a lot of diagnostics
output to sift through.  Filtering this by the hotness of the region can help to
focus the user on performance opportunities that are likely to pay off.
>> 
>> The approach I am thinking of taking is to install a wrapper as the
diagnostics handler that will only forward to the original handler if the region
of code is considered hot.  This will be installed by a new pass that will use
BlockFrequencyInfo to determine the top N hot regions.
>> 
>> This is at very early stage right now.  I would appreciate any
feedback.
>> 
>> Thanks,
>> Adam
>> 
>> [1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
>> [2] http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
>> [3]
http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

Hi Adam,

I think would be a really useful feature to have. I don't think that the
backend should be responsible for filtering, but should pass the relative
hotness information to the frontend. Given that these diagnostics are not just
going to be used for -Rpass and friends, but also for generating reports by
other tools (see the discussion around D19678, for example), I think it is
important to allow the frontend to filter. The frontend, or other tool, might
also want to collect information from different compilation jobs to provide the
user with an overall ranking.

The default diagnostic, furthermore, does not provide enough information to
identify a code "region". I think that the pass generating the
diagnostic needs to provide the information, however, we could certainly create
some utility functions that take a pointer to the BFI analysis and a Value* that
can do the right thing in most simple cases.

Thanks again,
Hal

----- Original Message -----
> From: "Adam Nemet" <anemet at apple.com>
> To: "llvm-dev (llvm-dev at lists.llvm.org)" <llvm-dev at
lists.llvm.org>
> Cc: "Hal Finkel" <hfinkel at anl.gov>
> Sent: Wednesday, May 4, 2016 1:12:17 PM
> Subject: Filter optimization remarks by the hotness of the code region
> 
> This idea came up a few times recently [1][2] so I’d like start
> prototyping it.  To summarize, we can emit optimization remarks
> using the -Rpass* options.  These are currently emitted by
> optimizations like vectorization[3], unrolling, inlining and since
> last week loop distribution.
> 
> For large programs however this can amount to a lot of diagnostics
> output to sift through.  Filtering this by the hotness of the region
> can help to focus the user on performance opportunities that are
> likely to pay off.
> 
> The approach I am thinking of taking is to install a wrapper as the
> diagnostics handler that will only forward to the original handler
> if the region of code is considered hot.  This will be installed by
> a new pass that will use BlockFrequencyInfo to determine the top N
> hot regions.
> 
> This is at very early stage right now.  I would appreciate any
> feedback.
> 
> Thanks,
> Adam
> 
> [1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
> [2] http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
> [3]
> http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

Hi Hal,
> On May 10, 2016, at 5:39 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> 
> Hi Adam,
> 
> I think would be a really useful feature to have. I don't think that
the backend should be responsible for filtering, but should pass the relative
hotness information to the frontend. Given that these diagnostics are not just
going to be used for -Rpass and friends, but also for generating reports by
other tools (see the discussion around D19678, for example), I think it is
important to allow the frontend to filter.
I am not sure I follow, can you please elaborate.  Are you saying that for
example in the listing use case we don’t want the filtered diagnostics?  In
other words it should be up to the remark handler to decide whether it wants
filtered or unfiltered remarks?
> The frontend, or other tool, might also want to collect information from
different compilation jobs to provide the user with an overall ranking.
Strictly speaking about PGO, the different compilation jobs get the same PGO
with the aggregated profile, thus the hotness calculated should be global.  I am
not sure why an extra aggregation step is necessary.
> The default diagnostic, furthermore, does not provide enough information to
identify a code "region". I think that the pass generating the
diagnostic needs to provide the information, however, we could certainly create
some utility functions that take a pointer to the BFI analysis and a Value* that
can do the right thing in most simple cases.
Good point.

Thanks for your feedback.

Adam
> 
> Thanks again,
> Hal
> 
> ----- Original Message -----
>> From: "Adam Nemet" <anemet at apple.com>
>> To: "llvm-dev (llvm-dev at lists.llvm.org)" <llvm-dev at
lists.llvm.org>
>> Cc: "Hal Finkel" <hfinkel at anl.gov>
>> Sent: Wednesday, May 4, 2016 1:12:17 PM
>> Subject: Filter optimization remarks by the hotness of the code region
>> 
>> This idea came up a few times recently [1][2] so I’d like start
>> prototyping it.  To summarize, we can emit optimization remarks
>> using the -Rpass* options.  These are currently emitted by
>> optimizations like vectorization[3], unrolling, inlining and since
>> last week loop distribution.
>> 
>> For large programs however this can amount to a lot of diagnostics
>> output to sift through.  Filtering this by the hotness of the region
>> can help to focus the user on performance opportunities that are
>> likely to pay off.
>> 
>> The approach I am thinking of taking is to install a wrapper as the
>> diagnostics handler that will only forward to the original handler
>> if the region of code is considered hot.  This will be installed by
>> a new pass that will use BlockFrequencyInfo to determine the top N
>> hot regions.
>> 
>> This is at very early stage right now.  I would appreciate any
>> feedback.
>> 
>> Thanks,
>> Adam
>> 
>> [1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098492.html
>> [2] http://lists.llvm.org/pipermail/cfe-dev/2016-April/048526.html
>> [3]
>> http://blog.llvm.org/2014/11/loop-vectorization-diagnostics-and.html
> 
> -- 
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory