llvm dev - Jun 2017 - Showing hotness in LLVM optimization remarks using AutoFDO sampling profile data?

Hello!
(+cc Adam Nemet, since he presented on optimization remarks at LLVM Dev Mtg
2016)

I have a large C++ program, which I am compiling using a sampling profile
generated via perf and AutoFDO. I'd like to use this profile in order to
show the hotness of each code path that is displayed in the new
optimization remarks viewer tool (
https://www.youtube.com/watch?v=qq0q1hfzidg).

It seems, however, that the '-fdiagnostics-show-hotness' option emits a
warning when used with '-fprofile-sample-use': "warning: argument
'-fdiagnostics-show-hotness' requires profile-guided optimization
information".

Looking at the code in Clang that generates the warning, it appears that
the warning is emitted unless instrumentation profile data is provided. Am
I correct in understanding that sampling profile data cannot be used to
determine the hotness of the code paths displayed via opt-viewer?

If this is correct, I am curious if this is a hard limitation, or whether I
can do some work on CodeGen or ORE to make it possible to use a sampling
profile. I ask because, while I'm excited to use optimization remarks,
getting an instrumentation profile of my program would be difficult -- it's
performance-critical and I can't release an instrumented build.

Sorry if this is a dumb question -- I don't have much experience using PGO
-- and thanks for all the work on opt-viewer! It's really fun to use.

- Brian Gesiak
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170609/46689fe0/attachment.html>

Hi Brian,

It’s great to hear that you find the opt-viewer useful.  To your questions,
specifically:
> On Jun 9, 2017, at 9:02 AM, Brian Gesiak <modocache at gmail.com>
wrote:
> 
> Hello!
> (+cc Adam Nemet, since he presented on optimization remarks at LLVM Dev Mtg
2016)
> 
> I have a large C++ program, which I am compiling using a sampling profile
generated via perf and AutoFDO. I'd like to use this profile in order to
show the hotness of each code path that is displayed in the new optimization
remarks viewer tool (https://www.youtube.com/watch?v=qq0q1hfzidg
<https://www.youtube.com/watch?v=qq0q1hfzidg>).
> 
> It seems, however, that the '-fdiagnostics-show-hotness' option
emits a warning when used with '-fprofile-sample-use': "warning:
argument '-fdiagnostics-show-hotness' requires profile-guided
optimization information”.
I think this is probably an oversight and we should accept -fprofile-sample-use
as well.  I think that simply no one has tried to use the opt-viewer with
sample-based profiling so far.  (We don’t have this support on macOS, so I’ve
only been using PGO.)
> 
> Looking at the code in Clang that generates the warning, it appears that
the warning is emitted unless instrumentation profile data is provided. Am I
correct in understanding that sampling profile data cannot be used to determine
the hotness of the code paths displayed via opt-viewer?
> 
> If this is correct, I am curious if this is a hard limitation, or whether I
can do some work on CodeGen or ORE to make it possible to use a sampling
profile. I ask because, while I'm excited to use optimization remarks,
getting an instrumentation profile of my program would be difficult -- it's
performance-critical and I can't release an instrumented build.
I don’t think there is any hard limitation.  The information from
instrumentation- and sample-based profiling are exposed through the same
interface/analysis (BlockFrequencyInfo) within the compiler which is what the
optimization remark facility uses.

So I’d say that this should just work modulo some differences in accuracy due
the sampling vs instrumentation.  Despite the warning, do we already get hotness
in the remarks?  It seems we should since we still set DiagnosticsWithHotness in
CodeGenOpts [1] and in turn would call setDiagnosticHotnessRequested with true
in the LLVM Context [2].  This is what then controls whether we emit hotness in
the remark in LLVM [3].

If this does not work, you probably wan to debug through this path of execution
and see where we fail.  Let me know if I can help further.
> 
> Sorry if this is a dumb question -- I don't have much experience using
PGO -- and thanks for all the work on opt-viewer! It's really fun to use.
Not dumb at all, thanks for bringing this up.

Adam
> 
> - Brian Gesiak
[1]
https://github.com/llvm-mirror/clang/blob/master/lib/Frontend/CompilerInvocation.cpp#L887
[2]
https://github.com/llvm-mirror/clang/blob/master/lib/CodeGen/CodeGenAction.cpp#L231
[3]
https://github.com/llvm-mirror/llvm/blob/master/lib/Analysis/OptimizationDiagnosticInfo.cpp#L179
<https://github.com/llvm-mirror/llvm/blob/master/lib/Analysis/OptimizationDiagnosticInfo.cpp#L179>

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170609/0e86aa31/attachment.html>

On Fri, Jun 9, 2017 at 11:15 AM, Adam Nemet <anemet at apple.com> wrote:
> Hi Brian,
>
> It’s great to hear that you find the opt-viewer useful.  To your
> questions, specifically:
>
> On Jun 9, 2017, at 9:02 AM, Brian Gesiak <modocache at gmail.com>
wrote:
>
> Hello!
> (+cc Adam Nemet, since he presented on optimization remarks at LLVM Dev
> Mtg 2016)
>
> I have a large C++ program, which I am compiling using a sampling profile
> generated via perf and AutoFDO. I'd like to use this profile in order
to
> show the hotness of each code path that is displayed in the new
> optimization remarks viewer tool (https://www.youtube.com/
> watch?v=qq0q1hfzidg).
>
> It seems, however, that the '-fdiagnostics-show-hotness' option
emits a
> warning when used with '-fprofile-sample-use': "warning:
argument
> '-fdiagnostics-show-hotness' requires profile-guided optimization
> information”.
>
>
> I think this is probably an oversight and we should accept
> -fprofile-sample-use as well.  I think that simply no one has tried to use
> the opt-viewer with sample-based profiling so far.  (We don’t have this
> support on macOS, so I’ve only been using PGO.)
>
>
Agreed. Patches to fix the problem are welcome :)

David

>
> Looking at the code in Clang that generates the warning, it appears that
> the warning is emitted unless instrumentation profile data is provided. Am
> I correct in understanding that sampling profile data cannot be used to
> determine the hotness of the code paths displayed via opt-viewer?
>
> If this is correct, I am curious if this is a hard limitation, or whether
> I can do some work on CodeGen or ORE to make it possible to use a sampling
> profile. I ask because, while I'm excited to use optimization remarks,
> getting an instrumentation profile of my program would be difficult --
it's
> performance-critical and I can't release an instrumented build.
>
>
> I don’t think there is any hard limitation.  The information from
> instrumentation- and sample-based profiling are exposed through the same
> interface/analysis (BlockFrequencyInfo) within the compiler which is what
> the optimization remark facility uses.
>
> So I’d say that this should just work modulo some differences in accuracy
> due the sampling vs instrumentation.  Despite the warning, do we already
> get hotness in the remarks?  It seems we should since we still set
> DiagnosticsWithHotness in CodeGenOpts [1] and in turn would call
> setDiagnosticHotnessRequested with true in the LLVM Context [2].  This is
> what then controls whether we emit hotness in the remark in LLVM [3].
>
> If this does not work, you probably wan to debug through this path of
> execution and see where we fail.  Let me know if I can help further.
>
>
> Sorry if this is a dumb question -- I don't have much experience using
PGO
> -- and thanks for all the work on opt-viewer! It's really fun to use.
>
>
> Not dumb at all, thanks for bringing this up.
>
> Adam
>
>
> - Brian Gesiak
>
>
> [1] https://github.com/llvm-mirror/clang/blob/master/lib/
> Frontend/CompilerInvocation.cpp#L887
> [2] https://github.com/llvm-mirror/clang/blob/master/lib/
> CodeGen/CodeGenAction.cpp#L231
> [3] https://github.com/llvm-mirror/llvm/blob/master/lib/Analysis/
> OptimizationDiagnosticInfo.cpp#L179
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170609/cbdaa5e0/attachment.html>

Adam, thanks!
> Despite the warning, do we already get hotness in the remarks?
You're absolutely right: despite the error, it looks like ORE does include
hotness in the output YAML. Fantastic! Thanks for pointing that out.

I'll try to submit a patch over the weekend to prevent the warning from
being printed in the case a sampling profile is used. Of course, feel free
to beat me to it. :)

- Brian Gesiak
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170609/0e802e1b/attachment.html>