llvm dev - Sep 2015 - RFC: PGO Late instrumentation for LLVM

This is a late reply -- the email somehow skipped my inbox.
> Philip and Sanjoy, out of curiosity do you guys use your own
instrumentation
> placement for PGO? Is an IR-level PGO infrastructure upstream something you
> guys would be interested in?
>
> I think that 2. is something that once we have 1. we will be able to
> evaluate better, but for now my opinion is that we should be able to make
> good progress without digging into that.
>
> I think that 3. is a no-brainer if it provides a really significant win,
but
> without 1. we can't really measure its effect in isolation. It also has
a
> usability problem since it requires feeding in an existing profile for the
> *instrumented* build, but if the benefit is very significant this may be
> worth it for some users. We will probably be able to easily refactor 1. as
> needed into an MST approach that degrades gracefully to using static
> heuristics in the absence of real profile information, so is not a
> maintenance burden (maybe even helps by providing a good framework in which
> to develop effective static heuristics).
Regarding 3, I am not sure what usability issue are you referring to.
Can you elaborate?

thanks,

David



>
> For the time being, I think we can avoid discussion of 2. and 3. until we
> have more of 1. working. So I think it would be most productive if we focus
> this discussion on 1.
>
>>
>> Additionally, some of the overhead imposed by FE instrumentation is not
>> really all that easy to get rid of.  You end up duplicating
functionality
>> that is more naturally implemented in the middle end.
>
>
> Yeah, I was looking into a couple of other simple approaches and quickly
> found out that I was basically replicating much of the sort of logic that
> the inliner already has.
>
> -- Sean Silva
>
>>
>>
>> I see the two approaches as supplementary, rather than complementary. 
One
>> does not negate the other.  Some of the optimizations we'd do in
the FE, may
>> hurt coverage.  Instead, by instrumenting in the middle end, you can
focus
>> exclusively on performance (coverage be damned).
>>
>>
>> Diego.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>

On Tue, Sep 1, 2015 at 2:10 PM, Xinliang David Li <davidxl at google.com>
wrote:
> This is a late reply -- the email somehow skipped my inbox.
>
> > Philip and Sanjoy, out of curiosity do you guys use your own
> instrumentation
> > placement for PGO? Is an IR-level PGO infrastructure upstream
something
> you
> > guys would be interested in?
> >
> > I think that 2. is something that once we have 1. we will be able to
> > evaluate better, but for now my opinion is that we should be able to
make
> > good progress without digging into that.
> >
> > I think that 3. is a no-brainer if it provides a really significant
win,
> but
> > without 1. we can't really measure its effect in isolation. It
also has a
> > usability problem since it requires feeding in an existing profile for
> the
> > *instrumented* build, but if the benefit is very significant this may
be
> > worth it for some users. We will probably be able to easily refactor
1.
> as
> > needed into an MST approach that degrades gracefully to using static
> > heuristics in the absence of real profile information, so is not a
> > maintenance burden (maybe even helps by providing a good framework in
> which
> > to develop effective static heuristics).
>
> Regarding 3, I am not sure what usability issue are you referring to.
> Can you elaborate?
>
A user will need to provide a profile do the "instr-generate" phase
and as
Rong points out the "instr-use" phase will need to also be supplied
that
same profile in order to interpret the counters. This adds significant
complexity to any PGO workflow and is not essential (it may be desirable at
some point, but it can be added to 1. as an incremental step at any time).

-- Sean Silva


>
> thanks,
>
> David
>
>
>
>
> >
> > For the time being, I think we can avoid discussion of 2. and 3. until
we
> > have more of 1. working. So I think it would be most productive if we
> focus
> > this discussion on 1.
> >
> >>
> >> Additionally, some of the overhead imposed by FE instrumentation
is not
> >> really all that easy to get rid of.  You end up duplicating
> functionality
> >> that is more naturally implemented in the middle end.
> >
> >
> > Yeah, I was looking into a couple of other simple approaches and
quickly
> > found out that I was basically replicating much of the sort of logic
that
> > the inliner already has.
> >
> > -- Sean Silva
> >
> >>
> >>
> >> I see the two approaches as supplementary, rather than
complementary.
> One
> >> does not negate the other.  Some of the optimizations we'd do
in the
> FE, may
> >> hurt coverage.  Instead, by instrumenting in the middle end, you
can
> focus
> >> exclusively on performance (coverage be damned).
> >>
> >>
> >> Diego.
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150901/e3a00f17/attachment.html>

On Tue, Sep 1, 2015 at 2:32 PM, Sean Silva <chisophugis at gmail.com>
wrote:
>
>
> On Tue, Sep 1, 2015 at 2:10 PM, Xinliang David Li <davidxl at
google.com>
> wrote:
>>
>> This is a late reply -- the email somehow skipped my inbox.
>>
>> > Philip and Sanjoy, out of curiosity do you guys use your own
>> > instrumentation
>> > placement for PGO? Is an IR-level PGO infrastructure upstream
something
>> > you
>> > guys would be interested in?
>> >
>> > I think that 2. is something that once we have 1. we will be able
to
>> > evaluate better, but for now my opinion is that we should be able
to
>> > make
>> > good progress without digging into that.
>> >
>> > I think that 3. is a no-brainer if it provides a really
significant win,
>> > but
>> > without 1. we can't really measure its effect in isolation. It
also has
>> > a
>> > usability problem since it requires feeding in an existing profile
for
>> > the
>> > *instrumented* build, but if the benefit is very significant this
may be
>> > worth it for some users. We will probably be able to easily
refactor 1.
>> > as
>> > needed into an MST approach that degrades gracefully to using
static
>> > heuristics in the absence of real profile information, so is not a
>> > maintenance burden (maybe even helps by providing a good framework
in
>> > which
>> > to develop effective static heuristics).
>>
>> Regarding 3, I am not sure what usability issue are you referring to.
>> Can you elaborate?
>
>
> A user will need to provide a profile do the "instr-generate"
phase
Late Instr-generate does not need a profile to work. It works the same
way (in terms of usability) as FE based instrumentation.  You can also
use opt tool to do instrumentation.
>and as
> Rong points out the "instr-use" phase will need to also be
supplied that
> same profile in order to interpret the counters. This adds significant
> complexity to any PGO workflow and is not essential (it may be desirable at
> some point, but it can be added to 1. as an incremental step at any time).
The PGO work flow does not change at all. The only requirement is that
profile annotation pass in profile-use needs to be at the same point
as where the instrumentation is inserted -- this is same as FE based
method too, but more flexible.

David
>
> -- Sean Silva
>
>
>>
>>
>> thanks,
>>
>> David
>>
>>
>>
>>
>> >
>> > For the time being, I think we can avoid discussion of 2. and 3.
until
>> > we
>> > have more of 1. working. So I think it would be most productive if
we
>> > focus
>> > this discussion on 1.
>> >
>> >>
>> >> Additionally, some of the overhead imposed by FE
instrumentation is not
>> >> really all that easy to get rid of.  You end up duplicating
>> >> functionality
>> >> that is more naturally implemented in the middle end.
>> >
>> >
>> > Yeah, I was looking into a couple of other simple approaches and
quickly
>> > found out that I was basically replicating much of the sort of
logic
>> > that
>> > the inliner already has.
>> >
>> > -- Sean Silva
>> >
>> >>
>> >>
>> >> I see the two approaches as supplementary, rather than
complementary.
>> >> One
>> >> does not negate the other.  Some of the optimizations we'd
do in the
>> >> FE, may
>> >> hurt coverage.  Instead, by instrumenting in the middle end,
you can
>> >> focus
>> >> exclusively on performance (coverage be damned).
>> >>
>> >>
>> >> Diego.
>> >>
>> >> _______________________________________________
>> >> LLVM Developers mailing list
>> >> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> >>
>> >
>
>