llvm dev - Jun 2018 - RFC: Pass Execution Instrumentation interface

Fedor Sergeev <fedor.sergeev at azul.com> writes:
> On 06/12/2018 12:04 AM, David A. Greene wrote:
>> // PIA - PassInstrumentationAnalysis
>> if (PIA->skipTransformation()) {
>>    return;
>> }
>> // Do it.
>> PIA->didTransformation();
> That should be easily doable (though the interface would be part of
> PassInstrumentation
> rather than PassInstrumentationAnalysis).
Ok.  The way I envision this working from a user standpoint is
-opt-bisect-limit <n> would mean "n applications of code
transformation." where "code transformation" could mean an entire
pass
run or individual transforms within a pass.  Each pass would decide what
it supports.

>> This kind of interface also encourages good pass design like doing all
>> the analysis for a transformation before actually doing the
>> transformation.  Some passes mix analysis with transformation and those
>> are much harder to instrument to support -pass-max operation.
> I'm not sure everybody would agree on this definition of a good pass
> design :)
> Ability to mix analysis with transformation might appear to be rather
useful
> when heavy analysis is only needed in a very special corner case of an
> overall
> transformation.
Yes, I'm sure there are exceptions.  I'm not referring to things like
instcombine that have individual rules that guard transformations and
the pass iterates applying transformations when rules are matched.
That's straightforward to instrument.

The harder cases are where the analysis phase itself does some
transformation (possily to facilitate analysis) and then decides the
larger-goal transformation is not viable.  If the pass then tries to
undo the first transformation, it's possible that -pass-max will result
in code that never would have been generated, because it could do the
first transformation but then not undo it because it hit the max number
of transforms.  Sometimes it's difficult to find where things are undone
and update the transformation index (basically allow the undo and
decrement the index to reflect the undo).

In code:

if (not hit max)
  do anlysis transform
  ++index

return

<some other function>

if (transform legal)
  if (not hit max)
    do big transform
    ++index

return

<some third function>
if (need to undo analysis transform)
  if (not hit max)
    undo it
    ++index

Sometimes it is not obvious that these three places are logically
connected.  Ideally we wouldn't increment the index for the analysis
transform or we would allow the undo and decrement the index, but it's
not always clear from the code that that is what should happen.

                            -David

On Wed, Jun 13, 2018 at 6:46 PM David A. Greene via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> [...]
>
> The harder cases are where the analysis phase itself does some
> transformation (possily to facilitate analysis) and then decides the
> larger-goal transformation is not viable.  If the pass then tries to
> undo the first transformation, it's possible that -pass-max will result
> in code that never would have been generated, because it could do the
> first transformation but then not undo it because it hit the max number
> of transforms.  Sometimes it's difficult to find where things are
undone
> and update the transformation index (basically allow the undo and
> decrement the index to reflect the undo).
>
It should be pointed out that analyses don't transform the IR. At least not
in the new PassManager, which I think we should focus on in this proposal.

Cheers,
Philip
>
> In code:
>
> if (not hit max)
>   do anlysis transform
>   ++index
>
> return
>
> <some other function>
>
> if (transform legal)
>   if (not hit max)
>     do big transform
>     ++index
>
> return
>
> <some third function>
> if (need to undo analysis transform)
>   if (not hit max)
>     undo it
>     ++index
>
> Sometimes it is not obvious that these three places are logically
> connected.  Ideally we wouldn't increment the index for the analysis
> transform or we would allow the undo and decrement the index, but it's
> not always clear from the code that that is what should happen.
>
>                             -David
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180613/dbfc8009/attachment.html>

On 06/13/2018 07:46 PM, David A. Greene wrote:
> Fedor Sergeev <fedor.sergeev at azul.com> writes:
>
>> On 06/12/2018 12:04 AM, David A. Greene wrote:
>>> // PIA - PassInstrumentationAnalysis
>>> if (PIA->skipTransformation()) {
>>>     return;
>>> }
>>> // Do it.
>>> PIA->didTransformation();
>> That should be easily doable (though the interface would be part of
>> PassInstrumentation
>> rather than PassInstrumentationAnalysis).
> Ok.  The way I envision this working from a user standpoint is
> -opt-bisect-limit <n> would mean "n applications of code
> transformation." where "code transformation" could mean an
entire pass
> run or individual transforms within a pass.  Each pass would decide what
> it supports.
I would rather not merge pass-execution and in-pass-transformation 
numbers into a single number.
It will only confuse users on what is being controlled.
Especially as in-pass control is going to be opt-in only.
>
>
>>> This kind of interface also encourages good pass design like doing
all
>>> the analysis for a transformation before actually doing the
>>> transformation.  Some passes mix analysis with transformation and
those
>>> are much harder to instrument to support -pass-max operation.
>> I'm not sure everybody would agree on this definition of a good
pass
>> design :)
>> Ability to mix analysis with transformation might appear to be rather
useful
>> when heavy analysis is only needed in a very special corner case of an
>> overall
>> transformation.
> Yes, I'm sure there are exceptions.  I'm not referring to things
like
> instcombine that have individual rules that guard transformations and
> the pass iterates applying transformations when rules are matched.
> That's straightforward to instrument.
>
> The harder cases are where the analysis phase itself does some
> transformation (possily to facilitate analysis) and then decides the
As Philip has already pointed out, analyses by design are expected to be 
non-mutating.

regards,
   Fedor.
> larger-goal transformation is not viable.  If the pass then tries to
> undo the first transformation, it's possible that -pass-max will result
> in code that never would have been generated, because it could do the
> first transformation but then not undo it because it hit the max number
> of transforms.  Sometimes it's difficult to find where things are
undone
> and update the transformation index (basically allow the undo and
> decrement the index to reflect the undo).
>
> In code:
>
> if (not hit max)
>    do anlysis transform
>    ++index
>
> return
>
> <some other function>
>
> if (transform legal)
>    if (not hit max)
>      do big transform
>      ++index
>
> return
>
> <some third function>
> if (need to undo analysis transform)
>    if (not hit max)
>      undo it
>      ++index
>
> Sometimes it is not obvious that these three places are logically
> connected.  Ideally we wouldn't increment the index for the analysis
> transform or we would allow the undo and decrement the index, but it's
> not always clear from the code that that is what should happen.
>
>                              -David

Philip Pfaffe <philip.pfaffe at gmail.com> writes:
>     The harder cases are where the analysis phase itself does some
>     transformation (possily to facilitate analysis) and then decides
>     the
>     larger-goal transformation is not viable. If the pass then tries
>     to
>     undo the first transformation, it's possible that -pass-max will
>     result
>     in code that never would have been generated, because it could do
>     the
>     first transformation but then not undo it because it hit the max
>     number
>     of transforms. Sometimes it's difficult to find where things are
>     undone
>     and update the transformation index (basically allow the undo and
>     decrement the index to reflect the undo).
> It should be pointed out that analyses don't transform the IR. At
> least not in the new PassManager, which I think we should focus on in
> this proposal.
I'm not talking about analysis passes as such.  I'm talking about
transformations passes that check various conditions before doing
transformations.  They have to check legality, profitability, etc.  Most
of the time this is well-separated but sometimes things can get pretty
convoluted and it's not always clear where the "logical changes"
are, as
opposed to component changes that make up a single logical change.

                            -David

Fedor Sergeev <fedor.sergeev at azul.com> writes:
>> Ok.  The way I envision this working from a user standpoint is
>> -opt-bisect-limit <n> would mean "n applications of code
>> transformation." where "code transformation" could mean
an entire pass
>> run or individual transforms within a pass.  Each pass would decide
what
>> it supports.
> I would rather not merge pass-execution and in-pass-transformation
> numbers into a single number.
> It will only confuse users on what is being controlled.
> Especially as in-pass control is going to be opt-in only.
Oh, ok.  I'm fine with that too.  Do we want this finer-grained control
on a global basis, or a per-pass basis?  For example, should something
like -transform-max=<n> apply over the whole compilation run, so that
every pass checks the limit, or should it work like
-transform-max=<pass>=<n>, where only pass <pass> checks the
limit?  If
the latter, then -opt-bisect-limit (or bugpoint) can identify the pass
and another run with -transform-max can identify the specific transform
within the pass.

The latter is how we have things set up here and it seems to work well,
but I can also see utility in a global limit because then you don't need
two separate runs to isolate the problem.

I'd like to start building this off the pass instrumentation stuff as
soon as it gets integrated.  Could you copy me on Phabricator when they
land there?  Thanks!
>> The harder cases are where the analysis phase itself does some
>> transformation (possily to facilitate analysis) and then decides the
> As Philip has already pointed out, analyses by design are expected to
> be non-mutating.
See my reply to Philip.  I'm talking about various analyses that happen
within transformation passes.

                               -David