llvm dev - Jun 2018 - RFC: Pass Execution Instrumentation interface

TL;DR
===
This RFC suggests an API to enable customizable instrumentation of pass 
execution.
The intent is to have a common machinery to implement all the 
pass-execution-debugging
features mentioned here.

Prime target of the interface is the new pass manager.
The overall approach and most of the implementation details should be 
equially applicable
to the legacy one though.


Background
=========
There are quite a few important debugging facilities in LLVM that affect
pass execution sequence:

    -print-after/-print-before[-all]
        execute IR-print pass before or after a particularly 
insteresting pass
        (or all passes)

    -verify-each
        execute verifier pass after each

    -opt-bisect-limit
        execute passes only up to a selected "pass-counter"

    -time-passes
        track execution time for each pass

There are also quite a few similar ideas floating around, i.e:
    -git-commit-after-all
    -debugify-each

All these facilities essentially require instrumentation of pass 
execution points
in the pass manager, each being implemented in a legacy pass manager 
through their
own custom way, e.g:
   * -time-passes has a bunch of dedicated code in each of the pass managers

   * -print-before/after/verify-each insert additional passes before/after
     the passes in the pipeline

And there is no implementation of any of these features for the new pass 
manager,
which obviously is a problem for new pass manager transition.

Proposal
=======
Main idea:
   - introduce an API that allows to instrument points of pass execution
   - access through LLVM Context (allows to control life-time and scope 
in multi-context execution)
   - wrap it into an analysis for easier access from pass managers


Details:
   1. introduce llvm::PassInstrumentation

      This is the main interface that handles the customization and 
provides instrumentation calls

      - resides in IR
      - is accessible through LLVMContext::getPassInstrumentation()
        (with context owning this object).

   2. every single point of Pass execution in the (new) PassManager(s) 
will query
      this analysis and run instrumentation call specific to a 
particular point.

      Instrumentation points:

         bool BeforePass (PassID, PassExecutionCounter);
         void AfterPass (PassID, PassExecutionCounter);

          Run before/after a particular pass execution
              BeforePass instrumentation call returns true if this 
execution is allowed to run.

             'PassID'
                  certain unique identifier for a pass (pass name?).

             'PassExecutionCounter'
                  a number that uniquely identifies this particular pass 
execution
                  in current pipeline, as tracked by Pass Manager.

         void StartPipeline()
         void EndPipeline()

          Run at the start/end of a pass pipeline execution.
          (useful for initialization/finalization purposes)


   3. custom callbacks are registered with 
PassInstrumentation::register* interfaces

      A sequence of registered callbacks is called at each 
instrumentation point as appropriate.

   4. introduce llvm::ExecutionCounter to track execution of passes

      (akin to DebugCounter, yet enabled in Release mode as well?)

      Note: it is somewhat nontrivial to uniquely track pass executions 
with counters in new pass
      manager as its pipeline schedule can be dynamic. Ideas are welcome 
on how to efficiently
      implement unique execution tracking that does not break in 
presence of fixed-point iteration
      passes like RepeatedPass/DevirtSCCRepeatedPass

      Also, the intent is for execution counters to be able provide 
thread-safety in multi-threaded
      pipeline execution (though no work planned for it yet).

   5. introduce a new analysis llvm::PassInstrumentationAnalysis

      This is a convenience wrapper to provide an access to 
PassInstrumentation via analysis framework.
      If using analysis is not convenient (?legacy) then 
PassInstrumentation can be queried
      directly from LLVMContext.


Additional goals
===============
   - layering problem
     Currently OptBisect/OptPassGate has layering issue - interface 
dependencies on all the "IR units",
     even those that are analyses - Loop, CallGraphSCC.

     Generic PassInstrumentation facilitiy allows to inject arbitrary 
call-backs in run-time,
     removing any compile-time interface dependencies on internals of 
those callbacks,
     effectively solving this layering issue.

   - life-time/scope control for multi-context execution

     Currently there are issues with multi-context execution of, say, 
-time-passes which store
     their data in global maps.

     With LLVMContext owning PassInstrumentation there should be no 
problem with multi-context execution
     (callbacks can be made owning the instrumentation data).

Open Questions
=============
   - whats the best way to handle ownership of PassInstrumentation

     Any problems with owning by LLVMContext?
     Something similar to TargetLibraryInfo (owned by 
TargetLibraryAnalysis/TargetLibraryInfoWrapperPass)?

   - using PassInstrumentationAnalysis or directly querying LLVMContext

     PassInstrumentationAnalysis appeared to be a nice idea, only until 
I tried querying it
     in new pass manager framework, and amount of hooplas to jump over 
makes me shiver a bit...

     Querying LLVMContext is plain and straightforward, but we do not 
have a generic way to access LLVMContext
     from a PassManager template (need to introduce generic 
IRUnit::getContext?)

Implementation
=============
PassInstrumentationAnalysis proof-of-concept unfinished prototype 
implementation:
(Heavily under construction, do not enter without wearing a hard hat...)

    https://reviews.llvm.org/D47858

Not sure how did I miss that, but this interface definitely lacks IRUnit.
Should be:

         bool BeforePass (PassID, IRUnit&, PassExecutionCounter);
         void AfterPass (PassID, IRUnit&, PassExecutionCounter);

regards,
   Fedor.

On 06/07/2018 03:00 AM, Fedor Sergeev via llvm-dev
wrote:
>   2. every single point of Pass execution in the (new) PassManager(s) 
> will query
>      this analysis and run instrumentation call specific to a 
> particular point.
>
>      Instrumentation points:
>
>         bool BeforePass (PassID, PassExecutionCounter);
>         void AfterPass (PassID, PassExecutionCounter);
>
>          Run before/after a particular pass execution
>              BeforePass instrumentation call returns true if this 
> execution is allowed to run.
>
>             'PassID'
>                  certain unique identifier for a pass (pass name?).
>
>             'PassExecutionCounter'
>                  a number that uniquely identifies this particular 
> pass execution
>                  in current pipeline, as tracked by Pass Manager.
>
>         void StartPipeline()
>         void EndPipeline()
>
>          Run at the start/end of a pass pipeline execution.
>          (useful for initialization/finalization purposes)

We had already talked about this, so unsurprisingly I'm generally in favor
of the direction. Some comments below.

On Thu, Jun 7, 2018 at 2:00 AM Fedor Sergeev <fedor.sergeev at azul.com>
wrote:
>    - access through LLVM Context (allows to control life-time and scope
> in multi-context execution)
>    - wrap it into an analysis for easier access from pass managers
>
Why not simply make it an analysis, and leave LLVM context out?

Because this is very pass specific, I think it would be substantially
cleaner for it to be more specifically based in the pass infrastructure.

I also think that this can be more cleanly designed by focusing on the new
PM. The legacy PM has reasonable solutions for these problems already, and
I think the desgin can be made somewhat simpler if we don't have to support
both in some way.

My hope would be that there are two basic "layers" to this. Along side
a
particular PassManager, we would have an analysis that instruments the
running passes. This would just expose the basic API to track and control
pass behavior and none of the "business logic".

Then I would hope that the Passes library can build an instance of this
analysis with callbacks (or a type parameter that gets type erased
internally) which handles all the business logic.

I think this will also address the layering issues around IR units because
I think that the generic code can use templates to generically lower the IR
unit down to something that can be cleanly handled by the Passes library. I
think it is generally fine for this layer to rapidly lose strong typing or
only have limited typed facilities because this is about instrumenting
things and shouldn't be having interesting (non-debug) behavioral effects.

>
>
> Details:
>    1. introduce llvm::PassInstrumentation
>
>       This is the main interface that handles the customization and
> provides instrumentation calls
>
>       - resides in IR
>       - is accessible through LLVMContext::getPassInstrumentation()
>         (with context owning this object).
>
>    2. every single point of Pass execution in the (new) PassManager(s)
> will query
>       this analysis and run instrumentation call specific to a
> particular point.
>
>       Instrumentation points:
>
>          bool BeforePass (PassID, PassExecutionCounter);
>          void AfterPass (PassID, PassExecutionCounter);
>
>           Run before/after a particular pass execution
>               BeforePass instrumentation call returns true if this
> execution is allowed to run.
>
>              'PassID'
>                   certain unique identifier for a pass (pass name?).
>
>              'PassExecutionCounter'
>                   a number that uniquely identifies this particular pass
> execution
>                   in current pipeline, as tracked by Pass Manager.
>
>          void StartPipeline()
>          void EndPipeline()
>
>           Run at the start/end of a pass pipeline execution.
>           (useful for initialization/finalization purposes)
>
>
>    3. custom callbacks are registered with
> PassInstrumentation::register* interfaces
>
>       A sequence of registered callbacks is called at each
> instrumentation point as appropriate.
>
>    4. introduce llvm::ExecutionCounter to track execution of passes
>
>       (akin to DebugCounter, yet enabled in Release mode as well?)
>
>       Note: it is somewhat nontrivial to uniquely track pass executions
> with counters in new pass
>       manager as its pipeline schedule can be dynamic. Ideas are welcome
> on how to efficiently
>       implement unique execution tracking that does not break in
> presence of fixed-point iteration
>       passes like RepeatedPass/DevirtSCCRepeatedPass
>
>       Also, the intent is for execution counters to be able provide
> thread-safety in multi-threaded
>       pipeline execution (though no work planned for it yet).
>
>    5. introduce a new analysis llvm::PassInstrumentationAnalysis
>
>       This is a convenience wrapper to provide an access to
> PassInstrumentation via analysis framework.
>       If using analysis is not convenient (?legacy) then
> PassInstrumentation can be queried
>       directly from LLVMContext.
>
>
> Additional goals
> ===============>
>    - layering problem
>      Currently OptBisect/OptPassGate has layering issue - interface
> dependencies on all the "IR units",
>      even those that are analyses - Loop, CallGraphSCC.
>
>      Generic PassInstrumentation facilitiy allows to inject arbitrary
> call-backs in run-time,
>      removing any compile-time interface dependencies on internals of
> those callbacks,
>      effectively solving this layering issue.
>
>    - life-time/scope control for multi-context execution
>
>      Currently there are issues with multi-context execution of, say,
> -time-passes which store
>      their data in global maps.
>
>      With LLVMContext owning PassInstrumentation there should be no
> problem with multi-context execution
>      (callbacks can be made owning the instrumentation data).
>
> Open Questions
> =============>
>    - whats the best way to handle ownership of PassInstrumentation
>
>      Any problems with owning by LLVMContext?
>      Something similar to TargetLibraryInfo (owned by
> TargetLibraryAnalysis/TargetLibraryInfoWrapperPass)?
>
>    - using PassInstrumentationAnalysis or directly querying LLVMContext
>
>      PassInstrumentationAnalysis appeared to be a nice idea, only until
> I tried querying it
>      in new pass manager framework, and amount of hooplas to jump over
> makes me shiver a bit...
>
>      Querying LLVMContext is plain and straightforward, but we do not
> have a generic way to access LLVMContext
>      from a PassManager template (need to introduce generic
> IRUnit::getContext?)
>
> Implementation
> =============>
> PassInstrumentationAnalysis proof-of-concept unfinished prototype
> implementation:
> (Heavily under construction, do not enter without wearing a hard hat...)
>
>     https://reviews.llvm.org/D47858
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180607/42548980/attachment-0001.html>

On 06/07/2018 06:11 PM, Chandler Carruth wrote:
> We had already talked about this, so unsurprisingly I'm generally in 
> favor of the direction. Some comments below.
>
> On Thu, Jun 7, 2018 at 2:00 AM Fedor Sergeev <fedor.sergeev at azul.com 
> <mailto:fedor.sergeev at azul.com>> wrote:
>
>        - access through LLVM Context (allows to control life-time and
>     scope
>     in multi-context execution)
>        - wrap it into an analysis for easier access from pass managers
>
>
> Why not simply make it an analysis, and leave LLVM context out?
I consider LLVM context to be a good reference point for 
"compilation-local singleton stuff".
My task is to provide a way to handle callbacks per-compilation-context, 
and preferably have a single copy of those
(possibly stateful) callbacks per compilation.

In my implementation (linked at the end of RFC) I'm using 
PassInstrumentationImpl to have a single copy of object.
What entity should *own* PassInstrumentationImpl object to make it 
unique per-compilation?

Again, in my implementation with Analysis-managed PassInstrumentation I 
put Impl into PassBuilder
which registers Analyses with a reference to its Impl.
However that makes Impl to be per-Builder unique, which is not the same 
as per-compilation.
>
> Because this is very pass specific, I think it would be substantially 
> cleaner for it to be more specifically based in the pass infrastructure.
>
> I also think that this can be more cleanly designed by focusing on the 
> new PM. The legacy PM has reasonable solutions for these problems 
> already, and I think the desgin can be made somewhat simpler if we 
> don't have to support both in some way.
That I kind of agree with.
And I do not plan to implement both at once.
So in a good case we just switch to new PM and go forward.
And in a bad case of postponing the switch we can use experience and 
details of implementation of new PM to solve problems with legacy PM
(but that is definitely a much lower priority for me).
>
> My hope would be that there are two basic "layers" to this. Along
side
> a particular PassManager, we would have an analysis that instruments 
> the running passes. This would just expose the basic API to track and 
> control pass behavior and none of the "business logic".
Yes. PassInstrumentation seems to provide that.
>
> Then I would hope that the Passes library can build an instance of 
> this analysis with callbacks (or a type parameter that gets type 
> erased internally) which handles all the business logic.
As an idea I do agree with this.
But practically I dont have  a clear picture on how to manage the 
instance(s).

regards,
   Fedor.
>
> I think this will also address the layering issues around IR units 
> because I think that the generic code can use templates to generically 
> lower the IR unit down to something that can be cleanly handled by the 
> Passes library. I think it is generally fine for this layer to rapidly 
> lose strong typing or only have limited typed facilities because this 
> is about instrumenting things and shouldn't be having interesting 
> (non-debug) behavioral effects.
>
>
>
>     Details:
>        1. introduce llvm::PassInstrumentation
>
>           This is the main interface that handles the customization and
>     provides instrumentation calls
>
>           - resides in IR
>           - is accessible through LLVMContext::getPassInstrumentation()
>             (with context owning this object).
>
>        2. every single point of Pass execution in the (new)
>     PassManager(s)
>     will query
>           this analysis and run instrumentation call specific to a
>     particular point.
>
>           Instrumentation points:
>
>              bool BeforePass (PassID, PassExecutionCounter);
>              void AfterPass (PassID, PassExecutionCounter);
>
>               Run before/after a particular pass execution
>                   BeforePass instrumentation call returns true if this
>     execution is allowed to run.
>
>                  'PassID'
>                       certain unique identifier for a pass (pass name?).
>
>                  'PassExecutionCounter'
>                       a number that uniquely identifies this
>     particular pass
>     execution
>                       in current pipeline, as tracked by Pass Manager.
>
>              void StartPipeline()
>              void EndPipeline()
>
>               Run at the start/end of a pass pipeline execution.
>               (useful for initialization/finalization purposes)
>
>
>        3. custom callbacks are registered with
>     PassInstrumentation::register* interfaces
>
>           A sequence of registered callbacks is called at each
>     instrumentation point as appropriate.
>
>        4. introduce llvm::ExecutionCounter to track execution of passes
>
>           (akin to DebugCounter, yet enabled in Release mode as well?)
>
>           Note: it is somewhat nontrivial to uniquely track pass
>     executions
>     with counters in new pass
>           manager as its pipeline schedule can be dynamic. Ideas are
>     welcome
>     on how to efficiently
>           implement unique execution tracking that does not break in
>     presence of fixed-point iteration
>           passes like RepeatedPass/DevirtSCCRepeatedPass
>
>           Also, the intent is for execution counters to be able provide
>     thread-safety in multi-threaded
>           pipeline execution (though no work planned for it yet).
>
>        5. introduce a new analysis llvm::PassInstrumentationAnalysis
>
>           This is a convenience wrapper to provide an access to
>     PassInstrumentation via analysis framework.
>           If using analysis is not convenient (?legacy) then
>     PassInstrumentation can be queried
>           directly from LLVMContext.
>
>
>     Additional goals
>     ===============>
>        - layering problem
>          Currently OptBisect/OptPassGate has layering issue - interface
>     dependencies on all the "IR units",
>          even those that are analyses - Loop, CallGraphSCC.
>
>          Generic PassInstrumentation facilitiy allows to inject arbitrary
>     call-backs in run-time,
>          removing any compile-time interface dependencies on internals of
>     those callbacks,
>          effectively solving this layering issue.
>
>        - life-time/scope control for multi-context execution
>
>          Currently there are issues with multi-context execution of, say,
>     -time-passes which store
>          their data in global maps.
>
>          With LLVMContext owning PassInstrumentation there should be no
>     problem with multi-context execution
>          (callbacks can be made owning the instrumentation data).
>
>     Open Questions
>     =============>
>        - whats the best way to handle ownership of PassInstrumentation
>
>          Any problems with owning by LLVMContext?
>          Something similar to TargetLibraryInfo (owned by
>     TargetLibraryAnalysis/TargetLibraryInfoWrapperPass)?
>
>        - using PassInstrumentationAnalysis or directly querying
>     LLVMContext
>
>          PassInstrumentationAnalysis appeared to be a nice idea, only
>     until
>     I tried querying it
>          in new pass manager framework, and amount of hooplas to jump
>     over
>     makes me shiver a bit...
>
>          Querying LLVMContext is plain and straightforward, but we do not
>     have a generic way to access LLVMContext
>          from a PassManager template (need to introduce generic
>     IRUnit::getContext?)
>
>     Implementation
>     =============>
>     PassInstrumentationAnalysis proof-of-concept unfinished prototype
>     implementation:
>     (Heavily under construction, do not enter without wearing a hard
>     hat...)
>
>     https://reviews.llvm.org/D47858
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180607/6e4ff131/attachment.html>

Hi Fedor,

Thanks for replying my questions about porting the OptBisecting to new PM.

This thread looks like a great improvement on what we currently have.
Though we are also trying to make opt-bisect more granular.

In particular, we think it would be helpful if we could have opt-bisect
work on a per-optimization level rather than per-pass level.
I believe this will be a more invasive change and we would like to do this
as a follow-up to this thread.

How difficult do you think it would be to support this use-case with your
design?

Thank you!
Zhizhou


On Wed, Jun 6, 2018 at 5:00 PM Fedor Sergeev via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> TL;DR
> ===>
> This RFC suggests an API to enable customizable instrumentation of pass
> execution.
> The intent is to have a common machinery to implement all the
> pass-execution-debugging
> features mentioned here.
>
> Prime target of the interface is the new pass manager.
> The overall approach and most of the implementation details should be
> equially applicable
> to the legacy one though.
>
>
> Background
> =========>
> There are quite a few important debugging facilities in LLVM that affect
> pass execution sequence:
>
>     -print-after/-print-before[-all]
>         execute IR-print pass before or after a particularly
> insteresting pass
>         (or all passes)
>
>     -verify-each
>         execute verifier pass after each
>
>     -opt-bisect-limit
>         execute passes only up to a selected "pass-counter"
>
>     -time-passes
>         track execution time for each pass
>
> There are also quite a few similar ideas floating around, i.e:
>     -git-commit-after-all
>     -debugify-each
>
> All these facilities essentially require instrumentation of pass
> execution points
> in the pass manager, each being implemented in a legacy pass manager
> through their
> own custom way, e.g:
>    * -time-passes has a bunch of dedicated code in each of the pass
> managers
>
>    * -print-before/after/verify-each insert additional passes before/after
>      the passes in the pipeline
>
> And there is no implementation of any of these features for the new pass
> manager,
> which obviously is a problem for new pass manager transition.
>
> Proposal
> =======>
> Main idea:
>    - introduce an API that allows to instrument points of pass execution
>    - access through LLVM Context (allows to control life-time and scope
> in multi-context execution)
>    - wrap it into an analysis for easier access from pass managers
>
>
> Details:
>    1. introduce llvm::PassInstrumentation
>
>       This is the main interface that handles the customization and
> provides instrumentation calls
>
>       - resides in IR
>       - is accessible through LLVMContext::getPassInstrumentation()
>         (with context owning this object).
>
>    2. every single point of Pass execution in the (new) PassManager(s)
> will query
>       this analysis and run instrumentation call specific to a
> particular point.
>
>       Instrumentation points:
>
>          bool BeforePass (PassID, PassExecutionCounter);
>          void AfterPass (PassID, PassExecutionCounter);
>
>           Run before/after a particular pass execution
>               BeforePass instrumentation call returns true if this
> execution is allowed to run.
>
>              'PassID'
>                   certain unique identifier for a pass (pass name?).
>
>              'PassExecutionCounter'
>                   a number that uniquely identifies this particular pass
> execution
>                   in current pipeline, as tracked by Pass Manager.
>
>          void StartPipeline()
>          void EndPipeline()
>
>           Run at the start/end of a pass pipeline execution.
>           (useful for initialization/finalization purposes)
>
>
>    3. custom callbacks are registered with
> PassInstrumentation::register* interfaces
>
>       A sequence of registered callbacks is called at each
> instrumentation point as appropriate.
>
>    4. introduce llvm::ExecutionCounter to track execution of passes
>
>       (akin to DebugCounter, yet enabled in Release mode as well?)
>
>       Note: it is somewhat nontrivial to uniquely track pass executions
> with counters in new pass
>       manager as its pipeline schedule can be dynamic. Ideas are welcome
> on how to efficiently
>       implement unique execution tracking that does not break in
> presence of fixed-point iteration
>       passes like RepeatedPass/DevirtSCCRepeatedPass
>
>       Also, the intent is for execution counters to be able provide
> thread-safety in multi-threaded
>       pipeline execution (though no work planned for it yet).
>
>    5. introduce a new analysis llvm::PassInstrumentationAnalysis
>
>       This is a convenience wrapper to provide an access to
> PassInstrumentation via analysis framework.
>       If using analysis is not convenient (?legacy) then
> PassInstrumentation can be queried
>       directly from LLVMContext.
>
>
> Additional goals
> ===============>
>    - layering problem
>      Currently OptBisect/OptPassGate has layering issue - interface
> dependencies on all the "IR units",
>      even those that are analyses - Loop, CallGraphSCC.
>
>      Generic PassInstrumentation facilitiy allows to inject arbitrary
> call-backs in run-time,
>      removing any compile-time interface dependencies on internals of
> those callbacks,
>      effectively solving this layering issue.
>
>    - life-time/scope control for multi-context execution
>
>      Currently there are issues with multi-context execution of, say,
> -time-passes which store
>      their data in global maps.
>
>      With LLVMContext owning PassInstrumentation there should be no
> problem with multi-context execution
>      (callbacks can be made owning the instrumentation data).
>
> Open Questions
> =============>
>    - whats the best way to handle ownership of PassInstrumentation
>
>      Any problems with owning by LLVMContext?
>      Something similar to TargetLibraryInfo (owned by
> TargetLibraryAnalysis/TargetLibraryInfoWrapperPass)?
>
>    - using PassInstrumentationAnalysis or directly querying LLVMContext
>
>      PassInstrumentationAnalysis appeared to be a nice idea, only until
> I tried querying it
>      in new pass manager framework, and amount of hooplas to jump over
> makes me shiver a bit...
>
>      Querying LLVMContext is plain and straightforward, but we do not
> have a generic way to access LLVMContext
>      from a PassManager template (need to introduce generic
> IRUnit::getContext?)
>
> Implementation
> =============>
> PassInstrumentationAnalysis proof-of-concept unfinished prototype
> implementation:
> (Heavily under construction, do not enter without wearing a hard hat...)
>
>     https://reviews.llvm.org/D47858
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180607/2b7f68d2/attachment.html>

Care to expand a bit on what you mean by per-optimization level? Preferably with
a use case.

To me optbisect is a low level developer tool and it doesn't cope well with
a crude user level hammer of optimization level.

F.



On Fri, Jun 8, 2018 at 9:12 AM +0300, "Zhizhou Yang" <zhizhouy at
google.com<mailto:zhizhouy at google.com>> wrote:

Hi Fedor,

Thanks for replying my questions about porting the OptBisecting to new PM.

This thread looks like a great improvement on what we currently have.
Though we are also trying to make opt-bisect more granular.

In particular, we think it would be helpful if we could have opt-bisect work on
a per-optimization level rather than per-pass level.
I believe this will be a more invasive change and we would like to do this as a
follow-up to this thread.

How difficult do you think it would be to support this use-case with your
design?

Thank you!
Zhizhou


On Wed, Jun 6, 2018 at 5:00 PM Fedor Sergeev via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
TL;DR
===
This RFC suggests an API to enable customizable instrumentation of pass
execution.
The intent is to have a common machinery to implement all the
pass-execution-debugging
features mentioned here.

Prime target of the interface is the new pass manager.
The overall approach and most of the implementation details should be
equially applicable
to the legacy one though.


Background
=========
There are quite a few important debugging facilities in LLVM that affect
pass execution sequence:

    -print-after/-print-before[-all]
        execute IR-print pass before or after a particularly
insteresting pass
        (or all passes)

    -verify-each
        execute verifier pass after each

    -opt-bisect-limit
        execute passes only up to a selected "pass-counter"

    -time-passes
        track execution time for each pass

There are also quite a few similar ideas floating around, i.e:
    -git-commit-after-all
    -debugify-each

All these facilities essentially require instrumentation of pass
execution points
in the pass manager, each being implemented in a legacy pass manager
through their
own custom way, e.g:
   * -time-passes has a bunch of dedicated code in each of the pass managers

   * -print-before/after/verify-each insert additional passes before/after
     the passes in the pipeline

And there is no implementation of any of these features for the new pass
manager,
which obviously is a problem for new pass manager transition.

Proposal
=======
Main idea:
   - introduce an API that allows to instrument points of pass execution
   - access through LLVM Context (allows to control life-time and scope
in multi-context execution)
   - wrap it into an analysis for easier access from pass managers


Details:
   1. introduce llvm::PassInstrumentation

      This is the main interface that handles the customization and
provides instrumentation calls

      - resides in IR
      - is accessible through LLVMContext::getPassInstrumentation()
        (with context owning this object).

   2. every single point of Pass execution in the (new) PassManager(s)
will query
      this analysis and run instrumentation call specific to a
particular point.

      Instrumentation points:

         bool BeforePass (PassID, PassExecutionCounter);
         void AfterPass (PassID, PassExecutionCounter);

          Run before/after a particular pass execution
              BeforePass instrumentation call returns true if this
execution is allowed to run.

             'PassID'
                  certain unique identifier for a pass (pass name?).

             'PassExecutionCounter'
                  a number that uniquely identifies this particular pass
execution
                  in current pipeline, as tracked by Pass Manager.

         void StartPipeline()
         void EndPipeline()

          Run at the start/end of a pass pipeline execution.
          (useful for initialization/finalization purposes)


   3. custom callbacks are registered with
PassInstrumentation::register* interfaces

      A sequence of registered callbacks is called at each
instrumentation point as appropriate.

   4. introduce llvm::ExecutionCounter to track execution of passes

      (akin to DebugCounter, yet enabled in Release mode as well?)

      Note: it is somewhat nontrivial to uniquely track pass executions
with counters in new pass
      manager as its pipeline schedule can be dynamic. Ideas are welcome
on how to efficiently
      implement unique execution tracking that does not break in
presence of fixed-point iteration
      passes like RepeatedPass/DevirtSCCRepeatedPass

      Also, the intent is for execution counters to be able provide
thread-safety in multi-threaded
      pipeline execution (though no work planned for it yet).

   5. introduce a new analysis llvm::PassInstrumentationAnalysis

      This is a convenience wrapper to provide an access to
PassInstrumentation via analysis framework.
      If using analysis is not convenient (?legacy) then
PassInstrumentation can be queried
      directly from LLVMContext.


Additional goals
===============
   - layering problem
     Currently OptBisect/OptPassGate has layering issue - interface
dependencies on all the "IR units",
     even those that are analyses - Loop, CallGraphSCC.

     Generic PassInstrumentation facilitiy allows to inject arbitrary
call-backs in run-time,
     removing any compile-time interface dependencies on internals of
those callbacks,
     effectively solving this layering issue.

   - life-time/scope control for multi-context execution

     Currently there are issues with multi-context execution of, say,
-time-passes which store
     their data in global maps.

     With LLVMContext owning PassInstrumentation there should be no
problem with multi-context execution
     (callbacks can be made owning the instrumentation data).

Open Questions
=============
   - whats the best way to handle ownership of PassInstrumentation

     Any problems with owning by LLVMContext?
     Something similar to TargetLibraryInfo (owned by
TargetLibraryAnalysis/TargetLibraryInfoWrapperPass)?

   - using PassInstrumentationAnalysis or directly querying LLVMContext

     PassInstrumentationAnalysis appeared to be a nice idea, only until
I tried querying it
     in new pass manager framework, and amount of hooplas to jump over
makes me shiver a bit...

     Querying LLVMContext is plain and straightforward, but we do not
have a generic way to access LLVMContext
     from a PassManager template (need to introduce generic
IRUnit::getContext?)

Implementation
=============
PassInstrumentationAnalysis proof-of-concept unfinished prototype
implementation:
(Heavily under construction, do not enter without wearing a hard hat...)

    https://reviews.llvm.org/D47858

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180608/012beb98/attachment.html>