llvm dev - May 2016 - Some questions about phase ordering in OPT and LLC

> >> You can look at AddOptimizationPasses() in opt.cpp.
> > 
> > As far as I understand, the two passmanager do not interleave their
> > passes. It first runs all the function passes and below. Then all the
> > module passes. So if you specify:
> > 
> >    opt -mymodulepass0 -myfunctionpass -mymodulepass1
> > 
> > What you actually get is:
> > 
> > 1. myfunctionpass on each function
> > 2. mymodulepass0 
> > 3. mymodulepass0 
> 
> (I assume your 3 was intended to be mymodulepass1 right?)
(yes)
> So AFAIK no, you should get the order you specified on the command line,
i.e.
> 
> 1. mymodulepass0 
> 2. myfunctionpass on each function
> 3. mymodulepass1
MMMh, from opt.cpp, there's a first call to:

  if (OptLevelO1 || OptLevelO2 || OptLevelOs || OptLevelOz || OptLevelO3) {
    FPasses->doInitialization();
    for (Function &F : *M)
      FPasses->run(F);
    FPasses->doFinalization();
  }

then a few lines later, a call to:

    Passes.run(*M);

where Passes is the Module pass Manager and FPasses is the Function Pass
Manager. Each is filled in AddOptimizationPasses with different passes.
I don't see the point where the two manage interleave their passes.

> On May 9, 2016, at 10:16 PM, serge guelton <sguelton at
quarkslab.com> wrote:
> 
>>>> You can look at AddOptimizationPasses() in opt.cpp.
>>> 
>>> As far as I understand, the two passmanager do not interleave their
>>> passes. It first runs all the function passes and below. Then all
the
>>> module passes. So if you specify:
>>> 
>>>   opt -mymodulepass0 -myfunctionpass -mymodulepass1
>>> 
>>> What you actually get is:
>>> 
>>> 1. myfunctionpass on each function
>>> 2. mymodulepass0 
>>> 3. mymodulepass0 
>> 
>> (I assume your 3 was intended to be mymodulepass1 right?)
> (yes)
>> So AFAIK no, you should get the order you specified on the command
line, i.e.
>> 
>> 1. mymodulepass0 
>> 2. myfunctionpass on each function
>> 3. mymodulepass1
> 
> MMMh, from opt.cpp, there's a first call to:
> 
>  if (OptLevelO1 || OptLevelO2 || OptLevelOs || OptLevelOz || OptLevelO3) {
>    FPasses->doInitialization();
>    for (Function &F : *M)
>      FPasses->run(F);
>    FPasses->doFinalization();
>  }
> 
> then a few lines later, a call to:
> 
>    Passes.run(*M);
> 
> where Passes is the Module pass Manager and FPasses is the Function Pass
> Manager. Each is filled in AddOptimizationPasses with different passes.
> I don't see the point where the two manage interleave their passes.
It seems like we need to clarify that we're talking about the same thing:
i.e. opt -O3 or opt -mypass?

The code you show correspond to the two separated pass managers I was mentioning
in my first answer about how to read the output of -debug-pass=Arguments for opt
-O3. As you can see in the if condition in the snippet you post, it is only used
when a -O is passed to opt, and not when you invoke opt using `  opt
-mymodulepass0 -myfunctionpass -mymodulepass1`.

The terminology is a bit confusing because "Function Pass Manager" can
also correspond to "class FunctionPassManager".
Indeed when you're invoking `opt -mymodulepass0 -myfunctionpass
-mymodulepass1`, here is what happens:
- a PassManager is created
- mymodulepass0 is inserted in the PassManager
- when myfunctionpass is added to the same PassManager, because it is a
FunctionPass, there is an implicit FunctionPassManager which is created to wrap
it (see FunctionPass::assignPassManager()). The FunctionPassManager offers a
"ModulePass" like interface and can be added in the same PassManager
as mymodulepass0.
- mymodulepass1 is inserted in the PassManager. If you added another function
pass before, it would be in the same FunctionPassManager (there is some magic to
find the most nested PassManager that can accommodate the pass you are adding,
see FunctionPass::assignPassManager() again).

-debug-pass=Structure helps to represent the pass manager nesting:


$ echo "" | opt -globalopt -instcombine -reassociate -globalopt
-debug-pass=Structure -o /dev/null
Pass Arguments:  -targetlibinfo -assumption-cache-tracker -globalopt -domtree
-instcombine -reassociate -globalopt -verify -verify-di
Target Library Information
Assumption Cache Tracker
  ModulePass Manager
    Global Variable Optimizer
    FunctionPass Manager
      Dominator Tree Construction
      Combine redundant instructions
      Reassociate expressions
    Global Variable Optimizer
    FunctionPass Manager
      Module Verifier
    Debug Info Verifier
    Bitcode Writer


You can see that "Combine redundant instructions" (-instcombine) and
"Reassociate expressions" (-reassociate) are handled by a
"FunctionPass Manager" nested in a "ModulePass Manager". You
can also see that "Global Variable Optimizer" (-globalopt) runs before
and after the two function passes.

Cheers,

Mehdi

On Tue, May 10, 2016 at 08:35:05AM -0700, Mehdi Amini
wrote:
> 
> > On May 9, 2016, at 10:16 PM, serge guelton <sguelton at
quarkslab.com> wrote:
> > 
> >>>> You can look at AddOptimizationPasses() in opt.cpp.
> >>> 
> >>> As far as I understand, the two passmanager do not interleave
their
> >>> passes. It first runs all the function passes and below. Then
all the
> >>> module passes. So if you specify:
> >>> 
> >>>   opt -mymodulepass0 -myfunctionpass -mymodulepass1
> >>> 
> >>> What you actually get is:
> >>> 
> >>> 1. myfunctionpass on each function
> >>> 2. mymodulepass0 
> >>> 3. mymodulepass0 
> >> 
> >> (I assume your 3 was intended to be mymodulepass1 right?)
> > (yes)
> >> So AFAIK no, you should get the order you specified on the command
line, i.e.
> >> 
> >> 1. mymodulepass0 
> >> 2. myfunctionpass on each function
> >> 3. mymodulepass1
> > 
> > MMMh, from opt.cpp, there's a first call to:
> > 
> >  if (OptLevelO1 || OptLevelO2 || OptLevelOs || OptLevelOz ||
OptLevelO3) {
> >    FPasses->doInitialization();
> >    for (Function &F : *M)
> >      FPasses->run(F);
> >    FPasses->doFinalization();
> >  }
> > 
> > then a few lines later, a call to:
> > 
> >    Passes.run(*M);
> > 
> > where Passes is the Module pass Manager and FPasses is the Function
Pass
> > Manager. Each is filled in AddOptimizationPasses with different
passes.
> > I don't see the point where the two manage interleave their
passes.
> 
> It seems like we need to clarify that we're talking about the same
thing: i.e. opt -O3 or opt -mypass?
> 
> The code you show correspond to the two separated pass managers I was
mentioning in my first answer about how to read the output of
-debug-pass=Arguments for opt -O3. As you can see in the if condition in the
snippet you post, it is only used when a -O is passed to opt, and not when you
invoke opt using `  opt -mymodulepass0 -myfunctionpass -mymodulepass1`.
> 
> The terminology is a bit confusing because "Function Pass
Manager" can also correspond to "class FunctionPassManager".
> Indeed when you're invoking `opt -mymodulepass0 -myfunctionpass
-mymodulepass1`, here is what happens:
> - a PassManager is created
> - mymodulepass0 is inserted in the PassManager
> - when myfunctionpass is added to the same PassManager, because it is a
FunctionPass, there is an implicit FunctionPassManager which is created to wrap
it (see FunctionPass::assignPassManager()). The FunctionPassManager offers a
"ModulePass" like interface and can be added in the same PassManager
as mymodulepass0.
> - mymodulepass1 is inserted in the PassManager. If you added another
function pass before, it would be in the same FunctionPassManager (there is some
magic to find the most nested PassManager that can accommodate the pass you are
adding, see FunctionPass::assignPassManager() again).
> 
> -debug-pass=Structure helps to represent the pass manager nesting:
> 
> 
> $ echo "" | opt -globalopt -instcombine -reassociate -globalopt
-debug-pass=Structure -o /dev/null
> Pass Arguments:  -targetlibinfo -assumption-cache-tracker -globalopt
-domtree -instcombine -reassociate -globalopt -verify -verify-di
> Target Library Information
> Assumption Cache Tracker
>   ModulePass Manager
>     Global Variable Optimizer
>     FunctionPass Manager
>       Dominator Tree Construction
>       Combine redundant instructions
>       Reassociate expressions
>     Global Variable Optimizer
>     FunctionPass Manager
>       Module Verifier
>     Debug Info Verifier
>     Bitcode Writer
> 
> 
> You can see that "Combine redundant instructions" (-instcombine)
and "Reassociate expressions" (-reassociate) are handled by a
"FunctionPass Manager" nested in a "ModulePass Manager". You
can also see that "Global Variable Optimizer" (-globalopt) runs before
and after the two function passes.
That's a perfect clarification, thanks a lot o/