llvm dev - Oct 2017 - New Pass Manager with flto[=thin] not enabled (??)

Hello,

I recently noticed that the new pass manager was not enabled at
regular/thin LTO link step even if '-fexperimental-new-pass-manager' was
specified in the compile step and link step commands.  Upon closer
inspection, it seems there's so real way to invoke the new pass manager
path ('runNewPMPasses' in lib/LTO/LTOBackend.cpp) during link step.

'Conf.UseNewPM' is used to select the new PM in LTOBackend.cpp, but only
two places in the code sets it.  The first is through the 'llvm-lto2'
tool,
and the second during the compile step with
'-fthinlto-index=<value>'.
Neither are invoked during LTO/thinLTO link.

tools/llvm-lto2/llvm-lto2.cpp
...
static cl::opt<bool>
    UseNewPM("use-new-pm",
             cl::desc("Run LTO passes using the new pass manager"),
             cl::init(false), cl::Hidden);
...
  Conf.UseNewPM = UseNewPM;
...

projects/clang/lib/CodeGen/BackendUtil.cpp
...
void clang::EmitBackendOutput( (...) {
...
      runThinLTOBackend(CombinedIndex.get(), M, HeaderOpts, CGOpts, TOpts,
                        LOpts, std::move(OS), CGOpts.SampleProfileFile,
Action);
      return;
...
}

static void runThinLTOBackend (...) {
...
  Conf.UseNewPM = CGOpts.ExperimentalNewPassManager;
...
}

My team and I were wondering if this is:
      intentional,
      if so, why do we not have an option to enable the new PM in the link
      step
      if not, should we pass an option via '-plugin-opt', or via some
other
      mechanism?  It's unclear to me what the preferred method of passing
      configuration options to LTO/thinLTO link step is.

Any insights/thoughts/comments would be greatly appreciated.

Cheers,

Graham Yiu
LLVM Compiler Development
IBM Toronto Software Lab
Office: (905) 413-4077      C2-707/8200/Markham
Email: gyiu at ca.ibm.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20171003/4c253cf7/attachment.html>

On Tue, Oct 3, 2017 at 11:57 AM, Graham Yiu via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> I recently noticed that the new pass manager was not enabled at
regular/thin
> LTO link step even if '-fexperimental-new-pass-manager' was
specified in the
> compile step and link step commands. Upon closer inspection, it seems
> there's so real way to invoke the new pass manager path
('runNewPMPasses' in
> lib/LTO/LTOBackend.cpp) during link step.
>
`-fexperimental-new-pass-manager` is a compile time flag and has no
effect on the link step IIRC.
You may want to teach the driver to pass the correct flag to the LTO
plugin in gold or to lld.

--
Davide

On Tue, Oct 3, 2017 at 12:08 PM, Davide Italiano <davide at freebsd.org>
wrote:
> On Tue, Oct 3, 2017 at 11:57 AM, Graham Yiu via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> Hello,
>>
>> I recently noticed that the new pass manager was not enabled at
regular/thin
>> LTO link step even if '-fexperimental-new-pass-manager' was
specified in the
>> compile step and link step commands. Upon closer inspection, it seems
>> there's so real way to invoke the new pass manager path
('runNewPMPasses' in
>> lib/LTO/LTOBackend.cpp) during link step.
>>
>
> `-fexperimental-new-pass-manager` is a compile time flag and has no
> effect on the link step IIRC.
> You may want to teach the driver to pass the correct flag to the LTO
> plugin in gold or to lld.
>
> --
> Davide
For lld in particular, currently there's only the ability to specifiy
a custom pass pipeline when passing `--lto-newpm-passes` to the
linker. This is because, historically, when I added this flag, the
pass manager work was still in progress and the `-O2/LTO` pipeline
wasn't fleshed entirely (missing passes & functionality).
You can probably add the option to lld now, if you can't/don't know
how to do it, I may take a look this weekend.

Thanks,

-- 
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare