llvm dev - Apr 2021 - new pass manager version of opt vs. legacy pass manager version

There certainly have been cases where we've accepted patches to
maintain non-default configurations passing tests (for instance I
think Sony changed the language version default of their clang fork
compared to upstream - and contributed a bunch of patches to clang
tests so they pass no matter which language version is the default (in
cases where it wasn't material/significant to the test in question))

So I could imagine it might be reasonable to accept patches that
update tests that aren't intended to test one pass manager or another
(but became over-constrained to the default NPM behavior) so they pass
with either enabled (if this is important to you, though, I'd strongly
advise setting up a buildbot to keep track of violations of this
invariant (though if it's a really uncommon config, which it seems it
is, probably best to not send fail-email to commiters, instead only to
your team who can triage and then submit a patch (directly or for
review, as needed) to generalize the test so it passes in all configs
(which, in some cases, might mean adding an explicit "use the new pass
manager" flag to the test))

I don't actually know if there's a CMake config to change the default
here - if there is, then it seems pretty reasonable to me to support
building/running the tests (& expecting them to pass) in that
configuration. If there isn't, then it's a bit more debateable.

On Thu, Apr 22, 2021 at 8:33 AM Snider, Todd via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hello All,
>
>
>
> My development group has been maintaining a downstream version of the
monorepo that stays in sync with the upstream “main” branch, but we are still
using the legacy pass manager in our local copy of the monorepo.
>
>
>
> We’ve recently encountered a few instances of lit tests that are failing
when run with the legacy pass manager version of opt, but pass when run with the
new pass manager version of opt.
>
>
>
> The situation raises a couple of questions:
>
> Is the legacy pass manager behavior being adequately tested by the
buildbots?
> What expectation should there be that legacy pass manager behavior will be
maintained in light of changes made to code that affects both the new pass
manager version and the legacy pass manager version of opt?
>
>
>
> I suspect that my group is not the only ones trying to stay in sync with
the upstream LLVM main branch and keep using the legacy pass manager, and I
anticipate the only long-term remedy for our situation is to move to using the
new pass manager as soon as we can.
>
>
>
> Thoughts?
>
>
>
> Regards.
>
>
>
> Todd Snider
>
> Texas Instruments Incorporated
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 4/22/21 11:57 AM, David Blaikie via llvm-dev wrote:
> There certainly have been cases where we've accepted patches to
> maintain non-default configurations passing tests (for instance I
> think Sony changed the language version default of their clang fork
> compared to upstream - and contributed a bunch of patches to clang
> tests so they pass no matter which language version is the default (in
> cases where it wasn't material/significant to the test in question))
>
> So I could imagine it might be reasonable to accept patches that
> update tests that aren't intended to test one pass manager or another
> (but became over-constrained to the default NPM behavior) so they pass
> with either enabled (if this is important to you, though, I'd strongly
> advise setting up a buildbot to keep track of violations of this
> invariant (though if it's a really uncommon config, which it seems it
> is, probably best to not send fail-email to commiters, instead only to
> your team who can triage and then submit a patch (directly or for
> review, as needed) to generalize the test so it passes in all configs
> (which, in some cases, might mean adding an explicit "use the new pass
> manager" flag to the test))
>
> I don't actually know if there's a CMake config to change the
default
> here - if there is, then it seems pretty reasonable to me to support
> building/running the tests (& expecting them to pass) in that
> configuration. If there isn't, then it's a bit more debateable.
I'd hope we move towards removing the old-PM completely in upstream.
Thus, I don't really think we should require all tests to pass with
the old-PM, e.g., new passes should be allowed that run only with the
new-PM. If this is a reasonable way forward, let's add an lit XFAIL
for the old-PM and allow such XFAILs to be upstreamed for easier
maintenance.

~ Johannes

> On Thu, Apr 22, 2021 at 8:33 AM Snider, Todd via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> Hello All,
>>
>>
>>
>> My development group has been maintaining a downstream version of the
monorepo that stays in sync with the upstream “main” branch, but we are still
using the legacy pass manager in our local copy of the monorepo.
>>
>>
>>
>> We’ve recently encountered a few instances of lit tests that are
failing when run with the legacy pass manager version of opt, but pass when run
with the new pass manager version of opt.
>>
>>
>>
>> The situation raises a couple of questions:
>>
>> Is the legacy pass manager behavior being adequately tested by the
buildbots?
>> What expectation should there be that legacy pass manager behavior will
be maintained in light of changes made to code that affects both the new pass
manager version and the legacy pass manager version of opt?
>>
>>
>>
>> I suspect that my group is not the only ones trying to stay in sync
with the upstream LLVM main branch and keep using the legacy pass manager, and I
anticipate the only long-term remedy for our situation is to move to using the
new pass manager as soon as we can.
>>
>>
>>
>> Thoughts?
>>
>>
>>
>> Regards.
>>
>>
>>
>> Todd Snider
>>
>> Texas Instruments Incorporated
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev