Björn Pettersson A via llvm-dev
2021-Feb-04 17:34 UTC
[llvm-dev] [RFC] Changing the default pass manager for the optimization pipeline
Hi! Are there any existing plans for making the help text for opt more easy to understand? Here are some things that have been bugging me so far: Currently it for example doesn’t mention that –O0, –O1, etc are designed for legacy-pm (with new-pm it is a bit of a mess since you get the wrong aa-pipeline when for example using –O3). So are those options supposed to be removed, or should they remain as a short-form for “-passes=default<O3> -aa-pipeline=default” etc? opt -help-hidden lists all the legacy passes (under “Optimizations available:”), but those now require -enable-new-pm=0. Maybe that is obvious. But I’m missing a similar list mentioning all things that are legal to put in the -passes string. And options like -debug-pass-manager and -debug-pass=Arguments only work depending on which PM that is used, but that is not mentioned in the help text so one need to figure it out by trial and error. Although most of the time you don’t get an error, you just don’t get the requested functionality. So it would be helpful if the help text mention in what situation those options can be used or not (or there should be errors if using them in the wrong context). Downstream we implement fuzzy testing by using “opt -O3 -debug-pass=Arguments” in order to get a list of passes that are used randomly on the opt command line. I haven’t figured out yet how to implement something similar with the new pm. Is there a way to make opt output the pass names available (that can be used in -passes string)? Regards, Björn From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Arthur Eubanks via llvm-dev Sent: den 4 februari 2021 01:59 To: llvm-dev <llvm-dev at lists.llvm.org> Subject: Re: [llvm-dev] [RFC] Changing the default pass manager for the optimization pipeline This has been submitted as https://reviews.llvm.org/D95380<https://protect2.fireeye.com/v1/url?k=b6307ee8-e9ab47a5-b6303e73-86b568293eb5-9d63c8955f4add4e&q=1&e=91e53f19-023f-4266-a55b-dee7b016e1c7&u=https%3A%2F%2Freviews.llvm.org%2FD95380>. Please file bugs for any regressions. On Tue, Feb 2, 2021 at 12:25 PM Arthur Eubanks <aeubanks at google.com<mailto:aeubanks at google.com>> wrote: There are a couple of failures that I hadn't noticed showing up in the presubmit, as well as some internally reported performance regressions due to NPM-related changes, so this will likely get pushed back. On Wed, Jan 27, 2021 at 9:03 AM Philip Reames <listmail at philipreames.com<mailto:listmail at philipreames.com>> wrote: +1 to the strategy and timeline. This has been a long time in the works and I'm thrilled to see us approaching this major milestone. minor comment inline below Philip On 1/26/21 9:17 AM, Arthur Eubanks via llvm-dev wrote: Hi all, We've been fixing the various remaining issues in order to turn on the new pass manager for the optimization pipeline, and it's about time to turn it on. (Thanks to everyone who has helped with testing and fixing the new pass manager!) https://reviews.llvm.org/D95380<https://protect2.fireeye.com/v1/url?k=af9e01e8-f00538a5-af9e4173-86b568293eb5-c296f480e2a068bb&q=1&e=91e53f19-023f-4266-a55b-dee7b016e1c7&u=https%3A%2F%2Freviews.llvm.org%2FD95380> is the change that would happen, which sets the CMake flag -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON by default. This affects anything that uses the LLVM_ENABLE_NEW_PASS_MANAGER macro, which includes opt's handling of the `opt -instcombine` syntax, clang, and ThinLTO in lld drivers. This does not affect the backend target-specific codegen pipeline since that's mostly not been ported to use the new PM infrastructure yet. Here<https://protect2.fireeye.com/v1/url?k=79624a01-26f9734c-79620a9a-86b568293eb5-ae133b8bb6f72d04&q=1&e=91e53f19-023f-4266-a55b-dee7b016e1c7&u=https%3A%2F%2Fbugs.llvm.org%2Fshow_bug.cgi%3Fid%3D46649> is the umbrella bug for turning on the new PM with blockers. The main one is loop unswitching on divergent loop conditions is unsafe<https://protect2.fireeye.com/v1/url?k=390fa701-66949e4c-390fe79a-86b568293eb5-9ec901df3f64c78b&q=1&e=91e53f19-023f-4266-a55b-dee7b016e1c7&u=https%3A%2F%2Fbugs.llvm.org%2Fshow_bug.cgi%3Fid%3D48819>, which is being looked into. There's also the LLVM C API and bugpoint that still use the legacy PM, which can be ported later on and only block the removal of the legacy PM. The C API can be worked through (we may need to introduce replacements to the legacy pass manager APIs), but bugpoint will be tricky since it has so many legacy PM-specific hacks and we may need to trim it down if we want it to work with the new PM. Anyway, I don't think any of the remaining blockers are large enough to block the switch (but comments welcome). I see no problem with having these two remain on the legacy pass manager for the moment. I do think we should expose a new C API for the NewPM and not try to shove the new one into the same API as the old one, but that's a weakly held opinion and easily discussed later. I'd like to turn on the new PM by default soonish, after the 12.x branch. Perhaps roughly a week from now barring any major newly discovered regressions? As for potential issues only uncovered after the switch, if there is a large issue I will roll it back, but for smaller issues I'd rather ask users to pin to the legacy PM while we fix the issues, either via the CMake flag -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=OFF, or the corresponding compiler flags, like -flegacy-pass-manager for clang. Any concerns/comments? _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://protect2.fireeye.com/v1/url?k=92d95a97-cd4263da-92d91a0c-86b568293eb5-1b0269a2781fa1d9&q=1&e=91e53f19-023f-4266-a55b-dee7b016e1c7&u=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fllvm-dev> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210204/13e9a930/attachment.html>
Arthur Eubanks via llvm-dev
2021-Feb-04 19:36 UTC
[llvm-dev] [RFC] Changing the default pass manager for the optimization pipeline
On Thu, Feb 4, 2021 at 9:34 AM Björn Pettersson A < bjorn.a.pettersson at ericsson.com> wrote:> Hi! > > > > Are there any existing plans for making the help text for opt more easy to > understand? > > Here are some things that have been bugging me so far: > > > > Currently it for example doesn’t mention that –O0, –O1, etc are designed > for legacy-pm (with new-pm it is a bit of a mess since you get the wrong > aa-pipeline when for example using –O3). > > So are those options supposed to be removed, or should they remain as a > short-form for “-passes=default<O3> -aa-pipeline=default” etc? >I'd like to remove these at some point and just go with -passes='default<O#>', but they'll likely remain until we decide to start removing the legacy PM and we start cleaning up tests to only use -passes=. I'll update the description for these flags.> > > opt -help-hidden lists all the legacy passes (under “Optimizations > available:”), but those now require -enable-new-pm=0. Maybe that is obvious. > > But I’m missing a similar list mentioning all things that are legal to put > in the -passes string. >The legacy PM registered passes to a global variable which made this easier to do in the legacy PM. The new PM doesn't have a global registry, but rather each PassBuilder is initialized with the list of passes on its own. I'll draft up something to try to get this working.> > > And options like -debug-pass-manager and -debug-pass=Arguments only work > depending on which PM that is used, but that is not mentioned in the help > text so one need to figure it out by trial and error. Although most of the > time you don’t get an error, you just don’t get the requested > functionality. So it would be helpful if the help text mention in what > situation those options can be used or not (or there should be errors if > using them in the wrong context). >I'll update the help text. Each pass manager's debug logging is separate, rather than being tied to a global variable like the legacy PM's -debug-pass, so it's a bit weird to check in each new PM PassManager if -debug-pass has been specified.> > > Downstream we implement fuzzy testing by using “opt -O3 > -debug-pass=Arguments” in order to get a list of passes that are used > randomly on the opt command line. > > I haven’t figured out yet how to implement something similar with the new > pm. Is there a way to make opt output the pass names available (that can be > used in -passes string)? >There was some discussion about similar things in the bugpoint + new PM thread. The way the new PM is designed doesn't really lend itself to this sort of functionality. It could be possible for each type of PassManager to print out the names of all the passes it contains, but we'd need some change with the way we handle pass names (e.g. remove PassInfoMixin and have each pass declare its own name). Rather, I think something like the PassInstrumentation approach that -opt-bisect-limit does is better, where you can turn off a specific set of passes. But I could be convinced of a different approach.> > > Regards, > > Björn >Thanks for the questions, these are good for making things clearer to users. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210204/48583222/attachment.html>