llvm dev - Feb 2021 - bugpoint and the new pass manager

On 1/27/21 1:58 AM, Florian Hahn via llvm-dev wrote:
>
>> On Jan 26, 2021, at 22:32, Fangrui Song <maskray at google.com>
wrote:
>>
>> Forked the thread and changed the subject to talk about bugpoint
specifically.
>> CCed folks on https://bugs.llvm.org/show_bug.cgi?id=48813
> Thanks!
>
>> On 2021-01-26, Arthur Eubanks via llvm-dev wrote:
>>> Hi all,
>>>
>>> We've been fixing the various remaining issues in order to turn
on the new
>>> pass manager for the optimization pipeline, and it's about time
to turn it
>>> on. (Thanks to everyone who has helped with testing and fixing the
new pass
>>> manager!)
>>>
>>> https://reviews.llvm.org/D95380 is the change that would happen,
which sets
>>> the CMake flag -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON by
default. This
>>> affects anything that uses the LLVM_ENABLE_NEW_PASS_MANAGER macro,
which
>>> includes opt's handling of the `opt -instcombine` syntax,
clang, and
>>> ThinLTO in lld drivers. This does not affect the backend
target-specific
>>> codegen pipeline since that's mostly not been ported to use the
new PM
>>> infrastructure yet.
>>>
>>> Here <https://bugs.llvm.org/show_bug.cgi?id=46649> is the
umbrella bug for
>>> turning on the new PM with blockers. The main one is loop
unswitching on
>>> divergent loop conditions is unsafe
>>> <https://bugs.llvm.org/show_bug.cgi?id=48819>, which is being
looked into.
>>> There's also the LLVM C API and bugpoint that still use the
legacy PM,
>>> which can be ported later on and only block the removal of the
legacy PM.
>> I have heard several users said they just use
>>
>>   bugpoint -compile-custom -compile-command=./run a.ll
>>   (this usage is similar to llvm-reduce)
>>
>> but not any other command listed on https://llvm.org/docs/Bugpoint.html
.
>> (I am among them. https://llvm.org/docs/Bugpoint.html and
>> https://llvm.org/docs/CommandGuide/bugpoint.html
>> have no usage examples so I resorted to an external article
>> http://logan.tw/posts/2014/11/26/llvm-bugpoint/ )
> FWIW I often use bugpoint’s crashing pass reduction feature ( e.g.
`bugpoint -O3 foo.bc` to reduce the sequence of passes exposing a crash) when
investigating crashes in LLVM optimizations. This is unfortunately one of the
features directly tied to the pass manager.
Same.
> My main concern is that `opt` and `bugpoint` having different defaults
seems to have potential to be quite confusing to user  (e.g. `opt
-passes=default<O3>` crashes, but `bug point -passes=default<O3>`
does not, `bug point -O3 ` works, but does not reproduce the failure, e.g. due
to difference in the pipeline).
>
> I don’t know how widely used bugpoint is, but it features prominently in
the documentation, e.g. https://llvm.org/docs/HowToSubmitABug.html , which is
linked from the getting started page and the documentation sidebar.
>
> I think before switching, it would be good to at least update
HowToSubmitABug. We should probably also add a note to Bugpoint’s docs and refer
the the documentation for llvm-reduce. It is great if people take time to reduce
the test cases for their bug reports and we should try to make the experience as
smooth as possible
Florian raises a great point here.  I'd be comfortable simply 
documenting the issue for the moment, but getting bugpoint properly 
ported to the new PM close to the point we switch seems very worthwhile 
for the reason Florian nicely describes.

https://reviews.llvm.org/D95578 for updating HowToSubmitABug with
alternatives to bugpoint (and just modernizing it in general).

https://reviews.llvm.org/D86657 (not yet submitted) will dump the IR on an
LLVM crash, which would be useful. Extending that to dumping the IR to a
file would be even better.

On Wed, Jan 27, 2021 at 9:06 AM Philip Reames <listmail at
philipreames.com>
wrote:
>
> On 1/27/21 1:58 AM, Florian Hahn via llvm-dev wrote:
> >
> >> On Jan 26, 2021, at 22:32, Fangrui Song <maskray at
google.com> wrote:
> >>
> >> Forked the thread and changed the subject to talk about bugpoint
> specifically.
> >> CCed folks on https://bugs.llvm.org/show_bug.cgi?id=48813
> > Thanks!
> >
> >> On 2021-01-26, Arthur Eubanks via llvm-dev wrote:
> >>> Hi all,
> >>>
> >>> We've been fixing the various remaining issues in order to
turn on the
> new
> >>> pass manager for the optimization pipeline, and it's about
time to
> turn it
> >>> on. (Thanks to everyone who has helped with testing and fixing
the new
> pass
> >>> manager!)
> >>>
> >>> https://reviews.llvm.org/D95380 is the change that would
happen,
> which sets
> >>> the CMake flag -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON by
default.
> This
> >>> affects anything that uses the LLVM_ENABLE_NEW_PASS_MANAGER
macro,
> which
> >>> includes opt's handling of the `opt -instcombine` syntax,
clang, and
> >>> ThinLTO in lld drivers. This does not affect the backend
> target-specific
> >>> codegen pipeline since that's mostly not been ported to
use the new PM
> >>> infrastructure yet.
> >>>
> >>> Here <https://bugs.llvm.org/show_bug.cgi?id=46649> is
the umbrella
> bug for
> >>> turning on the new PM with blockers. The main one is loop
unswitching
> on
> >>> divergent loop conditions is unsafe
> >>> <https://bugs.llvm.org/show_bug.cgi?id=48819>, which is
being looked
> into.
> >>> There's also the LLVM C API and bugpoint that still use
the legacy PM,
> >>> which can be ported later on and only block the removal of the
legacy
> PM.
> >> I have heard several users said they just use
> >>
> >>   bugpoint -compile-custom -compile-command=./run a.ll
> >>   (this usage is similar to llvm-reduce)
> >>
> >> but not any other command listed on
https://llvm.org/docs/Bugpoint.html
> .
> >> (I am among them. https://llvm.org/docs/Bugpoint.html and
> >> https://llvm.org/docs/CommandGuide/bugpoint.html
> >> have no usage examples so I resorted to an external article
> >> http://logan.tw/posts/2014/11/26/llvm-bugpoint/ )
> > FWIW I often use bugpoint’s crashing pass reduction feature ( e.g.
> `bugpoint -O3 foo.bc` to reduce the sequence of passes exposing a crash)
> when investigating crashes in LLVM optimizations. This is unfortunately one
> of the features directly tied to the pass manager.
> Same.
> > My main concern is that `opt` and `bugpoint` having different defaults
> seems to have potential to be quite confusing to user  (e.g. `opt
> -passes=default<O3>` crashes, but `bug point
-passes=default<O3>` does not,
> `bug point -O3 ` works, but does not reproduce the failure, e.g. due to
> difference in the pipeline).
> >
> > I don’t know how widely used bugpoint is, but it features prominently
in
> the documentation, e.g. https://llvm.org/docs/HowToSubmitABug.html ,
> which is linked from the getting started page and the documentation
sidebar.
> >
> > I think before switching, it would be good to at least update
> HowToSubmitABug. We should probably also add a note to Bugpoint’s docs and
> refer the the documentation for llvm-reduce. It is great if people take
> time to reduce the test cases for their bug reports and we should try to
> make the experience as smooth as possible
> Florian raises a great point here.  I'd be comfortable simply
> documenting the issue for the moment, but getting bugpoint properly
> ported to the new PM close to the point we switch seems very worthwhile
> for the reason Florian nicely describes.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210127/6079581c/attachment.html>

On 2021-01-27, Arthur Eubanks wrote:
>https://reviews.llvm.org/D95578 for updating HowToSubmitABug with
>alternatives to bugpoint (and just modernizing it in general).
>
>https://reviews.llvm.org/D86657 (not yet submitted) will dump the IR on an
>LLVM crash, which would be useful. Extending that to dumping the IR to a
>file would be even better.
>
>On Wed, Jan 27, 2021 at 9:06 AM Philip Reames <listmail at
philipreames.com>
>wrote:
Thanks to Authur for https://reviews.llvm.org/D95578 (improve HowToSubmitABug).
If anyone knows how to improve the bugpoint part or renovate the following
bugpoint documentation:

llvm/llvm/docs/Bugpoint.rst
llvm/llvm/docs/BugpointRedesign.rst
llvm/llvm/docs/CommandGuide/bugpoint.rst

That will be very nice.
>>
>> On 1/27/21 1:58 AM, Florian Hahn via llvm-dev wrote:
>> >
>> >> On Jan 26, 2021, at 22:32, Fangrui Song <maskray at
google.com> wrote:
>> >>
>> >> Forked the thread and changed the subject to talk about
bugpoint
>> specifically.
>> >> CCed folks on https://bugs.llvm.org/show_bug.cgi?id=48813
>> > Thanks!
>> >
>> >> On 2021-01-26, Arthur Eubanks via llvm-dev wrote:
>> >>> Hi all,
>> >>>
>> >>> We've been fixing the various remaining issues in
order to turn on the
>> new
>> >>> pass manager for the optimization pipeline, and it's
about time to
>> turn it
>> >>> on. (Thanks to everyone who has helped with testing and
fixing the new
>> pass
>> >>> manager!)
>> >>>
>> >>> https://reviews.llvm.org/D95380 is the change that would
happen,
>> which sets
>> >>> the CMake flag -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON
by default.
>> This
>> >>> affects anything that uses the
LLVM_ENABLE_NEW_PASS_MANAGER macro,
>> which
>> >>> includes opt's handling of the `opt -instcombine`
syntax, clang, and
>> >>> ThinLTO in lld drivers. This does not affect the backend
>> target-specific
>> >>> codegen pipeline since that's mostly not been ported
to use the new PM
>> >>> infrastructure yet.
>> >>>
>> >>> Here <https://bugs.llvm.org/show_bug.cgi?id=46649>
is the umbrella
>> bug for
>> >>> turning on the new PM with blockers. The main one is loop
unswitching
>> on
>> >>> divergent loop conditions is unsafe
>> >>> <https://bugs.llvm.org/show_bug.cgi?id=48819>, which
is being looked
>> into.
>> >>> There's also the LLVM C API and bugpoint that still
use the legacy PM,
>> >>> which can be ported later on and only block the removal of
the legacy
>> PM.
>> >> I have heard several users said they just use
>> >>
>> >>   bugpoint -compile-custom -compile-command=./run a.ll
>> >>   (this usage is similar to llvm-reduce)
>> >>
>> >> but not any other command listed on
https://llvm.org/docs/Bugpoint.html
>> .
>> >> (I am among them. https://llvm.org/docs/Bugpoint.html and
>> >> https://llvm.org/docs/CommandGuide/bugpoint.html
>> >> have no usage examples so I resorted to an external article
>> >> http://logan.tw/posts/2014/11/26/llvm-bugpoint/ )
>> > FWIW I often use bugpoint’s crashing pass reduction feature ( e.g.
>> `bugpoint -O3 foo.bc` to reduce the sequence of passes exposing a
crash)
>> when investigating crashes in LLVM optimizations. This is unfortunately
one
>> of the features directly tied to the pass manager.
>> Same.
>> > My main concern is that `opt` and `bugpoint` having different
defaults
>> seems to have potential to be quite confusing to user  (e.g. `opt
>> -passes=default<O3>` crashes, but `bug point
-passes=default<O3>` does not,
>> `bug point -O3 ` works, but does not reproduce the failure, e.g. due to
>> difference in the pipeline).
>> >
>> > I don’t know how widely used bugpoint is, but it features
prominently in
>> the documentation, e.g. https://llvm.org/docs/HowToSubmitABug.html ,
>> which is linked from the getting started page and the documentation
sidebar.
>> >
>> > I think before switching, it would be good to at least update
>> HowToSubmitABug. We should probably also add a note to Bugpoint’s docs
and
>> refer the the documentation for llvm-reduce. It is great if people take
>> time to reduce the test cases for their bug reports and we should try
to
>> make the experience as smooth as possible
>> Florian raises a great point here.  I'd be comfortable simply
>> documenting the issue for the moment, but getting bugpoint properly
>> ported to the new PM close to the point we switch seems very worthwhile
>> for the reason Florian nicely describes.
>>