llvm dev - Aug 2018 - bugpoint --tool-args and --safe-tool-args

This is what works for me, when I want to reduce an llc failure triggering a
failure with a "foo bar baz" in the assert message (the grep for the
error message avoids reducing to an unrelated failure):

$ cat run_llc.sh
#!/bin/sh
if ! $HOME/dev/llvm/build/bin/llc "$@" 2>&1 | grep "foo
bar baz"; then
	exit 0
else
	exit $?
fi

$ PATH=$HOME/path/to/my/llvm/build/bin:$PATH bugpoint -compile-custom
-compile-command ./run_llc.sh test.ll


- Matthias
> On Jul 31, 2018, at 11:03 AM, Friedman, Eli via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On 7/31/2018 10:10 AM, David Greene via llvm-dev wrote:
>> I have a failing test and bugpoint would be the perfect tool to help
>> narrow it down.
>> 
>> llc is the failing tool.  It fails with one set of options and passes
>> with another.  I was hoping to use bugpoint like this:
>> 
>> bugpoint -safe-llc -run-llc <testcase> -tool-args <failing
args> -safe-tool-args <passing args>
>> 
>> Unfortunately, this doesn't seem to be possible.  According to the
>> documentation both -tool-args and -safe-tool-args consume all remaining
>> arguments so bugpoint would never see -safe-tool-args above.
>> 
>> I tried it and indeed it looks like -safe-tool-args never makes it to
>> llc.
>> 
>> For now I can build another llc which makes <failing args>
default and
>> then just use -safe-tool-args above but it's a bit painful.
>> 
>> Has anyone tried to use bugpoint like this before?  Any tips?
> 
> I've occasionally used shell script wrappers when I couldn't get
bugpoint command-line parsing to work the way I wanted it to.
> 
> Looking at CommandLine.rst, PositionalEatsArgs is supposed to allow
specifying something like the command you wrote, but I'm not sure it ever
worked correctly; nothing other than bugpoint uses it, as far as I can tell.
> 
> -Eli
> 
> -- 
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux
Foundation Collaborative Project
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180731/8c9069b5/attachment.html>

> On Jul 31, 2018, at 11:11 AM, Matthias Braun <mbraun at apple.com>
wrote:
> 
> This is what works for me, when I want to reduce an llc failure triggering
a failure with a "foo bar baz" in the assert message (the grep for the
error message avoids reducing to an unrelated failure):
> 
> $ cat run_llc.sh
> #!/bin/sh
> if ! $HOME/dev/llvm/build/bin/llc "$@" 2>&1 | grep
"foo bar baz"; then
> 	exit 0
> else
> 	exit $?
> fi
> 
> $ PATH=$HOME/path/to/my/llvm/build/bin:$PATH bugpoint -compile-custom
-compile-command ./run_llc.sh test.ll
Also:
- Make sure the script is executable `chmod +x run_llc.sh`.
- Try first that your run script behaves as expected. If you see bugpoint
running for a long time without making real progress it's likely that the
returncodes of your script aren't as you expect.

> 
> 
> - Matthias
> 
>> On Jul 31, 2018, at 11:03 AM, Friedman, Eli via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> On 7/31/2018 10:10 AM, David Greene via llvm-dev wrote:
>>> I have a failing test and bugpoint would be the perfect tool to
help
>>> narrow it down.
>>> 
>>> llc is the failing tool.  It fails with one set of options and
passes
>>> with another.  I was hoping to use bugpoint like this:
>>> 
>>> bugpoint -safe-llc -run-llc <testcase> -tool-args <failing
args> -safe-tool-args <passing args>
>>> 
>>> Unfortunately, this doesn't seem to be possible.  According to
the
>>> documentation both -tool-args and -safe-tool-args consume all
remaining
>>> arguments so bugpoint would never see -safe-tool-args above.
>>> 
>>> I tried it and indeed it looks like -safe-tool-args never makes it
to
>>> llc.
>>> 
>>> For now I can build another llc which makes <failing args>
default and
>>> then just use -safe-tool-args above but it's a bit painful.
>>> 
>>> Has anyone tried to use bugpoint like this before?  Any tips?
>> 
>> I've occasionally used shell script wrappers when I couldn't
get bugpoint command-line parsing to work the way I wanted it to.
>> 
>> Looking at CommandLine.rst, PositionalEatsArgs is supposed to allow
specifying something like the command you wrote, but I'm not sure it ever
worked correctly; nothing other than bugpoint uses it, as far as I can tell.
>> 
>> -Eli
>> 
>> -- 
>> Employee of Qualcomm Innovation Center, Inc.
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180731/d4a30ce5/attachment.html>

Oh yeah, that makes sense.  Thanks!

                          -David

Matthias Braun via llvm-dev <llvm-dev at lists.llvm.org> writes:
> This is what works for me, when I want to reduce an llc failure
> triggering a failure with a "foo bar baz" in the assert message
(the
> grep for the error message avoids reducing to an unrelated failure):
>
> $ cat run_llc.sh
> #!/bin/sh
> if ! $HOME/dev/llvm/build/bin/llc "$@" 2>&1 | grep
"foo bar baz"; then
> exit 0
> else
> exit $?
> fi
>
> $ PATH=$HOME/path/to/my/llvm/build/bin:$PATH bugpoint -compile-custom -
> compile-command ./run_llc.sh test.ll
>
> - Matthias
>
>     On Jul 31, 2018, at 11:03 AM, Friedman, Eli via llvm-dev
>     <llvm-dev at lists.llvm.org> wrote:
>
>     
>     On 7/31/2018 10:10 AM, David Greene via llvm-dev wrote:
>     
>     I have a failing test and bugpoint would be the perfect tool to
>         help
>         narrow it down.
>         
>         llc is the failing tool. It fails with one set of options and
>         passes
>         with another. I was hoping to use bugpoint like this:
>         
>         bugpoint -safe-llc -run-llc <testcase> -tool-args <failing
>         args> -safe-tool-args <passing args>
>         
>         Unfortunately, this doesn't seem to be possible. According to
>         the
>         documentation both -tool-args and -safe-tool-args consume all
>         remaining
>         arguments so bugpoint would never see -safe-tool-args above.
>         
>         I tried it and indeed it looks like -safe-tool-args never
>         makes it to
>         llc.
>         
>         For now I can build another llc which makes <failing args>
>         default and
>         then just use -safe-tool-args above but it's a bit painful.
>         
>         Has anyone tried to use bugpoint like this before? Any tips?
>
>     I've occasionally used shell script wrappers when I couldn't
get
>     bugpoint command-line parsing to work the way I wanted it to.
>     
>     Looking at CommandLine.rst, PositionalEatsArgs is supposed to
>     allow specifying something like the command you wrote, but I'm not
>     sure it ever worked correctly; nothing other than bugpoint uses
>     it, as far as I can tell.
>     
>     -Eli
>     
>     -- 
>     Employee of Qualcomm Innovation Center, Inc.
>     Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>     a Linux Foundation Collaborative Project
>     
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Ugh.  Scripts won't work either, at least not by default.  The problem
is that I need non-default options for both the safe and unsafe
compiler.

I hacked in an -unsafe-path option to let me point to a script for the
"bad" compiler.  We'll see if that works.  If it does, would it be
a
useful addition?

                           -David

Matthias Braun <mbraun at apple.com> writes:
> This is what works for me, when I want to reduce an llc failure
> triggering a failure with a "foo bar baz" in the assert message
(the
> grep for the error message avoids reducing to an unrelated failure):
>
> $ cat run_llc.sh
> #!/bin/sh
> if ! $HOME/dev/llvm/build/bin/llc "$@" 2>&1 | grep
"foo bar baz"; then
> exit 0
> else
> exit $?
> fi
>
> $ PATH=$HOME/path/to/my/llvm/build/bin:$PATH bugpoint -compile-custom -
> compile-command ./run_llc.sh test.ll
>
> - Matthias
>
>     On Jul 31, 2018, at 11:03 AM, Friedman, Eli via llvm-dev
>     <llvm-dev at lists.llvm.org> wrote:
>
>     
>     On 7/31/2018 10:10 AM, David Greene via llvm-dev wrote:
>     
>     I have a failing test and bugpoint would be the perfect tool to
>         help
>         narrow it down.
>         
>         llc is the failing tool. It fails with one set of options and
>         passes
>         with another. I was hoping to use bugpoint like this:
>         
>         bugpoint -safe-llc -run-llc <testcase> -tool-args <failing
>         args> -safe-tool-args <passing args>
>         
>         Unfortunately, this doesn't seem to be possible. According to
>         the
>         documentation both -tool-args and -safe-tool-args consume all
>         remaining
>         arguments so bugpoint would never see -safe-tool-args above.
>         
>         I tried it and indeed it looks like -safe-tool-args never
>         makes it to
>         llc.
>         
>         For now I can build another llc which makes <failing args>
>         default and
>         then just use -safe-tool-args above but it's a bit painful.
>         
>         Has anyone tried to use bugpoint like this before? Any tips?
>
>     I've occasionally used shell script wrappers when I couldn't
get
>     bugpoint command-line parsing to work the way I wanted it to.
>     
>     Looking at CommandLine.rst, PositionalEatsArgs is supposed to
>     allow specifying something like the command you wrote, but I'm not
>     sure it ever worked correctly; nothing other than bugpoint uses
>     it, as far as I can tell.
>     
>     -Eli
>     
>     -- 
>     Employee of Qualcomm Innovation Center, Inc.
>     Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>     a Linux Foundation Collaborative Project
>     
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Have you read
http://blog.llvm.org/2015/11/reduce-your-testcases-with-bugpoint-and.html?m=1
?

That's how I always use bugpoint and have done things like this. I always
seem to have to brush up my bash skills every time I use it.

On Tue, Jul 31, 2018, 1:47 PM David A. Greene via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Ugh.  Scripts won't work either, at least not by default.  The problem
> is that I need non-default options for both the safe and unsafe
> compiler.
>
> I hacked in an -unsafe-path option to let me point to a script for the
> "bad" compiler.  We'll see if that works.  If it does, would
it be a
> useful addition?
>
>                            -David
>
> Matthias Braun <mbraun at apple.com> writes:
>
> > This is what works for me, when I want to reduce an llc failure
> > triggering a failure with a "foo bar baz" in the assert
message (the
> > grep for the error message avoids reducing to an unrelated failure):
> >
> > $ cat run_llc.sh
> > #!/bin/sh
> > if ! $HOME/dev/llvm/build/bin/llc "$@" 2>&1 | grep
"foo bar baz"; then
> > exit 0
> > else
> > exit $?
> > fi
> >
> > $ PATH=$HOME/path/to/my/llvm/build/bin:$PATH bugpoint -compile-custom
-
> > compile-command ./run_llc.sh test.ll
> >
> > - Matthias
> >
> >     On Jul 31, 2018, at 11:03 AM, Friedman, Eli via llvm-dev
> >     <llvm-dev at lists.llvm.org> wrote:
> >
> >
> >     On 7/31/2018 10:10 AM, David Greene via llvm-dev wrote:
> >
> >     I have a failing test and bugpoint would be the perfect tool to
> >         help
> >         narrow it down.
> >
> >         llc is the failing tool. It fails with one set of options and
> >         passes
> >         with another. I was hoping to use bugpoint like this:
> >
> >         bugpoint -safe-llc -run-llc <testcase> -tool-args
<failing
> >         args> -safe-tool-args <passing args>
> >
> >         Unfortunately, this doesn't seem to be possible. According
to
> >         the
> >         documentation both -tool-args and -safe-tool-args consume all
> >         remaining
> >         arguments so bugpoint would never see -safe-tool-args above.
> >
> >         I tried it and indeed it looks like -safe-tool-args never
> >         makes it to
> >         llc.
> >
> >         For now I can build another llc which makes <failing
args>
> >         default and
> >         then just use -safe-tool-args above but it's a bit
painful.
> >
> >         Has anyone tried to use bugpoint like this before? Any tips?
> >
> >     I've occasionally used shell script wrappers when I
couldn't get
> >     bugpoint command-line parsing to work the way I wanted it to.
> >
> >     Looking at CommandLine.rst, PositionalEatsArgs is supposed to
> >     allow specifying something like the command you wrote, but I'm
not
> >     sure it ever worked correctly; nothing other than bugpoint uses
> >     it, as far as I can tell.
> >
> >     -Eli
> >
> >     --
> >     Employee of Qualcomm Innovation Center, Inc.
> >     Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> >     a Linux Foundation Collaborative Project
> >
> >     _______________________________________________
> >     LLVM Developers mailing list
> >     llvm-dev at lists.llvm.org
> >     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180731/e36eb720/attachment.html>