Sean,
Thanks for looking at this. Knowing that the last instruction triggered the bug
is often not enough. I use bugpoint to reduce the failing test. The reason is
that some of the bugs may be caused by the interaction between several
instruction. Having said that, I think that the change that you proposed is a
good one. Can you send a patch ?
Thanks,
Nadav
From: Sean Silva [mailto:silvas at purdue.edu]
Sent: Monday, February 27, 2012 05:45
To: Hal Finkel
Cc: Rotem, Nadav; llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] llvm-stress for fuzzing llvm
I'm finding it useful to replace the main loop with:
for (unsigned i = 0, n = SizeCL/Modifiers.size(); i < n; ++i) {
Modifiers[i%Modifiers.size()]->Act();
}
That way, changing the size by 1 adds exactly one instruction, which makes delta
debugging MUCH easier. Maybe it would be worth changing?
--Sean Silva
On Sun, Feb 26, 2012 at 9:23 PM, Sean Silva <silvas at purdue.edu> wrote:
Wow, nifty tool! I've already found a couple crashes!
It is also really easy to pinpoint what is causing the error. Whenever you
trigger a bug, run llvm-stress with the same seed but a really small size that
doesn't trigger the bug (e.g. like 10). Then do binary search on the size.
Eventually you find exactly the cutoff of size that triggers the bug (e.g. 539
runs fine, but 540 crashes), and then you can diff the crashing and non-crashing
.ll files and there should only be a tiny difference.
--Sean Silva
On Sun, Feb 26, 2012 at 7:29 PM, Hal Finkel <hfinkel at anl.gov> wrote:
Nadav,
Thanks, this is neat! Here is a patch which optionally enables
generation of the other floating-point types. Please review.
-Hal
On Sun, 26 Feb 2012 11:51:04 +0000
"Rotem, Nadav" <nadav.rotem at intel.com> wrote:
> Hi,
>
> Compiling lots of 'junk' helps in catching bugs. I added a new
tool
> (located under llvm/tools/llvm-stress) for generating random LL
> files. The tool can be used to test different llvm components using
> various compilation flags. Until now, I only found bugs in the
> codegen, and not in general llvm optimizations. This probably means
> that the generated tests are currently too simple for the
> higher-level optimizations.
>
> The command line below generates a random ll file, and llc compiles
> this file. It often crashes.
>
> ./llvm-stress -seed $RANDOM -o tmp.ll -size 1000 ; ./llc tmp.ll
> -mcpu=corei7-avx -mattr=+avx -o /dev/null
>
> The "-seed" flag sets the initial seed to be used by the random
> function. I implemented a simple portable 'random' function so
that
> the result should be identical on all platforms. The initial seed
> also appears in the name of the generated function. The "-size"
> parameter sets the size of the generated random file.
>
> Nadav
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
--
Hal Finkel
Postdoctoral Appointee
Leadership Computing Facility
Argonne National Laboratory
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
---------------------------------------------------------------------
Intel Israel (74) Limited
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
Here is that patch. Btw, I've just been using bugpoint, and it's really nifty! --Sean Silva 2012/2/27 Rotem, Nadav <nadav.rotem at intel.com>> Sean, > > Thanks for looking at this. Knowing that the last instruction triggered > the bug is often not enough. I use bugpoint to reduce the failing test. > The reason is that some of the bugs may be caused by the interaction > between several instruction. Having said that, I think that the change > that you proposed is a good one. Can you send a patch ? > > Thanks, > Nadav > > > From: Sean Silva [mailto:silvas at purdue.edu] > Sent: Monday, February 27, 2012 05:45 > To: Hal Finkel > Cc: Rotem, Nadav; llvmdev at cs.uiuc.edu > Subject: Re: [LLVMdev] llvm-stress for fuzzing llvm > > I'm finding it useful to replace the main loop with: > for (unsigned i = 0, n = SizeCL/Modifiers.size(); i < n; ++i) { > Modifiers[i%Modifiers.size()]->Act(); > } > > That way, changing the size by 1 adds exactly one instruction, which makes > delta debugging MUCH easier. Maybe it would be worth changing? > > --Sean Silva > > On Sun, Feb 26, 2012 at 9:23 PM, Sean Silva <silvas at purdue.edu> wrote: > Wow, nifty tool! I've already found a couple crashes! > > It is also really easy to pinpoint what is causing the error. Whenever you > trigger a bug, run llvm-stress with the same seed but a really small size > that doesn't trigger the bug (e.g. like 10). Then do binary search on the > size. Eventually you find exactly the cutoff of size that triggers the bug > (e.g. 539 runs fine, but 540 crashes), and then you can diff the crashing > and non-crashing .ll files and there should only be a tiny difference. > > --Sean Silva > > On Sun, Feb 26, 2012 at 7:29 PM, Hal Finkel <hfinkel at anl.gov> wrote: > Nadav, > > Thanks, this is neat! Here is a patch which optionally enables > generation of the other floating-point types. Please review. > > -Hal > > On Sun, 26 Feb 2012 11:51:04 +0000 > "Rotem, Nadav" <nadav.rotem at intel.com> wrote: > > > Hi, > > > > Compiling lots of 'junk' helps in catching bugs. I added a new tool > > (located under llvm/tools/llvm-stress) for generating random LL > > files. The tool can be used to test different llvm components using > > various compilation flags. Until now, I only found bugs in the > > codegen, and not in general llvm optimizations. This probably means > > that the generated tests are currently too simple for the > > higher-level optimizations. > > > > The command line below generates a random ll file, and llc compiles > > this file. It often crashes. > > > > ./llvm-stress -seed $RANDOM -o tmp.ll -size 1000 ; ./llc tmp.ll > > -mcpu=corei7-avx -mattr=+avx -o /dev/null > > > > The "-seed" flag sets the initial seed to be used by the random > > function. I implemented a simple portable 'random' function so that > > the result should be identical on all platforms. The initial seed > > also appears in the name of the generated function. The "-size" > > parameter sets the size of the generated random file. > > > > Nadav > > --------------------------------------------------------------------- > > Intel Israel (74) Limited > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > > > > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > -- > Hal Finkel > Postdoctoral Appointee > Leadership Computing Facility > Argonne National Laboratory > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > --------------------------------------------------------------------- > Intel Israel (74) Limited > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20120227/6397736d/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: forloop.diff Type: text/x-patch Size: 764 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20120227/6397736d/attachment.bin>
Attached is a simple bash script I wrote to run llvm-stress several times, might be useful! Thanks Joey On 27 February 2012 19:14, Sean Silva <silvas at purdue.edu> wrote:> Here is that patch. > > Btw, I've just been using bugpoint, and it's really nifty! > > --Sean Silva > > 2012/2/27 Rotem, Nadav <nadav.rotem at intel.com> > > Sean, >> >> Thanks for looking at this. Knowing that the last instruction triggered >> the bug is often not enough. I use bugpoint to reduce the failing test. >> The reason is that some of the bugs may be caused by the interaction >> between several instruction. Having said that, I think that the change >> that you proposed is a good one. Can you send a patch ? >> >> Thanks, >> Nadav >> >> >> From: Sean Silva [mailto:silvas at purdue.edu] >> Sent: Monday, February 27, 2012 05:45 >> To: Hal Finkel >> Cc: Rotem, Nadav; llvmdev at cs.uiuc.edu >> Subject: Re: [LLVMdev] llvm-stress for fuzzing llvm >> >> I'm finding it useful to replace the main loop with: >> for (unsigned i = 0, n = SizeCL/Modifiers.size(); i < n; ++i) { >> Modifiers[i%Modifiers.size()]->Act(); >> } >> >> That way, changing the size by 1 adds exactly one instruction, which >> makes delta debugging MUCH easier. Maybe it would be worth changing? >> >> --Sean Silva >> >> On Sun, Feb 26, 2012 at 9:23 PM, Sean Silva <silvas at purdue.edu> wrote: >> Wow, nifty tool! I've already found a couple crashes! >> >> It is also really easy to pinpoint what is causing the error. Whenever >> you trigger a bug, run llvm-stress with the same seed but a really small >> size that doesn't trigger the bug (e.g. like 10). Then do binary search on >> the size. Eventually you find exactly the cutoff of size that triggers the >> bug (e.g. 539 runs fine, but 540 crashes), and then you can diff the >> crashing and non-crashing .ll files and there should only be a tiny >> difference. >> >> --Sean Silva >> >> On Sun, Feb 26, 2012 at 7:29 PM, Hal Finkel <hfinkel at anl.gov> wrote: >> Nadav, >> >> Thanks, this is neat! Here is a patch which optionally enables >> generation of the other floating-point types. Please review. >> >> -Hal >> >> On Sun, 26 Feb 2012 11:51:04 +0000 >> "Rotem, Nadav" <nadav.rotem at intel.com> wrote: >> >> > Hi, >> > >> > Compiling lots of 'junk' helps in catching bugs. I added a new tool >> > (located under llvm/tools/llvm-stress) for generating random LL >> > files. The tool can be used to test different llvm components using >> > various compilation flags. Until now, I only found bugs in the >> > codegen, and not in general llvm optimizations. This probably means >> > that the generated tests are currently too simple for the >> > higher-level optimizations. >> > >> > The command line below generates a random ll file, and llc compiles >> > this file. It often crashes. >> > >> > ./llvm-stress -seed $RANDOM -o tmp.ll -size 1000 ; ./llc tmp.ll >> > -mcpu=corei7-avx -mattr=+avx -o /dev/null >> > >> > The "-seed" flag sets the initial seed to be used by the random >> > function. I implemented a simple portable 'random' function so that >> > the result should be identical on all platforms. The initial seed >> > also appears in the name of the generated function. The "-size" >> > parameter sets the size of the generated random file. >> > >> > Nadav >> > --------------------------------------------------------------------- >> > Intel Israel (74) Limited >> > >> > This e-mail and any attachments may contain confidential material for >> > the sole use of the intended recipient(s). Any review or distribution >> > by others is strictly prohibited. If you are not the intended >> > recipient, please contact the sender and delete all copies. >> > >> > >> > _______________________________________________ >> > LLVM Developers mailing list >> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> >> -- >> Hal Finkel >> Postdoctoral Appointee >> Leadership Computing Facility >> Argonne National Laboratory >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> >> --------------------------------------------------------------------- >> Intel Israel (74) Limited >> >> This e-mail and any attachments may contain confidential material for >> the sole use of the intended recipient(s). Any review or distribution >> by others is strictly prohibited. If you are not the intended >> recipient, please contact the sender and delete all copies. >> > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20120301/c70a0268/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: run_stress.sh Type: application/x-sh Size: 306 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20120301/c70a0268/attachment.sh>