Haghighat, Mohammad R
2013-Jul-27 03:18 UTC
[LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
Hal, I ran the failing tests in the attachment to the bug 16431 on the latest clang trunk (version 3.4 trunk 187225). http://llvm.org/bugs/show_bug.cgi?id=16431 The following tests still fail: Tests in diff: t10236 t12206 t2581 t6734 t7788 t7820 t8069 t9982 All tests in InfLoopInClang: t19193 t22300 t25903 t27872 t33143 t8543 Meanwhile, I'll launch a new run of icFuzz and will post the results later. -moh -----Original Message----- From: Hal Finkel [mailto:hfinkel at anl.gov] Sent: Friday, July 26, 2013 8:33 AM To: Haghighat, Mohammad R Cc: llvmdev at cs.uiuc.edu Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk ----- Original Message -----> ----- Original Message ----- > > Great job, Hal! > > > > Sure. I'd be happy to run icFuzz and report the fails once these > > bugs > > are fixed and thereafter whenever people want new runs. Obviously, > > this can be automated, but the problem is that icFuzz is not > > currently open sourced. > > I would be happy to see this open sourced, but I think that we can > work something out regardless. > > Also, once we get the current set of things resolved, I think it > would be useful to test running with: > > -- -O3, LTO (-O4 or -flto), > -- -fslp-vectorize, -fslp-vectorize-aggressive (which are actually > separate optimizations) > -- -ffast-math (if you can do floating point with tolerances, or at > least -ffinite-math-only), -fno-math-errno > (and there are obviously a whole bunch of non-default > code-generation and target options). > > Is it feasible to set up runs with different flags? > > > Once there's a bug in the compiler, there's > > really no limit in the number of failing tests that can be > > generated, so it's more productive to run the generator after the > > previously reported bugs are fixed. > > Agreed. > > > > > We've also seen cases where the results of "clang -O2" are > > different > > on Mac vs. Linux/Windows. > > I recall an issue related to default settings for FP, and differences > with libm implementation. Are there non-floating-point cases? > > > > > Just let me know when you want a new run. > > Will do!Mohammad, Can you please re-run these now? I know that the original loop-vectorizer bugs causing the miscompiles have been fixed, and the others also seem to have been resolved as well. Thanks again, Hal> > -Hal > > > > > Cheers, > > -moh > > > > -----Original Message----- > > From: Hal Finkel [mailto:hfinkel at anl.gov] > > Sent: Wednesday, June 26, 2013 7:35 AM > > To: Haghighat, Mohammad R > > Cc: llvmdev at cs.uiuc.edu; Jim Grosbach > > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly > > generated tests that fail on clang 3.4 trunk > > > > ----- Original Message ----- > > > > > > Hi Moh, > > > > > > > > > Thanks for this. I’m really glad to see the work you’re doing in > > > this > > > area and believe it will be extremely helpful in improving the > > > quality of the compiler. > > > > > > > > > -Jim > > > > > > > > > > > > On Jun 24, 2013, at 4:10 PM, Haghighat, Mohammad R < > > > mohammad.r.haghighat at intel.com > wrote: > > > > > > > > > > > > > > > > > > Hi, > > > > > > I just submitted a bug report with a package containing 107 small > > > test cases that fail on the latest LLVM/clang 3.4 main trunk > > > (184563). Included are test sources, compilation commands, test > > > input files, and results at –O0 and –O2 when applicable. > > > > > > http://llvm.org/bugs/show_bug.cgi?id=16431 > > > > > > These tests have been automatically generated by an internal tool > > > at > > > Intel, the Intel Compiler fuzzer, icFuzz. The tests are typically > > > very small. For example, for the following simple loop (test > > > t5702) > > > on MacOS X, clang at –O2 generates a binary that crashes: > > > > > > // Test Loop Interchange > > > for (j = 2; j < 76; j++) { > > > for (jm = 1; jm < 30; jm++) { > > > h[j-1][jm-1] = j + 83; > > > } > > > } > > > > > > The tests are put in to two categories > > > - tests that have different runtime outputs when compiled at -O0 > > > and > > > -O2 (this category also includes runtime crashes) > > > - tests that cause infinite loops in the Clang optimizer > > > > > > Many of these failing tests could be due to the same bug, thus a > > > much > > > smaller number of root problems are expected. > > > > > > Any help with triaging these bugs would be highly appreciated. > > > > I've gone through all of the miscompile cases, used bugpoint to > > reduce them, and opened individual PRs for several distinct bugs. > > So > > far we have: PR16455 (loop vectorizer), PR16457 (sccp), PR16460 > > (instcombine). Thanks again for doing this! Do you plan on > > repeating > > this testing on a regular basis? Can it be automated? > > > > -Hal > > > > > > > > Thanks, > > > -moh > > > > > > _______________________________________________ > > > LLVM Developers mailing list > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > > _______________________________________________ > > > LLVM Developers mailing list > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > > > -- > > Hal Finkel > > Assistant Computational Scientist > > Leadership Computing Facility > > Argonne National Laboratory > > > > -- > Hal Finkel > Assistant Computational Scientist > Leadership Computing Facility > Argonne National Laboratory > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory
Haghighat, Mohammad R
2013-Jul-29 00:13 UTC
[LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
Hal, Just posted a package containing 214 small tests showing bugs in the latest Clang (3.4 trunk 187225) on MacOS X when compiled at -O2. http://llvm.org/bugs/show_bug.cgi?id=16431 These are new tests different from the previously posted ones, but their root causes could be the same as before or could actually be new bugs. Cheers, -moh -----Original Message----- From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Haghighat, Mohammad R Sent: Friday, July 26, 2013 8:19 PM To: Hal Finkel Cc: llvmdev at cs.uiuc.edu Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk Hal, I ran the failing tests in the attachment to the bug 16431 on the latest clang trunk (version 3.4 trunk 187225). http://llvm.org/bugs/show_bug.cgi?id=16431 The following tests still fail: Tests in diff: t10236 t12206 t2581 t6734 t7788 t7820 t8069 t9982 All tests in InfLoopInClang: t19193 t22300 t25903 t27872 t33143 t8543 Meanwhile, I'll launch a new run of icFuzz and will post the results later. -moh -----Original Message----- From: Hal Finkel [mailto:hfinkel at anl.gov] Sent: Friday, July 26, 2013 8:33 AM To: Haghighat, Mohammad R Cc: llvmdev at cs.uiuc.edu Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk ----- Original Message -----> ----- Original Message ----- > > Great job, Hal! > > > > Sure. I'd be happy to run icFuzz and report the fails once these > > bugs are fixed and thereafter whenever people want new runs. > > Obviously, this can be automated, but the problem is that icFuzz is > > not currently open sourced. > > I would be happy to see this open sourced, but I think that we can > work something out regardless. > > Also, once we get the current set of things resolved, I think it would > be useful to test running with: > > -- -O3, LTO (-O4 or -flto), > -- -fslp-vectorize, -fslp-vectorize-aggressive (which are actually > separate optimizations) > -- -ffast-math (if you can do floating point with tolerances, or at > least -ffinite-math-only), -fno-math-errno (and there are obviously a > whole bunch of non-default code-generation and target options). > > Is it feasible to set up runs with different flags? > > > Once there's a bug in the compiler, there's really no limit in the > > number of failing tests that can be generated, so it's more > > productive to run the generator after the previously reported bugs > > are fixed. > > Agreed. > > > > > We've also seen cases where the results of "clang -O2" are different > > on Mac vs. Linux/Windows. > > I recall an issue related to default settings for FP, and differences > with libm implementation. Are there non-floating-point cases? > > > > > Just let me know when you want a new run. > > Will do!Mohammad, Can you please re-run these now? I know that the original loop-vectorizer bugs causing the miscompiles have been fixed, and the others also seem to have been resolved as well. Thanks again, Hal> > -Hal > > > > > Cheers, > > -moh > > > > -----Original Message----- > > From: Hal Finkel [mailto:hfinkel at anl.gov] > > Sent: Wednesday, June 26, 2013 7:35 AM > > To: Haghighat, Mohammad R > > Cc: llvmdev at cs.uiuc.edu; Jim Grosbach > > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly > > generated tests that fail on clang 3.4 trunk > > > > ----- Original Message ----- > > > > > > Hi Moh, > > > > > > > > > Thanks for this. I’m really glad to see the work you’re doing in > > > this area and believe it will be extremely helpful in improving > > > the quality of the compiler. > > > > > > > > > -Jim > > > > > > > > > > > > On Jun 24, 2013, at 4:10 PM, Haghighat, Mohammad R < > > > mohammad.r.haghighat at intel.com > wrote: > > > > > > > > > > > > > > > > > > Hi, > > > > > > I just submitted a bug report with a package containing 107 small > > > test cases that fail on the latest LLVM/clang 3.4 main trunk > > > (184563). Included are test sources, compilation commands, test > > > input files, and results at –O0 and –O2 when applicable. > > > > > > http://llvm.org/bugs/show_bug.cgi?id=16431 > > > > > > These tests have been automatically generated by an internal tool > > > at Intel, the Intel Compiler fuzzer, icFuzz. The tests are > > > typically very small. For example, for the following simple loop > > > (test > > > t5702) > > > on MacOS X, clang at –O2 generates a binary that crashes: > > > > > > // Test Loop Interchange > > > for (j = 2; j < 76; j++) { > > > for (jm = 1; jm < 30; jm++) { > > > h[j-1][jm-1] = j + 83; > > > } > > > } > > > > > > The tests are put in to two categories > > > - tests that have different runtime outputs when compiled at -O0 > > > and > > > -O2 (this category also includes runtime crashes) > > > - tests that cause infinite loops in the Clang optimizer > > > > > > Many of these failing tests could be due to the same bug, thus a > > > much smaller number of root problems are expected. > > > > > > Any help with triaging these bugs would be highly appreciated. > > > > I've gone through all of the miscompile cases, used bugpoint to > > reduce them, and opened individual PRs for several distinct bugs. > > So > > far we have: PR16455 (loop vectorizer), PR16457 (sccp), PR16460 > > (instcombine). Thanks again for doing this! Do you plan on repeating > > this testing on a regular basis? Can it be automated? > > > > -Hal > > > > > > > > Thanks, > > > -moh > > > > > > _______________________________________________ > > > LLVM Developers mailing list > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > > _______________________________________________ > > > LLVM Developers mailing list > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > > > -- > > Hal Finkel > > Assistant Computational Scientist > > Leadership Computing Facility > > Argonne National Laboratory > > > > -- > Hal Finkel > Assistant Computational Scientist > Leadership Computing Facility > Argonne National Laboratory > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory _______________________________________________ LLVM Developers mailing list LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Hal Finkel
2013-Jul-29 04:01 UTC
[LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
----- Original Message -----> Hal, > > Just posted a package containing 214 small tests showing bugs in the > latest Clang (3.4 trunk 187225) on MacOS X when compiled at -O2. > http://llvm.org/bugs/show_bug.cgi?id=16431 > > These are new tests different from the previously posted ones, but > their root causes could be the same as before or could actually be > new bugs.Great, thanks! I'll go through them. -Hal> > Cheers, > -moh > > -----Original Message----- > From: llvmdev-bounces at cs.uiuc.edu > [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Haghighat, > Mohammad R > Sent: Friday, July 26, 2013 8:19 PM > To: Hal Finkel > Cc: llvmdev at cs.uiuc.edu > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly > generated tests that fail on clang 3.4 trunk > > Hal, > > I ran the failing tests in the attachment to the bug 16431 on the > latest clang trunk (version 3.4 trunk 187225). > http://llvm.org/bugs/show_bug.cgi?id=16431 > > The following tests still fail: > Tests in diff: t10236 t12206 t2581 t6734 t7788 > t7820 t8069 t9982 > All tests in InfLoopInClang: t19193 t22300 t25903 t27872 t33143 > t8543 > > Meanwhile, I'll launch a new run of icFuzz and will post the results > later. > > -moh > > -----Original Message----- > From: Hal Finkel [mailto:hfinkel at anl.gov] > Sent: Friday, July 26, 2013 8:33 AM > To: Haghighat, Mohammad R > Cc: llvmdev at cs.uiuc.edu > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly > generated tests that fail on clang 3.4 trunk > > ----- Original Message ----- > > ----- Original Message ----- > > > Great job, Hal! > > > > > > Sure. I'd be happy to run icFuzz and report the fails once these > > > bugs are fixed and thereafter whenever people want new runs. > > > Obviously, this can be automated, but the problem is that icFuzz > > > is > > > not currently open sourced. > > > > I would be happy to see this open sourced, but I think that we can > > work something out regardless. > > > > Also, once we get the current set of things resolved, I think it > > would > > be useful to test running with: > > > > -- -O3, LTO (-O4 or -flto), > > -- -fslp-vectorize, -fslp-vectorize-aggressive (which are actually > > separate optimizations) > > -- -ffast-math (if you can do floating point with tolerances, or > > at > > least -ffinite-math-only), -fno-math-errno (and there are > > obviously a > > whole bunch of non-default code-generation and target options). > > > > Is it feasible to set up runs with different flags? > > > > > Once there's a bug in the compiler, there's really no limit in > > > the > > > number of failing tests that can be generated, so it's more > > > productive to run the generator after the previously reported > > > bugs > > > are fixed. > > > > Agreed. > > > > > > > > We've also seen cases where the results of "clang -O2" are > > > different > > > on Mac vs. Linux/Windows. > > > > I recall an issue related to default settings for FP, and > > differences > > with libm implementation. Are there non-floating-point cases? > > > > > > > > Just let me know when you want a new run. > > > > Will do! > > Mohammad, > > Can you please re-run these now? I know that the original > loop-vectorizer bugs causing the miscompiles have been fixed, and > the others also seem to have been resolved as well. > > Thanks again, > Hal > > > > > -Hal > > > > > > > > Cheers, > > > -moh > > > > > > -----Original Message----- > > > From: Hal Finkel [mailto:hfinkel at anl.gov] > > > Sent: Wednesday, June 26, 2013 7:35 AM > > > To: Haghighat, Mohammad R > > > Cc: llvmdev at cs.uiuc.edu; Jim Grosbach > > > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing > > > randomly > > > generated tests that fail on clang 3.4 trunk > > > > > > ----- Original Message ----- > > > > > > > > Hi Moh, > > > > > > > > > > > > Thanks for this. I’m really glad to see the work you’re doing > > > > in > > > > this area and believe it will be extremely helpful in improving > > > > the quality of the compiler. > > > > > > > > > > > > -Jim > > > > > > > > > > > > > > > > On Jun 24, 2013, at 4:10 PM, Haghighat, Mohammad R < > > > > mohammad.r.haghighat at intel.com > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > I just submitted a bug report with a package containing 107 > > > > small > > > > test cases that fail on the latest LLVM/clang 3.4 main trunk > > > > (184563). Included are test sources, compilation commands, test > > > > input files, and results at –O0 and –O2 when applicable. > > > > > > > > http://llvm.org/bugs/show_bug.cgi?id=16431 > > > > > > > > These tests have been automatically generated by an internal > > > > tool > > > > at Intel, the Intel Compiler fuzzer, icFuzz. The tests are > > > > typically very small. For example, for the following simple > > > > loop > > > > (test > > > > t5702) > > > > on MacOS X, clang at –O2 generates a binary that crashes: > > > > > > > > // Test Loop Interchange > > > > for (j = 2; j < 76; j++) { > > > > for (jm = 1; jm < 30; jm++) { > > > > h[j-1][jm-1] = j + 83; > > > > } > > > > } > > > > > > > > The tests are put in to two categories > > > > - tests that have different runtime outputs when compiled at > > > > -O0 > > > > and > > > > -O2 (this category also includes runtime crashes) > > > > - tests that cause infinite loops in the Clang optimizer > > > > > > > > Many of these failing tests could be due to the same bug, thus > > > > a > > > > much smaller number of root problems are expected. > > > > > > > > Any help with triaging these bugs would be highly appreciated. > > > > > > I've gone through all of the miscompile cases, used bugpoint to > > > reduce them, and opened individual PRs for several distinct bugs. > > > So > > > far we have: PR16455 (loop vectorizer), PR16457 (sccp), PR16460 > > > (instcombine). Thanks again for doing this! Do you plan on > > > repeating > > > this testing on a regular basis? Can it be automated? > > > > > > -Hal > > > > > > > > > > > Thanks, > > > > -moh > > > > > > > > _______________________________________________ > > > > LLVM Developers mailing list > > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > > > > _______________________________________________ > > > > LLVM Developers mailing list > > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > > > > > > -- > > > Hal Finkel > > > Assistant Computational Scientist > > > Leadership Computing Facility > > > Argonne National Laboratory > > > > > > > -- > > Hal Finkel > > Assistant Computational Scientist > > Leadership Computing Facility > > Argonne National Laboratory > > > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > -- > Hal Finkel > Assistant Computational Scientist > Leadership Computing Facility > Argonne National Laboratory > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory
Possibly Parallel Threads
- [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
- [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
- [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
- [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
- [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk