On Wed, Jul 15, 2009 at 3:43 PM, Eli Friedman<eli.friedman at gmail.com> wrote:> On Wed, Jul 15, 2009 at 3:01 PM, Bill Wendling<isanbard at gmail.com> wrote: >> The core problem, in my opinion, is that people *don't* pay attention >> to the build bot failure messages that come along. > > That's largely because of the number of false positives. >There have been fewer and fewer of these in recent times. -bw
That depends on what you call a false positive. The public buildbot regularly fails because of mailing Frontend tests, and I have had continues failures of some DejaGNU tests for a long time on some builders. Its not a false positive per se, but one starts to ignore the failures because they aren't unexpected. - Daniel On Wed, Jul 15, 2009 at 4:10 PM, Bill Wendling<isanbard at gmail.com> wrote:> On Wed, Jul 15, 2009 at 3:43 PM, Eli Friedman<eli.friedman at gmail.com> wrote: >> On Wed, Jul 15, 2009 at 3:01 PM, Bill Wendling<isanbard at gmail.com> wrote: >>> The core problem, in my opinion, is that people *don't* pay attention >>> to the build bot failure messages that come along. >> >> That's largely because of the number of false positives. >> > There have been fewer and fewer of these in recent times. > > -bw > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >
On Jul 15, 2009, at 4:48 PMPDT, Daniel Dunbar wrote:> That depends on what you call a false positive. The public buildbot > regularly fails because of mailing Frontend tests, and I have had > continues failures of some DejaGNU tests for a long time on some > builders. Its not a false positive per se, but one starts to ignore > the failures because they aren't unexpected.Yes. Probably the only way this will work better is if we get the testsuite to 0 failures, everywhere, conditionalizing as necessary to get rid of expected failures. Then regressions will be more visible. I doubt that will happen unless we freeze the tree for a while and get everybody to fix bugs, or disable tests, instead of doing new stuff (at least, that was the case for gcc).> - Daniel > > On Wed, Jul 15, 2009 at 4:10 PM, Bill Wendling<isanbard at gmail.com> > wrote: >> On Wed, Jul 15, 2009 at 3:43 PM, Eli >> Friedman<eli.friedman at gmail.com> wrote: >>> On Wed, Jul 15, 2009 at 3:01 PM, Bill Wendling<isanbard at gmail.com> >>> wrote: >>>> The core problem, in my opinion, is that people *don't* pay >>>> attention >>>> to the build bot failure messages that come along. >>> >>> That's largely because of the number of false positives. >>> >> There have been fewer and fewer of these in recent times. >> >> -bw >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On Wednesday 15 July 2009 18:48, Daniel Dunbar wrote:> That depends on what you call a false positive. The public buildbot > regularly fails because of mailing Frontend tests, and I have had > continues failures of some DejaGNU tests for a long time on some > builders. Its not a false positive per se, but one starts to ignore > the failures because they aren't unexpected.I've experienced that in my own local copies. For example, I've had 19-21 unexpected failures for weeks that no one else seems to see. Something about our test infrastructure is fragile to the point that changing environments somehow causes different results. -Dave