llvm dev - Aug 2015 - buildbot failure in LLVM on clang-native-arm-cortex-a9

On 08/26/2015 09:38 AM, Renato Golin wrote:
> On 26 August 2015 at 17:30, Philip Reames <listmail at
philipreames.com> wrote:
>> To say this differently, we will revert a *change* which is
problematic.
>> Why shouldn't we "revert" a bot?
> I don't disagree, just don't want to do that lightly. Most
certainly
> not before we have comments from the bot owner.
Why?  This is not our policy for commits; why should it be different for 
bots?  Comments within a reasonable time window (2 hours?) sure, but an 
unresponsive owner can simply re-enable when they get around to it.  
Just like the commit author can re-apply at a later
time.
>
>
>> As an illustrative example, I submitted some documentation changes
earlier
>> this week and got 5 unique build failure notices.  In this case, I
ignored
>> them, but if that had been a small code change, that would have cost me
at
>> least an hour of productivity.
> I have to say, I never spent more than a few minutes looking up
> failing bots. If there's nothing that I can find in 30 seconds of
> looking at the bot screen, I rely on the bot owners to ping me, revert
> my patches, let me know what's wrong.
The key point I was trying to make was the interruption factor.  Not all 
of the notices come it at once.  If I could batch process, it would take 
a lot less time.

Also, there are simply some usability issues; finding the actual build 
error on a small screen - from a phone while in a meeting say - is 
rather challenging.  I usually end up having to move to a laptop before 
being able to identify something as a false positive.
>
> I'll make your words, mine:
>
>> If all bot owners were doing this, having a unstable list which
doesn't
>> actively notify would be completely workable.  If not all bot owners
are
>> doing this, I can't say I really care about the status of those
bots.
> :D
>
> cheers,
> --renato

On 26 August 2015 at 17:43, Philip Reames <listmail at philipreames.com>
wrote:
> Why?  This is not our policy for commits; why should it be different for
> bots?  Comments within a reasonable time window (2 hours?) sure, but an
> unresponsive owner can simply re-enable when they get around to it.  Just
> like the commit author can re-apply at a later time.
>From this comment, I infer that you don't own any bots... :)
Once a bot goes red, it's hard to make it back green again. Once it's
gone red for a few days, the time it consumes is immense. I could
spend hours describing all sorts of issues that I had to deal with red
bots picking up new failures and not reporting, but suffice to say
that reapplying a patch is orders of magnitude easier than re-enabling
a build bot, especially in architectures that not many people have. We
cannot have one decision model to rule them all.

2 hours is satisfactory for commits, 2 days would be satisfactory for
bot owners. We can fiddle with the numbers, but I'd like to give at
least one order of magnitude more to bots than to commits. Also, rarer
and slower bots get larger time-frames than more common rapid-fire
ones.

If we take all that into consideration, I think we can write up a
community guidelines for "reverting" bots and commits.

cheers,
--renato

On 08/26/2015 09:50 AM, Renato Golin wrote:
> On 26 August 2015 at 17:43, Philip Reames <listmail at
philipreames.com> wrote:
>> Why?  This is not our policy for commits; why should it be different
for
>> bots?  Comments within a reasonable time window (2 hours?) sure, but an
>> unresponsive owner can simply re-enable when they get around to it. 
Just
>> like the commit author can re-apply at a later time.
>  From this comment, I infer that you don't own any bots... :)
Not public facing ones, no.  My internal ones use an entirely unrelated 
infrastructure with its own set of problems.  :)
>
> Once a bot goes red, it's hard to make it back green again. Once
it's
> gone red for a few days, the time it consumes is immense. I could
> spend hours describing all sorts of issues that I had to deal with red
> bots picking up new failures and not reporting, but suffice to say
> that reapplying a patch is orders of magnitude easier than re-enabling
> a build bot, especially in architectures that not many people have. We
> cannot have one decision model to rule them all.
>
> 2 hours is satisfactory for commits, 2 days would be satisfactory for
> bot owners. We can fiddle with the numbers, but I'd like to give at
> least one order of magnitude more to bots than to commits. Also, rarer
> and slower bots get larger time-frames than more common rapid-fire
> ones.
2 days seems fine to me.  I don't care what the specific threshold is as 
long as there is one.  :)

I'll note for the record that I was describing time to response, not 
time to fix, but that doesn't really change anything
material.
>
> If we take all that into consideration, I think we can write up a
> community guidelines for "reverting" bots and commits.
+1
>
> cheers,
> --renato