On Nov 9, 2015, at 11:42 AM, David Blaikie via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > > On Mon, Nov 9, 2015 at 4:32 AM, Renato Golin <renato.golin at linaro.org <mailto:renato.golin at linaro.org>> wrote: > Hi David/Galina, > > FYI, I found one big reason why self-hosting bots fail long after the > offending commit is tested: dirty builds. > > In a dirty build (no make clean), when a file X is changed that breaks > file Y, the bot will only accuse the failure when file Y is changed, > too, so that stage1 with modified X will compile file Y, and break. > > That sounds like a bug in the build system - do you have any examples of this? We should fix them. > > The build should be reliable - any case where a clean build produces a different result from a dirty build reduces/removes confidence in incremental builds & hurts developer productivity (just like flakey tests, etc).I think he meant that the /second-stage/ build should always be done as a clean build -- to make sure that all of the source gets rebuilt with the newly-compiled compiler, instead of only rebuilding the modified sources with the newly-built compiler. Build systems typically aren't expected to watch out for the compiler itself changing out from under them. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151109/d6897b8e/attachment.html>
On Mon, Nov 9, 2015 at 8:53 AM, James Y Knight via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On Nov 9, 2015, at 11:42 AM, David Blaikie via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > > > On Mon, Nov 9, 2015 at 4:32 AM, Renato Golin <renato.golin at linaro.org> > wrote: > >> Hi David/Galina, >> >> FYI, I found one big reason why self-hosting bots fail long after the >> offending commit is tested: dirty builds. >> >> In a dirty build (no make clean), when a file X is changed that breaks >> file Y, the bot will only accuse the failure when file Y is changed, >> too, so that stage1 with modified X will compile file Y, and break. >> > > That sounds like a bug in the build system - do you have any examples of > this? We should fix them. > > The build should be reliable - any case where a clean build produces a > different result from a dirty build reduces/removes confidence in > incremental builds & hurts developer productivity (just like flakey tests, > etc). > > > I think he meant that the /second-stage/ build should always be done as a > clean build -- to make sure that all of the source gets rebuilt with the > newly-compiled compiler, instead of only rebuilding the modified sources > with the newly-built compiler. > > Build systems typically aren't expected to watch out for the compiler > itself changing out from under them. >+1, I know that my self-hosting Windows buildbot does not do a clean second stage build, and I've been meaning to fix that for some time. Renato, I would rather fix this in our buildbot configs. We should just make stage2 always be clean, and that will fix my builders as well as yours. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151109/197eba68/attachment.html>
On Mon, Nov 9, 2015 at 10:27 AM, Reid Kleckner <rnk at google.com> wrote:> On Mon, Nov 9, 2015 at 8:53 AM, James Y Knight via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> On Nov 9, 2015, at 11:42 AM, David Blaikie via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >> >> >> On Mon, Nov 9, 2015 at 4:32 AM, Renato Golin <renato.golin at linaro.org> >> wrote: >> >>> Hi David/Galina, >>> >>> FYI, I found one big reason why self-hosting bots fail long after the >>> offending commit is tested: dirty builds. >>> >>> In a dirty build (no make clean), when a file X is changed that breaks >>> file Y, the bot will only accuse the failure when file Y is changed, >>> too, so that stage1 with modified X will compile file Y, and break. >>> >> >> That sounds like a bug in the build system - do you have any examples of >> this? We should fix them. >> >> The build should be reliable - any case where a clean build produces a >> different result from a dirty build reduces/removes confidence in >> incremental builds & hurts developer productivity (just like flakey tests, >> etc). >> >> >> I think he meant that the /second-stage/ build should always be done as a >> clean build -- to make sure that all of the source gets rebuilt with the >> newly-compiled compiler, instead of only rebuilding the modified sources >> with the newly-built compiler. >> >> Build systems typically aren't expected to watch out for the compiler >> itself changing out from under them. >> > > +1, I know that my self-hosting Windows buildbot does not do a clean > second stage build, and I've been meaning to fix that for some time. > > Renato, I would rather fix this in our buildbot configs. We should just > make stage2 always be clean, and that will fix my builders as well as yours. >Yeah, sorry, a stage2 is always necessarily clean - because there's a new compiler. ccache isn't even relevant/useful here, is it (I mean it should be a no-op, if it hashes the compiler itself and all the compiler's dependencies too - but I don't know if it does that?)? We want to actually do the real work of compiling the code with the new compiler for everything. Nothing should be a cache/build hit. (fancy build system like Bazel could still get a cache hit on the link steps - if the compiler produced identical object files, etc) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151109/3094b11c/attachment.html>