On Thursday 16 July 2009 14:04, Daniel Dunbar wrote:> 2009/7/15 Török Edwin <edwintorok at gmail.com>: > > I'm not too keen about seeing buildbots play with trunk ;) > > > > How about starting simple, and just auto-tagging builds that work? > > Could be done per OS/arch, and one global tag when all buildbots pass. > > I don't know anything about svn performance. Would this negatively > impact llvm.org, which is already pretty strained?Tags are cheap.> If this is reasonable I think I can update my buildbots to do this > fairly easily. The global tag is difficult since it requires > coordination, but individual tags would be easy. It would be nice to > get the mails from them filtered out, otherwise it will make the > commit list very noisy.Thanks to Törok, I think I have the validator buildbot working. The recipe I have does this: * Build llvm * Run make check on llvm * Build llvm-gcc, pointed to the just-built llvm * Install llvm-gcc in a temporary locationo * Build llvm again, pointing to the just-installed llvm-gcc * Run make check on llvm * Run llvm-test If all of these pass, the validation is considered to succeed. I can easily add make check on llvm-gcc if/when we support it. I'd like to take advantage of any tagging infrastructure you add. It would be nice to auto-tag individual validation builds as working. I agree that global tagging can't be automated within the buildbot. Perhaps a daemon could listed to e-mails from the buildbots and tag when a particular revision passes all of the buildbot. That'll come later, though. In the meantime, should I attempt to merge my recipe into your buildbot config when you check it into the llvm repository? I'd make separate buildbot factories to run the two recipes (nightly and validation) but the smae slave pool could be used for both. Or should these remain as completely separate buildbots? One long-term goal for me is to make validation even more target-sensitive. So for example if we had a way to set default llvm and/or llvm-gcc options to compile for specific implementations (Barcelona vs. Nehalem, for example), we could more rigorously test the optimizer and code generator. I know that for us this would be a big win. But AFAIK there's no way to set default compiler flags like this right now. -Dave
On Jul 16, 2009, at 1:55 PM, David Greene wrote:> On Thursday 16 July 2009 14:04, Daniel Dunbar wrote: >> 2009/7/15 Török Edwin <edwintorok at gmail.com>: >>> I'm not too keen about seeing buildbots play with trunk ;) >>> >>> How about starting simple, and just auto-tagging builds that work? >>> Could be done per OS/arch, and one global tag when all buildbots >>> pass. >> >> I don't know anything about svn performance. Would this negatively >> impact llvm.org, which is already pretty strained? > > Tags are cheap. >Sure, but if you have thousands of them, they become somewhat meaningless. How often are you thinking about tagging? If its every commit, that seems excessive.>> If this is reasonable I think I can update my buildbots to do this >> fairly easily. The global tag is difficult since it requires >> coordination, but individual tags would be easy. It would be nice to >> get the mails from them filtered out, otherwise it will make the >> commit list very noisy. > > Thanks to Törok, I think I have the validator buildbot working. The > recipe > I have does this: > > * Build llvm > * Run make check on llvm > * Build llvm-gcc, pointed to the just-built llvm > * Install llvm-gcc in a temporary locationo > * Build llvm again, pointing to the just-installed llvm-gcc > * Run make check on llvm > * Run llvm-test > > If all of these pass, the validation is considered to succeed. >How do you determine if llvm-test passes? You need some post processing to compare it to some known valid build (ie. a release). -Tanya> I can easily add make check on llvm-gcc if/when we support it. > > I'd like to take advantage of any tagging infrastructure you add. > It would > be nice to auto-tag individual validation builds as working. I > agree that > global tagging can't be automated within the buildbot. Perhaps a > daemon > could listed to e-mails from the buildbots and tag when a particular > revision passes all of the buildbot. That'll come later, though. > > In the meantime, should I attempt to merge my recipe into your > buildbot config > when you check it into the llvm repository? I'd make separate > buildbot > factories to run the two recipes (nightly and validation) but the > smae slave > pool could be used for both. Or should these remain as completely > separate > buildbots? > > One long-term goal for me is to make validation even more target- > sensitive. > So for example if we had a way to set default llvm and/or llvm-gcc > options > to compile for specific implementations (Barcelona vs. Nehalem, for > example), > we could more rigorously test the optimizer and code generator. I > know that > for us this would be a big win. But AFAIK there's no way to set > default > compiler flags like this right now. > > -Dave > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090716/45edd181/attachment.html>
On Thursday 16 July 2009 17:20, Tanya Lattner wrote:> Sure, but if you have thousands of them, they become somewhat > meaningless. How often are you thinking about tagging? If its every > commit, that seems excessive.Agreed. The validator will only run about once every 100 commits, and maybe less frequent than that. I don't know what Daniel's plans are.> > If all of these pass, the validation is considered to succeed. > > How do you determine if llvm-test passes? You need some post > processing to compare it to some known valid build (ie. a release).I'm not going to get into that game. For now we may ave to forego llvm-test. Only when it passes EVERYTHING will I make it a validation criteria. Is there some fundamental reason llvm-test doesn't pass everything or is it just a matter of getting people to fix bugs? If it's the former perhaps we should XFAIL some tests. -Dave
On Thu, Jul 16, 2009 at 1:55 PM, David Greene<dag at cray.com> wrote:> On Thursday 16 July 2009 14:04, Daniel Dunbar wrote: >> 2009/7/15 Török Edwin <edwintorok at gmail.com>: >> > I'm not too keen about seeing buildbots play with trunk ;) >> > >> > How about starting simple, and just auto-tagging builds that work? >> > Could be done per OS/arch, and one global tag when all buildbots pass. >> >> I don't know anything about svn performance. Would this negatively >> impact llvm.org, which is already pretty strained? > > Tags are cheap. > >> If this is reasonable I think I can update my buildbots to do this >> fairly easily. The global tag is difficult since it requires >> coordination, but individual tags would be easy. It would be nice to >> get the mails from them filtered out, otherwise it will make the >> commit list very noisy. > > Thanks to Törok, I think I have the validator buildbot working. The recipe > I have does this: > > * Build llvm > * Run make check on llvmYou should run the "make check" on llvm *after* building llvm-gcc. Otherwise, you will be using an older llvm-gcc for the Frontend* tests. -bw
On Jul 16, 2009, at 3:32 PM, Bill Wendling wrote:> On Thu, Jul 16, 2009 at 1:55 PM, David Greene<dag at cray.com> wrote: >> On Thursday 16 July 2009 14:04, Daniel Dunbar wrote: >>> 2009/7/15 Török Edwin <edwintorok at gmail.com>: >>>> I'm not too keen about seeing buildbots play with trunk ;) >>>> >>>> How about starting simple, and just auto-tagging builds that work? >>>> Could be done per OS/arch, and one global tag when all buildbots >>>> pass. >>> >>> I don't know anything about svn performance. Would this negatively >>> impact llvm.org, which is already pretty strained? >> >> Tags are cheap. >> >>> If this is reasonable I think I can update my buildbots to do this >>> fairly easily. The global tag is difficult since it requires >>> coordination, but individual tags would be easy. It would be nice to >>> get the mails from them filtered out, otherwise it will make the >>> commit list very noisy. >> >> Thanks to Törok, I think I have the validator buildbot working. >> The recipe >> I have does this: >> >> * Build llvm >> * Run make check on llvm > > You should run the "make check" on llvm *after* building llvm-gcc. > Otherwise, you will be using an older llvm-gcc for the Frontend* > tests.Not sure off hand how to do it in the existing harness, but something that might be useful is to have a test that checks the version string output of, e.g. llc and llvm-gcc to make sure that they match and stops testing if they don't? -eric
On Thursday 16 July 2009 17:32, Bill Wendling wrote:> > * Build llvm > > * Run make check on llvm > > You should run the "make check" on llvm *after* building llvm-gcc. > Otherwise, you will be using an older llvm-gcc for the Frontend* > tests.Then that's a problem with the llvm test base. As has been mentioned elsewhere, those tests belong in llvm-gcc. -Dave