Am Samstag, den 08.03.2008, 14:02 -0800 schrieb Tanya Lattner:> - ability to check out llvm-gcc or update llvm-gcc and build it before > running tests. In addition to using a prebuilt binary.Does it need a prebuilt binary? I have been suspecting so since it has been failing with BUILD ERROR for me. I just haven't found the time to verify that yet. I have yet to try building llvm-gcc from sources though. The instructions seem to be clear enough, but it's still possible that I'll hit a hidden snag. Well, I'll try that anyway. I like my software well-done after all :-)> If you fix the two things above then you should be able to just rerun > the script multiple times for different targets like you want. It will > also solve problems many others have experienced.Ensuring that the tree is pristine could be a bit of a challenge. It's always possible that the user inadvertently changed or deleted a file, giving a false alarm for all future checks. Options to avoid that seem to be: 1. Do a 'make dist-clean' before running the tests. Of course, if dist-clean has bugs, this may create trouble. On the other hand, it would exercise dist-clean on a regular basis, which might be a Good Thing. 2. Recreate $BUILDDIR as a copy of a pristine check-out directory each time the tester is run. This would be much faster than a download, but still a lot of unnecessary load. 3. For each test, first run svn revert, then ask svn about any unversioned files and delete them, finally svn update. (2) would be easiest. (3) would be most failsafe. (1) might be more useful for the project because it could uncover bugs in the dist-clean target though. (OTOH one could argue that this should be a separate regression test.)> A -noreport option is probably a good idea too.OK, that should be easy to add.> Just be sure not to change what it sends to the website. That needs to > stay the same.Sure. I think wrapping an if statement around the HTTP send should be easy; the code of NewNightlyTester.pl isn't very complicated. On a tangent: What's the best way to submit patches: Bugzilla? Something else? Regards, Jo
On Mar 8, 2008, at 2:45 PM, Joachim Durchholz wrote:> > Am Samstag, den 08.03.2008, 14:02 -0800 schrieb Tanya Lattner: >> - ability to check out llvm-gcc or update llvm-gcc and build it >> before >> running tests. In addition to using a prebuilt binary. > > Does it need a prebuilt binary? > I have been suspecting so since it has been failing with BUILD ERROR > for > me. I just haven't found the time to verify that yet. > > I have yet to try building llvm-gcc from sources though. The > instructions seem to be clear enough, but it's still possible that > I'll > hit a hidden snag. > Well, I'll try that anyway. I like my software well-done after all :-) >You need to have an llvm-gcc, so you either build it yourself or use one of the prebuilt binaries. The nightly tester just needs to know where its located, but it doesn't know anything about llvm-gcc otherwise. Since most people are testing TOT and llvm/llvm-gcc are tied together it makes sense to update llvm-gcc before running the nightly tester. However, it takes quite a bit of time to checkout and build llvm-gcc, so having the option to just update existing sources would be nice.> >> If you fix the two things above then you should be able to just rerun >> the script multiple times for different targets like you want. It >> will >> also solve problems many others have experienced. > > Ensuring that the tree is pristine could be a bit of a challenge. It's > always possible that the user inadvertently changed or deleted a file, > giving a false alarm for all future checks. > Options to avoid that seem to be: > 1. Do a 'make dist-clean' before running the tests. Of course, if > dist-clean has bugs, this may create trouble. On the other hand, it > would exercise dist-clean on a regular basis, which might be a Good > Thing. > 2. Recreate $BUILDDIR as a copy of a pristine check-out directory each > time the tester is run. This would be much faster than a download, but > still a lot of unnecessary load. > 3. For each test, first run svn revert, then ask svn about any > unversioned files and delete them, finally svn update. > > (2) would be easiest. (3) would be most failsafe. (1) might be more > useful for the project because it could uncover bugs in the dist-clean > target though. (OTOH one could argue that this should be a separate > regression test.) >I'd rather not use the nightly tester to test make dist-clean. I'm not sure how many people actually use make dist anything right now so it probably has bugs too. This should probably be investigated separately. I think the only way to ensure the tree is pristine is to do a clean checkout like it does now. So I think that should be the preferred way. However, I think its worthwhile to be able to update your tree and build using the nightly tester. Checking out and building llvm can be a time consuming process for people with slow network connections and slow machines. This is worthwhile in my opinion, but its not necessarily the solution to your problem if you want to make sure your tree is perfect. However, I think #2 is the better solution for your particular problem and I don't think you need to do any changes to the nightly tester, just tell it to use a new build directory.>> A -noreport option is probably a good idea too. > > OK, that should be easy to add. > >> Just be sure not to change what it sends to the website. That needs >> to >> stay the same. > > Sure. I think wrapping an if statement around the HTTP send should be > easy; the code of NewNightlyTester.pl isn't very complicated. >Cool.> On a tangent: > What's the best way to submit patches: Bugzilla? Something else? >Send to llvm-commits or llvm-dev (as long as its not too big). -Tanya P.S. I'm not sure if someone has gotten a chance to look at your dejagnu bugs you filed, but if not.. I'll try to take a look this weekend. Unfortunately, I've been preoccupied this week with family matters.> Regards, > Jo > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Am Samstag, den 08.03.2008, 15:11 -0800 schrieb Tanya Lattner:> >> A -noreport option is probably a good idea too. > > > > OK, that should be easy to add. > > > >> Just be sure not to change what it sends to the website. That needs > >> to > >> stay the same. > > > > Sure. I think wrapping an if statement around the HTTP send should be > > easy; the code of NewNightlyTester.pl isn't very complicated. > > > > Cool.Actually it's trivial. There was an undocumented $TESTING variable that already did exactly what -noreport was supposed to do. Patch is attached. Option is now named -nosubmit, in line with -submit-server and -submit-script.> P.S. I'm not sure if someone has gotten a chance to look at your > dejagnu bugs you filed,No, the bugs have remained undisturbed. (Except that maybe somebody changed the priority.)> but if not.. I'll try to take a look this > weekend. Unfortunately, I've been preoccupied this week with family > matters.I have to admit that I was beginning to wonder how fast bug reports and fixes would get recognized, but I kept reminding myself that all sorts of things can interfere (seems I guessed right with that). Oh, and take your time. Some things are more important than others :-) Regards, Jo -------------- next part -------------- A non-text attachment was scrubbed... Name: NewNightlyTest.pl.nosubmit.patch Type: text/x-patch Size: 1824 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20080309/cec5f941/attachment.bin>
On Mar 8, 2008, at 2:45 PM, Joachim Durchholz wrote:> Am Samstag, den 08.03.2008, 14:02 -0800 schrieb Tanya Lattner: >> - ability to check out llvm-gcc or update llvm-gcc and build it >> before >> running tests. In addition to using a prebuilt binary. > > Does it need a prebuilt binary? > I have been suspecting so since it has been failing with BUILD ERROR > for > me. I just haven't found the time to verify that yet. > > I have yet to try building llvm-gcc from sources though. The > instructions seem to be clear enough, but it's still possible that > I'll > hit a hidden snag. > Well, I'll try that anyway.Thanks!, this will be very useful.> I like my software well-done after all :-)- Devang