Hi all, when looking at NewNightlyTester.pl, I see it has several distinct phases: 1. Get a current tree 2. ./configure 3. Build 4. Run a selection of tests 7. Send test results to web site 8. Clean up Wouldn't it make sense to allow each phase to be activated individually? Or possibly a subset of phases, something along the lines of -from-phase=test -to-phase=report With that, I could do all kinds of nifty things, such as checking out once and running the test site for i386 and amd64; or testing compiler options without having to re-download the source tree; or avoiding sending irrelevant test results because the test suite isn't properly installed yet. Does that make sense? Problems? I'd be willing to implement this if something like that is welcomed. Regards, Jo
On Mar 8, 2008, at 10:57 AM, Joachim Durchholz wrote:> Hi all, > > when looking at NewNightlyTester.pl, I see it has several distinct > phases: > > 1. Get a current tree > 2. ./configure > 3. Build > 4. Run a selection of tests > 7. Send test results to web site > 8. Clean up > > Wouldn't it make sense to allow each phase to be activated > individually? > Or possibly a subset of phases, something along the lines of > > -from-phase=test -to-phase=report > > With that, I could do all kinds of nifty things, such as checking out > once and running the test site for i386 and amd64; or testing compiler > options without having to re-download the source tree; or avoiding > sending irrelevant test results because the test suite isn't properly > installed yet. > > Does that make sense? Problems? > I'd be willing to implement this if something like that is welcomed. >Its been on my todo for awhile, but this is what I would like to see. - ability to run the nightly tester from a tree thats already checked out and updated. I think right now it forces you to always check out a new tree and if you don't it reports a build error. - ability to check out llvm-gcc or update llvm-gcc and build it before running tests. In addition to using a prebuilt binary. If you fix the two things above then you should be able to just rerun the script multiple times for different targets like you want. It will also solve problems many others have experienced. A -noreport option is probably a good idea too. Just be sure not to change what it sends to the website. That needs to stay the same. -Tanya> Regards, > Jo > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Am Samstag, den 08.03.2008, 14:02 -0800 schrieb Tanya Lattner:> - ability to check out llvm-gcc or update llvm-gcc and build it before > running tests. In addition to using a prebuilt binary.Does it need a prebuilt binary? I have been suspecting so since it has been failing with BUILD ERROR for me. I just haven't found the time to verify that yet. I have yet to try building llvm-gcc from sources though. The instructions seem to be clear enough, but it's still possible that I'll hit a hidden snag. Well, I'll try that anyway. I like my software well-done after all :-)> If you fix the two things above then you should be able to just rerun > the script multiple times for different targets like you want. It will > also solve problems many others have experienced.Ensuring that the tree is pristine could be a bit of a challenge. It's always possible that the user inadvertently changed or deleted a file, giving a false alarm for all future checks. Options to avoid that seem to be: 1. Do a 'make dist-clean' before running the tests. Of course, if dist-clean has bugs, this may create trouble. On the other hand, it would exercise dist-clean on a regular basis, which might be a Good Thing. 2. Recreate $BUILDDIR as a copy of a pristine check-out directory each time the tester is run. This would be much faster than a download, but still a lot of unnecessary load. 3. For each test, first run svn revert, then ask svn about any unversioned files and delete them, finally svn update. (2) would be easiest. (3) would be most failsafe. (1) might be more useful for the project because it could uncover bugs in the dist-clean target though. (OTOH one could argue that this should be a separate regression test.)> A -noreport option is probably a good idea too.OK, that should be easy to add.> Just be sure not to change what it sends to the website. That needs to > stay the same.Sure. I think wrapping an if statement around the HTTP send should be easy; the code of NewNightlyTester.pl isn't very complicated. On a tangent: What's the best way to submit patches: Bugzilla? Something else? Regards, Jo
Am Samstag, den 08.03.2008, 14:02 -0800 schrieb Tanya Lattner:> - ability to check out llvm-gcc or update llvm-gcc and build it before > running tests.This seems to be a bit more complicated than I thought. There are variations in the build process depending on whether it's a Darwin system or not, installed gcc version, and presence or absence of multilib extensions. I'm not sure that a simple-minded script can reliably detect all the differences. The alternative would be options and letting the user configure. Writing a script would be a difficult task, since it would have to be tested on as many architectures and configurations as possible, with the developer having to rely on error reports to fix things (difficult, lots of work, slow progress). Options, on the other hand, would tend to perpetuate current entry barriers. I'm not too happy about either option, but probably a choice must be made. Which shall it be?> In addition to using a prebuilt binary.This would be simple, of course. Regards, Jo