llvm dev - Mar 2008 - [LLVMdev] Proposal for GSoC project for improving llvm-test testsuite

On Mar 19, 2008, at 1:39 AM, Duncan Sands wrote:
> I would like to see:
>
> (1) LLVM testsuite run every night for all targets, target variants  
> and
> operating systems, for example by using an emulator to emulate targets
> for which the hardware is not available.  The gcc project has a  
> compile
> farm which might be usable for this, but we would need to ask them  
> if it
> is ok.
> (2) Likewise for the gcc testsuite, with results emailed to the  
> nightly
> test mailing list.  I would like the Fortran and Ada testsuites to  
> be run
> as well as the C/C++/ObjC tests.
>
> Just some thoughts!
I agree this would be useful. However these two items require access  
to lots of machine cycles. Do you anticipate lots of work, enough to  
justify for a GSoC proposal, in existing test setup to accommodate  
these two specific goals ?
>
>
> Ciao,
>
> Duncan.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-
Devang

Hi,
> I agree this would be useful. However these two items require access  
> to lots of machine cycles.
that's why I suggested using gcc's cfarm compile farm.
> Do you anticipate lots of work, enough to   
> justify for a GSoC proposal, in existing test setup to accommodate  
> these two specific goals ?
I think it would take at least two weeks to set this up.  I don't
know how long a SoC project is supposed to take, but presumably more
than this.  So this could be part of a larger testing framework.  I
must admit that setting this kind of thing up isn't very exciting or
cutting edge, but it is important.  My impression is that LLVM isn't
tested half as hard as it needs to be.

It would also be nice to have some kind of automated patch queue,
to which patches can be submitted and which runs the testsuite with
them applied.

Ciao,

Duncan.

While on the subject of the testsuite, some random thoughts:

- show how test results changed from one week and one month
ago, and compared to the last release, rather than just
comparing with the previous run as now.  Good for spotting
trends.
- fix the testsuite so that buildlogs are really available:
I occasionally see that a nightly tester failed the build, and
in the message it gives a url where you can get the build log.
Only the logs aren't there...

Ciao,

Duncan.

Hello every body,
Thank you all for your great suggestions and feedbacks, I'd forward the
detailed proposal within couple of days. And also if I need to clarify any
thing I'll ask in the list. Thanks again.
Regards,
Rajika

On Thu, Mar 20, 2008 at 10:35 PM, Duncan Sands <baldrick at free.fr>
wrote:
> While on the subject of the testsuite, some random thoughts:
>
> - show how test results changed from one week and one month
> ago, and compared to the last release, rather than just
> comparing with the previous run as now.  Good for spotting
> trends.
> - fix the testsuite so that buildlogs are really available:
> I occasionally see that a nightly tester failed the build, and
> in the message it gives a url where you can get the build log.
> Only the logs aren't there...
>
> Ciao,
>
> Duncan.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>


-- 
comp.lang.c - http://groups.google.com/group/comp.lang.c/topics
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20080322/037e4d40/attachment.html>