llvm dev - Jul 2010 - [LLVMdev] I would like to merge PARSEC into test-suite

From here: http://parsec.cs.princeton.edu

It is open-source and redistributable, and I have added LICENSE.TXT 
files to all the pieces I want to merge.  These are blackscholes, 
canneal, dedup, fluidanimate, freqmine, streamcluster, and swaptions.  I 
will disable the tests by default on the initial merge and test 
thoroughly on Linux and MacOS before enabling them.  May I please commit 
my changes directly to test-suite?

Thanks,
--Patrick

-- 
If I'm not here, I've gone out to find myself.  If I get back before I
return, please keep me here.

Patrick,

On Jul 20, 2010, at 1:18 AM, Patrick Simmons wrote:
> It is open-source and redistributable, and I have added LICENSE.TXT 
> files to all the pieces I want to merge.  These are blackscholes, 
> canneal, dedup, fluidanimate, freqmine, streamcluster, and swaptions.  I 
> will disable the tests by default on the initial merge and test 
> thoroughly on Linux and MacOS before enabling them.  May I please commit 
> my changes directly to test-suite?
I'm not convinced that the PARSEC tests are appropriate for addition the
LLVM testsuite, because they are multithreaded tests.  The testsuite needs to be
low-volatility in terms of performance variation, as well as easy to verify
correctness, and easy for the compiler developers to debug miscompilations.   I
don't think PARSEC is a good match on these fronts.

--Owen

On Tue, Jul 20, 2010 at 1:30 AM, Owen Anderson <resistor at mac.com>
wrote:
> Patrick,
>
> On Jul 20, 2010, at 1:18 AM, Patrick Simmons wrote:
>> It is open-source and redistributable, and I have added LICENSE.TXT
>> files to all the pieces I want to merge.  These are blackscholes,
>> canneal, dedup, fluidanimate, freqmine, streamcluster, and swaptions.
 I
>> will disable the tests by default on the initial merge and test
>> thoroughly on Linux and MacOS before enabling them.  May I please
commit
>> my changes directly to test-suite?
Awesome! Can we see a patch?

The PARSEC webpage mentions that the benchmark suite is quite large.
Does your patch actually commit a specific version to the test-suite,
or is it setup like the externals tests where the user is expected to
download PARSEC on the side if they wish to test it?
> I'm not convinced that the PARSEC tests are appropriate for addition
the LLVM testsuite, because they are multithreaded tests.  The testsuite needs
to be low-volatility in terms of performance variation, as well as easy to
verify correctness, and easy for the compiler developers to debug
miscompilations.   I don't think PARSEC is a good match on these fronts.
I don't agree, more tests are always better. We can always invent
infrastructure to ignore the performance of certain tests if that
becomes an issue.

 - Daniel
> --Owen
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>