llvm dev - Sep 2019 - RFC: Adding GCC C Torture Suite to External Test Suites

My somewhat ancient experience suggests that this is an excellent idea;
C-Torture files have an uncanny ability to elicit interesting bugs even in an
unrelated compiler.  The eeriest one it elicited during testing of the
PalmSource ARM compiler occurred at the same optimization level (O2) as in GCC,
even though our optimization levels meant something completely different (they
went all the way up to O5, rather than O3), and our back- and middle-ends were
licensed from well-respected commercial vendors.  (Details in my Software:
Practice and Experience article [0].)
    One more painful piece of experience testing later compilers and languages: 
Be cautious, and remain calm, when updating to the latest third-party tests. 
Sometimes a clever and tricky new file, which elicited a bug now fixed in the
target compiler, will also elicit a bug in yours.  It’s easy to overreact to
this, as though it’s a sudden regression, rather than an old problem newly
revealed.

___
[0] http://onlinelibrary.wiley.com/doi/10.1002/spe.812/abstract, 
preprint at http://pobox.com/~flash/Practical_Testing_of_C99.pdf, 
compiler testing bibliography:
http://pobox.com/~flash/compiler_testing_bibliography.html.

- - -
Flash Sheridan
Compiler Quality Assurance Lead at Axoni for AxLang (http://axoni.com/axlang)
http://pobox.com/~flash
http://www.linkedin.com/in/flashsheridan

We’re hiring, e.g. compiler engineer:
https://jobs.lever.co/axoni/1356bb24-c82f-4a0f-a6f5-783d6a265d25?lever-via=9vamXsNUHv
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190906/b8231555/attachment.html>