Philip Reames via llvm-dev
2021-Dec-10 19:03 UTC
[llvm-dev] [beta] test case reduction service
I want to announce the beta availability of a public service for reducing and triaging LLVM test cases. If you have a bunch of test cases you've been meaning to reduce, please post a pull request against the repo <https://github.com/preames/llvm-auto-triage-corpus>. The service can currently handle reducing crashes in llvm tools triggered by IR inputs, crashes in clang triggered by C/C++ inputs, and miscompiles which alive2 can confirm. It does not, and probably will never, handle any failure which requires executing binaries or grepping through results in ad-hoc manners. For those, you're still on your own. The other major restriction is that test cases reduced must be fairly short running, and otherwise not require a bunch of resources on the machine doing the reduction. I may loosen those restrictions eventually. I do need to emphasize that this is currently in early beta. A bunch of it is held together with hacky scripts, cron jobs, and manual work on my part. If this gets a bunch of uptake, please be patient. I'm announcing this now mostly because I need a bunch of test cases to help drive work on reducers, and the easiest way to get those is to publish the repo. On a technical level, there's really not too much exciting to this. I'm just wrapping existing reducers, and some ad-hoc python scripting to automate test case reduction. The focus of this project is around the workflow aspects of making it easy to reduce, and encourage the creation of a public corpus of reduced test cases waiting to be fixed. Code can be found here <https://github.com/preames/llvm-auto-triage-scripting> for anyone who wants to see ugly scripting. Philip -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20211210/9bb1e72b/attachment.html>