Hi all, I would like to get some feedback on who in the community is using the LLVM test-suite repository, and in what ways. I am talking specifically about the "test-suite" SVN project, e.g.: http://llvm.org/svn/llvm-project/test-suite/trunk/ not about the "test (no-dash) suite" that is part of LLVM. If you are using the LLVM test-suite project directly (as opposed to via LNT), I'd appreciate it if you could reply to me personally with the following information: 1. What facilities of the project do you use: [ ] User defined test functionality [ ] Ability to execute two compilers (baseline, modified) [ ] Ability to use Makefile's to modify and rerun small parts of tests [ ] Test suite ability to work with LLVM bit code files and tools (as opposed to native object files) [ ] Ability to test SPEC [ ] Ability to execute tests on a remote target [ ] Ability to test Fortran codes 2. What compilers do you use the project to test: [ ] clang [ ] llvm-gcc [ ] dragonegg [ ] Other LLVM compilers (?) [ ] Other non-LLVM compilers 3. What TEST variants do you use: [ ] I do not know what this means [ ] nightly [ ] simple [ ] Other, please specify: _________ - Daniel
On 25 October 2012 16:04, Daniel Dunbar <daniel at zuster.org> wrote:> Hi all, > > I would like to get some feedback on who in the community is using the > LLVM test-suite repository, and in what ways. > > I am talking specifically about the "test-suite" SVN project, e.g.: > http://llvm.org/svn/llvm-project/test-suite/trunk/ > not about the "test (no-dash) suite" that is part of LLVM. > > If you are using the LLVM test-suite project directly (as opposed to > via LNT), I'd appreciate it if you could reply to me personally with > the following information: > > 1. What facilities of the project do you use: > [X] User defined test functionality > [X] Ability to execute two compilers (baseline, modified) > [X] Ability to use Makefile's to modify and rerun small parts of tests > [X] Test suite ability to work with LLVM bit code files and tools (as > opposed to native object files) > [X] Ability to test SPEC > [ ] Ability to execute tests on a remote target > [ ] Ability to test Fortran codes > > 2. What compilers do you use the project to test: > [X] clang > [ ] llvm-gcc > [ ] dragonegg > [ ] Other LLVM compilers (?) > [ ] Other non-LLVM compilers > > 3. What TEST variants do you use: > [ ] I do not know what this means > [X] nightly > [ ] simple > [X] Other, please specify: _All_sorts_of_custom_TEST_files_ > > - Daniel > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20121025/33fb0d2a/attachment.html>
Yes, I use the test-suite (TEST.simple) as a check-in criteria on anything non trivial. On Oct 25, 2012, at 4:04 PM, Daniel Dunbar <daniel at zuster.org> wrote:> Hi all, > > I would like to get some feedback on who in the community is using the > LLVM test-suite repository, and in what ways. > > I am talking specifically about the "test-suite" SVN project, e.g.: > http://llvm.org/svn/llvm-project/test-suite/trunk/ > not about the "test (no-dash) suite" that is part of LLVM. > > If you are using the LLVM test-suite project directly (as opposed to > via LNT), I'd appreciate it if you could reply to me personally with > the following information: > > 1. What facilities of the project do you use: > [ ] User defined test functionality > [ ] Ability to execute two compilers (baseline, modified) > [X] Ability to use Makefile's to modify and rerun small parts of testsI run 'make' in sub directories to parts of the build.> [ ] Test suite ability to work with LLVM bit code files and tools (as > opposed to native object files) > [X ] Ability to test SPECYes.> [ ] Ability to execute tests on a remote target > [ ] Ability to test Fortran codes > > 2. What compilers do you use the project to test: > [ X] clang > [ ] llvm-gcc > [ ] dragonegg > [ ] Other LLVM compilers (?) > [ ] Other non-LLVM compilers >Just Clang.> 3. What TEST variants do you use: > [ ] I do not know what this means > [ ] nightly > [x ] simple > [ ] Other, please specify: _________ >Just Simple.> - Daniel > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On 10/25/12 6:04 PM, Daniel Dunbar wrote:> Hi all, > > I would like to get some feedback on who in the community is using the > LLVM test-suite repository, and in what ways. > > I am talking specifically about the "test-suite" SVN project, e.g.: > http://llvm.org/svn/llvm-project/test-suite/trunk/ > not about the "test (no-dash) suite" that is part of LLVM. > > If you are using the LLVM test-suite project directly (as opposed to > via LNT), I'd appreciate it if you could reply to me personally with > the following information: > > 1. What facilities of the project do you use: > [ X] User defined test functionality > [ X] Ability to execute two compilers (baseline, modified) > [ ] Ability to use Makefile's to modify and rerun small parts of tests > [ X] Test suite ability to work with LLVM bit code files and tools (as > opposed to native object files) > [ X] Ability to test SPEC > [ ] Ability to execute tests on a remote target > [ X] Ability to test Fortran codesThe ability to run Fortran code is something we've used in the past, and we may want to use it in the future (at least if there's Fortran code in SPEC 2006). However, I think it's a very low priority for us.> > 2. What compilers do you use the project to test: > [ X] clang > [ ] llvm-gcc > [ ] dragonegg > [ X] Other LLVM compilers (?)By other compilers, I mean research prototypes. Some are modified versions of Clang; others are passes that are loaded into opt. Others might use llvm-gcc (although I don't know of any off-hand that do that we care about).> [ ] Other non-LLVM compilers > > 3. What TEST variants do you use: > [ ] I do not know what this means > [ ] nightly > [ ] simple > [ X] Other, please specify: _________We use hand-written variants in SAFECode, Poolalloc, and in internal research projects. These Makefiles are out-of-tree but plug into the test suite using an extension mechanism that Chris Lattner originally wrote for the DSA/Poolalloc project. We have used test-suite heavily in our research experiments and open-source project testing. The two features that are most beneficial to us are: 1) We can plug into the LLVM test suite to do customized experiments without modifying the core test-suite Makefiles. 2) The API doesn't change, so we continue using the same infrastructure with each new LLVM release. The SAFECode and Poolalloc tests were written years ago but have continued working with only minor modification. Many of our internal projects have just copied and modified what they've done. Experimental infrastructure is a pain to set up, so having most of the work done is a real benefit. -- John T.> > - Daniel > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Hi Daniel,> I would like to get some feedback on who in the community is using the > LLVM test-suite repository, and in what ways. > > I am talking specifically about the "test-suite" SVN project, e.g.: > http://llvm.org/svn/llvm-project/test-suite/trunk/ > not about the "test (no-dash) suite" that is part of LLVM. > > If you are using the LLVM test-suite project directly (as opposed to > via LNT), I'd appreciate it if you could reply to me personally with > the following information:I have a buildbot running the nightly testsuite. Besides that:> 1. What facilities of the project do you use: > [ ] User defined test functionality > [ ] Ability to execute two compilers (baseline, modified) > [ ] Ability to use Makefile's to modify and rerun small parts of tests > [ ] Test suite ability to work with LLVM bit code files and tools (as > opposed to native object files) > [X] Ability to test SPEC > [ ] Ability to execute tests on a remote target > [X] Ability to test Fortran codes^ OK, I've not recently used the ability to test Fortran, but I would like to improve the testsuite to use gfortran+dragonegg for this.> 2. What compilers do you use the project to test: > [ ] clang > [ ] llvm-gcc > [X] dragonegg > [ ] Other LLVM compilers (?) > [ ] Other non-LLVM compilers > > 3. What TEST variants do you use: > [ ] I do not know what this means > [X] nightly > [X] simple > [ ] Other, please specify: _________ >Ciao, Duncan.
Hey Daniel, My usage hasn't changed much recently, with the exception that I use 'simple' more than 'nightly' these days. I only use nightly when I want to exercise the baseline vs. modified functionality via LLC_BETA. I run pretty much exclusively via the Makefiles. -Jim On Oct 25, 2012, at 4:04 PM, Daniel Dunbar <daniel at zuster.org> wrote:> Hi all, > > I would like to get some feedback on who in the community is using the > LLVM test-suite repository, and in what ways. > > I am talking specifically about the "test-suite" SVN project, e.g.: > http://llvm.org/svn/llvm-project/test-suite/trunk/ > not about the "test (no-dash) suite" that is part of LLVM. > > If you are using the LLVM test-suite project directly (as opposed to > via LNT), I'd appreciate it if you could reply to me personally with > the following information: > > 1. What facilities of the project do you use: > [ ] User defined test functionality > [ x] Ability to execute two compilers (baseline, modified) > [ x] Ability to use Makefile's to modify and rerun small parts of tests > [ x] Test suite ability to work with LLVM bit code files and tools (as > opposed to native object files) > [ x] Ability to test SPEC > [ x] Ability to execute tests on a remote target > [ ] Ability to test Fortran codes > > 2. What compilers do you use the project to test: > [ x] clang > [ ] llvm-gcc > [ ] dragonegg > [ ] Other LLVM compilers (?) > [ ] Other non-LLVM compilers > > 3. What TEST variants do you use: > [ ] I do not know what this means > [ x] nightly > [ x] simple > [ ] Other, please specify: _________ > > - Daniel > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Hi Daniel, 1. What facilities of the project do you use: [ ] User defined test functionality [X] Ability to execute two compilers (baseline, modified) [X] Ability to use Makefile's to modify and rerun small parts of tests [X] Test suite ability to work with LLVM bit code files and tools (as opposed to native object files) [ ] Ability to test SPEC [ ] Ability to execute tests on a remote target [ ] Ability to test Fortran codes I'll generally test on a subset while doing development, and run on the full suite when I'm doing something else. It is very likely that I'll be interested in SPEC and Fortran in the near future. 2. What compilers do you use the project to test: [X] clang [ ] llvm-gcc [ ] dragonegg [ ] Other LLVM compilers (?) [X] Other non-LLVM compilers By 'clang' I really mean 'opt'. I never actually modify clang. The non-LLVM compilers part isn't quite true (yet), but I'll be using test-suite to test a custom GCC plugin at some point in the next two weeks (I'm 95% sure I can make this work, somehow). Not the intended purpose of test-suite, I know. 3. What TEST variants do you use: [ ] I do not know what this means [ ] nightly [ ] simple [X] Other, please specify: __profile__ Currently I mostly use my own TEST.profile.[Makefile,report]. This has been sent to llvm-commits and is awaiting review (I'll be resending very soon as I just had to update it due to the RUNSAFELY changes.) Regards, Alastair. On 25/10/12 19:04, Daniel Dunbar wrote:> Hi all, > > I would like to get some feedback on who in the community is using the > LLVM test-suite repository, and in what ways. > > I am talking specifically about the "test-suite" SVN project, e.g.: > http://llvm.org/svn/llvm-project/test-suite/trunk/ > not about the "test (no-dash) suite" that is part of LLVM. > > If you are using the LLVM test-suite project directly (as opposed to > via LNT), I'd appreciate it if you could reply to me personally with > the following information: > > 1. What facilities of the project do you use: > [ ] User defined test functionality > [ ] Ability to execute two compilers (baseline, modified) > [ ] Ability to use Makefile's to modify and rerun small parts of tests > [ ] Test suite ability to work with LLVM bit code files and tools (as > opposed to native object files) > [ ] Ability to test SPEC > [ ] Ability to execute tests on a remote target > [ ] Ability to test Fortran codes > > 2. What compilers do you use the project to test: > [ ] clang > [ ] llvm-gcc > [ ] dragonegg > [ ] Other LLVM compilers (?) > [ ] Other non-LLVM compilers > > 3. What TEST variants do you use: > [ ] I do not know what this means > [ ] nightly > [ ] simple > [ ] Other, please specify: _________ > > - Daniel > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >