Star Tan
2013-Jun-06 18:17 UTC
[LLVMdev] [Polly] Set up performance tester for GSOC2013 FastPolly project
Hi Tobias, I am recently trying to set up the performance tester for FastPolly project. According to your suggestion, I plan to use the LNT infrastructure to set up the performance tester. For this purpose, I think I should do this job in three steps: First, I will add PolyBench to LLVM test-suite since PolyBench is the critical benchmarks for FastPolly. I have adjust the PolyBench-c-3.2 so we can put it into LLVM test-suite/SingleSource/Benchmarks/. Please find the attached PolyBench_for_LLVM_test_suite.tgz. Second, I think I should select some benchmarks for FastPolly testing. Running all LLVM test-suite benchmarks is very time-consuming and unnecessary. My idea is still to run the mediebench and PolyBench using LNT infrastructure. Please find the attached Prliminary PolyBench evaluation results (LNT_PolyBench.html) using LNT infrastructure. Third, I think I should evaluate the compile-time and running time in both Polly-enabling case and Polly-disabling case. Currently, I am not clear how to configurate LNT so it can test the same benchmark using different parameters in the same runtime test, but I will investigate it. Do you think it is a good way to set up a performance tester for the FastPolly project? If you agree with me, I will try to commit the PolyBench to llvm test-suite in the next step. Best wishes, Star Tan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130607/89278521/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: PolyBench_For_LLVM_Test_suite.tgz Type: application/octet-stream Size: 22691 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130607/89278521/attachment.obj> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130607/89278521/attachment-0001.html>
Tobias Grosser
2013-Jun-09 07:09 UTC
[LLVMdev] [Polly] Set up performance tester for GSOC2013 FastPolly project
On 06/06/2013 11:17 AM, Star Tan wrote:> Hi Tobias, > > > I am recently trying to set up the performance tester for FastPolly project. According to your suggestion, I plan to use the LNT infrastructure to set up the performance tester. For this purpose, I think I should do this job in three steps: > > > First, I will add PolyBench to LLVM test-suite since PolyBench is the critical benchmarks for FastPolly. I have adjust the PolyBench-c-3.2 so we can put it into LLVM test-suite/SingleSource/Benchmarks/. Please find the attached PolyBench_for_LLVM_test_suite.tgz.Polybench should already be part of the LLVM test suite. Please have a source at SingleSource/Benchmarks/Polybench/> Second, I think I should select some benchmarks for FastPolly testing. Running all LLVM test-suite benchmarks is very time-consuming and unnecessary. My idea is still to run the mediebench and PolyBench using LNT infrastructure. Please find the attached Prliminary PolyBench evaluation results (LNT_PolyBench.html) using LNT infrastructure.I believe in the end we should run all benchmarks. It is important to see the performance over a wider range of input programs. To get started you can obviously focus on specific programs or problems.> Third, I think I should evaluate the compile-time and running time in both Polly-enabling case and Polly-disabling case. Currently, I am not clear how to configurate LNT so it can test the same benchmark using different parameters in the same runtime test, but I will investigate it.The LNT infrastructure will provide you with compile-time and run-time statistics. We should set up the tester with 'clang -O3' and 'polly-clang -O3' and compare the compile-time of both runs against each other.> Do you think it is a good way to set up a performance tester for the FastPolly project? If you agree with me, I will try to commit the PolyBench to llvm test-suite in the next step.You can skip the step with committing the test suite. A the moment I do not have dedicated machines available to set up public performance testers, so we need to rely on Sebastian's internal testers or your own experiments. I may be able to get some machines in the not so distance future. Until we reach this point, I propose you make yourself familiar with LNT, the LLVM test suite and possibly also the buildbot system we have. If you have a machine where you can run this locally, this would already be a good start. Cheers, Tobias
Star Tan
2013-Jun-09 07:25 UTC
[LLVMdev] [Polly] Set up performance tester for GSOC2013 FastPolly project
At 2013-06-09 15:09:54,"Tobias Grosser" <tobias at grosser.es> wrote:>On 06/06/2013 11:17 AM, Star Tan wrote: >> Hi Tobias, >> >> >> I am recently trying to set up the performance tester for FastPolly project. According to your suggestion, I plan to use the LNT infrastructure to set up the performance tester. For this purpose, I think I should do this job in three steps: >> >> >> First, I will add PolyBench to LLVM test-suite since PolyBench is the critical benchmarks for FastPolly. I have adjust the PolyBench-c-3.2 so we can put it into LLVM test-suite/SingleSource/Benchmarks/. Please find the attached PolyBench_for_LLVM_test_suite.tgz. > >Polybench should already be part of the LLVM test suite. Please have a >source at SingleSource/Benchmarks/Polybench/I see. Yes, the Polybench has been added to LLVM test suite. My previous investigation is based on test-suite-3.0, which is not up-to-date.> >> Second, I think I should select some benchmarks for FastPolly testing. Running all LLVM test-suite benchmarks is very time-consuming and unnecessary. My idea is still to run the mediebench and PolyBench using LNT infrastructure. Please find the attached Prliminary PolyBench evaluation results (LNT_PolyBench.html) using LNT infrastructure. > >I believe in the end we should run all benchmarks. It is important to >see the performance over a wider range of input programs. To get started >you can obviously focus on specific programs or problems. >OK, I can run all benchmark once every half month. In other testing, we can mainly focus on Polybench and mediabench.>> Third, I think I should evaluate the compile-time and running time in both Polly-enabling case and Polly-disabling case. Currently, I am not clear how to configurate LNT so it can test the same benchmark using different parameters in the same runtime test, but I will investigate it. > >The LNT infrastructure will provide you with compile-time and run-time >statistics. We should set up the tester with 'clang -O3' and >'polly-clang -O3' and compare the compile-time of both runs against each >other. > >> Do you think it is a good way to set up a performance tester for the FastPolly project? If you agree with me, I will try to commit the PolyBench to llvm test-suite in the next step. > >You can skip the step with committing the test suite. > >A the moment I do not have dedicated machines available to set up public >performance testers, so we need to rely on Sebastian's internal testers >or your own experiments. I may be able to get some machines in the not >so distance future. Until we reach this point, I propose you make >yourself familiar with LNT, the LLVM test suite and possibly also the >buildbot system we have. If you have a machine where you can run this >locally, this would already be a good start.Thanks. Yes, I will run all tests on my own computer recently.> >Cheers, >Tobias > >Star Tan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130609/27c6de50/attachment.html>
Apparently Analagous Threads
- [LLVMdev] [Polly] Set up performance tester for GSOC2013 FastPolly project
- [LLVMdev] [Polly][GSOC2013] FastPolly -- SCOP Detection Pass
- [LLVMdev] [Polly][GSOC2013] FastPolly -- SCOP Detection Pass
- [LLVMdev] [Polly][GSOC2013] FastPolly -- SCOP Detection Pass
- [LLVMdev] [FastPolly]: Update of Polly's performance on LLVM test-suite