Kristof Beyls
2014-Dec-16 10:53 UTC
[LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?
I've been wondering too about how to get better ARM v6m compile-and-execute testing going. As you say Jon, the non-execution-based regression tests are surprisingly good at catching issues; but they're no full substitute for executing the code produced by the backend for a reasonably-sized test suite. If somehow it would be possible to compile and run the LLVM test-suite for v6m, I think that would be a good step forward. It would also allow to get a buildbot going without too much effort continuously checking basic correctness of v6m code generation. My guess is that the biggest hurdle would be to get linux or a similar operating system going on a v6m/thumb1 simulator. Does anyone have an idea if this is feasible or completely impossible? Thanks, Kristof> -----Original Message----- > From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] > On Behalf Of Jonathan Roelofs > Sent: 15 December 2014 18:31 > To: Bjoern Haase; llvmdev at cs.uiuc.edu > Subject: Re: [LLVMdev] Newbee question: LLVM backend regression tests > for thumb1 targets on simulator possible? > > > > On 12/15/14 4:29 AM, Bjoern Haase wrote: > > > > Hello, > > > > as a newbee, I'd appreciate some support on regression test setup. > > > > Specifically, I am interrested in the feature of tail call > > optimizations for the ARM v6m targets. This feature currently seems to > > be completely deactivated at the moment (v6m being based on thumb1 > > ?!). According to my code-reading, this feature will involve some > > modifications in epilogue generation. > > > > My work on a gcc backend did show me that for a beginner like me, it > > is rather likely that the first attempts will break something. > :) > > > > Thus, being completely new to the llvm project I think that it's > > essential to first establish a suitable regression test setup. My plan > > is to run the tests for the v4t platform in thumb-only mode where free > > simulators are available and which behaves (at least with respect to > > the > > epilogue) very much like v6m. > Why not test your v6m changes on a Cortex-m0 QEMU? Semihosted > applications are pretty easy to get going, something like: > > $ qemu-system-arm -semihosting -M integratorcp -cpu cortex-m0 -kernel > a.out > > Thumv4t support is a bit spotty in llvm as it's not very well tested. > The problem I run into most (and fixed several times), is that thumbv4t > doesn't have a lo->lo mov instruction that doesn't clobber cpsr. That > being said, it does work for the most part... just something to watch > out for. > > > > From former work on gcc I'm used to the possibilty to run regression > > tests in a simulator for the target platform (there with the expect > > script mechanism). In the documentation on the web, I did, however, > > not yet find information on a comparable feature for llvm. > > > > Therefore the specific questions: > > > > - Is there a mechanism to run backend testsuite runs in a simulation > > framework and if yes, could you give me a hint on how to find > > information > Not that I know of, no. > > I routinely run the libc++ & libc++abi test suites in QEMU, but that's > more of a whole-toolchain test, rather that just a backend test. You can > get surprisingly far without simulator testing, so for most things, LLVM > doesn't use it for testing. This makes it easier for any random > developer to check out the llvm sources, build them, and run the > testsuite, and not have to have simulators installed for all the > different backends. > > - If not, how is regression testing for the thumb1 targets currently > > implemented? > We use the LIT framework http://llvm.org/docs/CommandGuide/lit.html to > test the compiler. Mostly this means feeding in llvm IR, and using > FileCheck http://llvm.org/docs/CommandGuide/FileCheck.html (which is > basically a glorified > grep) to verify that the assembly produced is the same as what the test > expects to be generated. > > To run these tests, use the 'make check-all' target from the build > directory. > > > Cheers, > > Jon > > > > Yours, > > > > Björn Haase > > > > P.S.: Of course, any hint with respect to sibling call optimization > > would also be appreciated in a second step. I actually already found > > some helpful information in the sources. > > > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > -- > Jon Roelofs > jonathan at codesourcery.com > CodeSourcery / Mentor Embedded > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Bjoern Haase
2014-Dec-16 11:03 UTC
[LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?
Am 16.12.2014 11:53, schrieb Kristof Beyls:> If somehow it would be possible to compile and run the LLVM test-suite > for v6m, I think that would be a good step forward. It would also allow > to get a buildbot going without too much effort continuously checking > basic correctness of v6m code generation.IIRC, any linux will be requiring some MMU unit? If I understand correctly, the v6m instruction set is just a binary compatible subset of v7. So what about generating code with restrictions for v6m and running tests on some linux on, say, A15 machine? Björn.
Jonathan Roelofs
2014-Dec-16 14:27 UTC
[LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?
On 12/16/14 3:53 AM, Kristof Beyls wrote:> I've been wondering too about how to get better ARM v6m compile-and-execute > testing going. > > As you say Jon, the non-execution-based regression tests are surprisingly > good at catching issues; but they're no full substitute for executing the > code produced by the backend for a reasonably-sized test suite. > > If somehow it would be possible to compile and run the LLVM test-suite > for v6m, I think that would be a good step forward. It would also allow > to get a buildbot going without too much effort continuously checking > basic correctness of v6m code generation. > > My guess is that the biggest hurdle would be to get linux or a similar > operating system going on a v6m/thumb1 simulator. Does anyone have an idea > if this is feasible or completely impossible?I'm working on a baremetal cross toolchain, and I would like to set up such a buildbot. There are several pieces needed before that can happen though, and the important one is remote testing support in LIT (which I'm working on)... in due time :) Jon -- Jon Roelofs jonathan at codesourcery.com CodeSourcery / Mentor Embedded
Bjoern Haase
2014-Dec-16 16:32 UTC
[LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?
Am 16.12.2014 15:27, schrieb Jonathan Roelofs:> I'm working on a baremetal cross toolchain, and I would like to set up > such a buildbot. There are several pieces needed before that can > happen though, and the important one is remote testing support in LIT > (which I'm working on)... in due time :) > > JonThis would be perfect. Actually, you suggested to use > $qemu-system-arm -semihosting -M integratorcp -cpu cortex-m0 -kernel a.out You have some special version of qemu? My default distribution's qemu package did not have a cortex-M0 nor has qemu head freshly taken from git: $ qemu-arm -cpu ? Available CPUs: arm926 arm946 arm1026 arm1136 arm1136-r2 arm1176 arm11mpcore cortex-m3 cortex-a8 cortex-a8-r2 cortex-a9 cortex-a15 ti925t pxa250 sa1100 sa1110 pxa255 pxa260 pxa261 pxa262 pxa270 pxa270-a0 pxa270-a1 pxa270-b0 pxa270-b1 xa270-c0 pxa270-c5 any Still your procedure will work, just by using the m3 instead :-). Yours, Björn.
Eric Christopher
2014-Dec-17 19:10 UTC
[LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?
On Wed Dec 17 2014 at 10:51:51 AM Kristof Beyls <kristof.beyls at arm.com> wrote:> I've been wondering too about how to get better ARM v6m compile-and-execute > testing going. > > As you say Jon, the non-execution-based regression tests are surprisingly > good at catching issues; but they're no full substitute for executing the > code produced by the backend for a reasonably-sized test suite. > > If somehow it would be possible to compile and run the LLVM test-suite > for v6m, I think that would be a good step forward. It would also allow > to get a buildbot going without too much effort continuously checking > basic correctness of v6m code generation. > > My guess is that the biggest hurdle would be to get linux or a similar > operating system going on a v6m/thumb1 simulator. Does anyone have an idea > if this is feasible or completely impossible? > >Not sure feasibility of the particular implementation, but qemu + linux works pretty well for testing other targets. -eric> Thanks, > > Kristof > > > -----Original Message----- > > From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] > > On Behalf Of Jonathan Roelofs > > Sent: 15 December 2014 18:31 > > To: Bjoern Haase; llvmdev at cs.uiuc.edu > > Subject: Re: [LLVMdev] Newbee question: LLVM backend regression tests > > for thumb1 targets on simulator possible? > > > > > > > > On 12/15/14 4:29 AM, Bjoern Haase wrote: > > > > > > Hello, > > > > > > as a newbee, I'd appreciate some support on regression test setup. > > > > > > Specifically, I am interrested in the feature of tail call > > > optimizations for the ARM v6m targets. This feature currently seems to > > > be completely deactivated at the moment (v6m being based on thumb1 > > > ?!). According to my code-reading, this feature will involve some > > > modifications in epilogue generation. > > > > > > My work on a gcc backend did show me that for a beginner like me, it > > > is rather likely that the first attempts will break something. > > :) > > > > > > Thus, being completely new to the llvm project I think that it's > > > essential to first establish a suitable regression test setup. My plan > > > is to run the tests for the v4t platform in thumb-only mode where free > > > simulators are available and which behaves (at least with respect to > > > the > > > epilogue) very much like v6m. > > Why not test your v6m changes on a Cortex-m0 QEMU? Semihosted > > applications are pretty easy to get going, something like: > > > > $ qemu-system-arm -semihosting -M integratorcp -cpu cortex-m0 -kernel > > a.out > > > > Thumv4t support is a bit spotty in llvm as it's not very well tested. > > The problem I run into most (and fixed several times), is that thumbv4t > > doesn't have a lo->lo mov instruction that doesn't clobber cpsr. That > > being said, it does work for the most part... just something to watch > > out for. > > > > > > From former work on gcc I'm used to the possibilty to run regression > > > tests in a simulator for the target platform (there with the expect > > > script mechanism). In the documentation on the web, I did, however, > > > not yet find information on a comparable feature for llvm. > > > > > > Therefore the specific questions: > > > > > > - Is there a mechanism to run backend testsuite runs in a simulation > > > framework and if yes, could you give me a hint on how to find > > > information > > Not that I know of, no. > > > > I routinely run the libc++ & libc++abi test suites in QEMU, but that's > > more of a whole-toolchain test, rather that just a backend test. You can > > get surprisingly far without simulator testing, so for most things, LLVM > > doesn't use it for testing. This makes it easier for any random > > developer to check out the llvm sources, build them, and run the > > testsuite, and not have to have simulators installed for all the > > different backends. > > > - If not, how is regression testing for the thumb1 targets currently > > > implemented? > > We use the LIT framework http://llvm.org/docs/CommandGuide/lit.html to > > test the compiler. Mostly this means feeding in llvm IR, and using > > FileCheck http://llvm.org/docs/CommandGuide/FileCheck.html (which is > > basically a glorified > > grep) to verify that the assembly produced is the same as what the test > > expects to be generated. > > > > To run these tests, use the 'make check-all' target from the build > > directory. > > > > > > Cheers, > > > > Jon > > > > > > Yours, > > > > > > Björn Haase > > > > > > P.S.: Of course, any hint with respect to sibling call optimization > > > would also be appreciated in a second step. I actually already found > > > some helpful information in the sources. > > > > > > _______________________________________________ > > > LLVM Developers mailing list > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > -- > > Jon Roelofs > > jonathan at codesourcery.com > > CodeSourcery / Mentor Embedded > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141217/8a5f547e/attachment.html>