llvm dev - Dec 2014 - [LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?

> You have some special version of qemu? My default distribution's qemu
> package did not have a cortex-M0 nor has qemu head freshly taken from git:
The trouble with Cortex-M0 (from a virtualised testing perspective) is
that it doesn't have an MMU, so any "real" operating system
environment will be far more artificial than we'd like, and probably
very low down on qemu's list of priorities to boot.

Testing a Thumb1 CPU with an MMU (I'm sure there are ARM11* CPUs with
MMU but before Thumb2, but would need to dig into which is closest to
v6m) would cover *most* issues, but obviously not the ways in which
Cortex-M0 is closer to v7 than v6.

Tim.

On 18 December 2014 at 04:02, Tim Northover <t.p.northover at gmail.com>
wrote:
>> You have some special version of qemu? My default distribution's
qemu
>> package did not have a cortex-M0 nor has qemu head freshly taken from
git:
>
> The trouble with Cortex-M0 (from a virtualised testing perspective) is
> that it doesn't have an MMU, so any "real" operating system
> environment will be far more artificial than we'd like, and probably
> very low down on qemu's list of priorities to boot.
>
> Testing a Thumb1 CPU with an MMU (I'm sure there are ARM11* CPUs with
> MMU but before Thumb2, but would need to dig into which is closest to
> v6m) would cover *most* issues, but obviously not the ways in which
> Cortex-M0 is closer to v7 than v6.
Why not test on real hardware?

https://developer.mbed.org/platforms/DipCortex-M0/

cheers,
--renato

Am 19.12.2014 11:00, schrieb Renato Golin:
> On 18 December 2014 at 04:02, Tim Northover <t.p.northover at
gmail.com> wrote:
>>> You have some special version of qemu? My default
distribution's qemu
>>> package did not have a cortex-M0 nor has qemu head freshly taken
from git:
>> The trouble with Cortex-M0 (from a virtualised testing perspective) is
>> that it doesn't have an MMU, so any "real" operating
system
>> environment will be far more artificial than we'd like, and
probably
>> very low down on qemu's list of priorities to boot.
>>
>>
>> Why not test on real hardware?
>>
>> https://developer.mbed.org/platforms/DipCortex-M0/
>>
>> cheers,
>> --renato
>>
Of course, it would not be bad to run also some limited tests on 
real-world hardware. However, the answer why "why not"... is in my
opinion:

Because you will have big difficulties matching existing regression test 
code (designed for bigger targets) to a system being constrained to say 
16k Flash and 4k RAM. My experience with the testsuite of gcc being run 
on microcontrollers (AVR in this case) is that you encounter so many 
pseudo failures just because the test cases don't account for the 
limitations of the DUT that it's a tedious work to get any useful 
information out of the results. Imagine, e.g. a test that uses alloca 
and tests for correct stack pointer re-adjustment from frame pointers 
and just allocate two buffers of 2k RAM.

In my opinion there is more need for a regression test framework in a 
simulation environment in a first step. Of course an optional, second 
step with tests on real-world HW would not hurt. Actually qemu does not 
seem to be this bad for this purpose.

Björn.