llvm dev - Sep 2014 - [LLVMdev] QEMU testing for LIT execution tests

On 22 August 2014 01:48, Jim Grosbach <grosbach at apple.com>
wrote:
> No specific comments at the moment, but I do want to say that I very much
believe this is worth pursuing. Better testing infrastructure for remote targets
will be an absolutely fantastic improvement. Thank you and Dan both for pushing
forward on the topic.
+1

I remember Greg  (cc'd) did some of that in compiler-rt (or was it
sanitizers?). He's not active in LLVM any more, but maybe you guys
could share some thoughts.

cheers,
--renato

Hi Jon,

+1 on adding support to run cross-compiled libcxx tests.  +Marshall

To add cross-compilation support to the compiler-rt test suite, I
added '%run' variable to the 'RUN:' lines in each lit test.  For
example:

https://github.com/llvm-mirror/compiler-rt/blob/master/test/asan/TestCases/atexit_stats.cc


and substituted '%run' with 'config.emulator' in the lit
configuration file:

https://github.com/llvm-mirror/compiler-rt/blob/master/test/lit.common.cfg


which is set via CMake option COMPILER_RT_EMULATOR:

https://github.com/llvm-mirror/compiler-rt/blob/master/test/lit.common.configured.in


To configure the build to use QEMU (or adb, scp, etc):

$ cmake -DCOMPILER_RT_EMULATOR=qemu-aarch64

If you need additional parameters, you can either add them to the
COMPILER_RT_EMULATOR string or add a wrapper shell script.


To ensure new tests include '%run', I added the script
'litlint.py'
and invoke it within the CMake build:

https://github.com/llvm-mirror/compiler-rt/blob/master/lib/sanitizer_common/scripts/litlint.py


Let me know if you have any questions.  Feel free to include me on
Phabricator reviews.

-Greg


On Fri, Aug 22, 2014 at 7:30 AM, Renato Golin <renato.golin at linaro.org>
wrote:
> On 22 August 2014 01:48, Jim Grosbach <grosbach at apple.com> wrote:
>> No specific comments at the moment, but I do want to say that I very
much believe this is worth pursuing. Better testing infrastructure for remote
targets will be an absolutely fantastic improvement. Thank you and Dan both for
pushing forward on the topic.
>
> +1
>
> I remember Greg  (cc'd) did some of that in compiler-rt (or was it
> sanitizers?). He's not active in LLVM any more, but maybe you guys
> could share some thoughts.
>
> cheers,
> --renato

So I started prototyping remote testing for libcxx over the weekend:
   https://github.com/jroelofs/libcxx/tree/remote_test

The SSHExecutor isn't quite finished yet, but this should give an idea of
what
I've got in mind.

Does this look generic enough to work for other projects under the llvm-umbrella
(LNT, test-suite, compiler_rt, libcxxabi, etc)?  Are there use cases that you 
see where this will not work?


Jon

On 8/25/14, 6:48 PM, Greg Fitzgerald wrote:
> Hi Jon,
>
> +1 on adding support to run cross-compiled libcxx tests.  +Marshall
>
> To add cross-compilation support to the compiler-rt test suite, I
> added '%run' variable to the 'RUN:' lines in each lit test.
For
> example:
>
>
https://github.com/llvm-mirror/compiler-rt/blob/master/test/asan/TestCases/atexit_stats.cc
>
>
> and substituted '%run' with 'config.emulator' in the lit
configuration file:
>
> https://github.com/llvm-mirror/compiler-rt/blob/master/test/lit.common.cfg
>
>
> which is set via CMake option COMPILER_RT_EMULATOR:
>
>
https://github.com/llvm-mirror/compiler-rt/blob/master/test/lit.common.configured.in
>
>
> To configure the build to use QEMU (or adb, scp, etc):
>
> $ cmake -DCOMPILER_RT_EMULATOR=qemu-aarch64
>
> If you need additional parameters, you can either add them to the
> COMPILER_RT_EMULATOR string or add a wrapper shell script.
>
>
> To ensure new tests include '%run', I added the script
'litlint.py'
> and invoke it within the CMake build:
>
>
https://github.com/llvm-mirror/compiler-rt/blob/master/lib/sanitizer_common/scripts/litlint.py
>
>
> Let me know if you have any questions.  Feel free to include me on
> Phabricator reviews.
>
> -Greg
>
>
> On Fri, Aug 22, 2014 at 7:30 AM, Renato Golin <renato.golin at
linaro.org> wrote:
>> On 22 August 2014 01:48, Jim Grosbach <grosbach at apple.com>
wrote:
>>> No specific comments at the moment, but I do want to say that I
very much believe this is worth pursuing. Better testing infrastructure for
remote targets will be an absolutely fantastic improvement. Thank you and Dan
both for pushing forward on the topic.
>>
>> +1
>>
>> I remember Greg  (cc'd) did some of that in compiler-rt (or was it
>> sanitizers?). He's not active in LLVM any more, but maybe you guys
>> could share some thoughts.
>>
>> cheers,
>> --renato
-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded