Dan/Daniel/Eric,
I'm testing a bare-metal ARM toolchain, and I've hacked up my local copy
of lit
to get it to run libcxx tests on qemu. I wanted to pick your collective brains
to see if there was a better way of doing this.
What I have implemented is here (don't take this as a formal review request,
we
can do that later once we figure out the best direction to go):
lit part: http://reviews.llvm.org/D5014
libcxx part: http://reviews.llvm.org/D5015
Talking with Dan Albert earlier, he pointed out an existing review
(http://reviews.llvm.org/D4594) where he has implemented something vaguely
similar for Android. Looking at that, I've realized that maybe it makes
sense to
make an interface for the different actions of:
* Compiling the test
o Maybe the compiler isn't on the machine that runs the test?
o Maybe the compile jobs could be distributed to different machines?
* Running the test
o Perhaps the test needs to be sent out to a remote device?
o Perhaps the test can be scheduled on more than one remote device?
Then that interface could be implemented in a couple of generic ways:
* scp & ssh for remote hosts (or QEMU with an OS image)
* Wrapper around Android's adb
* Thin wrapper around "run it in this machine's shell" (what
we currently have)
What are your thoughts on this? Is this worth pursuing? What other use-cases can
we support by structuring LIT this way? I don't yet have a concrete proposal
for
what this would look like, but I can throw one together if there's interest.
Cheers,
Jon
--
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded
No specific comments at the moment, but I do want to say that I very much believe this is worth pursuing. Better testing infrastructure for remote targets will be an absolutely fantastic improvement. Thank you and Dan both for pushing forward on the topic. Jim> On Aug 21, 2014, at 5:42 PM, Jonathan Roelofs <jonathan at codesourcery.com> wrote: > > Dan/Daniel/Eric, > > I'm testing a bare-metal ARM toolchain, and I've hacked up my local copy of lit to get it to run libcxx tests on qemu. I wanted to pick your collective brains to see if there was a better way of doing this. > > What I have implemented is here (don't take this as a formal review request, we can do that later once we figure out the best direction to go): > lit part: http://reviews.llvm.org/D5014 > libcxx part: http://reviews.llvm.org/D5015 > > > > Talking with Dan Albert earlier, he pointed out an existing review (http://reviews.llvm.org/D4594) where he has implemented something vaguely similar for Android. Looking at that, I've realized that maybe it makes sense to make an interface for the different actions of: > * Compiling the test > o Maybe the compiler isn't on the machine that runs the test? > o Maybe the compile jobs could be distributed to different machines? > * Running the test > o Perhaps the test needs to be sent out to a remote device? > o Perhaps the test can be scheduled on more than one remote device? > > Then that interface could be implemented in a couple of generic ways: > * scp & ssh for remote hosts (or QEMU with an OS image) > * Wrapper around Android's adb > * Thin wrapper around "run it in this machine's shell" (what we currently have) > > What are your thoughts on this? Is this worth pursuing? What other use-cases can we support by structuring LIT this way? I don't yet have a concrete proposal for what this would look like, but I can throw one together if there's interest. > > > Cheers, > > Jon > > -- > Jon Roelofs > jonathan at codesourcery.com > CodeSourcery / Mentor Embedded > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On 22 August 2014 01:48, Jim Grosbach <grosbach at apple.com> wrote:> No specific comments at the moment, but I do want to say that I very much believe this is worth pursuing. Better testing infrastructure for remote targets will be an absolutely fantastic improvement. Thank you and Dan both for pushing forward on the topic.+1 I remember Greg (cc'd) did some of that in compiler-rt (or was it sanitizers?). He's not active in LLVM any more, but maybe you guys could share some thoughts. cheers, --renato
Hi Jon, I'm doing work in a similar vein as yours at the moment. We have several arm platforms (hardware and qemu) running either Linux or Android and want to run test-suite on them using x86->arm/aarch64 cross compilers dispatched via LNT. I've had some success bypassing lit and LNT by modifying RunSafely.sh within test-suite. It's likely this won't completely solve your problem but I think it would be a good idea to find the right path forward. I posted the patch I made in another thread on the list, titled under "test-suite RunSafely.sh remote execution with test copy". I'm using a variant on the method you outlined talking with Dan Albert to accomplish it. Like your code it's a bit of a hack, but I've gotten it to work provided the remote host has ssh and scp. I decided against the adb path for simplicity and consistency in the code modifications. Were a solution provided that had both ssh/scp and adb I'd be thrilled. Finally, I've done some work on getting Ubuntu 14.04LTS aarch64 running on Qemu but have not yet posted the instructions online. If you (or anyone) is interested please let me know and I'll make it a priority to document the steps I used. -Brian On Thu, Aug 21, 2014 at 7:42 PM, Jonathan Roelofs <jonathan at codesourcery.com> wrote:> Dan/Daniel/Eric, > > I'm testing a bare-metal ARM toolchain, and I've hacked up my local copy > of lit to get it to run libcxx tests on qemu. I wanted to pick your > collective brains to see if there was a better way of doing this. > > What I have implemented is here (don't take this as a formal review > request, we can do that later once we figure out the best direction to go): > lit part: http://reviews.llvm.org/D5014 > libcxx part: http://reviews.llvm.org/D5015 > > > > Talking with Dan Albert earlier, he pointed out an existing review ( > http://reviews.llvm.org/D4594) where he has implemented something vaguely > similar for Android. Looking at that, I've realized that maybe it makes > sense to make an interface for the different actions of: > * Compiling the test > o Maybe the compiler isn't on the machine that runs the test? > o Maybe the compile jobs could be distributed to different machines? > * Running the test > o Perhaps the test needs to be sent out to a remote device? > o Perhaps the test can be scheduled on more than one remote device? > > Then that interface could be implemented in a couple of generic ways: > * scp & ssh for remote hosts (or QEMU with an OS image) > * Wrapper around Android's adb > * Thin wrapper around "run it in this machine's shell" (what we > currently have) > > What are your thoughts on this? Is this worth pursuing? What other > use-cases can we support by structuring LIT this way? I don't yet have a > concrete proposal for what this would look like, but I can throw one > together if there's interest. > > > Cheers, > > Jon > > -- > Jon Roelofs > jonathan at codesourcery.com > CodeSourcery / Mentor Embedded > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140825/2fba1d99/attachment.html>
I hope this isn't considered spamming the list as I think it's relevant to QEMU and lit testing on ARM-64 platforms. After a few people contacted me for the instructions to setup Ubuntu aarch64 on QEMU I wrote them up here: http://rzycki.blogspot.com/2014/08/using-qemu-to-run-ubuntu-arm-64-bit.html Thanks, -Brian On Mon, Aug 25, 2014 at 10:57 AM, Brian M. Rzycki <brzycki at gmail.com> wrote:> Hi Jon, > > I'm doing work in a similar vein as yours at the moment. We have several > arm platforms (hardware and qemu) running either Linux or Android and want > to run test-suite on them using x86->arm/aarch64 cross compilers dispatched > via LNT. > > I've had some success bypassing lit and LNT by modifying RunSafely.sh > within test-suite. It's likely this won't completely solve your problem but > I think it would be a good idea to find the right path forward. I posted > the patch I made in another thread on the list, titled under "test-suite > RunSafely.sh remote execution with test copy". I'm using a variant on the > method you outlined talking with Dan Albert to accomplish it. Like your > code it's a bit of a hack, but I've gotten it to work provided the remote > host has ssh and scp. I decided against the adb path for simplicity and > consistency in the code modifications. Were a solution provided that had > both ssh/scp and adb I'd be thrilled. > > Finally, I've done some work on getting Ubuntu 14.04LTS aarch64 running on > Qemu but have not yet posted the instructions online. If you (or anyone) is > interested please let me know and I'll make it a priority to document the > steps I used. > > -Brian > > > On Thu, Aug 21, 2014 at 7:42 PM, Jonathan Roelofs < > jonathan at codesourcery.com> wrote: > >> Dan/Daniel/Eric, >> >> I'm testing a bare-metal ARM toolchain, and I've hacked up my local copy >> of lit to get it to run libcxx tests on qemu. I wanted to pick your >> collective brains to see if there was a better way of doing this. >> >> What I have implemented is here (don't take this as a formal review >> request, we can do that later once we figure out the best direction to go): >> lit part: http://reviews.llvm.org/D5014 >> libcxx part: http://reviews.llvm.org/D5015 >> >> >> >> Talking with Dan Albert earlier, he pointed out an existing review ( >> http://reviews.llvm.org/D4594) where he has implemented something >> vaguely similar for Android. Looking at that, I've realized that maybe it >> makes sense to make an interface for the different actions of: >> * Compiling the test >> o Maybe the compiler isn't on the machine that runs the test? >> o Maybe the compile jobs could be distributed to different machines? >> * Running the test >> o Perhaps the test needs to be sent out to a remote device? >> o Perhaps the test can be scheduled on more than one remote device? >> >> Then that interface could be implemented in a couple of generic ways: >> * scp & ssh for remote hosts (or QEMU with an OS image) >> * Wrapper around Android's adb >> * Thin wrapper around "run it in this machine's shell" (what we >> currently have) >> >> What are your thoughts on this? Is this worth pursuing? What other >> use-cases can we support by structuring LIT this way? I don't yet have a >> concrete proposal for what this would look like, but I can throw one >> together if there's interest. >> >> >> Cheers, >> >> Jon >> >> -- >> Jon Roelofs >> jonathan at codesourcery.com >> CodeSourcery / Mentor Embedded >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140825/76c4d6b8/attachment.html>