llvm dev - Aug 2014 - [LLVMdev] test-suite RunSafely.sh remote execution with test copy

Hello everyone,

I am working to run the LLVM Nightly Testsuite on embedded arm devices. Not
all of these devices can mount NFS to share a common directory tree. To
mitigate this I have patched test-suite/RunSafely.sh (attached) to create a
temporary directory, run the test remotely, collect the output, and cleanup
after itself. The patch breaks compatibility with the current remote device
model in the following ways:

1. uses scp/rcp to copy files from the host running test-suite instead of
assuming an identical tree on the remote device.
2. expects the remote device to have the mktemp command to create the
temporary workspace (usually under /tmp/*)
3. expects the remote device to have timeit installed in its path. If
timeit were compiled statically I could scp the one passed into
RunSafely.sh to the destination. Hosts with a different libc than the
tested compiler (Android, embedded uClibc devices) cannot run the
dynamically-linked timeit.
4. It's a bit slower due to all the network copies for the entire test
suite.

I'd appreciate feedback and suggestions as to how I can rework this model
for inclusion upstream. I'm sure there are others out there that would find
this kind of testing useful. I'm fine with a new remote-copy mode that can
be passed in via lnt nt command line invocation.

I hope others find this useful.
-Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140815/ae71ed76/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RunSafely-rhost-copy.patch
Type: application/octet-stream
Size: 2616 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140815/ae71ed76/attachment.obj>

On Aug 15, 2014 10:37 PM, "Brian M. Rzycki" <brzycki at
gmail.com> wrote:
>
> Hello everyone,
>
> I am working to run the LLVM Nightly Testsuite on embedded arm devices.
Not all of these devices can mount NFS to share a common directory tree. To
mitigate this I have patched test-suite/RunSafely.sh (attached) to create a
temporary directory, run the test remotely, collect the output, and cleanup
after itself. The patch breaks compatibility with the current remote device
model in the following ways:
>
> 1. uses scp/rcp to copy files from the host running test-suite instead of
assuming an identical tree on the remote device.
> 2. expects the remote device to have the mktemp command to create the
temporary workspace (usually under /tmp/*)
> 3. expects the remote device to have timeit installed in its path. If
timeit were compiled statically I could scp the one passed into
RunSafely.sh to the destination. Hosts with a different libc than the
tested compiler (Android, embedded uClibc devices) cannot run the
dynamically-linked timeit.
> 4. It's a bit slower due to all the network copies for the entire test
suite.
>
> I'd appreciate feedback and suggestions as to how I can rework this
model
for inclusion upstream. I'm sure there are others out there that would find
this kind of testing useful. I'm fine with a new remote-copy mode that can
be passed in via lnt nt command line invocation.
>
> I hope others find this useful.
> -Brian
Have you tried sshfs.
I does much the same as nfs mount but uses ssh instead.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140816/813d7d54/attachment.html>

Hi James,
Thanks for the feedback. Yes I did look into sshfs and It's a great choice
for devices running Linux distributions. Unfortunately some of our devices
are Android based and have no plans for Linux support. I started examining
cross-compiling libfuse for Android but ran into complications there due to
the very limited nature of the device's root filesystem libraries,
binaries, and compiled kernel options.

It was much simpler cross-compiling dropbear + scp for Android which lead
to the relatively simple scp patch in RunSafely.sh.

I will take another stab at sshfs to see if there may be a way to get it
working on Android with minimal pain.

I'd be curious to know if others doing similar work rely mostly on NFS,
SMB, sshfs or some other network filesystem to accomplish remote execution
testing.

-Brian


On Sat, Aug 16, 2014 at 1:29 AM, James Courtier-Dutton <
james.dutton at gmail.com> wrote:
> On Aug 15, 2014 10:37 PM, "Brian M. Rzycki" <brzycki at
gmail.com> wrote:
> >
> > Hello everyone,
> >
> > I am working to run the LLVM Nightly Testsuite on embedded arm
devices.
> Not all of these devices can mount NFS to share a common directory tree. To
> mitigate this I have patched test-suite/RunSafely.sh (attached) to create a
> temporary directory, run the test remotely, collect the output, and cleanup
> after itself. The patch breaks compatibility with the current remote device
> model in the following ways:
> >
> > 1. uses scp/rcp to copy files from the host running test-suite instead
> of assuming an identical tree on the remote device.
> > 2. expects the remote device to have the mktemp command to create the
> temporary workspace (usually under /tmp/*)
> > 3. expects the remote device to have timeit installed in its path. If
> timeit were compiled statically I could scp the one passed into
> RunSafely.sh to the destination. Hosts with a different libc than the
> tested compiler (Android, embedded uClibc devices) cannot run the
> dynamically-linked timeit.
> > 4. It's a bit slower due to all the network copies for the entire
test
> suite.
> >
> > I'd appreciate feedback and suggestions as to how I can rework
this
> model for inclusion upstream. I'm sure there are others out there that
> would find this kind of testing useful. I'm fine with a new remote-copy
> mode that can be passed in via lnt nt command line invocation.
> >
> > I hope others find this useful.
> > -Brian
>
> Have you tried sshfs.
> I does much the same as nfs mount but uses ssh instead.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140818/297b4d65/attachment.html>