[llvm-dev] Wrapping lit commands for device testing

George Karpenkov via llvm-dev
Mon, 23 Apr 2018 14:09:42 -0000

Hi,

Tests in compiler-rt run under both iOS simulators and Android devices, which
requires “wrapping” the run line with the “adb” command for Android, and adding
a prefix / setting some environment variables for simulators/
In case of Android this is done by changing the compilation command to produce a
wrapper which already calls “adb”, and in case of iOS simulators by explicitly
adding the “%run” command in front of run lines.
The solution is far from perfect though: many tests (e.g. libFuzzer) extensively
rely on using arbitrary shell commands to create and manipulate temporary text
files.
Performing those commands in a different environment is done in a very ad-hoc
manner: from what I see, for sanitizer tests those temporary files are then
usually references in the environment variables present in the run command,
which are then parsed to carry those files into the executable environment.
Doing this for libFuzzer would be very difficult and would require a lot of
modifications to test.

I would prefer a more transparent approach: if LIT configuration was to provide
a hook to perform arbitrary replacements/wrapping on each of the executed lines,
the wrapping could be performed without modifying tests at all.

My implementation preference would be adding a method  `def wrap(self, s)` (str
-> str) to the format class (lit/formats/*), and then using it (if non-empty)
on each line being run.
I think that is preferable to a method on a `config` object, since line
transformation is a very heavyweight operation, akin to other methods available
on the `format` class (test discovery semantics and test execution semantics).
Would there be any objections against such an implementation plan?

Regards,
George