llvm dev - Apr 2018 - Multi-architecture (ELF or other) loaders and dynamic linkers available?

Hi,

I'm looking for some sort of "universal loader and dynamic
linker",
capable of loading in memory an executable (ELF or other, the format
is not a requisite at this time) for any of the "major" architectures
that have LLVM backends (namely: x86, x86_64, ARM, ARM64, PPC, PPC64,
MIPS32, MIPS64), including also relocation, and loading of symbols
from dynamic libraries that the executable depends on.

The final memory representation of the executable should be "ready to
be executed". Before you ask "what OS?, what about syscalls?",
let's
assume the executable doesn't use any library from the OS, and that
there are only a couple of basic syscalls for terminal I/O only, and
that such syscalls are already hooked in advance by a tweaked C
runtime (i.e.: the executable won't use any syscall protocol, just
call hooks in the loader, thanks to the tweaked C runtime).

Does LLVM provide any infrastructure ready for doing this?

Or do you know of any other open source project that could be used for this?

After some weeks searching, my best matches have been:

1- Google NaCl : I think all or almost all of the architectures listed
above are supported. However, NaCl seems to be facing an abandonware
status (they're migrating to WebAssembly, but wasm cannot manage
native code from the architectures above if I'm reading the specs
correctly). Maybe NaCl would be the way to go, but their migration
plan makes me doubt.

2- The Linux compatibility layer in NetBSD. Its main difficulties
would be that not all of the 8 architectures listed above are
supported (I believe only x86, x86_64, ARM32 and PPC32 are supported),
and that it's more complex than necessary because it performs syscall
protocol emulation. Also another difficulty is that it depends on the
NetBSD kernel to work, but, anyway, it's a start, and it could
probably be done from it.

3- Today I just found a project I didn't know: Angr
(http://angr.io/index.html) which has CLE (a loader that stands for
"CLE Loads Everything", and that is capable of loading not only ELF,
but also PE, and some subset of Mach-o). However, I only gave a quick
glimpse at it, and I'm not really sure that CLE is capable of putting
the executable in memory so that it's "ready to be executed" (with
all
relocation, linked to required symbols from dynamic libraries,
etc...).

Would you recommend any of these 3 approaches that I found? Or do you
know of any other, better suited for the task?

Or maybe LLVM already has infrastructure for this?

Thanks!!

ardi

On 4/3/2018 9:33 AM, ardi via llvm-dev wrote:
> Hi,
>
> I'm looking for some sort of "universal loader and dynamic
linker",
> capable of loading in memory an executable (ELF or other, the format
> is not a requisite at this time) for any of the "major"
architectures
> that have LLVM backends (namely: x86, x86_64, ARM, ARM64, PPC, PPC64,
> MIPS32, MIPS64), including also relocation, and loading of symbols
> from dynamic libraries that the executable depends on.
>
> The final memory representation of the executable should be "ready to
> be executed". Before you ask "what OS?, what about
syscalls?", let's
> assume the executable doesn't use any library from the OS, and that
> there are only a couple of basic syscalls for terminal I/O only, and
> that such syscalls are already hooked in advance by a tweaked C
> runtime (i.e.: the executable won't use any syscall protocol, just
> call hooks in the loader, thanks to the tweaked C runtime).
>
> Does LLVM provide any infrastructure ready for doing this?
RuntimeDyld is a runtime linker which supports all the architectures you 
listed. And it works cross-target, which might be hard to find 
otherwise.  But it's meant for use with objects generated with MCJIT, 
not arbitrary executables, so it might be missing some features you 
need. See https://llvm.org/docs/MCJITDesignAndImplementation.html .

-Eli

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux
Foundation Collaborative Project

On Tue, Apr 3, 2018 at 7:58 PM, Friedman, Eli <efriedma at codeaurora.org>
wrote:
> On 4/3/2018 9:33 AM, ardi via llvm-dev wrote:
>>
>> Hi,
>>
>> I'm looking for some sort of "universal loader and dynamic
linker",
>> capable of loading in memory an executable (ELF or other, the format
>> is not a requisite at this time) for any of the "major"
architectures
>> that have LLVM backends (namely: x86, x86_64, ARM, ARM64, PPC, PPC64,
>> MIPS32, MIPS64), including also relocation, and loading of symbols
>> from dynamic libraries that the executable depends on.
>>
>> The final memory representation of the executable should be "ready
to
>> be executed". Before you ask "what OS?, what about
syscalls?", let's
>> assume the executable doesn't use any library from the OS, and that
>> there are only a couple of basic syscalls for terminal I/O only, and
>> that such syscalls are already hooked in advance by a tweaked C
>> runtime (i.e.: the executable won't use any syscall protocol, just
>> call hooks in the loader, thanks to the tweaked C runtime).
>>
>> Does LLVM provide any infrastructure ready for doing this?
>
>
> RuntimeDyld is a runtime linker which supports all the architectures you
> listed. And it works cross-target, which might be hard to find otherwise.
> But it's meant for use with objects generated with MCJIT, not arbitrary
> executables, so it might be missing some features you need. See
> https://llvm.org/docs/MCJITDesignAndImplementation.html .
Thanks a lot, Eli. Certainly, it has the functionality I need,
although I don't see it clear how I should create the needed data
objects from an executable file. The MCJIT obviously expects source,
but I guess there must be some way of "fooling" RuntimeDyld by parsing
an executable file (ELF or whatever), loading the proper data, and
creating the MCJIT objects that RuntimeDyld expects.

Anyway, I think RuntimeDyld is what I should use, as I'm using clang
as compiler, so with RuntimeDyld it's quite likely that things will
tend to work more easily than if I mix code from different systems and
compilers...

ardi