[LLVMdev] JIT and libgcc_s.so

Jim Grosbach
Tue, 02 Oct 2012 17:14:08 -0000

On Oct 2, 2012, at 12:39 AM, James Molloy <James.Molloy at arm.com> wrote:

> Hi Eric,
>
> This makes total sense, thanks.
>
> One thing though; it seems a lot of functionality is being foisted onto
> the client. This is all well and good, but it seems that clients are
> expected to reinvent the wheel quite a bit for no real purpose - an
> example is the implementation of getPointerToNamedFunction,
> allocate{Code,Data}Section and invalidateInstructionCache in lli.cpp.
>
> Many clients will just want "a standard implementation that works but
> can be customised" for many of these functions, and this is currently
> not available. So they have to copy-paste code from lli.cpp and keep it
> in sync when bugs gets found.
>
> Would it not be nicer to hoist this behaviour that could be customised
> but will suit most people's needs (when JITting on host on a unix OS)
> into some superclass that could be subclassed in a client?
>

That's the end-goal, yes. Right now the focus is on making the division of
responsibility a very clear one and get the MCJIT functionality fleshed out.
Doing lots of both at once leads to intermingling of functionality out of
convenience. For example, consider the many patches proposed that are rejected
due to violations of the client/target layering. By making all clients do some
work now, it helps ensure everyone thinks about the needs of the broader
architecture, not just what will make things work for their use-case (which, as
you say, is typically a self-hosted Unix-ish environment).

Put another way, the old JIT intermingled lots of host and client logic, and
that was one of its major design flaws. We're fixing that, but it's
going to take a while for everyone's thinking to adjust and internalize the
new model. That's totally reasonable. The idea is to structure the code and
enforce divisions of labor in the code via client vs. library that help
reinforce the new, better layered, design.

If you really want to get things moving on this sooner rather than later, a step
in that direction would be to flesh out the lli support for the
"-remote-mcjit" memory manager so that all of the ExecutionEngine
tests pass with that enabled. As it stands, that's primarily good support
for external symbol resolution, I believe, perhaps with a bit of additional
relocation support. If we can turn that on for testing in all regression test
runs, that will help identify at "make check" time any gratuitous
layering violations, as they often don't survive the address translations
involved. Then we can start building up a set of memory managers that support
the common cases that can be plug-and-play used by MCJIT clients.

-Jim

> Cheers,
>
> James
>
> On Mon, 2012-10-01 at 21:16 +0100, Eric Christopher wrote:
>> So this is likely just an accident rather than on purpose. There's
>> totally room for that to happen, but it'll be the job of the client
>> and not MCJIT itself.
>>
>> Basically whomever should call dlopen if they want to and it's the
>> problem of the client application (which could be lli as the canonical
>> mcjit example) to link in the correct bits.
>>
>> Make sense? If you can see a use case in particular that would have
>> this functionality being linked into mcjit we'd love to hear it,
but I
>> haven't been able to come up with one.
>>
>> -eric
>>
>> On Mon, Oct 1, 2012 at 10:06 AM, Jim Grosbach <grosbach at
apple.com> wrote:
>>> Hi James,
>>>
>>> In that scenario, it would be the responsibility of the client to
implement a memory manager for the MCJIT that knows how to import those symbols
from the relevant shared library (or resolve them directly to the statically
compiled symbols).
>>>
>>> -Jim
>>>
>>> On Oct 1, 2012, at 3:09 AM, James Molloy <James.Molloy at
arm.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> There are symbols in libgcc (and compiler-rt) that JIT-compiled
modules
>>>> may need. These are currently linked correctly because lli and
friends
>>>> are linked against libgcc_s.so (i.e. shared library version of
libgcc,
>>>> so dlopen/dlsym works).
>>>>
>>>> However if a consumer links statically, dlsym won't find
all compiler
>>>> required functions (only the ones that were required by the
JIT-compiler
>>>> itself will be linked).
>>>>
>>>> So the question is, do we want to support users of the MCJIT
linking
>>>> statically to libgcc/compiler-rt? Or explicitly not handle this
>>>> use-case?
>>>>
>>>> Cheers,
>>>>
>>>> James
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
>
>