llvm dev - Feb 2018 - Is it possible to execute Objective-C code via LLVM JIT?

On 12 Feb 2018, at 22:31, Stanislav Pankevich via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Specifically I explored the latest objc4-723
> from Apple Open Source and it looks like all of the APIs that allow
> the registration of Objective-C classes, selectors, etc. are all very
> private.
The Objective-C runtime provides public APIs for doing all of this.  They’re
even documented.  They are also more or less standard and so work with all
runtime implementations, not just the Apple one.  I was using them for JIT’d
code on macOS and FreeBSD 10 years ago.
> One year ago you said you could help anyone interested in working on
> this. Let me check here again as a volunteer (if this work can ever be
> accomplished by someone outside Apple).
As I said in the earlier thread, the best way of doing this is to add a new
subclass of CGObjCRuntime that generates the code using the public APIs.  If
you’re running in the same process as the JIT, you could register the selectors
in the host environment and just inject the values as symbols (this is what I
did).  I’d be happy to help out someone who wants to do this.

David

On Tue, Feb 13, 2018 at 12:18 PM, David Chisnall
<David.Chisnall at cl.cam.ac.uk> wrote:
> On 12 Feb 2018, at 22:31, Stanislav Pankevich via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>
>> Specifically I explored the latest objc4-723
>> from Apple Open Source and it looks like all of the APIs that allow
>> the registration of Objective-C classes, selectors, etc. are all very
>> private.
>
> The Objective-C runtime provides public APIs for doing all of this. 
They’re even documented.  They are also more or less standard and so work with
all runtime implementations, not just the Apple one.  I was using them for JIT’d
code on macOS and FreeBSD 10 years ago.
Which methods are you referring to? For example of class registration,
do you mean objc_allocateClassPair/objc_registerClassPair or something
else?
>> One year ago you said you could help anyone interested in working on
>> this. Let me check here again as a volunteer (if this work can ever be
>> accomplished by someone outside Apple).
>
> As I said in the earlier thread, the best way of doing this is to add a new
subclass of CGObjCRuntime that generates the code using the public APIs.
Let me get this right. What clang::CodeGen:: CGObjCRuntime has to do
with this? My understanding of Lang's hint was that one has to extend
llvm's classes like RuntimeDyldMachO to parse Mach-O, find classes,
selectors, categories etc and register them all manually. Are you
saying that something has to be be added to CodeGen/*?
> If you’re running in the same process as the JIT, you could register the
selectors in the host environment and just inject the values as symbols (this is
what I did).  I’d be happy to help out someone who wants to do this.
It would be nice to get this working without embedding any of
Objective-C to the host process this is why I am particularly
interested in knowing how to do the work that objc4 does in the
methods such as: objc4/_objc_init, objc4/map_images_nolock and
objc4/_read_images.

My understanding of the goal is to make the lli example from this
thread working:
https://stackoverflow.com/questions/10375324/all-selectors-unrecognised-when-invoking-objective-c-methods-using-the-llvm-exec.

I would be happy to get a hint on which functions of Objective-C
Runtime's public API should I use to get that simple example working
in a quick and dirty way.

Thanks.

> On 13 Feb 2018, at 17:42, Stanislav Pankevich <s.pankevich at
gmail.com> wrote:
> 
> On Tue, Feb 13, 2018 at 12:18 PM, David Chisnall
> <David.Chisnall at cl.cam.ac.uk> wrote:
>> On 12 Feb 2018, at 22:31, Stanislav Pankevich via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>> 
>>> Specifically I explored the latest objc4-723
>>> from Apple Open Source and it looks like all of the APIs that allow
>>> the registration of Objective-C classes, selectors, etc. are all
very
>>> private.
>> 
>> The Objective-C runtime provides public APIs for doing all of this. 
They’re even documented.  They are also more or less standard and so work with
all runtime implementations, not just the Apple one.  I was using them for JIT’d
code on macOS and FreeBSD 10 years ago.
> 
> Which methods are you referring to? For example of class registration,
> do you mean objc_allocateClassPair/objc_registerClassPair or something
> else?
Yes, those set of APIs.  They provide an interface for building classes,
protocols, and so on.
>>> One year ago you said you could help anyone interested in working
on
>>> this. Let me check here again as a volunteer (if this work can ever
be
>>> accomplished by someone outside Apple).
>> 
>> As I said in the earlier thread, the best way of doing this is to add a
new subclass of CGObjCRuntime that generates the code using the public APIs.
> 
> Let me get this right. What clang::CodeGen:: CGObjCRuntime has to do
> with this? My understanding of Lang's hint was that one has to extend
> llvm's classes like RuntimeDyldMachO to parse Mach-O, find classes,
> selectors, categories etc and register them all manually. Are you
> saying that something has to be be added to CodeGen/*?
You have two options:

1) Hack up something in RuntimeDyldMachO to handle the data structures currently
generated by clang.  This is fragile, because the interface between the compiler
and the runtime is not documented, and is unique to each runtime.  This code
will be different on i386 and ARM, for example.

2) Create a new CGObjCRuntime subclass that creates a module init function that
constructs all of the classes using the public APIs, by adding something like
-fobjc-runtime=jit to the clang flags.  This is not particularly difficult and
means that the same code can be used with any Objective-C runtime.
>> If you’re running in the same process as the JIT, you could register
the selectors in the host environment and just inject the values as symbols
(this is what I did).  I’d be happy to help out someone who wants to do this.
> 
> It would be nice to get this working without embedding any of
> Objective-C to the host process this is
It’s an optimisation, not a compulsory part of the process.
> why I am particularly
> interested in knowing how to do the work that objc4 does in the
> methods such as: objc4/_objc_init, objc4/map_images_nolock and
> objc4/_read_images.
> 
> My understanding of the goal is to make the lli example from this
> thread working:
>
https://stackoverflow.com/questions/10375324/all-selectors-unrecognised-when-invoking-objective-c-methods-using-the-llvm-exec.
> 
> I would be happy to get a hint on which functions of Objective-C
> Runtime's public API should I use to get that simple example working
> in a quick and dirty way.
You seem to have decided that you want to use unmodified IR from a specific
version of Apple's Objective-C implementation.  I can’t help you there.

David