llvm dev - Jul 2012 - [LLVMdev] RFC: MCJIT enhancements

Following up and expanding on an earlier conversation, I'd like to discuss
making several non-trivial changes to the MCJIT engine and related objects. 
There may be some interdependencies between these changes, but I think that they
can be logically grouped as follows:

* Lazy module compilation

* Enhance the JIT memory manager interface to enable section-based memory
protection

* Clean up object/memory ownership problems

* Introduce a mechanism for caching generated objects

* Support JIT events for MCJIT

* Support ELF generation on Windows


Now let me expand on each of these a bit.

---------------------
Lazy compilation
---------------------

Right now a Module is passed to the EngineBuilder::create call and is compiled
immediately within the MCJIT constructor.  I don't believe it is possible to
perform lazy compilation on a per function basis as was done in the legacy JIT
interface, but there's no reason that module compilation and loading
can't be deferred until a function is requested.  I think this is a fairly
straight-forward and non-controversial change.


----------------------------------
Memory manager interface
----------------------------------

The memory manager changes are a bit more complicated.  In the current MCJIT
implementation, MCJIT clients may specify a memory manager through the
EngineBuilder and this memory manager will be passed along to the MCJIT engine. 
The  JITMemoryManager interface was originally created to suit the needs of the
legacy JIT engine.  As the MCJIT engine has come into existence, its memory
needs have been somewhat wedged in to the old JITMemoryManager interface.  The
result has been that there are effectively two interfaces existing side-by-side
in the same abstract base class and implementations will typically only support
one or the other set of functions.  This is further complicated by the
RuntimeDyldMemoryManager, which doesn't inherit from the JITMemoryManager
but does share a subset of functions and is implemented as a wrapper around the
client-supplied memory manager in the current MCJIT code.  There's also an
MCJITmemoryManager class which derives from RuntimeDyld and is used by MCJIT but
isn't exposed to the client at all.

I'd like to see this situation cleaned up as we move forward with a way for
the client to specify an MCJITMemoryManager directly and a separation of the
legacy memory manager interface from the new interface.  In addition, I'd
like the new memory manager interface to be extended with the functions that the
RuntimeDyld needs to manage setting of section-specific permissions including
non-writable code and read-only data.


----------------------
Object ownership
----------------------

Within the current MCJIT implementation there are a number of ugly cross
dependencies in object/memory life cycle, making the
MCJIT/RuntimeDyld/JITMemoryManager relationship fragile and very dependent upon
a particular order of object destruction.

In the current implementation, the buffer into which code is generated is owned
by the MCJIT component, but when sections from the generated code are loaded,
they are loaded into memory owned by the JITMemoryManager.  During object
loading, an ObjectFile is created within the RuntimeDyld which references both
of these buffers.  The situation is further compilicated in the case where the
RuntimeDyldELF object attempts to register the generated object with GDB,
because the GDB-interface requires a reference to both memory buffers.  The
GDB_required references are currently maintained in an ObjectImage instance
which is held by RuntimeDyldELF.

For those who are visually oriented, I am attaching a diagram which shows the
object relationships.

I would like to change this by introducing an ObjectBuffer which would be
allocated by the MCJIT object at compilation time and then passed to the
RuntimeDyld::loadObject.  RuntimeDyld::loadObject would hand this ObjectBuffer
off to the new ObjectImage instance (which it already creates today).  The
ObjectImage would be returned from the RuntimeDyld::loadObject to MCJIT and
MCJIT would own that object.

Again, a diagram is attached.


-------------------
Object caching
-------------------

I would like to introduce a mechanism whereby JITed objects could be cached and
loaded from cache at a future time.  It seems that the caching should logically
be performed transparently within the MCJIT engine, but it also seems that the
caching mechanism will need to be provided by the client (or at least under
client control).  I think that the best way to accomplish this is the introduce
a new interface (ObjectManager?), similar to the memory manager, that can be
specified by the client and used by the MCJIT engine.  What I have in mind is
that when MCJIT is about to compile a module, it will first check with the
ObjectManager to see if a pre-compiled image is available.  If so, it will
simply pass that image to the RuntimeDyld for loading.  If not, it will compile
as usual.  Likewise, after compilation (but before loading) MCJIT would offer
the compiled image to the ObjectManager and the ObjectManager could save the
image to a file cache (or whatever).


-----------------
MCJIT Events
-----------------

The legacy JIT interface supported a JITEventListener interface that would
provide notification when new functions were JITed.  That interface was used to
enable source-level profiling of JITed code.  There is currently no equivalent
for the MCJIT engine.  I'd like to correct that.

In order to get to the function level, some code somewhere will need to parse
the format-specific object code that is generated.  I think that it makes sense
for that to be pushed to the event listener itself since some listeners may be
satisfied with the raw object itself as input.

Most of the support necessary to parse object images to obtain source-level
debug information is already present in LLVM somewhere, though not all of it is
handily exposed through interfaces in the include tree.  In order to implement a
default event listener that provides the basic functionality of the existing JIT
event listeners, some extensions to the existing DIContext and related DebugInfo
classes will be necessary.  The development of an interface to expose more DWARF
information may prove worthy of its own discussion, as it would also enable
better testing, but for now I'll just mention it as a requirement here.

It may make sense to subsume the object loading event under the ObjectManager
interface described in the "object caching" section above rather than
overloading the existing JITEventListener interface within MCJIT, as the other
events in that interface don't properly apply to MCJIT.


-------------------------------
ELF Support on Windows
-------------------------------

There are various reasons that it would be nice to be able to support generation
of ELF objects on Windows through the MCJIT interface, one of which is that it
would allow users to debug JITed code with GDB (i.e. MinGW).  There was a
proposal by Eli Bendersky to enable ELF generation on Windows by extending the
target triple handling.  We have been using that approach, and it works. 
However, at the time it was proposed there seemed to be a general feeling that
it would be preferable to have a general way to specify a non-default object
format for any platform rather than a Windows/ELF-specific extension.

The original discussion can be found here:

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120130/136053.html

Unfortunately, I don't believe a consensus was ever reached as to what the
correct way to implement this would be.  I'd like to re-open that discussion
now so that we can find a solution that would be generally accepted for
inclusion in LLVM.

We can, of course, discuss any and all of these in more details as the
implementation unfolds, but I wanted to get all of it out there first as a sort
of road map to get general input on the overall direction of things in the MCJIT
space.

Thanks to all who took the time to read this, and thank you in advance for your
feedback.

-Andy

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120724/67098657/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mcjit-current-ownership.png
Type: image/png
Size: 40537 bytes
Desc: mcjit-current-ownership.png
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120724/67098657/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mcjit-proposed-ownership.png
Type: image/png
Size: 35977 bytes
Desc: mcjit-proposed-ownership.png
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120724/67098657/attachment-0001.png>

Hi Andy,

This all sounds fantastic. I'm sure we'll have some spirited discussions
about the details, but I completely agree with the approach and goals. Good
stuff, and thanks for working on this!

-Jim

On Jul 24, 2012, at 3:24 PM, "Kaylor, Andrew" <andrew.kaylor at
intel.com> wrote:
> Following up and expanding on an earlier conversation, I'd like to
discuss making several non-trivial changes to the MCJIT engine and related
objects.  There may be some interdependencies between these changes, but I think
that they can be logically grouped as follows:
>  
> * Lazy module compilation
>  
> * Enhance the JIT memory manager interface to enable section-based memory
protection
>  
> * Clean up object/memory ownership problems
>  
> * Introduce a mechanism for caching generated objects
>  
> * Support JIT events for MCJIT
>  
> * Support ELF generation on Windows
>  
>  
> Now let me expand on each of these a bit.
>  
> ---------------------
> Lazy compilation
> ---------------------
>  
> Right now a Module is passed to the EngineBuilder::create call and is
compiled immediately within the MCJIT constructor.  I don't believe it is
possible to perform lazy compilation on a per function basis as was done in the
legacy JIT interface, but there's no reason that module compilation and
loading can't be deferred until a function is requested.  I think this is a
fairly straight-forward and non-controversial change.
>  
Yep, also effectively a pre-requisite for meaningful lazy JITing of multiple
modules.
>  
> ----------------------------------
> Memory manager interface
> ----------------------------------
>  
> The memory manager changes are a bit more complicated.  In the current
MCJIT implementation, MCJIT clients may specify a memory manager through the
EngineBuilder and this memory manager will be passed along to the MCJIT engine. 
The  JITMemoryManager interface was originally created to suit the needs of the
legacy JIT engine.  As the MCJIT engine has come into existence, its memory
needs have been somewhat wedged in to the old JITMemoryManager interface.  The
result has been that there are effectively two interfaces existing side-by-side
in the same abstract base class and implementations will typically only support
one or the other set of functions.  This is further complicated by the
RuntimeDyldMemoryManager, which doesn't inherit from the JITMemoryManager
but does share a subset of functions and is implemented as a wrapper around the
client-supplied memory manager in the current MCJIT code.  There's also an
MCJITmemoryManager class which derives from RuntimeDyld and is used by MCJIT but
isn't exposed to the client at all.
>  
> I'd like to see this situation cleaned up as we move forward with a way
for the client to specify an MCJITMemoryManager directly and a separation of the
legacy memory manager interface from the new interface.  In addition, I'd
like the new memory manager interface to be extended with the functions that the
RuntimeDyld needs to manage setting of section-specific permissions including
non-writable code and read-only data.
The current situation is a bit ad-hoc as we try to minimize the impact of MCJIT
bring up on any clients of the old JIT. Now that the MCJIT is becoming more
mature, we can start being a bit more aggressive about that and making the
interfaces conform to what the MCJIT really needs.
>  
>  
> ----------------------
> Object ownership
> ----------------------
>  
> Within the current MCJIT implementation there are a number of ugly cross
dependencies in object/memory life cycle, making the
MCJIT/RuntimeDyld/JITMemoryManager relationship fragile and very dependent upon
a particular order of object destruction.
>  
> In the current implementation, the buffer into which code is generated is
owned by the MCJIT component, but when sections from the generated code are
loaded, they are loaded into memory owned by the JITMemoryManager.  During
object loading, an ObjectFile is created within the RuntimeDyld which references
both of these buffers.  The situation is further compilicated in the case where
the RuntimeDyldELF object attempts to register the generated object with GDB,
because the GDB-interface requires a reference to both memory buffers.  The
GDB_required references are currently maintained in an ObjectImage instance
which is held by RuntimeDyldELF.
>  
> For those who are visually oriented, I am attaching a diagram which shows
the object relationships.
>  
> I would like to change this by introducing an ObjectBuffer which would be
allocated by the MCJIT object at compilation time and then passed to the
RuntimeDyld::loadObject.  RuntimeDyld::loadObject would hand this ObjectBuffer
off to the new ObjectImage instance (which it already creates today).  The
ObjectImage would be returned from the RuntimeDyld::loadObject to MCJIT and
MCJIT would own that object.
>  
> Again, a diagram is attached.
>  
>  
Sounds reasonable.
> -------------------
> Object caching
> -------------------
>  
> I would like to introduce a mechanism whereby JITed objects could be cached
and loaded from cache at a future time.  It seems that the caching should
logically be performed transparently within the MCJIT engine, but it also seems
that the caching mechanism will need to be provided by the client (or at least
under client control).  I think that the best way to accomplish this is the
introduce a new interface (ObjectManager?), similar to the memory manager, that
can be specified by the client and used by the MCJIT engine.  What I have in
mind is that when MCJIT is about to compile a module, it will first check with
the ObjectManager to see if a pre-compiled image is available.  If so, it will
simply pass that image to the RuntimeDyld for loading.  If not, it will compile
as usual.  Likewise, after compilation (but before loading) MCJIT would offer
the compiled image to the ObjectManager and the ObjectManager could save the
image to a file cache (or whatever).
>  
>  
> -----------------
> MCJIT Events
> -----------------
>  
> The legacy JIT interface supported a JITEventListener interface that would
provide notification when new functions were JITed.  That interface was used to
enable source-level profiling of JITed code.  There is currently no equivalent
for the MCJIT engine.  I'd like to correct that.
>  
> In order to get to the function level, some code somewhere will need to
parse the format-specific object code that is generated.  I think that it makes
sense for that to be pushed to the event listener itself since some listeners
may be satisfied with the raw object itself as input.
>  
> Most of the support necessary to parse object images to obtain source-level
debug information is already present in LLVM somewhere, though not all of it is
handily exposed through interfaces in the include tree.  In order to implement a
default event listener that provides the basic functionality of the existing JIT
event listeners, some extensions to the existing DIContext and related DebugInfo
classes will be necessary.  The development of an interface to expose more DWARF
information may prove worthy of its own discussion, as it would also enable
better testing, but for now I'll just mention it as a requirement here.
>  
> It may make sense to subsume the object loading event under the
ObjectManager interface described in the "object caching" section
above rather than overloading the existing JITEventListener interface within
MCJIT, as the other events in that interface don't properly apply to MCJIT.
>  
>  
> -------------------------------
> ELF Support on Windows
> -------------------------------
>  
> There are various reasons that it would be nice to be able to support
generation of ELF objects on Windows through the MCJIT interface, one of which
is that it would allow users to debug JITed code with GDB (i.e. MinGW).  There
was a proposal by Eli Bendersky to enable ELF generation on Windows by extending
the target triple handling.  We have been using that approach, and it works. 
However, at the time it was proposed there seemed to be a general feeling that
it would be preferable to have a general way to specify a non-default object
format for any platform rather than a Windows/ELF-specific extension.
>   
> The original discussion can be found here:
>  
>
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120130/136053.html
>  
> Unfortunately, I don't believe a consensus was ever reached as to what
the correct way to implement this would be.  I'd like to re-open that
discussion now so that we can find a solution that would be generally accepted
for inclusion in LLVM.
>  
> We can, of course, discuss any and all of these in more details as the
implementation unfolds, but I wanted to get all of it out there first as a sort
of road map to get general input on the overall direction of things in the MCJIT
space.
>  
> Thanks to all who took the time to read this, and thank you in advance for
your feedback.
>  
> -Andy
>  
>
<mcjit-current-ownership.png><mcjit-proposed-ownership.png>_______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120726/be639cda/attachment.html>

I have some vision about Object Caching.

A client, which is interested in object caching, will need some help from
compiler and LLVM side. The object file, located on disk, should contain
information that allows to decide – rebuild or not rebuild.
The client (a backend engine, which translates IR to JIT) compares data from
object file, found on disk  and module IR.
The information that object file may contain I call “Signature”. It can be
stored in “.comment” section.
Signature includes:

·         Name of source file, full path. (It may be .ll or .cl or .cpp or any
other extension, the compilation was started from)

·         Time stamp of the source file.

·         Name and version of the compiler.

·         Compilation flags.

·         Environment variables (all or those who affect compilation process)
The compiler can generate this data and put it in metadata.

If we’ll take Intel OpenCL product, the clang can generate this special
metadata. MCJIT, while building an object file, will compose a “ .comment” with
a signature, described above.


Ø  I would like to introduce a mechanism whereby JITed objects could be cached
and loaded from cache at a future time.  It seems that the caching should
logically be performed transparently within the MCJIT engine, but it also seems
that the caching mechanism will need to be provided by the client (or at least
under client control).  I think that the best way to accomplish this is the
introduce a new interface (ObjectManager?), similar to the memory manager, that
can be specified by the client and used by the MCJIT engine.  What I have in
mind is that when MCJIT is about to compile a module, it will first check with
the ObjectManager to see if a pre-compiled image is available.  If so, it will
simply pass that image to the RuntimeDyld for loading.  If not, it will compile
as usual.  Likewise, after compilation (but before loading) MCJIT would offer
the compiled image to the ObjectManager and the ObjectManager could save the
image to a file cache (or whatever).


- Elena
From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
Behalf Of Kaylor, Andrew
Sent: Wednesday, July 25, 2012 01:25
To: llvmdev at cs.uiuc.edu
Subject: [LLVMdev] RFC: MCJIT enhancements

Following up and expanding on an earlier conversation, I'd like to discuss
making several non-trivial changes to the MCJIT engine and related objects. 
There may be some interdependencies between these changes, but I think that they
can be logically grouped as follows:

* Lazy module compilation

* Enhance the JIT memory manager interface to enable section-based memory
protection

* Clean up object/memory ownership problems

* Introduce a mechanism for caching generated objects

* Support JIT events for MCJIT

* Support ELF generation on Windows


Now let me expand on each of these a bit.

---------------------
Lazy compilation
---------------------

Right now a Module is passed to the EngineBuilder::create call and is compiled
immediately within the MCJIT constructor.  I don't believe it is possible to
perform lazy compilation on a per function basis as was done in the legacy JIT
interface, but there's no reason that module compilation and loading
can't be deferred until a function is requested.  I think this is a fairly
straight-forward and non-controversial change.


----------------------------------
Memory manager interface
----------------------------------

The memory manager changes are a bit more complicated.  In the current MCJIT
implementation, MCJIT clients may specify a memory manager through the
EngineBuilder and this memory manager will be passed along to the MCJIT engine. 
The  JITMemoryManager interface was originally created to suit the needs of the
legacy JIT engine.  As the MCJIT engine has come into existence, its memory
needs have been somewhat wedged in to the old JITMemoryManager interface.  The
result has been that there are effectively two interfaces existing side-by-side
in the same abstract base class and implementations will typically only support
one or the other set of functions.  This is further complicated by the
RuntimeDyldMemoryManager, which doesn't inherit from the JITMemoryManager
but does share a subset of functions and is implemented as a wrapper around the
client-supplied memory manager in the current MCJIT code.  There's also an
MCJITmemoryManager class which derives from RuntimeDyld and is used by MCJIT but
isn't exposed to the client at all.

I'd like to see this situation cleaned up as we move forward with a way for
the client to specify an MCJITMemoryManager directly and a separation of the
legacy memory manager interface from the new interface.  In addition, I'd
like the new memory manager interface to be extended with the functions that the
RuntimeDyld needs to manage setting of section-specific permissions including
non-writable code and read-only data.


----------------------
Object ownership
----------------------

Within the current MCJIT implementation there are a number of ugly cross
dependencies in object/memory life cycle, making the
MCJIT/RuntimeDyld/JITMemoryManager relationship fragile and very dependent upon
a particular order of object destruction.

In the current implementation, the buffer into which code is generated is owned
by the MCJIT component, but when sections from the generated code are loaded,
they are loaded into memory owned by the JITMemoryManager.  During object
loading, an ObjectFile is created within the RuntimeDyld which references both
of these buffers.  The situation is further compilicated in the case where the
RuntimeDyldELF object attempts to register the generated object with GDB,
because the GDB-interface requires a reference to both memory buffers.  The
GDB_required references are currently maintained in an ObjectImage instance
which is held by RuntimeDyldELF.

For those who are visually oriented, I am attaching a diagram which shows the
object relationships.

I would like to change this by introducing an ObjectBuffer which would be
allocated by the MCJIT object at compilation time and then passed to the
RuntimeDyld::loadObject.  RuntimeDyld::loadObject would hand this ObjectBuffer
off to the new ObjectImage instance (which it already creates today).  The
ObjectImage would be returned from the RuntimeDyld::loadObject to MCJIT and
MCJIT would own that object.

Again, a diagram is attached.


-------------------
Object caching
-------------------

I would like to introduce a mechanism whereby JITed objects could be cached and
loaded from cache at a future time.  It seems that the caching should logically
be performed transparently within the MCJIT engine, but it also seems that the
caching mechanism will need to be provided by the client (or at least under
client control).  I think that the best way to accomplish this is the introduce
a new interface (ObjectManager?), similar to the memory manager, that can be
specified by the client and used by the MCJIT engine.  What I have in mind is
that when MCJIT is about to compile a module, it will first check with the
ObjectManager to see if a pre-compiled image is available.  If so, it will
simply pass that image to the RuntimeDyld for loading.  If not, it will compile
as usual.  Likewise, after compilation (but before loading) MCJIT would offer
the compiled image to the ObjectManager and the ObjectManager could save the
image to a file cache (or whatever).


-----------------
MCJIT Events
-----------------

The legacy JIT interface supported a JITEventListener interface that would
provide notification when new functions were JITed.  That interface was used to
enable source-level profiling of JITed code.  There is currently no equivalent
for the MCJIT engine.  I'd like to correct that.

In order to get to the function level, some code somewhere will need to parse
the format-specific object code that is generated.  I think that it makes sense
for that to be pushed to the event listener itself since some listeners may be
satisfied with the raw object itself as input.

Most of the support necessary to parse object images to obtain source-level
debug information is already present in LLVM somewhere, though not all of it is
handily exposed through interfaces in the include tree.  In order to implement a
default event listener that provides the basic functionality of the existing JIT
event listeners, some extensions to the existing DIContext and related DebugInfo
classes will be necessary.  The development of an interface to expose more DWARF
information may prove worthy of its own discussion, as it would also enable
better testing, but for now I'll just mention it as a requirement here.

It may make sense to subsume the object loading event under the ObjectManager
interface described in the "object caching" section above rather than
overloading the existing JITEventListener interface within MCJIT, as the other
events in that interface don't properly apply to MCJIT.


-------------------------------
ELF Support on Windows
-------------------------------

There are various reasons that it would be nice to be able to support generation
of ELF objects on Windows through the MCJIT interface, one of which is that it
would allow users to debug JITed code with GDB (i.e. MinGW).  There was a
proposal by Eli Bendersky to enable ELF generation on Windows by extending the
target triple handling.  We have been using that approach, and it works. 
However, at the time it was proposed there seemed to be a general feeling that
it would be preferable to have a general way to specify a non-default object
format for any platform rather than a Windows/ELF-specific extension.

The original discussion can be found here:

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120130/136053.html

Unfortunately, I don't believe a consensus was ever reached as to what the
correct way to implement this would be.  I'd like to re-open that discussion
now so that we can find a solution that would be generally accepted for
inclusion in LLVM.

We can, of course, discuss any and all of these in more details as the
implementation unfolds, but I wanted to get all of it out there first as a sort
of road map to get general input on the overall direction of things in the MCJIT
space.

Thanks to all who took the time to read this, and thank you in advance for your
feedback.

-Andy

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120730/074b981d/attachment.html>

This sounds interesting, but strikes me as significantly outside the scope of
the MCJIT itself. That is, this is the sort of thing a client application or
framework would build on top of the MCJIT.

-Jim

On Jul 30, 2012, at 6:20 AM, "Demikhovsky, Elena"
<elena.demikhovsky at intel.com> wrote:
> I have some vision about Object Caching.
>  
> A client, which is interested in object caching, will need some help from
compiler and LLVM side. The object file, located on disk, should contain
information that allows to decide – rebuild or not rebuild.
> The client (a backend engine, which translates IR to JIT) compares data
from object file, found on disk  and module IR.
> The information that object file may contain I call “Signature”. It can be
stored in “.comment” section.
> Signature includes:
> ·         Name of source file, full path. (It may be .ll or .cl or .cpp or
any other extension, the compilation was started from)
> ·         Time stamp of the source file.
> ·         Name and version of the compiler.
> ·         Compilation flags.
> ·         Environment variables (all or those who affect compilation
process)
> The compiler can generate this data and put it in metadata.
>  
> If we’ll take Intel OpenCL product, the clang can generate this special
metadata. MCJIT, while building an object file, will compose a “ .comment” with
a signature, described above.
>  
> Ø  I would like to introduce a mechanism whereby JITed objects could be
cached and loaded from cache at a future time.  It seems that the caching should
logically be performed transparently within the MCJIT engine, but it also seems
that the caching mechanism will need to be provided by the client (or at least
under client control).  I think that the best way to accomplish this is the
introduce a new interface (ObjectManager?), similar to the memory manager, that
can be specified by the client and used by the MCJIT engine.  What I have in
mind is that when MCJIT is about to compile a module, it will first check with
the ObjectManager to see if a pre-compiled image is available.  If so, it will
simply pass that image to the RuntimeDyld for loading.  If not, it will compile
as usual.  Likewise, after compilation (but before loading) MCJIT would offer
the compiled image to the ObjectManager and the ObjectManager could save the
image to a file cache (or whatever).
>  
>  
> - Elena
> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at
cs.uiuc.edu] On Behalf Of Kaylor, Andrew
> Sent: Wednesday, July 25, 2012 01:25
> To: llvmdev at cs.uiuc.edu
> Subject: [LLVMdev] RFC: MCJIT enhancements
>  
> Following up and expanding on an earlier conversation, I'd like to
discuss making several non-trivial changes to the MCJIT engine and related
objects.  There may be some interdependencies between these changes, but I think
that they can be logically grouped as follows:
>  
> * Lazy module compilation
>  
> * Enhance the JIT memory manager interface to enable section-based memory
protection
>  
> * Clean up object/memory ownership problems
>  
> * Introduce a mechanism for caching generated objects
>  
> * Support JIT events for MCJIT
>  
> * Support ELF generation on Windows
>  
>  
> Now let me expand on each of these a bit.
>  
> ---------------------
> Lazy compilation
> ---------------------
>  
> Right now a Module is passed to the EngineBuilder::create call and is
compiled immediately within the MCJIT constructor.  I don't believe it is
possible to perform lazy compilation on a per function basis as was done in the
legacy JIT interface, but there's no reason that module compilation and
loading can't be deferred until a function is requested.  I think this is a
fairly straight-forward and non-controversial change.
>  
>  
> ----------------------------------
> Memory manager interface
> ----------------------------------
>  
> The memory manager changes are a bit more complicated.  In the current
MCJIT implementation, MCJIT clients may specify a memory manager through the
EngineBuilder and this memory manager will be passed along to the MCJIT engine. 
The  JITMemoryManager interface was originally created to suit the needs of the
legacy JIT engine.  As the MCJIT engine has come into existence, its memory
needs have been somewhat wedged in to the old JITMemoryManager interface.  The
result has been that there are effectively two interfaces existing side-by-side
in the same abstract base class and implementations will typically only support
one or the other set of functions.  This is further complicated by the
RuntimeDyldMemoryManager, which doesn't inherit from the JITMemoryManager
but does share a subset of functions and is implemented as a wrapper around the
client-supplied memory manager in the current MCJIT code.  There's also an
MCJITmemoryManager class which derives from RuntimeDyld and is used by MCJIT but
isn't exposed to the client at all.
>  
> I'd like to see this situation cleaned up as we move forward with a way
for the client to specify an MCJITMemoryManager directly and a separation of the
legacy memory manager interface from the new interface.  In addition, I'd
like the new memory manager interface to be extended with the functions that the
RuntimeDyld needs to manage setting of section-specific permissions including
non-writable code and read-only data.
>  
>  
> ----------------------
> Object ownership
> ----------------------
>  
> Within the current MCJIT implementation there are a number of ugly cross
dependencies in object/memory life cycle, making the
MCJIT/RuntimeDyld/JITMemoryManager relationship fragile and very dependent upon
a particular order of object destruction.
>  
> In the current implementation, the buffer into which code is generated is
owned by the MCJIT component, but when sections from the generated code are
loaded, they are loaded into memory owned by the JITMemoryManager.  During
object loading, an ObjectFile is created within the RuntimeDyld which references
both of these buffers.  The situation is further compilicated in the case where
the RuntimeDyldELF object attempts to register the generated object with GDB,
because the GDB-interface requires a reference to both memory buffers.  The
GDB_required references are currently maintained in an ObjectImage instance
which is held by RuntimeDyldELF.
>  
> For those who are visually oriented, I am attaching a diagram which shows
the object relationships.
>  
> I would like to change this by introducing an ObjectBuffer which would be
allocated by the MCJIT object at compilation time and then passed to the
RuntimeDyld::loadObject.  RuntimeDyld::loadObject would hand this ObjectBuffer
off to the new ObjectImage instance (which it already creates today).  The
ObjectImage would be returned from the RuntimeDyld::loadObject to MCJIT and
MCJIT would own that object.
>  
> Again, a diagram is attached.
>  
>  
> -------------------
> Object caching
> -------------------
>  
> I would like to introduce a mechanism whereby JITed objects could be cached
and loaded from cache at a future time.  It seems that the caching should
logically be performed transparently within the MCJIT engine, but it also seems
that the caching mechanism will need to be provided by the client (or at least
under client control).  I think that the best way to accomplish this is the
introduce a new interface (ObjectManager?), similar to the memory manager, that
can be specified by the client and used by the MCJIT engine.  What I have in
mind is that when MCJIT is about to compile a module, it will first check with
the ObjectManager to see if a pre-compiled image is available.  If so, it will
simply pass that image to the RuntimeDyld for loading.  If not, it will compile
as usual.  Likewise, after compilation (but before loading) MCJIT would offer
the compiled image to the ObjectManager and the ObjectManager could save the
image to a file cache (or whatever).
>  
>  
> -----------------
> MCJIT Events
> -----------------
>  
> The legacy JIT interface supported a JITEventListener interface that would
provide notification when new functions were JITed.  That interface was used to
enable source-level profiling of JITed code.  There is currently no equivalent
for the MCJIT engine.  I'd like to correct that.
>  
> In order to get to the function level, some code somewhere will need to
parse the format-specific object code that is generated.  I think that it makes
sense for that to be pushed to the event listener itself since some listeners
may be satisfied with the raw object itself as input.
>  
> Most of the support necessary to parse object images to obtain source-level
debug information is already present in LLVM somewhere, though not all of it is
handily exposed through interfaces in the include tree.  In order to implement a
default event listener that provides the basic functionality of the existing JIT
event listeners, some extensions to the existing DIContext and related DebugInfo
classes will be necessary.  The development of an interface to expose more DWARF
information may prove worthy of its own discussion, as it would also enable
better testing, but for now I'll just mention it as a requirement here.
>  
> It may make sense to subsume the object loading event under the
ObjectManager interface described in the "object caching" section
above rather than overloading the existing JITEventListener interface within
MCJIT, as the other events in that interface don't properly apply to MCJIT.
>  
>  
> -------------------------------
> ELF Support on Windows
> -------------------------------
>  
> There are various reasons that it would be nice to be able to support
generation of ELF objects on Windows through the MCJIT interface, one of which
is that it would allow users to debug JITed code with GDB (i.e. MinGW).  There
was a proposal by Eli Bendersky to enable ELF generation on Windows by extending
the target triple handling.  We have been using that approach, and it works. 
However, at the time it was proposed there seemed to be a general feeling that
it would be preferable to have a general way to specify a non-default object
format for any platform rather than a Windows/ELF-specific extension.
>   
> The original discussion can be found here:
>  
>
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120130/136053.html
>  
> Unfortunately, I don't believe a consensus was ever reached as to what
the correct way to implement this would be.  I'd like to re-open that
discussion now so that we can find a solution that would be generally accepted
for inclusion in LLVM.
>  
> We can, of course, discuss any and all of these in more details as the
implementation unfolds, but I wanted to get all of it out there first as a sort
of road map to get general input on the overall direction of things in the MCJIT
space.
>  
> Thanks to all who took the time to read this, and thank you in advance for
your feedback.
>  
> -Andy
>  
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
> 
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all
copies._______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120730/c82fb534/attachment.html>

On Wed, Jul 25, 2012 at 12:24 AM, Kaylor, Andrew <andrew.kaylor at
intel.com>wrote:
>
>  -------------------------------****
>
> ELF Support on Windows****
>
> -------------------------------****
>
>  ****
>
> There are various reasons that it would be nice to be able to support
> generation of ELF objects on Windows through the MCJIT interface, one of
> which is that it would allow users to debug JITed code with GDB (i.e.
> MinGW).  There was a proposal by Eli Bendersky to enable ELF generation on
> Windows by extending the target triple handling.  We have been using that
> approach, and it works.  However, at the time it was proposed there seemed
> to be a general feeling that it would be preferable to have a general way
> to specify a non-default object format for any platform rather than a
> Windows/ELF-specific extension.****
>
>   ****
>
> The original discussion can be found here:****
>
>  ****
>
>
>
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120130/136053.html
> ****
>
>  ****
>
> Unfortunately, I don't believe a consensus was ever reached as to what
the
> correct way to implement this would be.  I'd like to re-open that
> discussion now so that we can find a solution that would be generally
> accepted for inclusion in LLVM.
>
I'm using this patch mentioned and it works just fine.
However, I would like to see it implemented in more generic way:

1. When a container type is explicitly specified in triple it should be
used by MCJIT.
2. When a container type is not specified it is deduced from OS.
3. Use ELF by default.

"Environment" part of the triple may be used to specify the container.
Some
options are already there (like MachO). But there may conflicts exist where
both environment and container must have been specified together (e.g.
"arm-apple-ios-eabi-macho").

- Paweł
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120816/90a05e3a/attachment.html>

Hi Paweł,

Thanks for continuing this discussion.

I like the simplicity of your suggestion.  My only concern involves the
ambiguity of what is meant by “environment”.  Presently there are functions in
the llvm::Triple class to access the environment as an enumeration of a fixed
set of values.  It seems that some non-enumerated values are already in use, but
introducing possible combinations of ABI and object format would seem to strain
the API.

I couldn’t find an explanation anywhere of what is meant by environment in the
context of the triple, and it isn’t clear from looking at the possible values
whether object format properly belongs to the environment concept or whether it
should be a new triple component.  Looking at the uses in the LLVM code base, it
seems that the environment element has been used specifically to enable
generation of MachO objects on non-Darwin OSes.  I see that clang  recognizes
“iphoneos” as a value (though it’s not in the Triple::EnvironmentType enum), and
it’s not clear to me what it uses it for.  Beyond that, the environment seems to
be used to influence ABI selection, which would seem to have some degree of
overlap with object format selection while being essentially different.

In other words, environment seems to be a bit of a catch-all dumping ground at
the moment.  That’s not entirely bad, I suppose, but it does make the task of
cleanly extending the target triple more complicated.

It would seem that at some point the target triple has evolved into a target
quadruple with the addition of environment.  The question now, I think, is
whether it makes sense to extend it further into a target quintuple, adding
object format as a well-defined, but optional, component with full API support. 
The difficulty, of course, is that with two optional extensions the expected
canonical form becomes ambiguous, but I don’t think that’s much of a problem. 
Given that object format is a relatively small enumeration, the triple parser
should be able to handle it presence or absence while parsing or normalizing.

So here’s what I would suggest.


1.       Define the target triple as follows (adapted from Triple.h):



/// Target triples are strings in the canonical form:

///   ARCHITECTURE-VENDOR-OPERATING_SYSTEM

/// or

///   ARCHITECTURE-VENDOR-OPERATING_SYSTEM-OBJECT_FORMAT

/// or

///   ARCHITECTURE-VENDOR-OPERATING_SYSTEM-ENVIRONMENT

/// or

///   ARCHITECTURE-VENDOR-OPERATING_SYSTEM-OBJECT_FORMAT-ENVIRONMENT



2.       Add an llvm::Triple::ObjectFormatType enum with all llvm-supported
object formats.

3.       Remove “MachO” from the llvm::Triple::EnvironmentType enum.

4.       Add the following methods to llvm::Triple:



Triple(const Twine &ArchStr, const Twine &VendorStr, const Twine
&OSStr, const Twine &ObjFmtStr)

Triple(const Twine &ArchStr, const Twine &VendorStr, const Twine
&OSStr, const Twine &ObjFmtStr, const Twine &EnvStr)

bool hasExplicitObjectFormat() const

ObjectFormatType getObjectFormat() const

StringRef getObjectFormatName() const

bool isOutputObjectFormatELF() const

bool isOutputObjectFormatCOFF() const

bool isOutputObjectFormatMachO() const

static const char * getObjectFormatName(ObjectFormatType Kind)



5.       When an object format is specified, it will be used by MC and MCJIT if
supported for the specified architecture

6.       When an unsupport object format is specified, a fatal error will be
thrown

7.       When an object format and environment are both specified but are
incompatible, a fatal error will be thrown

8.       When an object format is not specified everything will behave as it
currently does

How does that sound?

-Andy


From: Paweł Bylica [mailto:pawel.bylica at ibs.org.pl]
Sent: Thursday, August 16, 2012 4:03 AM
To: Kaylor, Andrew
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] RFC: MCJIT enhancements


On Wed, Jul 25, 2012 at 12:24 AM, Kaylor, Andrew <andrew.kaylor at
intel.com<mailto:andrew.kaylor at intel.com>> wrote:
-------------------------------
ELF Support on Windows
-------------------------------

There are various reasons that it would be nice to be able to support generation
of ELF objects on Windows through the MCJIT interface, one of which is that it
would allow users to debug JITed code with GDB (i.e. MinGW).  There was a
proposal by Eli Bendersky to enable ELF generation on Windows by extending the
target triple handling.  We have been using that approach, and it works. 
However, at the time it was proposed there seemed to be a general feeling that
it would be preferable to have a general way to specify a non-default object
format for any platform rather than a Windows/ELF-specific extension.

The original discussion can be found here:

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120130/136053.html

Unfortunately, I don't believe a consensus was ever reached as to what the
correct way to implement this would be.  I'd like to re-open that discussion
now so that we can find a solution that would be generally accepted for
inclusion in LLVM.

I'm using this patch mentioned and it works just fine.
However, I would like to see it implemented in more generic way:

1. When a container type is explicitly specified in triple it should be used by
MCJIT.
2. When a container type is not specified it is deduced from OS.
3. Use ELF by default.

"Environment" part of the triple may be used to specify the container.
Some options are already there (like MachO). But there may conflicts exist where
both environment and container must have been specified together (e.g.
"arm-apple-ios-eabi-macho").

- Paweł
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120816/3dfb897b/attachment.html>