llvm dev - Nov 2012 - [LLVMdev] Lazy compilation in MCJIT (was RE: Old JIT Status (i.e., can we delete it?))

On 11/24/2012 11:05 AM, Benjamin Kramer wrote:
> The last major roadblock is lazy compilation. I don't think we can
remove the old JIT until MCJIT supports that.
I agree. But that alone may not be enough to support dynamic languages.
In the Pure JIT I often need to be able to replace the IR of the body of
an already lowered function and generate new code for it. This also
needs to be fast, so recompiling the entire module when this happens is
not an option. Will this be possible with MCJIT?

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr.Graef at t-online.de, ag at muwiinfa.geschichte.uni-mainz.de
WWW:    http://www.musikinformatik.uni-mainz.de/ag

So when you say 'lazy compilation' you mean 'deferred compilation of
individual functions within a module until that function is needed,' right? 
That's certainly what 'lazy compilation' meant in the legacy JIT.

I don't think that will ever be possible in MCJIT.  If you look at the MCJIT
engine you'll see that it delegates absolutely everything.  In particular,
compilation is done by the MC code, and it is done in exactly the same way that
static compilation of objects is done.  Rather than producing a collection of
loosely coupled pieces of executable memory containing machine code for
functions, MCJIT produces executable object images and prepares those for
execution.  I don't see any way for a 'partial' object to be
produced without drastically changing the design of the MCJIT engine.

I'd like to suggest that instead of talking about how we can get MCJIT to do
'lazy compilation' in the old way, we should be talking about how to
solve the same problem within the constraints of what MCJIT does.  I'm not
sure I understand the constraints of the client side well enough to address this
problem thoroughly, but I'll take a stab at first steps.

I think the way to approach this problem is to deconstruct large modules into
smaller modules which we can afford to re-JIT, and then optimize the process of
linking modules together.  Currently, a single instance of MCJIT only supports
handling of a single module, but I did recently add a mechanism to delay
compilation of the module so that it doesn't happen when the MCJIT engine is
constructed.

Once MCJIT supports multiple modules, if it had a mechanism for determining
dependencies between modules and managing lazy compilation of those dependencies
would that be acceptable?

This still leaves a bit of a problem with regard to function replacement, but I
think it suggests a direction for making that happen too.

-Andy

-----Original Message-----
From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
Behalf Of Albert Graef
Sent: Saturday, November 24, 2012 6:47 AM
To: LLVM Developers Mailing List
Subject: Re: [LLVMdev] Old JIT Status (i.e., can we delete it?)

On 11/24/2012 11:05 AM, Benjamin Kramer wrote:
> The last major roadblock is lazy compilation. I don't think we can
remove the old JIT until MCJIT supports that.
I agree. But that alone may not be enough to support dynamic languages.
In the Pure JIT I often need to be able to replace the IR of the body of an
already lowered function and generate new code for it. This also needs to be
fast, so recompiling the entire module when this happens is not an option. Will
this be possible with MCJIT?

--
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr.Graef at t-online.de, ag at muwiinfa.geschichte.uni-mainz.de
WWW:    http://www.musikinformatik.uni-mainz.de/ag
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On 11/26/2012 10:35 PM, Kaylor, Andrew wrote:
> So when you say 'lazy compilation' you mean 'deferred
compilation of individual functions within a module until that function is
needed,' right?  That's certainly what 'lazy compilation' meant
in the legacy JIT.
Yes. Let's not forget what JIT actually means; this *is* what makes a
compiler a JIT compiler. In Pure you can add new functions and change
existing ones at any time, by just typing an equation at the command
prompt of Pure's interactive frontend. Other dynamic languages work in a
similar fashion. The old JIT makes this very easy to do; you just
(re)compile the function to IR on the fly, and the JIT will then take
care of lowering the function to native code when it's first needed. For
me this has always been one of the coolest features of LLVM, a feature
which really sets it apart from traditional compiler technology.

If the MCJIT lacks this feature then it's not a real JIT compiler for
me, it's an in-memory compiler/linker/loader. That's sufficient for many
applications, but if you're striving to eventually replace the old JIT
then you'll have to find a way to replicate the lazy compilation bit in
some way. Otherwise we'd need the old JIT to stick around forever to
support this kind of application (and then we'd also need a working ARM
backend for it).
> I don't think that will ever be possible in MCJIT.
That's bad news. :(
> I think the way to approach this problem is to deconstruct large modules
into smaller modules which we can afford to re-JIT, and then optimize the
process of linking modules together.  Currently, a single instance of MCJIT only
supports handling of a single module, but I did recently add a mechanism to
delay compilation of the module so that it doesn't happen when the MCJIT
engine is constructed.
In Pure this would basically mean that each function would have to be in
its own module. (More precisely, each global Pure function. Pure also
has lexical closures, these could of course be in the same module as the
global functions they belong to.)
> Once MCJIT supports multiple modules, if it had a mechanism for determining
dependencies between modules and managing lazy compilation of those dependencies
would that be acceptable?
Yes, this might be an acceptable (if somewhat awkward) solution. But I'm
not really convinced that it's feasible. It would need a linker that's
able to cope with thousands of modules, add new modules and replace
existing ones at any time, and do this *fast*. Do you think that this
will be possible with MCJIT? Maintaining all these modules and the
linker tables in memory might add significant overhead compared to the
old JIT, right? Will it be at least as fast as the old JIT?

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr.Graef at t-online.de, ag at muwiinfa.geschichte.uni-mainz.de
WWW:    http://www.musikinformatik.uni-mainz.de/ag