llvm dev - Aug 2014 - [LLVMdev] Heuristic for choosing between MCJIT and Interpreter

I'm facing a situation where I have generated IR that only needs to be
executed once. I've noticed for simple IR it's faster to run the
interpreter on it, but for complex IR it's much better to JIT compile and
execute it. I'm seeking suggestions for a good heuristic to decide which
approach to take for any given IR. I'm leaning in favor of deciding based
on the presence/absence of loops.
-Josh
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140809/f3049acd/attachment.html>

On 9 August 2014 21:33, Josh Klontz <josh.klontz at gmail.com>
wrote:
> I'm facing a situation where I have generated IR that only needs to be
> executed once. I've noticed for simple IR it's faster to run the
interpreter
> on it, but for complex IR it's much better to JIT compile and execute
it.
> I'm seeking suggestions for a good heuristic to decide which approach
to
> take for any given IR. I'm leaning in favor of deciding based on the
> presence/absence of loops.
Hi Josh,

In the past, when facing a similar situation, I've decided based on
maintenance, not IR heuristics. Maybe maintaining two versions and
making sure they both get updated when you change something would be
worse than having MCJIT as a default and then try to improve the JIT
to make it fast and reliable, rather than avoid the issues altogether
and mud things up in the future.

cheers,
--renato

Hi Josh,

On 9 Aug 2014, at 21:33, Josh Klontz <josh.klontz at gmail.com> wrote:
> I'm facing a situation where I have generated IR that only needs to be
executed once. I've noticed for simple IR it's faster to run the
interpreter on it, but for complex IR it's much better to JIT compile and
execute it. I'm seeking suggestions for a good heuristic to decide which
approach to take for any given IR. I'm leaning in favor of deciding based on
the presence/absence of loops.
What are you generating IR from?  You may find that an AST interpreter, although
slow, will be faster than going to the effort of generating LLVM IR and then
interpreting it.  LLVM IR is much slower than bytecodes for high-level languages
because you have the overhead of interpreting for things that often map to a
single machine instruction, whereas high-level bytecodes tend to amortise the
interpretation cost by having complex operations.

I've found having a working AST interpreter to be good for testing an
LLVM-based JIT, as you can run the same code with both check that the same
actions externally visible happen in the same order.

You can probably find some heuristics at the AST layer that will work well.  For
example, does it contain any loops with more than N iterations (where N is a
number you determine experimentally)?  Does it operate on data over a certain
size?

David

On 08/11/2014 04:21 AM, David Chisnall wrote:
> Hi Josh,
>
> On 9 Aug 2014, at 21:33, Josh Klontz <josh.klontz at gmail.com>
wrote:
>
>> I'm facing a situation where I have generated IR that only needs to
be executed once. I've noticed for simple IR it's faster to run the
interpreter on it, but for complex IR it's much better to JIT compile and
execute it. I'm seeking suggestions for a good heuristic to decide which
approach to take for any given IR. I'm leaning in favor of deciding based on
the presence/absence of loops.
> What are you generating IR from?  You may find that an AST interpreter,
although slow, will be faster than going to the effort of generating LLVM IR and
then interpreting it.  LLVM IR is much slower than bytecodes for high-level
languages because you have the overhead of interpreting for things that often
map to a single machine instruction, whereas high-level bytecodes tend to
amortise the interpretation cost by having complex operations.
>
> I've found having a working AST interpreter to be good for testing an
LLVM-based JIT, as you can run the same code with both check that the same
actions externally visible happen in the same order.
David makes a good set of points here.  The only reason not to take the 
approach he is suggesting is that it does involve writing and 
maintaining additional code.  If you don't really care about the 
performance of your cold code, using the LLVM interpreter is definite an 
option.  (Keep in mind that there's little active development happening 
on the interpreter and it may have started to bitrot.)

Given your original question, I'm guessing you do care about the 
performance of your cold code.  Given that, David has suggested the best 
approach.
>
> You can probably find some heuristics at the AST layer that will work well.
For example, does it contain any loops with more than N iterations (where N is a
number you determine experimentally)?  Does it operate on data over a certain
size?
>
> David
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev