llvm dev - Oct 2009 - [LLVMdev] Should LLVM JIT default to lazy or non-lazy?

In r85295, in response to the discussion at http://llvm.org/PR5184
(Lazy JIT ain't thread-safe), I changed the default JIT from lazy to
non-lazy. It has since come to my attention that this may have been
the wrong change, so I wanted to ask you guys.

A couple reasons to make the default non-lazy compilation:
* The lack of thread-safety surprises new users
* Crashes due to this will be rare and so hard to track down
* The current lazy scheme is almost never the right answer for performance
* It's only one line of code to turn on lazy compilation when it is
the right answer for you.

And a couple to default to lazy compilation:
* It's safe for single-threaded code.
* There are existing users who have assumed this default.
* PPC and ARM don't support non-lazy compilation yet (the tests
currently run the lazy jit).
* Gratuitous changes are bad.

Thoughts?

We can choose the default for lli separately from the JIT's default if we
want.

On Wed, Oct 28, 2009 at 11:41 AM, Jeffrey Yasskin <jyasskin at google.com>
wrote:
> And a couple to default to lazy compilation:
> * It's safe for single-threaded code.
> * There are existing users who have assumed this default.
And what might break if this assumption doesn't hold?

On Oct 28, 2009, at 9:41 AM, Jeffrey Yasskin wrote:
> In r85295, in response to the discussion at http://llvm.org/PR5184
> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy to
> non-lazy. It has since come to my attention that this may have been
> the wrong change, so I wanted to ask you guys.
>
> A couple reasons to make the default non-lazy compilation:
> * The lack of thread-safety surprises new users
> * Crashes due to this will be rare and so hard to track down
> * The current lazy scheme is almost never the right answer for  
> performance
> * It's only one line of code to turn on lazy compilation when it is
> the right answer for you.
>
> And a couple to default to lazy compilation:
> * It's safe for single-threaded code.
> * There are existing users who have assumed this default.
> * PPC and ARM don't support non-lazy compilation yet (the tests
> currently run the lazy jit).
> * Gratuitous changes are bad.
If the objection was about changing the sense of a magic bool, why not  
change the argument to be an enum instead?  That should make it  
extremely clear in the source what behavior is desired.

-Chris

On Wed, Oct 28, 2009 at 9:48 AM, Kenneth Uildriks <kennethuil at
gmail.com> wrote:
> On Wed, Oct 28, 2009 at 11:41 AM, Jeffrey Yasskin <jyasskin at
google.com> wrote:
>> And a couple to default to lazy compilation:
>> * It's safe for single-threaded code.
>> * There are existing users who have assumed this default.
>
> And what might break if this assumption doesn't hold?
They'll wind up compiling the whole tree of transitive calls at
startup, including calls that are dynamically never taken, instead of
compiling only the taken calls gradually over the running time of the
app.

On Wed, Oct 28, 2009 at 9:50 AM, Chris Lattner <clattner at apple.com>
wrote:
>
> On Oct 28, 2009, at 9:41 AM, Jeffrey Yasskin wrote:
>
>> In r85295, in response to the discussion at http://llvm.org/PR5184
>> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy
to
>> non-lazy. It has since come to my attention that this may have been
>> the wrong change, so I wanted to ask you guys.
>>
>> A couple reasons to make the default non-lazy compilation:
>> * The lack of thread-safety surprises new users
>> * Crashes due to this will be rare and so hard to track down
>> * The current lazy scheme is almost never the right answer for
performance
>> * It's only one line of code to turn on lazy compilation when it is
>> the right answer for you.
>>
>> And a couple to default to lazy compilation:
>> * It's safe for single-threaded code.
>> * There are existing users who have assumed this default.
>> * PPC and ARM don't support non-lazy compilation yet (the tests
>> currently run the lazy jit).
>> * Gratuitous changes are bad.
>
> If the objection was about changing the sense of a magic bool, why not
> change the argument to be an enum instead?  That should make it extremely
> clear in the source what behavior is desired.
No, the magic bools all keep the same sense. The change affects the
behavior of programs without a call to
JIT->DisableLazyCompilation(bool). Anyone who's already calling it
with any parameter keeps their current behavior.

I'd be reasonably happy making that call required (so no default) if
people want that instead.

>From where I sit, this boils down to a very simple question (modulo
Chris's point): Either choice will surprise some users. Which surprise
is worse? Personally, I'd always prefer correct but slow behavior by
default, and explicitly enabling dangerous (but in some cases fast)
behavior.

I would also point out that it seems that most of the people new to
the JIT are surprised by the current behavior, where as those who
would be surprised by needing to enable lazy JIT are those long
familiar with past behavior. In the OSS world, I always favor easing
adoption over maintaining the status quo.

My meager 2 cents.
-Chandler

On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at google.com>
wrote:
> In r85295, in response to the discussion at http://llvm.org/PR5184
> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy to
> non-lazy. It has since come to my attention that this may have been
> the wrong change, so I wanted to ask you guys.
>
> A couple reasons to make the default non-lazy compilation:
> * The lack of thread-safety surprises new users
> * Crashes due to this will be rare and so hard to track down
> * The current lazy scheme is almost never the right answer for performance
> * It's only one line of code to turn on lazy compilation when it is
> the right answer for you.
>
> And a couple to default to lazy compilation:
> * It's safe for single-threaded code.
> * There are existing users who have assumed this default.
> * PPC and ARM don't support non-lazy compilation yet (the tests
> currently run the lazy jit).
> * Gratuitous changes are bad.
>
> Thoughts?
>
> We can choose the default for lli separately from the JIT's default if
we want.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Jeffrey Yasskin <jyasskin at google.com> writes:

[snip]
> Thoughts?
I didn't notice the existence of a parameter for disabling JIT
laziness. What I do on my code is

    for (Module::iterator it = M->begin(); it != M->end(); ++it)
      EE->getPointerToFunction(&*it);

thus forcing code emission for all functions.

Native code creation on x86/x86_64 not only is very slow, its time
complexity for N LLVM instructions is N^x (x > 1). For a project I'm
working on, the JIT takes almost 1 minute to emit native code for all
functions on a low-end machine. That application acts as a server, and
clients will time-out if it takes more than a few seconds to respond,
which is very likely to happen with a lazy JIT. This is almost as bad as
the thread-safety issue, when the time required for lazy JITting is
similar to the time-out threshold.

IMHO it would be better for LLVM to default to non-lazy JITting.

-- 
Óscar

On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
> From where I sit, this boils down to a very simple question (modulo
> Chris's point): Either choice will surprise some users. Which surprise
> is worse? Personally, I'd always prefer correct but slow behavior by
> default, and explicitly enabling dangerous (but in some cases fast)
> behavior.
The behavior is only dangerous because people are using it in new and different
ways.
> 
> I would also point out that it seems that most of the people new to
> the JIT are surprised by the current behavior, where as those who
> would be surprised by needing to enable lazy JIT are those long
> familiar with past behavior. In the OSS world, I always favor easing
> adoption over maintaining the status quo.
This argues for better documentation. I'd prefer for EE to abort if user is
asking for a known dangerous configuration (multi-threaded and lazy).

The biggest argument I have for staying with lazy is llvm JIT is not a light
weight JIT. It's designed to do all the codegen optimizations a normal
static compiler would do. Non-lazy JIT is too slow.

I'd prefer not to change the behavior. If we want to start using it in new
and interesting ways, we should just design a new JIT.

Evan
> 
> My meager 2 cents.
> -Chandler
> 
> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at
google.com> wrote:
>> In r85295, in response to the discussion at http://llvm.org/PR5184
>> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy
to
>> non-lazy. It has since come to my attention that this may have been
>> the wrong change, so I wanted to ask you guys.
>> 
>> A couple reasons to make the default non-lazy compilation:
>> * The lack of thread-safety surprises new users
>> * Crashes due to this will be rare and so hard to track down
>> * The current lazy scheme is almost never the right answer for
performance
>> * It's only one line of code to turn on lazy compilation when it is
>> the right answer for you.
>> 
>> And a couple to default to lazy compilation:
>> * It's safe for single-threaded code.
>> * There are existing users who have assumed this default.
>> * PPC and ARM don't support non-lazy compilation yet (the tests
>> currently run the lazy jit).
>> * Gratuitous changes are bad.
>> 
>> Thoughts?
>> 
>> We can choose the default for lli separately from the JIT's default
if we want.
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev