llvm dev - Oct 2009 - [LLVMdev] Should LLVM JIT default to lazy or non-lazy?

On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
> From where I sit, this boils down to a very simple question (modulo
> Chris's point): Either choice will surprise some users. Which surprise
> is worse? Personally, I'd always prefer correct but slow behavior by
> default, and explicitly enabling dangerous (but in some cases fast)
> behavior.
The behavior is only dangerous because people are using it in new and different
ways.
> 
> I would also point out that it seems that most of the people new to
> the JIT are surprised by the current behavior, where as those who
> would be surprised by needing to enable lazy JIT are those long
> familiar with past behavior. In the OSS world, I always favor easing
> adoption over maintaining the status quo.
This argues for better documentation. I'd prefer for EE to abort if user is
asking for a known dangerous configuration (multi-threaded and lazy).

The biggest argument I have for staying with lazy is llvm JIT is not a light
weight JIT. It's designed to do all the codegen optimizations a normal
static compiler would do. Non-lazy JIT is too slow.

I'd prefer not to change the behavior. If we want to start using it in new
and interesting ways, we should just design a new JIT.

Evan
> 
> My meager 2 cents.
> -Chandler
> 
> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at
google.com> wrote:
>> In r85295, in response to the discussion at http://llvm.org/PR5184
>> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy
to
>> non-lazy. It has since come to my attention that this may have been
>> the wrong change, so I wanted to ask you guys.
>> 
>> A couple reasons to make the default non-lazy compilation:
>> * The lack of thread-safety surprises new users
>> * Crashes due to this will be rare and so hard to track down
>> * The current lazy scheme is almost never the right answer for
performance
>> * It's only one line of code to turn on lazy compilation when it is
>> the right answer for you.
>> 
>> And a couple to default to lazy compilation:
>> * It's safe for single-threaded code.
>> * There are existing users who have assumed this default.
>> * PPC and ARM don't support non-lazy compilation yet (the tests
>> currently run the lazy jit).
>> * Gratuitous changes are bad.
>> 
>> Thoughts?
>> 
>> We can choose the default for lli separately from the JIT's default
if we want.
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at apple.com>
wrote:
> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
>> From where I sit, this boils down to a very simple question (modulo
>> Chris's point): Either choice will surprise some users. Which
surprise
>> is worse? Personally, I'd always prefer correct but slow behavior
by
>> default, and explicitly enabling dangerous (but in some cases fast)
>> behavior.
>
> The behavior is only dangerous because people are using it in new and
different ways.
The fact that an interface, when used in new ways, exposes bugs this
severe and subtle indicates that it is a poor interface. Your argument
doesn't lend much weight, because the same can be said of computed
goto, the fork() system call, etc. Surviving new and different uses
*is* the purpose of a good interface.
>> I would also point out that it seems that most of the people new to
>> the JIT are surprised by the current behavior, where as those who
>> would be surprised by needing to enable lazy JIT are those long
>> familiar with past behavior. In the OSS world, I always favor easing
>> adoption over maintaining the status quo.
>
> This argues for better documentation. I'd prefer for EE to abort if
user is asking for a known dangerous configuration (multi-threaded and lazy).
I do not think you can guarantee that the abort occurs if multiple
threads are present. I would also much rather a speed limit sign than
a dashboard full of speeding tickets to teach me how to drive. (Even
if in that case I opt for both.... ;])
> The biggest argument I have for staying with lazy is llvm JIT is not a
light weight JIT. It's designed to do all the codegen optimizations a normal
static compiler would do. Non-lazy JIT is too slow.
The JIT doesn't get faster by being lazy, its slowness is just
amortized over the runtime. As several have pointed out, that's not
always desirable, and in some cases is outright terrible. We always
take the same amount of time to actually JIT the code.

At best, the lazy JIT simply sees less code, but for most dynamic
languages, the only code ever given to the JIT is what is already
known to be needed.
> I'd prefer not to change the behavior. If we want to start using it in
new and interesting ways, we should just design a new JIT.
We clearly do want use it in new and interesting ways, there is no
'if'. I'm not sure what you mean by 'design a new JIT'...

-Chandler

On Oct 28, 2009, at 12:41 PM, Chandler Carruth wrote:
> On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at
apple.com> wrote:
>> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
>>> From where I sit, this boils down to a very simple question (modulo
>>> Chris's point): Either choice will surprise some users. Which
surprise
>>> is worse? Personally, I'd always prefer correct but slow
behavior by
>>> default, and explicitly enabling dangerous (but in some cases fast)
>>> behavior.
>> 
>> The behavior is only dangerous because people are using it in new and
different ways.
> 
> The fact that an interface, when used in new ways, exposes bugs this
> severe and subtle indicates that it is a poor interface. Your argument
> doesn't lend much weight, because the same can be said of computed
> goto, the fork() system call, etc. Surviving new and different uses
> *is* the purpose of a good interface.
And your point is? We are talking about changing the default. That fixes the
interface?

I'd like to see some concrete proposal rather than flipping default
behaviors. That does actually fix the problem. That just hides the problem.
There won't be a consensus since we are lots of users with different needs.
LLVM community has always let people who are doing to work make design
decisions. Let's stick with that policy. Whoever is signed on to tackle the
thread safety issue can decide what to do.

Evan
> 
>>> I would also point out that it seems that most of the people new to
>>> the JIT are surprised by the current behavior, where as those who
>>> would be surprised by needing to enable lazy JIT are those long
>>> familiar with past behavior. In the OSS world, I always favor
easing
>>> adoption over maintaining the status quo.
>> 
>> This argues for better documentation. I'd prefer for EE to abort if
user is asking for a known dangerous configuration (multi-threaded and lazy).
> 
> I do not think you can guarantee that the abort occurs if multiple
> threads are present. I would also much rather a speed limit sign than
> a dashboard full of speeding tickets to teach me how to drive. (Even
> if in that case I opt for both.... ;])
> 
>> The biggest argument I have for staying with lazy is llvm JIT is not a
light weight JIT. It's designed to do all the codegen optimizations a normal
static compiler would do. Non-lazy JIT is too slow.
> 
> The JIT doesn't get faster by being lazy, its slowness is just
> amortized over the runtime. As several have pointed out, that's not
> always desirable, and in some cases is outright terrible. We always
> take the same amount of time to actually JIT the code.
> 
> At best, the lazy JIT simply sees less code, but for most dynamic
> languages, the only code ever given to the JIT is what is already
> known to be needed.
> 
>> I'd prefer not to change the behavior. If we want to start using it
in new and interesting ways, we should just design a new JIT.
> 
> We clearly do want use it in new and interesting ways, there is no
> 'if'. I'm not sure what you mean by 'design a new
JIT'...
> 
> -Chandler

2009/10/28 Chandler Carruth <chandlerc at
google.com>:
> Surviving new and different uses
> *is* the purpose of a good interface.
I don't have much to add to this interesting discussion, just like to
remind people that changes in the default behaviour are only
acceptable (IMHO) when everybody (but a few legacy code) is NOT using
(or should not use) the current default behaviour.

The counter example (and a good point when not to change the default
even when it's dangerous) is the strcpy() in the libc. It's dangerous,
malicious, bearing stupid, but still there. You have the strncpy (and
all associated variants) but it's not compulsory nor the default
behaviour.

Creating another method (getLazyFunctionPointer) or passing a boolean,
enum, whatever seems like the best course of action right now.

I understand that passing lots of Function*s to the JIT and only
generating what's really executed would be "nice" but (as was
pointed
out before) this is signal of bad code. You should only pass to the
JIT what's really going to be executed, and I guess it's not that hard
to do the "lazy evaluation" on your side.

My 2 cents...

cheers,
--renato

Reclaim your digital rights, eliminate DRM, learn more at
http://www.defectivebydesign.org/what_is_drm

On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at apple.com>
wrote:
>
> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
>
>> From where I sit, this boils down to a very simple question (modulo
>> Chris's point): Either choice will surprise some users. Which
surprise
>> is worse? Personally, I'd always prefer correct but slow behavior
by
>> default, and explicitly enabling dangerous (but in some cases fast)
>> behavior.
>
> The behavior is only dangerous because people are using it in new and
different ways.
I'd point out that reid thought he made the JIT thread-safe in r22404
back in 2005. Calling it from multiple threads isn't new and
different, it's just subtly broken. I suggested changing the default
because most people who run into this problem don't think they're
doing anything unusual, and in fact their code works fine with the
eager JIT. People shouldn't stumble into broken behavior, and defaults
are good to prevent stumbling.

To avoid misconceptions, Unladen Swallow added the line to turn off
the lazy jit months ago, and we'll keep that line whatever the
decision is here. We might take advantage of a thread-safe
code-patching facility eventually, but we've been designing assuming
nobody else will implement that for us. I favor changing the default
because it'll help newbies, not because we need it for anything.
>> I would also point out that it seems that most of the people new to
>> the JIT are surprised by the current behavior, where as those who
>> would be surprised by needing to enable lazy JIT are those long
>> familiar with past behavior. In the OSS world, I always favor easing
>> adoption over maintaining the status quo.
>
> This argues for better documentation. I'd prefer for EE to abort if
user is asking for a known dangerous configuration (multi-threaded and lazy).
I think that'd be a decent fix for the thread-safety problem. It'd
require an extra check on each call to runJITOnFunctionUnlocked since
laziness is set by a call, not on construction. Or, we could use the
JIT lock to assert that it's never entered concurrently. On the other
hand, Nicolas Geoffray objected when I suggested that in the bug.
> The biggest argument I have for staying with lazy is llvm JIT is not a
light weight JIT. It's designed to do all the codegen optimizations a normal
static compiler would do. Non-lazy JIT is too slow.
Óscar used the cost of the JIT as an argument _against_ the lazy JIT.
Could you elaborate on why you think it's an argument in favor of lazy
JITting?
> I'd prefer not to change the behavior.
I'm guessing, based on your vagueness, that there are some internal
single-threaded Apple users who think that the lazy JIT is the best
performing option for them and who don't want to add a line to their
apps overriding the default. But it's hard for me or anyone else
outside Apple to judge whether they ought to drive the default without
a little more information about why the lazy JIT is good for them.
Could you describe any features of their use that make the lazy JIT a
good idea for them?
> If we want to start using it in new and interesting ways, we should just
design a new JIT.
>> My meager 2 cents.
>> -Chandler
>>
>> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at
google.com> wrote:
>>> In r85295, in response to the discussion at http://llvm.org/PR5184
>>> (Lazy JIT ain't thread-safe), I changed the default JIT from
lazy to
>>> non-lazy. It has since come to my attention that this may have been
>>> the wrong change, so I wanted to ask you guys.
>>>
>>> A couple reasons to make the default non-lazy compilation:
>>> * The lack of thread-safety surprises new users
>>> * Crashes due to this will be rare and so hard to track down
>>> * The current lazy scheme is almost never the right answer for
performance
>>> * It's only one line of code to turn on lazy compilation when
it is
>>> the right answer for you.
>>>
>>> And a couple to default to lazy compilation:
>>> * It's safe for single-threaded code.
>>> * There are existing users who have assumed this default.
>>> * PPC and ARM don't support non-lazy compilation yet (the tests
>>> currently run the lazy jit).
>>> * Gratuitous changes are bad.
>>>
>>> Thoughts?
>>>
>>> We can choose the default for lli separately from the JIT's
default if we want.
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>

There is nothing here that prevents users from enabling or disabling  
lazy JIT. We're talk about flipping the default behavior. That does  
not make the API any better or worse. Nor does it fix the inherent  
thread safety issue.

I can tell you there are no Apple clients that depend on the lazy JIT.  
Please do not imply I am being secretive. I am simply busy. My  
objection is simple. Changing API simply for the sake of flipping  
default behavior does not seem like a good trade off to me. You may  
not feel the pain, but there are existing customers out there that are  
using previous llvm release who will not appreciate the API change.

To me, the decision is simple. People who have done the work in the  
past have made their choices. We better respect their choices unless  
we are replacing them with something better. Flipping the default  
isn't better. If someone wants to sign up to design a new API, that  
person(s) get to decide on the default behavior. It's simple as that.

For behavior / API change like this, we better extend llvm-config (or  
some other means) to expose that information. So the existing clients  
can at least ifdef the code.

Evan

On Oct 28, 2009, at 1:31 PM, Jeffrey Yasskin wrote:
> On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at
apple.com>
> wrote:
>>
>> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
>>
>>> From where I sit, this boils down to a very simple question (modulo
>>> Chris's point): Either choice will surprise some users. Which  
>>> surprise
>>> is worse? Personally, I'd always prefer correct but slow
behavior by
>>> default, and explicitly enabling dangerous (but in some cases fast)
>>> behavior.
>>
>> The behavior is only dangerous because people are using it in new  
>> and different ways.
>
> I'd point out that reid thought he made the JIT thread-safe in r22404
> back in 2005. Calling it from multiple threads isn't new and
> different, it's just subtly broken. I suggested changing the default
> because most people who run into this problem don't think they're
> doing anything unusual, and in fact their code works fine with the
> eager JIT. People shouldn't stumble into broken behavior, and defaults
> are good to prevent stumbling.
>
> To avoid misconceptions, Unladen Swallow added the line to turn off
> the lazy jit months ago, and we'll keep that line whatever the
> decision is here. We might take advantage of a thread-safe
> code-patching facility eventually, but we've been designing assuming
> nobody else will implement that for us. I favor changing the default
> because it'll help newbies, not because we need it for anything.
>
>>> I would also point out that it seems that most of the people new to
>>> the JIT are surprised by the current behavior, where as those who
>>> would be surprised by needing to enable lazy JIT are those long
>>> familiar with past behavior. In the OSS world, I always favor
easing
>>> adoption over maintaining the status quo.
>>
>> This argues for better documentation. I'd prefer for EE to abort if
>> user is asking for a known dangerous configuration (multi-threaded  
>> and lazy).
>
> I think that'd be a decent fix for the thread-safety problem. It'd
> require an extra check on each call to runJITOnFunctionUnlocked since
> laziness is set by a call, not on construction. Or, we could use the
> JIT lock to assert that it's never entered concurrently. On the other
> hand, Nicolas Geoffray objected when I suggested that in the bug.
>
>> The biggest argument I have for staying with lazy is llvm JIT is  
>> not a light weight JIT. It's designed to do all the codegen  
>> optimizations a normal static compiler would do. Non-lazy JIT is  
>> too slow.
>
> Óscar used the cost of the JIT as an argument _against_ the lazy JIT.
> Could you elaborate on why you think it's an argument in favor of lazy
> JITting?
>
>> I'd prefer not to change the behavior.
>
> I'm guessing, based on your vagueness, that there are some internal
> single-threaded Apple users who think that the lazy JIT is the best
> performing option for them and who don't want to add a line to their
> apps overriding the default. But it's hard for me or anyone else
> outside Apple to judge whether they ought to drive the default without
> a little more information about why the lazy JIT is good for them.
> Could you describe any features of their use that make the lazy JIT a
> good idea for them?
>
>> If we want to start using it in new and interesting ways, we should  
>> just design a new JIT.
>
>>> My meager 2 cents.
>>> -Chandler
>>>
>>> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at
google.com
>>> > wrote:
>>>> In r85295, in response to the discussion at
http://llvm.org/PR5184
>>>> (Lazy JIT ain't thread-safe), I changed the default JIT
from lazy
>>>> to
>>>> non-lazy. It has since come to my attention that this may have
been
>>>> the wrong change, so I wanted to ask you guys.
>>>>
>>>> A couple reasons to make the default non-lazy compilation:
>>>> * The lack of thread-safety surprises new users
>>>> * Crashes due to this will be rare and so hard to track down
>>>> * The current lazy scheme is almost never the right answer for
>>>> performance
>>>> * It's only one line of code to turn on lazy compilation
when it is
>>>> the right answer for you.
>>>>
>>>> And a couple to default to lazy compilation:
>>>> * It's safe for single-threaded code.
>>>> * There are existing users who have assumed this default.
>>>> * PPC and ARM don't support non-lazy compilation yet (the
tests
>>>> currently run the lazy jit).
>>>> * Gratuitous changes are bad.
>>>>
>>>> Thoughts?
>>>>
>>>> We can choose the default for lli separately from the JIT's
>>>> default if we want.
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>