Nicolas Geoffray
2009-Oct-29 21:30 UTC
[LLVMdev] Should LLVM JIT default to lazy or non-lazy?
Hi Jeffrey, Jeffrey Yasskin wrote:> Cool, I'll start implementing it. >Great! Thanks. Just to clarify things: on my end, it doesn't really matter what is the default behavior, as long as vmkit can continue to have the existing behavior of lazy compilation. With Chris' solution, I was wondering how you would implement the getPointerToFunction{Eager, Lazy} functions when the getPointerToFunction is called by the JIT, not the user. For example, when Function F calls Function G and the JIT needs an address for G (either a callback or the function address), how will it know if it must call getPointerToFunctionEager or getPointerToFunctionLazy? Do you plan on continuing having a flag that enables/disables lazy compilation and poll this flag on each function call? How is that different than the existing system? Thanks, Nicolas> Thanks all for the decision! > > On Thu, Oct 29, 2009 at 11:03 AM, Evan Cheng <evan.cheng at apple.com> wrote: > >> I have no objection to Chris' proposal. >> >> Evan >> >> On Oct 29, 2009, at 9:45 AM, Jeffrey Yasskin wrote: >> >> >>> Are you objecting to Chris's proposal? I was waiting to implement it >>> until you replied so I wouldn't have to implement two things. I >>> disagree with a lot of what you wrote below, but it's not worth >>> arguing about if there's a compromise we can both live with. >>> >>> On Thu, Oct 29, 2009 at 12:36 AM, Evan Cheng <evan.cheng at apple.com> wrote: >>> >>>> There is nothing here that prevents users from enabling or disabling lazy >>>> JIT. We're talk about flipping the default behavior. That does not make the >>>> API any better or worse. Nor does it fix the inherent thread safety issue. >>>> >>>> I can tell you there are no Apple clients that depend on the lazy JIT. >>>> Please do not imply I am being secretive. I am simply busy. My objection is >>>> simple. Changing API simply for the sake of flipping default behavior does >>>> not seem like a good trade off to me. You may not feel the pain, but there >>>> are existing customers out there that are using previous llvm release who >>>> will not appreciate the API change. >>>> >>>> To me, the decision is simple. People who have done the work in the past >>>> have made their choices. We better respect their choices unless we are >>>> replacing them with something better. Flipping the default isn't better. If >>>> someone wants to sign up to design a new API, that person(s) get to decide >>>> on the default behavior. It's simple as that. >>>> >>>> For behavior / API change like this, we better extend llvm-config (or some >>>> other means) to expose that information. So the existing clients can at >>>> least ifdef the code. >>>> >>>> Evan >>>> >>>> On Oct 28, 2009, at 1:31 PM, Jeffrey Yasskin wrote: >>>> >>>> >>>>> On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at apple.com> wrote: >>>>> >>>>>> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote: >>>>>> >>>>>> >>>>>>> From where I sit, this boils down to a very simple question (modulo >>>>>>> Chris's point): Either choice will surprise some users. Which surprise >>>>>>> is worse? Personally, I'd always prefer correct but slow behavior by >>>>>>> default, and explicitly enabling dangerous (but in some cases fast) >>>>>>> behavior. >>>>>>> >>>>>> The behavior is only dangerous because people are using it in new and >>>>>> different ways. >>>>>> >>>>> I'd point out that reid thought he made the JIT thread-safe in r22404 >>>>> back in 2005. Calling it from multiple threads isn't new and >>>>> different, it's just subtly broken. I suggested changing the default >>>>> because most people who run into this problem don't think they're >>>>> doing anything unusual, and in fact their code works fine with the >>>>> eager JIT. People shouldn't stumble into broken behavior, and defaults >>>>> are good to prevent stumbling. >>>>> >>>>> To avoid misconceptions, Unladen Swallow added the line to turn off >>>>> the lazy jit months ago, and we'll keep that line whatever the >>>>> decision is here. We might take advantage of a thread-safe >>>>> code-patching facility eventually, but we've been designing assuming >>>>> nobody else will implement that for us. I favor changing the default >>>>> because it'll help newbies, not because we need it for anything. >>>>> >>>>> >>>>>>> I would also point out that it seems that most of the people new to >>>>>>> the JIT are surprised by the current behavior, where as those who >>>>>>> would be surprised by needing to enable lazy JIT are those long >>>>>>> familiar with past behavior. In the OSS world, I always favor easing >>>>>>> adoption over maintaining the status quo. >>>>>>> >>>>>> This argues for better documentation. I'd prefer for EE to abort if user >>>>>> is asking for a known dangerous configuration (multi-threaded and lazy). >>>>>> >>>>> I think that'd be a decent fix for the thread-safety problem. It'd >>>>> require an extra check on each call to runJITOnFunctionUnlocked since >>>>> laziness is set by a call, not on construction. Or, we could use the >>>>> JIT lock to assert that it's never entered concurrently. On the other >>>>> hand, Nicolas Geoffray objected when I suggested that in the bug. >>>>> >>>>> >>>>>> The biggest argument I have for staying with lazy is llvm JIT is not a >>>>>> light weight JIT. It's designed to do all the codegen optimizations a normal >>>>>> static compiler would do. Non-lazy JIT is too slow. >>>>>> >>>>> Óscar used the cost of the JIT as an argument _against_ the lazy JIT. >>>>> Could you elaborate on why you think it's an argument in favor of lazy >>>>> JITting? >>>>> >>>>> >>>>>> I'd prefer not to change the behavior. >>>>>> >>>>> I'm guessing, based on your vagueness, that there are some internal >>>>> single-threaded Apple users who think that the lazy JIT is the best >>>>> performing option for them and who don't want to add a line to their >>>>> apps overriding the default. But it's hard for me or anyone else >>>>> outside Apple to judge whether they ought to drive the default without >>>>> a little more information about why the lazy JIT is good for them. >>>>> Could you describe any features of their use that make the lazy JIT a >>>>> good idea for them? >>>>> >>>>> >>>>>> If we want to start using it in new and interesting ways, we should just >>>>>> design a new JIT. >>>>>> >>>>>>> My meager 2 cents. >>>>>>> -Chandler >>>>>>> >>>>>>> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at google.com> >>>>>>> wrote: >>>>>>> >>>>>>>> In r85295, in response to the discussion at http://llvm.org/PR5184 >>>>>>>> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy to >>>>>>>> non-lazy. It has since come to my attention that this may have been >>>>>>>> the wrong change, so I wanted to ask you guys. >>>>>>>> >>>>>>>> A couple reasons to make the default non-lazy compilation: >>>>>>>> * The lack of thread-safety surprises new users >>>>>>>> * Crashes due to this will be rare and so hard to track down >>>>>>>> * The current lazy scheme is almost never the right answer for >>>>>>>> performance >>>>>>>> * It's only one line of code to turn on lazy compilation when it is >>>>>>>> the right answer for you. >>>>>>>> >>>>>>>> And a couple to default to lazy compilation: >>>>>>>> * It's safe for single-threaded code. >>>>>>>> * There are existing users who have assumed this default. >>>>>>>> * PPC and ARM don't support non-lazy compilation yet (the tests >>>>>>>> currently run the lazy jit). >>>>>>>> * Gratuitous changes are bad. >>>>>>>> >>>>>>>> Thoughts? >>>>>>>> >>>>>>>> We can choose the default for lli separately from the JIT's default if >>>>>>>> we want. >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> LLVM Developers mailing list >>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>>>>>> >>>>>> >>>> >> > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >
Jeffrey Yasskin
2009-Oct-29 21:55 UTC
[LLVMdev] Should LLVM JIT default to lazy or non-lazy?
On Thu, Oct 29, 2009 at 2:30 PM, Nicolas Geoffray <nicolas.geoffray at lip6.fr> wrote:> Hi Jeffrey, > > Jeffrey Yasskin wrote: >> >> Cool, I'll start implementing it. >> > > Great! Thanks. > > Just to clarify things: on my end, it doesn't really matter what is the > default behavior, as long as vmkit can continue to have the existing > behavior of lazy compilation. With Chris' solution, I was wondering how you > would implement the getPointerToFunction{Eager, Lazy} functions when the > getPointerToFunction is called by the JIT, not the user. For example, when > Function F calls Function G and the JIT needs an address for G (either a > callback or the function address), how will it know if it must call > getPointerToFunctionEager or getPointerToFunctionLazy? Do you plan on > continuing having a flag that enables/disables lazy compilation and poll > this flag on each function call? How is that different than the existing > system?Semantically, I'll thread the flag through all the calls that may eventually need to recursively call getPointerToFunction. To implement that without having to modify lots of calls, I'll probably replace the current public default eager/lazy setting with a private flag with values {Unknown, Lazy, Eager}, set it on entry and exit of getPointerToFunction, and check it on each internal recursive call. The difference with the current system is that the user is forced to set the flag to their desired value whenever they call into the JIT, rather than relying on a default. That choice then propagates through the whole recursive tree of codegens, without affecting the next tree. Note that I'm using getPointerToFunction as an abbreviation for the 3ish public functions that'll need to take this option.>> Thanks all for the decision! >> >> On Thu, Oct 29, 2009 at 11:03 AM, Evan Cheng <evan.cheng at apple.com> wrote: >> >>> >>> I have no objection to Chris' proposal. >>> >>> Evan >>> >>> On Oct 29, 2009, at 9:45 AM, Jeffrey Yasskin wrote: >>> >>> >>>> >>>> Are you objecting to Chris's proposal? I was waiting to implement it >>>> until you replied so I wouldn't have to implement two things. I >>>> disagree with a lot of what you wrote below, but it's not worth >>>> arguing about if there's a compromise we can both live with. >>>> >>>> On Thu, Oct 29, 2009 at 12:36 AM, Evan Cheng <evan.cheng at apple.com> >>>> wrote: >>>> >>>>> >>>>> There is nothing here that prevents users from enabling or disabling >>>>> lazy >>>>> JIT. We're talk about flipping the default behavior. That does not make >>>>> the >>>>> API any better or worse. Nor does it fix the inherent thread safety >>>>> issue. >>>>> >>>>> I can tell you there are no Apple clients that depend on the lazy JIT. >>>>> Please do not imply I am being secretive. I am simply busy. My >>>>> objection is >>>>> simple. Changing API simply for the sake of flipping default behavior >>>>> does >>>>> not seem like a good trade off to me. You may not feel the pain, but >>>>> there >>>>> are existing customers out there that are using previous llvm release >>>>> who >>>>> will not appreciate the API change. >>>>> >>>>> To me, the decision is simple. People who have done the work in the >>>>> past >>>>> have made their choices. We better respect their choices unless we are >>>>> replacing them with something better. Flipping the default isn't >>>>> better. If >>>>> someone wants to sign up to design a new API, that person(s) get to >>>>> decide >>>>> on the default behavior. It's simple as that. >>>>> >>>>> For behavior / API change like this, we better extend llvm-config (or >>>>> some >>>>> other means) to expose that information. So the existing clients can at >>>>> least ifdef the code. >>>>> >>>>> Evan >>>>> >>>>> On Oct 28, 2009, at 1:31 PM, Jeffrey Yasskin wrote: >>>>> >>>>> >>>>>> >>>>>> On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at apple.com> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote: >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> From where I sit, this boils down to a very simple question (modulo >>>>>>>> Chris's point): Either choice will surprise some users. Which >>>>>>>> surprise >>>>>>>> is worse? Personally, I'd always prefer correct but slow behavior by >>>>>>>> default, and explicitly enabling dangerous (but in some cases fast) >>>>>>>> behavior. >>>>>>>> >>>>>>> >>>>>>> The behavior is only dangerous because people are using it in new and >>>>>>> different ways. >>>>>>> >>>>>> >>>>>> I'd point out that reid thought he made the JIT thread-safe in r22404 >>>>>> back in 2005. Calling it from multiple threads isn't new and >>>>>> different, it's just subtly broken. I suggested changing the default >>>>>> because most people who run into this problem don't think they're >>>>>> doing anything unusual, and in fact their code works fine with the >>>>>> eager JIT. People shouldn't stumble into broken behavior, and defaults >>>>>> are good to prevent stumbling. >>>>>> >>>>>> To avoid misconceptions, Unladen Swallow added the line to turn off >>>>>> the lazy jit months ago, and we'll keep that line whatever the >>>>>> decision is here. We might take advantage of a thread-safe >>>>>> code-patching facility eventually, but we've been designing assuming >>>>>> nobody else will implement that for us. I favor changing the default >>>>>> because it'll help newbies, not because we need it for anything. >>>>>> >>>>>> >>>>>>>> >>>>>>>> I would also point out that it seems that most of the people new to >>>>>>>> the JIT are surprised by the current behavior, where as those who >>>>>>>> would be surprised by needing to enable lazy JIT are those long >>>>>>>> familiar with past behavior. In the OSS world, I always favor easing >>>>>>>> adoption over maintaining the status quo. >>>>>>>> >>>>>>> >>>>>>> This argues for better documentation. I'd prefer for EE to abort if >>>>>>> user >>>>>>> is asking for a known dangerous configuration (multi-threaded and >>>>>>> lazy). >>>>>>> >>>>>> >>>>>> I think that'd be a decent fix for the thread-safety problem. It'd >>>>>> require an extra check on each call to runJITOnFunctionUnlocked since >>>>>> laziness is set by a call, not on construction. Or, we could use the >>>>>> JIT lock to assert that it's never entered concurrently. On the other >>>>>> hand, Nicolas Geoffray objected when I suggested that in the bug. >>>>>> >>>>>> >>>>>>> >>>>>>> The biggest argument I have for staying with lazy is llvm JIT is not >>>>>>> a >>>>>>> light weight JIT. It's designed to do all the codegen optimizations a >>>>>>> normal >>>>>>> static compiler would do. Non-lazy JIT is too slow. >>>>>>> >>>>>> >>>>>> Óscar used the cost of the JIT as an argument _against_ the lazy JIT. >>>>>> Could you elaborate on why you think it's an argument in favor of lazy >>>>>> JITting? >>>>>> >>>>>> >>>>>>> >>>>>>> I'd prefer not to change the behavior. >>>>>>> >>>>>> >>>>>> I'm guessing, based on your vagueness, that there are some internal >>>>>> single-threaded Apple users who think that the lazy JIT is the best >>>>>> performing option for them and who don't want to add a line to their >>>>>> apps overriding the default. But it's hard for me or anyone else >>>>>> outside Apple to judge whether they ought to drive the default without >>>>>> a little more information about why the lazy JIT is good for them. >>>>>> Could you describe any features of their use that make the lazy JIT a >>>>>> good idea for them? >>>>>> >>>>>> >>>>>>> >>>>>>> If we want to start using it in new and interesting ways, we should >>>>>>> just >>>>>>> design a new JIT. >>>>>>> >>>>>>>> >>>>>>>> My meager 2 cents. >>>>>>>> -Chandler >>>>>>>> >>>>>>>> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin >>>>>>>> <jyasskin at google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> In r85295, in response to the discussion at http://llvm.org/PR5184 >>>>>>>>> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy >>>>>>>>> to >>>>>>>>> non-lazy. It has since come to my attention that this may have been >>>>>>>>> the wrong change, so I wanted to ask you guys. >>>>>>>>> >>>>>>>>> A couple reasons to make the default non-lazy compilation: >>>>>>>>> * The lack of thread-safety surprises new users >>>>>>>>> * Crashes due to this will be rare and so hard to track down >>>>>>>>> * The current lazy scheme is almost never the right answer for >>>>>>>>> performance >>>>>>>>> * It's only one line of code to turn on lazy compilation when it is >>>>>>>>> the right answer for you. >>>>>>>>> >>>>>>>>> And a couple to default to lazy compilation: >>>>>>>>> * It's safe for single-threaded code. >>>>>>>>> * There are existing users who have assumed this default. >>>>>>>>> * PPC and ARM don't support non-lazy compilation yet (the tests >>>>>>>>> currently run the lazy jit). >>>>>>>>> * Gratuitous changes are bad. >>>>>>>>> >>>>>>>>> Thoughts? >>>>>>>>> >>>>>>>>> We can choose the default for lli separately from the JIT's default >>>>>>>>> if >>>>>>>>> we want. >>>>>>>>> _______________________________________________ >>>>>>>>> LLVM Developers mailing list >>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>> >>> >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > >
2009/10/29 Nicolas Geoffray <nicolas.geoffray at lip6.fr>:> example, when Function F calls Function G and the JIT needs an address > for G (either a callback or the function address), how will it know if > it must call getPointerToFunctionEager or getPointerToFunctionLazy? DoWell, if a JIT function only refers to another function, I would assume propagating the caller's behaviour should be used. So, when the call is actually made, you can evaluate (lazy), or had it pre-evaluated (eager). This behaviour can easily be recursive. cheers, --renato Reclaim your digital rights, eliminate DRM, learn more at http://www.defectivebydesign.org/what_is_drm
On 2009-10-29 23:55, Jeffrey Yasskin wrote:> On Thu, Oct 29, 2009 at 2:30 PM, Nicolas Geoffray > <nicolas.geoffray at lip6.fr> wrote: > >> Hi Jeffrey, >> >> Jeffrey Yasskin wrote: >> >>> Cool, I'll start implementing it. >>> >>> >> Great! Thanks. >> >> Just to clarify things: on my end, it doesn't really matter what is the >> default behavior, as long as vmkit can continue to have the existing >> behavior of lazy compilation. With Chris' solution, I was wondering how you >> would implement the getPointerToFunction{Eager, Lazy} functions when the >> getPointerToFunction is called by the JIT, not the user. For example, when >> Function F calls Function G and the JIT needs an address for G (either a >> callback or the function address), how will it know if it must call >> getPointerToFunctionEager or getPointerToFunctionLazy? Do you plan on >> continuing having a flag that enables/disables lazy compilation and poll >> this flag on each function call? How is that different than the existing >> system? >> > > Semantically, I'll thread the flag through all the calls that may > eventually need to recursively call getPointerToFunction. To implement > that without having to modify lots of calls, I'll probably replace the > current public default eager/lazy setting with a private flag with > values {Unknown, Lazy, Eager}, set it on entry and exit of > getPointerToFunction, and check it on each internal recursive call. > The difference with the current system is that the user is forced to > set the flag to their desired value whenever they call into the JIT, > rather than relying on a default. That choice then propagates through > the whole recursive tree of codegens, without affecting the next tree. > > Note that I'm using getPointerToFunction as an abbreviation for the > 3ish public functions that'll need to take this option.The documentation should also be updated (http://llvm.org/docs/ProgrammersManual.html#threading) to reflect what one needs to do, to ensure thread-safe JITing. Also does every JIT target support non-lazy JITing now? See PR4816, last time I checked (r83242) it only worked on X86, and failed on PPC; so I had to keep lazy JITing enabled even if its not what I want for many reasons. Also perhaps the lazy compilation stub should spin waiting on a lock (implemented using atomics), and the compilation callback should execute while holding the lock just before patching the callsite, so it would look like this in pseudocode: callsite_patch_state = 0;// for each callsite one byte of memory callsite: if (atomic_load(&callsite_patch_state) == 2) { //fast-path: already compiled and patched patchsite: jmp <nop nop nop nop nop nop nop nop> // will be patched } // not yet patched, it may already be compiling if (atomic_test_and_set(&callsite_patch_state, 0, 1) == 0) { // not yet compiling, set state to compiling, and start compiling call CompilationCallBack // set state to patched atomic_set(&callsite_patch_state, 2) } // wait for patched state while (atomic_load(&callsite_patch_state) != 2) { waitJIT(); } // serialize CPUID patchsite2: // execute new code jmp <nop nop nop nop nop nop nop nop> // will be patched waitJIT: jitLock() jitUnlock() ^This should be consistent with the Intel Manual's requirements on XMC, which has a similar algorithm, except for the fast-path. CompilationCallBack: jitLock(); if (isJITed(F)) {jitUnlock(); return;} JIT function patch_callsite(&patchsite, compiledFunctionAddress); patch_callsite(&patchsite2, compiledFunctionAddress); setJITed(F, true); jitUnlock(); This way once it is compiled the callsite will only execute: atomic_load(&callsite_patch_state) == 2 jmp compiledFunctionAddress Best regards, --Edwin
Apparently Analagous Threads
- [LLVMdev] Should LLVM JIT default to lazy or non-lazy?
- [LLVMdev] Should LLVM JIT default to lazy or non-lazy?
- [LLVMdev] Should LLVM JIT default to lazy or non-lazy?
- [LLVMdev] Should LLVM JIT default to lazy or non-lazy?
- [LLVMdev] Proposal for a new LLVM concurrency memory model