llvm dev - Mar 2015 - [LLVMdev] How will OrcJIT guarantee thread-safety when a function is asked to be re generated?

Hi Sanjoy,
> You need the hijack-return-pc approach *in addition* to a call-site
> patching approach.  Modifying the return PC lets you guarantee that
> nothing will *return* into the old generated code. To guarantee that
> nothing will *call* into it either you could use a double indirection
> (all calls go through a trampoline) or patchpoints.
You need to hijack the return addresses if you want to delete the original
function body immediately, but what if you just leave the original in-place
until you return through it? That is the scheme that Pete and I had been
talking about. On the one-hand that means that the old code may live for an
arbitrarily long time, on the other hand it saves you from implementing
some complicated infrastructure. I suspect that in most JIT use-cases the
cost of keeping the duplicate function around will be minimal, but I don't
actually have any data to back that up. :)

It's worth noting that, as per its design goals, Orc is agnostic about all
this. Either scheme, or both, should be able to be implemented in the new
framework. It's just that nobody has implemented it yet.

- Lang.

On Fri, Mar 20, 2015 at 4:55 AM, Sanjoy Das <sanjoy at
playingwithpointers.com>
wrote:
> > That all makes sense. What are your thoughts on the trade-offs of this
vs
> > the patchpoint approach though? If you can modify previously
executable
> > memory it seems like the patchpoint approach would have lower
overhead,
> > unless you have a truly huge number of callsites to update?
>
> You need the hijack-return-pc approach *in addition* to a call-site
> patching approach.  Modifying the return PC lets you guarantee that
> nothing will *return* into the old generated code. To guarantee that
> nothing will *call* into it either you could use a double indirection
> (all calls go through a trampoline) or patchpoints.
>
> -- Sanjoy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150320/fabfa31d/attachment.html>

On Thu, Mar 19, 2015 at 3:28 PM, Lang Hames <lhames at gmail.com>
wrote:
> Hi Sanjoy,
>
>> You need the hijack-return-pc approach *in addition* to a call-site
>> patching approach.  Modifying the return PC lets you guarantee that
>> nothing will *return* into the old generated code. To guarantee that
>> nothing will *call* into it either you could use a double indirection
>> (all calls go through a trampoline) or patchpoints.
>
> You need to hijack the return addresses if you want to delete the original
> function body immediately, but what if you just leave the original in-place
> until you return through it? That is the scheme that Pete and I had been
> talking about. On the one-hand that means that the old code may live for an
> arbitrarily long time, on the other hand it saves you from implementing
some
> complicated infrastructure. I suspect that in most JIT use-cases the cost
of
> keeping the duplicate function around will be minimal, but I don't
actually
> have any data to back that up. :)
I agree that the cost of keeping around the old generated code is
likely to be small.

However, you may need to ensure that nothing returns into the original
code for correctness:  the reason you're replacing an old compilation
with a new one could very well be that you want to do something that
makes the old piece of code incorrect to execute.  For instance, in
the old compilation you may have assumed that class X does not have
any subclasses (and you devirtualized some methods based on this
assumption), but now you're about to dlopen a shared object that will
introduce a class subclassing from X.  Your devirtualization decisions
will be invalid after you've run dlopen, and it would no longer be
correct to execute the old bit of code.

You could also do this by appending all calls / invokes with

  some_other_method();
  if (!this_method->is_still_valid) {
    do_on_stack_replacement(); // noreturn
    unreachable;
  }
  // continue normal execution

and that's perfectly sound because you'll always check validity before
re-entry into a stack frame.  Is this what you meant by frame
residence counting?
> It's worth noting that, as per its design goals, Orc is agnostic about
all
> this. Either scheme, or both, should be able to be implemented in the new
> framework. It's just that nobody has implemented it yet.
So this is the Patches Welcome(TM) part. :P

-- Sanjoy

Sanjoy, so does the JVM do what you described? I always thought the JVM has
a trampoline as methods are HotSpot compiled.

On Thu, Mar 19, 2015 at 3:39 PM, Sanjoy Das <sanjoy at
playingwithpointers.com>
wrote:
> On Thu, Mar 19, 2015 at 3:28 PM, Lang Hames <lhames at gmail.com>
wrote:
> > Hi Sanjoy,
> >
> >> You need the hijack-return-pc approach *in addition* to a
call-site
> >> patching approach.  Modifying the return PC lets you guarantee
that
> >> nothing will *return* into the old generated code. To guarantee
that
> >> nothing will *call* into it either you could use a double
indirection
> >> (all calls go through a trampoline) or patchpoints.
> >
> > You need to hijack the return addresses if you want to delete the
> original
> > function body immediately, but what if you just leave the original
> in-place
> > until you return through it? That is the scheme that Pete and I had
been
> > talking about. On the one-hand that means that the old code may live
for
> an
> > arbitrarily long time, on the other hand it saves you from
implementing
> some
> > complicated infrastructure. I suspect that in most JIT use-cases the
> cost of
> > keeping the duplicate function around will be minimal, but I don't
> actually
> > have any data to back that up. :)
>
> I agree that the cost of keeping around the old generated code is
> likely to be small.
>
> However, you may need to ensure that nothing returns into the original
> code for correctness:  the reason you're replacing an old compilation
> with a new one could very well be that you want to do something that
> makes the old piece of code incorrect to execute.  For instance, in
> the old compilation you may have assumed that class X does not have
> any subclasses (and you devirtualized some methods based on this
> assumption), but now you're about to dlopen a shared object that will
> introduce a class subclassing from X.  Your devirtualization decisions
> will be invalid after you've run dlopen, and it would no longer be
> correct to execute the old bit of code.
>
> You could also do this by appending all calls / invokes with
>
>   some_other_method();
>   if (!this_method->is_still_valid) {
>     do_on_stack_replacement(); // noreturn
>     unreachable;
>   }
>   // continue normal execution
>
> and that's perfectly sound because you'll always check validity
before
> re-entry into a stack frame.  Is this what you meant by frame
> residence counting?
>
> > It's worth noting that, as per its design goals, Orc is agnostic
about
> all
> > this. Either scheme, or both, should be able to be implemented in the
new
> > framework. It's just that nobody has implemented it yet.
>
> So this is the Patches Welcome(TM) part. :P
>
> -- Sanjoy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150319/fa9cb32c/attachment.html>

Hi Sanjoy,

I hadn't started thinking about deoptimization yet. That makes perfect
sense though.
> So this is the Patches Welcome(TM) part. :P
Yep. :)

Cheers,
Lang.

On Fri, Mar 20, 2015 at 9:39 AM, Sanjoy Das <sanjoy at
playingwithpointers.com>
wrote:
> On Thu, Mar 19, 2015 at 3:28 PM, Lang Hames <lhames at gmail.com>
wrote:
> > Hi Sanjoy,
> >
> >> You need the hijack-return-pc approach *in addition* to a
call-site
> >> patching approach.  Modifying the return PC lets you guarantee
that
> >> nothing will *return* into the old generated code. To guarantee
that
> >> nothing will *call* into it either you could use a double
indirection
> >> (all calls go through a trampoline) or patchpoints.
> >
> > You need to hijack the return addresses if you want to delete the
> original
> > function body immediately, but what if you just leave the original
> in-place
> > until you return through it? That is the scheme that Pete and I had
been
> > talking about. On the one-hand that means that the old code may live
for
> an
> > arbitrarily long time, on the other hand it saves you from
implementing
> some
> > complicated infrastructure. I suspect that in most JIT use-cases the
> cost of
> > keeping the duplicate function around will be minimal, but I don't
> actually
> > have any data to back that up. :)
>
> I agree that the cost of keeping around the old generated code is
> likely to be small.
>
> However, you may need to ensure that nothing returns into the original
> code for correctness:  the reason you're replacing an old compilation
> with a new one could very well be that you want to do something that
> makes the old piece of code incorrect to execute.  For instance, in
> the old compilation you may have assumed that class X does not have
> any subclasses (and you devirtualized some methods based on this
> assumption), but now you're about to dlopen a shared object that will
> introduce a class subclassing from X.  Your devirtualization decisions
> will be invalid after you've run dlopen, and it would no longer be
> correct to execute the old bit of code.
>
> You could also do this by appending all calls / invokes with
>
>   some_other_method();
>   if (!this_method->is_still_valid) {
>     do_on_stack_replacement(); // noreturn
>     unreachable;
>   }
>   // continue normal execution
>
> and that's perfectly sound because you'll always check validity
before
> re-entry into a stack frame.  Is this what you meant by frame
> residence counting?
>
> > It's worth noting that, as per its design goals, Orc is agnostic
about
> all
> > this. Either scheme, or both, should be able to be implemented in the
new
> > framework. It's just that nobody has implemented it yet.
>
> So this is the Patches Welcome(TM) part. :P
>
> -- Sanjoy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150325/cc0e1c88/attachment.html>