llvm dev - May 2014 - [LLVMdev] Finding safe thread suspension points while JIT-ing (was: Add pass run listeners to the pass manager.)

Sounds good. Lets get started by nailing down the C API and semantics for this
first.

I mirrored the C API for the LLVM context diagnostic handler and used Andy’s
suggested name for the callback.
The opaque handle was suggested by Duncan and can provide optional user
specified information that is
provided back during the callback (i.e. barrier, etc).

Cheers,
Juergen 

Core.h:
typedef void (*LLVMMaySuspendCallback)(LLVMContextRef, void *);

/**
 * Set the may-suspend callback function for this context.
 *
 * @see LLVMContext::setMaySuspendCallback()
 */
void LLVMContextSetMaySuspendCallback(LLVMContextRef C,
                                      LLVMMaySuspendCallback Callback,
                                      void *OpaqueHandle);

LLVMContext.h:
/// \brief Registers a may-suspend callback with the context.
///
/// The may-suspend callback function may be called by LLVM to transfer
/// control back to the client that invoked the LLVM compilation. The client
/// is not garantueed to ever receive this callback. It is at the sole
/// discretion of LLVM to do so and only if it can guarantee that suspending
/// the thread won't block any forward progress in other LLVM contexts.
void setMaySuspendCallback(MaySuspendCallbackTy Callback, void *OpaqueHandle);

/// \brief Calls the may-suspend callback (if applicable).
///
/// This transfers control back to the client, which may suspend the current
/// thread. Only call this method when LLVM doesn't hold any global mutex or
/// cannot block the execution in another LLVM context.
void callMaySuspendCallback();

On May 12, 2014, at 5:26 PM, Nick Lewycky <nlewycky at google.com> wrote:
> Would you (or anyone) oppose a simple maySuspendContext() callback API? It
would mean nothing more than the thread(s) for a given LLVM context can be
suspended independent from other contexts.
> 
> I think this is the right approach. So a given thread hits a safe point, it
optionally calls a "suspend check" or "i an safe to suspend right
now" callback if set. It doesn't stop other threads, it doesn't
continue until the function returns.
> 
> If you want to stop all threads then the user callback may contain a
barrier and count down how many threads have stopped until it sees all of them.
> 
> Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140513/2ea7888c/attachment.html>

Given the use case (user mode scheduling), I'm not going to oppose this 
proposal.  I would like to see a couple of things clarified 
documentation wise:
- When is this interface valid?  (i.e. the single thread case)
- If a context does have multiple threads, is this called once per 
thread?  Or once per thread group after internal coordination? (you can 
write this out of scope if desired)
- If we later introduce multiple threads, and this mechanism doesn't 
support it, what will happen?  Will the function just not be called?
- You hint at this already, but clarifying the state of the current 
context at a suspend point would be helpful.

Here's a possible draft that includes the above:
The may-suspend callback function may be called by LLVM to transfer 
control back to the client that invoked the LLVM compilation. This can 
be used to yield control of the thread, or perform periodic work needed 
by the client.  There is no guaranteed frequency at which callbacks must 
occur; in fact, the client is not guaranteed to ever receive this 
callback. It is at the sole discretion of LLVM to do so and only if it 
can guarantee that suspending the thread won't block any forward 
progress in other LLVM contexts.

At a suspend point, the state of the current LLVM context is 
intentionally undefined.  No assumptions about it can or should be 
made.  In particular, call backs into the context are not supported 
until the suspend function returns control to LLVM.  Other LLVM contexts 
are unaffected.

Currently, LLVM assumes one thread per LLVM context.  If, or when, we 
introduce multiple threads, this interface will not be available for 
contexts which opt-in to the thread pool model.  We may extend this 
interface at a later time to support thread pools, but for the moment, 
that use case is explicitly unsupported.


p.s. Bikeshed wise, might "yield" be a better term than
"suspend" here?

Philip

On 05/13/2014 11:49 AM, Juergen Ributzka wrote:
> Sounds good. Lets get started by nailing down the C API and semantics 
> for this first.
>
> I mirrored the C API for the LLVM context diagnostic handler and used 
> Andy's suggested name for the callback.
> The opaque handle was suggested by Duncan and can provide optional 
> user specified information that is
> provided back during the callback (i.e. barrier, etc).
>
> Cheers,
> Juergen
>
> Core.h:
> typedef void (*LLVMMaySuspendCallback)(LLVMContextRef, void *);
>
> /**
>  * Set the may-suspend callback function for this context.
>  *
>  * @see LLVMContext::setMaySuspendCallback()
>  */
> void LLVMContextSetMaySuspendCallback(LLVMContextRef C,
>                                       LLVMMaySuspendCallback Callback,
>                                       void *OpaqueHandle);
>
> LLVMContext.h:
> /// \brief Registers a may-suspend callback with the context.
> ///
> /// The may-suspend callback function may be called by LLVM to transfer
> /// control back to the client that invoked the LLVM compilation. The 
> client
> /// is not garantueed to ever receive this callback. It is at the sole
> /// discretion of LLVM to do so and only if it can guarantee that 
> suspending
> /// the thread won't block any forward progress in other LLVM contexts.
> void setMaySuspendCallback(MaySuspendCallbackTy Callback, void 
> *OpaqueHandle);
>
> /// \brief Calls the may-suspend callback (if applicable).
> ///
> /// This transfers control back to the client, which may suspend the 
> current
> /// thread. Only call this method when LLVM doesn't hold any global 
> mutex or
> /// cannot block the execution in another LLVM context.
> void callMaySuspendCallback();
>
> On May 12, 2014, at 5:26 PM, Nick Lewycky <nlewycky at google.com 
> <mailto:nlewycky at google.com>> wrote:
>
>>     Would you (or anyone) oppose a simple maySuspendContext()
>>     callback API? It would mean nothing more than the thread(s) for a
>>     given LLVM context can be suspended independent from other
contexts.
>>
>>
>> I think this is the right approach. So a given thread hits a safe 
>> point, it optionally calls a "suspend check" or "i an
safe to suspend
>> right now" callback if set. It doesn't stop other threads, it
doesn't
>> continue until the function returns.
>>
>> If you want to stop all threads then the user callback may contain a 
>> barrier and count down how many threads have stopped until it sees 
>> all of them.
>>
>> Nick
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/b28f7a32/attachment.html>

On Tue, May 13, 2014 at 12:49 PM, Juergen Ributzka <juergen at
apple.com>wrote:
> Sounds good. Lets get started by nailing down the C API and semantics for
> this first.

If we're going in this direction, you might want to back out the pass
callback changes sooner rather than later...
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/b03a8122/attachment.html>

I generally like the API being proposed, especially structurally.

On Thu, May 15, 2014 at 10:50 AM, Philip Reames
<listmail at philipreames.com>wrote:
>  Given the use case (user mode scheduling), I'm not going to oppose
this
> proposal.  I would like to see a couple of things clarified documentation
> wise:
> - When is this interface valid?  (i.e. the single thread case)
> - If a context does have multiple threads, is this called once per
> thread?  Or once per thread group after internal coordination?  (you can
> write this out of scope if desired)
> - If we later introduce multiple threads, and this mechanism doesn't
> support it, what will happen?  Will the function just not be called?
> - You hint at this already, but clarifying the state of the current
> context at a suspend point would be helpful.
>
+1, i think clarifying a lot of this context and getting it baked into the
docs is key.

>
> Here's a possible draft that includes the above:
> The may-suspend callback function may be called by LLVM to transfer
> control back to the client that invoked the LLVM compilation. This can be
> used to yield control of the thread, or perform periodic work needed by the
> client.  There is no guaranteed frequency at which callbacks must occur; in
> fact, the client is not guaranteed to ever receive this callback. It is at
> the sole discretion of LLVM to do so and only if it can guarantee that
> suspending the thread won't block any forward progress in other LLVM
> contexts.
>
> At a suspend point, the state of the current LLVM context is intentionally
> undefined.  No assumptions about it can or should be made.  In particular,
> call backs into the context are not supported until the suspend function
> returns control to LLVM.  Other LLVM contexts are unaffected.
>
> Currently, LLVM assumes one thread per LLVM context.  If, or when, we
> introduce multiple threads, this interface will not be available for
> contexts which opt-in to the thread pool model.  We may extend this
> interface at a later time to support thread pools, but for the moment, that
> use case is explicitly unsupported.
>
>
> p.s. Bikeshed wise, might "yield" be a better term than
"suspend" here?
>
I have to say I like yield better. I'm not set on it.

I don't think we need the "may" prefix either way. Wether or not
(or for
how long) a suspension happened is (necessarily) not something that LLVM
should be observing. I think the caller should be assuming that it *will*
happen, and so the API needn't be cagey about it.

>
> Philip
>
>
> On 05/13/2014 11:49 AM, Juergen Ributzka wrote:
>
> Sounds good. Lets get started by nailing down the C API and semantics for
> this first.
>
>  I mirrored the C API for the LLVM context diagnostic handler and used
> Andy’s suggested name for the callback.
> The opaque handle was suggested by Duncan and can provide optional user
> specified information that is
> provided back during the callback (i.e. barrier, etc).
>
>  Cheers,
> Juergen
>
>  Core.h:
> typedef void (*LLVMMaySuspendCallback)(LLVMContextRef, void *);
>
>  /**
>  * Set the may-suspend callback function for this context.
>  *
>  * @see LLVMContext::setMaySuspendCallback()
>  */
> void LLVMContextSetMaySuspendCallback(LLVMContextRef C,
>                                       LLVMMaySuspendCallback Callback,
>                                       void *OpaqueHandle);
>
>  LLVMContext.h:
> /// \brief Registers a may-suspend callback with the context.
> ///
> /// The may-suspend callback function may be called by LLVM to transfer
> /// control back to the client that invoked the LLVM compilation. The
> client
> /// is not garantueed to ever receive this callback. It is at the sole
> /// discretion of LLVM to do so and only if it can guarantee that
> suspending
> /// the thread won't block any forward progress in other LLVM contexts.
> void setMaySuspendCallback(MaySuspendCallbackTy Callback, void
> *OpaqueHandle);
>
> /// \brief Calls the may-suspend callback (if applicable).
> ///
> /// This transfers control back to the client, which may suspend the
> current
> /// thread. Only call this method when LLVM doesn't hold any global
mutex
> or
> /// cannot block the execution in another LLVM context.
> void callMaySuspendCallback();
>
>  On May 12, 2014, at 5:26 PM, Nick Lewycky <nlewycky at google.com>
wrote:
>
>  Would you (or anyone) oppose a simple maySuspendContext() callback API?
>> It would mean nothing more than the thread(s) for a given LLVM context
can
>> be suspended independent from other contexts.
>>
>
>  I think this is the right approach. So a given thread hits a safe point,
> it optionally calls a "suspend check" or "i an safe to
suspend right now"
> callback if set. It doesn't stop other threads, it doesn't continue
until
> the function returns.
>
>  If you want to stop all threads then the user callback may contain a
> barrier and count down how many threads have stopped until it sees all of
> them.
>
>  Nick
>
>
>
>
> _______________________________________________
> LLVM Developers mailing listLLVMdev at cs.uiuc.edu        
http://llvm.cs.uiuc.eduhttp://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/258008e3/attachment.html>

On 05/15/2014 09:56 AM, Chandler Carruth wrote:
>
> On Tue, May 13, 2014 at 12:49 PM, Juergen Ributzka <juergen at apple.com
> <mailto:juergen at apple.com>> wrote:
>
>     Sounds good. Lets get started by nailing down the C API and
>     semantics for this first.
>
>
> If we're going in this direction, you might want to back out the pass 
> callback changes sooner rather than later...
Strongly agreed.  I have no problem with a separate pass progress 
mechanism btw, but it should be clearly separate.

Philip
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/0cc6858f/attachment.html>

On May 15, 2014, at 9:50 AM, Philip Reames <listmail at philipreames.com>
wrote:
> Given the use case (user mode scheduling), I'm not going to oppose this
proposal.  I would like to see a couple of things clarified documentation wise:
> - When is this interface valid?  (i.e. the single thread case)
> - If a context does have multiple threads, is this called once per thread? 
Or once per thread group after internal coordination?  (you can write this out
of scope if desired)
> - If we later introduce multiple threads, and this mechanism doesn't
support it, what will happen?  Will the function just not be called?
> - You hint at this already, but clarifying the state of the current context
at a suspend point would be helpful.
> 
> Here's a possible draft that includes the above:
> The may-suspend callback function may be called by LLVM to transfer control
back to the client that invoked the LLVM compilation. This can be used to yield
control of the thread, or perform periodic work needed by the client.  There is
no guaranteed frequency at which callbacks must occur; in fact, the client is
not guaranteed to ever receive this callback. It is at the sole discretion of
LLVM to do so and only if it can guarantee that suspending the thread won't
block any forward progress in other LLVM contexts.
> 
> At a suspend point, the state of the current LLVM context is intentionally
undefined.  No assumptions about it can or should be made.  In particular, call
backs into the context are not supported until the suspend function returns     
control to LLVM.  Other LLVM contexts are unaffected.
Great.
> Currently, LLVM assumes one thread per LLVM context.  If, or when, we
introduce multiple threads, this interface will not be available for contexts
which opt-in to the thread pool model.  We may extend this interface at a later
time to support thread pools, but for the moment, that use case is explicitly
unsupported.
Correct. We should avoid mentioning multi-thread contexts in the API docs. It is
very misleading to describe a feature that LLVM does not support. We can instead
add a statement to the docs explaining that the callback is only synchronous
with respect to the calling thread and places no guarantee on the state of other
threads, regardless of their context.
> p.s. Bikeshed wise, might "yield" be a better term than
"suspend" here?
I personally like calling it “yield” because it is more intuitive and describes
the use case. I proposed maySuspend because I wanted to be accurate. It is
really the client deciding what to do with the callback. LLVM should make no
assumption that it’s actually yielding.

Chandler likes “yield” too, so lets go with that unless anyone else wants to
weigh in.

On the commits list, Juergen introduced our current use case, along with a
couple other future use cases for this API. I’m sorry I neglected to clearly
reiterate our usage, but when it comes to documenting the C API I intentionally
try not to limit its potential to a narrow use case.

-Andy
> Philip
> 
> On 05/13/2014 11:49 AM, Juergen Ributzka wrote:
>> Sounds good. Lets get started by nailing down the C API and semantics
for this first.
>> 
>> I mirrored the C API for the LLVM context diagnostic handler and used
Andy’s suggested name for the callback.
>> The opaque handle was suggested by Duncan and can provide optional user
specified information that is
>> provided back during the callback (i.e. barrier, etc).
>> 
>> Cheers,
>> Juergen 
>> 
>> Core.h:
>> typedef void (*LLVMMaySuspendCallback)(LLVMContextRef, void *);
>> 
>> /**
>>  * Set the may-suspend callback function for this context.
>>  *
>>  * @see LLVMContext::setMaySuspendCallback()
>>  */
>> void LLVMContextSetMaySuspendCallback(LLVMContextRef C,
>>                                       LLVMMaySuspendCallback Callback,
>>                                       void *OpaqueHandle);
>> 
>> LLVMContext.h:
>> /// \brief Registers a may-suspend callback with the context.
>> ///
>> /// The may-suspend callback function may be called by LLVM to transfer
>> /// control back to the client that invoked the LLVM compilation. The
client
>> /// is not garantueed to ever receive this callback. It is at the sole
>> /// discretion of LLVM to do so and only if it can guarantee that
suspending
>> /// the thread won't block any forward progress in other LLVM
contexts.
>> void setMaySuspendCallback(MaySuspendCallbackTy Callback, void
*OpaqueHandle);
>> 
>> /// \brief Calls the may-suspend callback (if applicable).
>> ///
>> /// This transfers control back to the client, which may suspend the
current
>> /// thread. Only call this method when LLVM doesn't hold any global
mutex or
>> /// cannot block the execution in another LLVM context.
>> void callMaySuspendCallback();
>> 
>> On May 12, 2014, at 5:26 PM, Nick Lewycky <nlewycky at
google.com> wrote:
>> 
>>> Would you (or anyone) oppose a simple maySuspendContext() callback
API? It would mean nothing more than the thread(s) for a given LLVM context can
be suspended independent from other contexts.
>>> 
>>> I think this is the right approach. So a given thread hits a safe
point, it optionally calls a "suspend check" or "i an safe to
suspend right now" callback if set. It doesn't stop other threads, it
doesn't continue until the function returns.
>>> 
>>> If you want to stop all threads then the user callback may contain
a barrier and count down how many threads have stopped until it sees all of
them.
>>> 
>>> Nick
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/57eb312f/attachment.html>