Andrew Trick
2014-May-15 18:32 UTC
[LLVMdev] Finding safe thread suspension points while JIT-ing (was: Add pass run listeners to the pass manager.)
On May 15, 2014, at 9:50 AM, Philip Reames <listmail at philipreames.com> wrote:> Given the use case (user mode scheduling), I'm not going to oppose this proposal. I would like to see a couple of things clarified documentation wise: > - When is this interface valid? (i.e. the single thread case) > - If a context does have multiple threads, is this called once per thread? Or once per thread group after internal coordination? (you can write this out of scope if desired) > - If we later introduce multiple threads, and this mechanism doesn't support it, what will happen? Will the function just not be called? > - You hint at this already, but clarifying the state of the current context at a suspend point would be helpful. > > Here's a possible draft that includes the above: > The may-suspend callback function may be called by LLVM to transfer control back to the client that invoked the LLVM compilation. This can be used to yield control of the thread, or perform periodic work needed by the client. There is no guaranteed frequency at which callbacks must occur; in fact, the client is not guaranteed to ever receive this callback. It is at the sole discretion of LLVM to do so and only if it can guarantee that suspending the thread won't block any forward progress in other LLVM contexts. > > At a suspend point, the state of the current LLVM context is intentionally undefined. No assumptions about it can or should be made. In particular, call backs into the context are not supported until the suspend function returns control to LLVM. Other LLVM contexts are unaffected.Great.> Currently, LLVM assumes one thread per LLVM context. If, or when, we introduce multiple threads, this interface will not be available for contexts which opt-in to the thread pool model. We may extend this interface at a later time to support thread pools, but for the moment, that use case is explicitly unsupported.Correct. We should avoid mentioning multi-thread contexts in the API docs. It is very misleading to describe a feature that LLVM does not support. We can instead add a statement to the docs explaining that the callback is only synchronous with respect to the calling thread and places no guarantee on the state of other threads, regardless of their context.> p.s. Bikeshed wise, might "yield" be a better term than "suspend" here?I personally like calling it “yield” because it is more intuitive and describes the use case. I proposed maySuspend because I wanted to be accurate. It is really the client deciding what to do with the callback. LLVM should make no assumption that it’s actually yielding. Chandler likes “yield” too, so lets go with that unless anyone else wants to weigh in. On the commits list, Juergen introduced our current use case, along with a couple other future use cases for this API. I’m sorry I neglected to clearly reiterate our usage, but when it comes to documenting the C API I intentionally try not to limit its potential to a narrow use case. -Andy> Philip > > On 05/13/2014 11:49 AM, Juergen Ributzka wrote: >> Sounds good. Lets get started by nailing down the C API and semantics for this first. >> >> I mirrored the C API for the LLVM context diagnostic handler and used Andy’s suggested name for the callback. >> The opaque handle was suggested by Duncan and can provide optional user specified information that is >> provided back during the callback (i.e. barrier, etc). >> >> Cheers, >> Juergen >> >> Core.h: >> typedef void (*LLVMMaySuspendCallback)(LLVMContextRef, void *); >> >> /** >> * Set the may-suspend callback function for this context. >> * >> * @see LLVMContext::setMaySuspendCallback() >> */ >> void LLVMContextSetMaySuspendCallback(LLVMContextRef C, >> LLVMMaySuspendCallback Callback, >> void *OpaqueHandle); >> >> LLVMContext.h: >> /// \brief Registers a may-suspend callback with the context. >> /// >> /// The may-suspend callback function may be called by LLVM to transfer >> /// control back to the client that invoked the LLVM compilation. The client >> /// is not garantueed to ever receive this callback. It is at the sole >> /// discretion of LLVM to do so and only if it can guarantee that suspending >> /// the thread won't block any forward progress in other LLVM contexts. >> void setMaySuspendCallback(MaySuspendCallbackTy Callback, void *OpaqueHandle); >> >> /// \brief Calls the may-suspend callback (if applicable). >> /// >> /// This transfers control back to the client, which may suspend the current >> /// thread. Only call this method when LLVM doesn't hold any global mutex or >> /// cannot block the execution in another LLVM context. >> void callMaySuspendCallback(); >> >> On May 12, 2014, at 5:26 PM, Nick Lewycky <nlewycky at google.com> wrote: >> >>> Would you (or anyone) oppose a simple maySuspendContext() callback API? It would mean nothing more than the thread(s) for a given LLVM context can be suspended independent from other contexts. >>> >>> I think this is the right approach. So a given thread hits a safe point, it optionally calls a "suspend check" or "i an safe to suspend right now" callback if set. It doesn't stop other threads, it doesn't continue until the function returns. >>> >>> If you want to stop all threads then the user callback may contain a barrier and count down how many threads have stopped until it sees all of them. >>> >>> Nick >> >> >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/57eb312f/attachment.html>
Juergen Ributzka
2014-May-15 20:43 UTC
[LLVMdev] Finding safe thread suspension points while JIT-ing (was: Add pass run listeners to the pass manager.)
The addition about the undefined state of the current context is a very good point. Although I wouldn’t like to shut the door completely and allow for future extensions to relax this constraint for APIs that explicitly state that they are safe to use during a yield callback. I agree with Andy that we shouldn’t mention multi-threaded context at all in the API docs, because we are not even supporting it right now. I think this feature should be opt-in anyways, so extending the API documentation once we support it should be sufficient. What I would like to see of course is that we would still be able to support this callback for a multi-threaded context/pass manager design in such a way, that it is possible to suspend any thread at a suspension point and it won’t block any other thread even in the same context. But I don’t want to write this in stone now. This gives us all the flexibility we need for developing a good concurrent design. Having the ability to dynamically add and remove threads from an active compilation would be a nice feature that we should try to incorporate when possible and that would work nicely with the yield callback. Anyways, I am digressing to much. As I said before we should only specify the multi-threaded context constrains in the API doc once they are actually available. For now this is simply undefined behavior and not even supported. I extended the API with Philip’s suggestions and updated the doc. Please ignore proper 80col formatting for now ;-) Does this look good to everyone? -Juergen Core.h: typedef void (*LLVMYieldCallback)(LLVMContextRef, void *); /** * Set the yield callback function for this context. * * @see LLVMContext::setYieldCallback() */ void LLVMContextSetYieldCallback(LLVMContextRef C, LLVMYieldCallback Callback, void *OpaqueHandle); LLVMContext.h: /// \brief Registers a yield callback with the given context. /// /// The yield callback function may be called by LLVM to transfer /// control back to the client that invoked the LLVM compilation. This can /// be used to yield control of the thread, or perform periodic work needed /// by the client. There is no guaranteed frequency at which callbacks must /// occur; in fact, the client is not guaranteed to ever receive this callback. /// It is at the sole discretion of LLVM to do so and only if it can guarantee /// that suspending the thread won't block any forward progress in other LLVM /// contexts in the same process. /// /// At a suspend point, the state of the current LLVM context is intentionally /// undefined. No assumptions about it can or should be made. Only LLVM context /// API calls that explicitly state that they can be used during a yield callback /// are allowed to be used. Any other API calls into the context are not supported /// until the yield callback function returns control to LLVM. Other LLVM contexts /// are unaffected by this restriction. void setYieldCallback(YieldCallbackTy Callback, void *OpaqueHandle); /// \brief Calls the yield callback (if applicable). /// /// This transfers control back to the client, which may suspend the current /// thread. Only call this method when LLVM doesn't hold any global mutex or /// cannot block the execution in another LLVM context. void yieldToClient(); On May 15, 2014, at 11:32 AM, Andrew Trick <atrick at apple.com> wrote:> > On May 15, 2014, at 9:50 AM, Philip Reames <listmail at philipreames.com> wrote: > >> Given the use case (user mode scheduling), I'm not going to oppose this proposal. I would like to see a couple of things clarified documentation wise: >> - When is this interface valid? (i.e. the single thread case) >> - If a context does have multiple threads, is this called once per thread? Or once per thread group after internal coordination? (you can write this out of scope if desired) >> - If we later introduce multiple threads, and this mechanism doesn't support it, what will happen? Will the function just not be called? >> - You hint at this already, but clarifying the state of the current context at a suspend point would be helpful. >> >> Here's a possible draft that includes the above: >> The may-suspend callback function may be called by LLVM to transfer control back to the client that invoked the LLVM compilation. This can be used to yield control of the thread, or perform periodic work needed by the client. There is no guaranteed frequency at which callbacks must occur; in fact, the client is not guaranteed to ever receive this callback. It is at the sole discretion of LLVM to do so and only if it can guarantee that suspending the thread won't block any forward progress in other LLVM contexts. >> >> At a suspend point, the state of the current LLVM context is intentionally undefined. No assumptions about it can or should be made. In particular, call backs into the context are not supported until the suspend function returns control to LLVM. Other LLVM contexts are unaffected. > > Great. > >> Currently, LLVM assumes one thread per LLVM context. If, or when, we introduce multiple threads, this interface will not be available for contexts which opt-in to the thread pool model. We may extend this interface at a later time to support thread pools, but for the moment, that use case is explicitly unsupported. > > Correct. We should avoid mentioning multi-thread contexts in the API docs. It is very misleading to describe a feature that LLVM does not support. We can instead add a statement to the docs explaining that the callback is only synchronous with respect to the calling thread and places no guarantee on the state of other threads, regardless of their context. > >> p.s. Bikeshed wise, might "yield" be a better term than "suspend" here? > > I personally like calling it “yield” because it is more intuitive and describes the use case. I proposed maySuspend because I wanted to be accurate. It is really the client deciding what to do with the callback. LLVM should make no assumption that it’s actually yielding. > > Chandler likes “yield” too, so lets go with that unless anyone else wants to weigh in. > > On the commits list, Juergen introduced our current use case, along with a couple other future use cases for this API. I’m sorry I neglected to clearly reiterate our usage, but when it comes to documenting the C API I intentionally try not to limit its potential to a narrow use case. > > -Andy > >> Philip >> >> On 05/13/2014 11:49 AM, Juergen Ributzka wrote: >>> Sounds good. Lets get started by nailing down the C API and semantics for this first. >>> >>> I mirrored the C API for the LLVM context diagnostic handler and used Andy’s suggested name for the callback. >>> The opaque handle was suggested by Duncan and can provide optional user specified information that is >>> provided back during the callback (i.e. barrier, etc). >>> >>> Cheers, >>> Juergen >>> >>> Core.h: >>> typedef void (*LLVMMaySuspendCallback)(LLVMContextRef, void *); >>> >>> /** >>> * Set the may-suspend callback function for this context. >>> * >>> * @see LLVMContext::setMaySuspendCallback() >>> */ >>> void LLVMContextSetMaySuspendCallback(LLVMContextRef C, >>> LLVMMaySuspendCallback Callback, >>> void *OpaqueHandle); >>> >>> LLVMContext.h: >>> /// \brief Registers a may-suspend callback with the context. >>> /// >>> /// The may-suspend callback function may be called by LLVM to transfer >>> /// control back to the client that invoked the LLVM compilation. The client >>> /// is not garantueed to ever receive this callback. It is at the sole >>> /// discretion of LLVM to do so and only if it can guarantee that suspending >>> /// the thread won't block any forward progress in other LLVM contexts. >>> void setMaySuspendCallback(MaySuspendCallbackTy Callback, void *OpaqueHandle); >>> >>> /// \brief Calls the may-suspend callback (if applicable). >>> /// >>> /// This transfers control back to the client, which may suspend the current >>> /// thread. Only call this method when LLVM doesn't hold any global mutex or >>> /// cannot block the execution in another LLVM context. >>> void callMaySuspendCallback(); >>> >>> On May 12, 2014, at 5:26 PM, Nick Lewycky <nlewycky at google.com> wrote: >>> >>>> Would you (or anyone) oppose a simple maySuspendContext() callback API? It would mean nothing more than the thread(s) for a given LLVM context can be suspended independent from other contexts. >>>> >>>> I think this is the right approach. So a given thread hits a safe point, it optionally calls a "suspend check" or "i an safe to suspend right now" callback if set. It doesn't stop other threads, it doesn't continue until the function returns. >>>> >>>> If you want to stop all threads then the user callback may contain a barrier and count down how many threads have stopped until it sees all of them. >>>> >>>> Nick >>> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/e985af4a/attachment.html>
Chandler Carruth
2014-May-15 20:51 UTC
[LLVMdev] Finding safe thread suspension points while JIT-ing (was: Add pass run listeners to the pass manager.)
LGTM Tiny nit: On Thu, May 15, 2014 at 2:43 PM, Juergen Ributzka <juergen at apple.com> wrote:> /// \brief Calls the yield callback (if applicable). > > /// > /// This transfers control back to the client, which may suspend the > current > /// thread. Only call this method when LLVM doesn't hold any global mutex > or > /// cannot block the execution in another LLVM context. > void yieldToClient();I would just call it "yield" as we're not passing in a particular client. This makes me think of a designated yield to some other thread in a UMS style system. Anyways, this is down at the dim bottoms of tiny nits and bikesheds. =D Ship it. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/b9ceea2b/attachment.html>
Philip Reames
2014-May-15 22:31 UTC
[LLVMdev] Finding safe thread suspension points while JIT-ing
Slight nitpick: control of *what*? Your comment should read something like "transfer control of the current thread" or something similar. Otherwise, LGTM. Philip On 05/15/2014 01:43 PM, Juergen Ributzka wrote:> The addition about the undefined state of the current context is a > very good point. Although I wouldn’t like to shut the door completely > and allow for future extensions to relax this constraint for APIs that > explicitly state that they are safe to use during a yield callback. > > I agree with Andy that we shouldn’t mention multi-threaded context at > all in the API docs, because we are not even supporting it right now. > I think this feature should be opt-in anyways, so extending the API > documentation once we support it should be sufficient. > > What I would like to see of course is that we would still be able to > support this callback for a multi-threaded context/pass manager design > in such a way, that it is possible to suspend any thread at a > suspension point and it won’t block any other thread even in the same > context. But I don’t want to write this in stone now. This gives us > all the flexibility we need for developing a good concurrent design. > > Having the ability to dynamically add and remove threads from an > active compilation would be a nice feature that we should try to > incorporate when possible and that would work nicely with the yield > callback. > > Anyways, I am digressing to much. As I said before we should only > specify the multi-threaded context constrains in the API doc once they > are actually available. For now this is simply undefined behavior and > not even supported. > > I extended the API with Philip’s suggestions and updated the doc. > Please ignore proper 80col formatting for now ;-) > > Does this look good to everyone? > > -Juergen > > Core.h: > typedef void (*LLVMYieldCallback)(LLVMContextRef, void *); > > /** > * Set the yield callback function for this context. > * > * @see LLVMContext::setYieldCallback() > */ > void LLVMContextSetYieldCallback(LLVMContextRef C, > LLVMYieldCallback Callback, > void *OpaqueHandle); > > LLVMContext.h: > /// \brief Registers a yield callback with the given context. > /// > /// The yield callback function may be called by LLVM to transfer > /// control back to the client that invoked the LLVM compilation. This can > /// be used to yield control of the thread, or perform periodic work > needed > /// by the client. There is no guaranteed frequency at which > callbacks must > /// occur; in fact, the client is not guaranteed to ever receive this > callback. > /// It is at the sole discretion of LLVM to do so and only if it > can guarantee > /// that suspending the thread won't block any forward progress in > other LLVM > /// contexts in the same process. > /// > /// At a suspend point, the state of the current LLVM context is > intentionally > /// undefined. No assumptions about it can or should be made. Only > LLVM context > /// API calls that explicitly state that they can be used during a > yield callback > /// are allowed to be used. Any other API calls into the context are > not supported > /// until the yield callback function returns control to LLVM. Other > LLVM contexts > /// are unaffected by this restriction. > void setYieldCallback(YieldCallbackTy Callback, void *OpaqueHandle); > > /// \brief Calls the yield callback (if applicable). > /// > /// This transfers control back to the client, which may suspend the > current > /// thread. Only call this method when LLVM doesn't hold any global > mutex or > /// cannot block the execution in another LLVM context. > void yieldToClient(); > > > On May 15, 2014, at 11:32 AM, Andrew Trick <atrick at apple.com > <mailto:atrick at apple.com>> wrote: > >> >> On May 15, 2014, at 9:50 AM, Philip Reames <listmail at philipreames.com >> <mailto:listmail at philipreames.com>> wrote: >> >>> Given the use case (user mode scheduling), I'm not going to oppose >>> this proposal. I would like to see a couple of things clarified >>> documentation wise: >>> - When is this interface valid? (i.e. the single thread case) >>> - If a context does have multiple threads, is this called once per >>> thread? Or once per thread group after internal coordination? (you >>> can write this out of scope if desired) >>> - If we later introduce multiple threads, and this mechanism doesn't >>> support it, what will happen? Will the function just not be called? >>> - You hint at this already, but clarifying the state of the current >>> context at a suspend point would be helpful. >>> >>> Here's a possible draft that includes the above: >>> The may-suspend callback function may be called by LLVM to transfer >>> control back to the client that invoked the LLVM compilation. This >>> can be used to yield control of the thread, or perform periodic work >>> needed by the client. There is no guaranteed frequency at which >>> callbacks must occur; in fact, the client is not guaranteed to ever >>> receive this callback. It is at the sole discretion of LLVM to do so >>> and only if it can guarantee that suspending the thread won't block >>> any forward progress in other LLVM contexts. >>> >>> At a suspend point, the state of the current LLVM context is >>> intentionally undefined. No assumptions about it can or should be >>> made. In particular, call backs into the context are not supported >>> until the suspend function returns control to LLVM. Other LLVM >>> contexts are unaffected. >> >> Great. >> >>> Currently, LLVM assumes one thread per LLVM context. If, or when, >>> we introduce multiple threads, this interface will not be available >>> for contexts which opt-in to the thread pool model. We may extend >>> this interface at a later time to support thread pools, but for the >>> moment, that use case is explicitly unsupported. >> >> Correct. We should avoid mentioning multi-thread contexts in the API >> docs. It is very misleading to describe a feature that LLVM does not >> support. We can instead add a statement to the docs explaining that >> the callback is only synchronous with respect to the calling thread >> and places no guarantee on the state of other threads, regardless of >> their context. >> >>> p.s. Bikeshed wise, might "yield" be a better term than "suspend" here? >> >> I personally like calling it “yield” because it is more intuitive and >> describes the use case. I proposed maySuspend because I wanted to be >> accurate. It is really the client deciding what to do with the >> callback. LLVM should make no assumption that it’s actually yielding. >> >> Chandler likes “yield” too, so lets go with that unless anyone else >> wants to weigh in. >> >> On the commits list, Juergen introduced our current use case, along >> with a couple other future use cases for this API. I’m sorry I >> neglected to clearly reiterate our usage, but when it comes to >> documenting the C API I intentionally try not to limit its potential >> to a narrow use case. >> >> -Andy >> >>> Philip >>> >>> On 05/13/2014 11:49 AM, Juergen Ributzka wrote: >>>> Sounds good. Lets get started by nailing down the C API and >>>> semantics for this first. >>>> >>>> I mirrored the C API for the LLVM context diagnostic handler and >>>> used Andy’s suggested name for the callback. >>>> The opaque handle was suggested by Duncan and can provide optional >>>> user specified information that is >>>> provided back during the callback (i.e. barrier, etc). >>>> >>>> Cheers, >>>> Juergen >>>> >>>> Core.h: >>>> typedef void (*LLVMMaySuspendCallback)(LLVMContextRef, void *); >>>> >>>> /** >>>> * Set the may-suspend callback function for this context. >>>> * >>>> * @see LLVMContext::setMaySuspendCallback() >>>> */ >>>> void LLVMContextSetMaySuspendCallback(LLVMContextRef C, >>>> LLVMMaySuspendCallback Callback, >>>> void *OpaqueHandle); >>>> >>>> LLVMContext.h: >>>> /// \brief Registers a may-suspend callback with the context. >>>> /// >>>> /// The may-suspend callback function may be called by LLVM to transfer >>>> /// control back to the client that invoked the LLVM compilation. >>>> The client >>>> /// is not garantueed to ever receive this callback. It is at the sole >>>> /// discretion of LLVM to do so and only if it can guarantee that >>>> suspending >>>> /// the thread won't block any forward progress in other LLVM contexts. >>>> void setMaySuspendCallback(MaySuspendCallbackTy Callback, void >>>> *OpaqueHandle); >>>> >>>> /// \brief Calls the may-suspend callback (if applicable). >>>> /// >>>> /// This transfers control back to the client, which may suspend >>>> the current >>>> /// thread. Only call this method when LLVM doesn't hold any global >>>> mutex or >>>> /// cannot block the execution in another LLVM context. >>>> void callMaySuspendCallback(); >>>> >>>> On May 12, 2014, at 5:26 PM, Nick Lewycky <nlewycky at google.com >>>> <mailto:nlewycky at google.com>> wrote: >>>> >>>>> Would you (or anyone) oppose a simple maySuspendContext() >>>>> callback API? It would mean nothing more than the thread(s) >>>>> for a given LLVM context can be suspended independent from >>>>> other contexts. >>>>> >>>>> >>>>> I think this is the right approach. So a given thread hits a safe >>>>> point, it optionally calls a "suspend check" or "i an safe to >>>>> suspend right now" callback if set. It doesn't stop other threads, >>>>> it doesn't continue until the function returns. >>>>> >>>>> If you want to stop all threads then the user callback may contain >>>>> a barrier and count down how many threads have stopped until it >>>>> sees all of them. >>>>> >>>>> Nick >>>> >>>> >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu >>> <mailto:LLVMdev at cs.uiuc.edu>http://llvm.cs.uiuc.edu >>> <http://llvm.cs.uiuc.edu/> >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140515/a0bbbd17/attachment.html>