Chris Bieneman via llvm-dev
2018-Nov-06 18:28 UTC
[llvm-dev] [RFC] Enable thread specific cl::opt values for multi-threaded support
It seems relevant to point out there have been several efforts over the years to solve this problem by eliminating the static initializers in LLVM and migrate cl::opts to storing their values in a context object. One of those efforts was made by me back in 2014 (see https://reviews.llvm.org/D6207 <https://reviews.llvm.org/D6207>). One of the perennial problems with cl::opt is that many of the values are declared in passes, and we rely on static initialization to register the options. My understanding is the new pass manager doesn't have pass registration, so that complicates how option initialization would work in a pass. Personally I very much prefer the idea of having *Context objects be responsible for options rather than thread-local storage, but in order to not break how cl::opt works today we need an answer to the initialization story. Maybe per-library initialization? We already have that for most of our libraries. -Chris> On Oct 31, 2018, at 12:11 AM, Yevgeny Rouban via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > >>! In D53424#1273950 <tel:1273950>, @delcypher wrote: > >>>! In D53424#1273737 <tel:1273737>, @yrouban wrote: > >>>>! In D53424#1273728 <tel:1273728>, @jfb wrote: > >>> Was there a commitment from the community, with some time horizon for moving away from this patch? > … > > While I agree that a transition away from global `cl::opt`s is going to be tough I think we need to get agreement from the community on some sort of plan before we land this. Otherwise we'll end up with this patch becoming a permanent part of LLVM's internal API which makes our technical debt even worse. > > My suggestion would be to create a new RFC on llvm-dev that proposes a plan to move away from `cl::opt`s in LLVM's codebase (and probably other sub-projects). Given that you're looking at this issue you're probably in good position to discuss the technical problems with the available replacement APIs in LLVM's codebase. > > I would not say that the patch https://reviews.llvm.org/D53424 <https://reviews.llvm.org/D53424> makes our technical debt worse. That is because it just allows us to set options per thread without almost any API change. Setting options locally (per thread, per pipeline, per LLVMContext, … etc.) is the one of the goal of the future changes you have in mind. When the future changes are ready it will not be difficult to start using them instead of D53424. At least it will be much easier than to implement the future changes themselves. Given that the future changes are just thoughts that need much effort and nobody has committed yet I would suggest that we accept the patch D53424. This can solve some problems. > > From technical point of view I would say that the patch D53424 just slightly changes the angle of view on the cl::opts. They are still global options that should be changed only during the configuration stage. The only change proposed is the definition of the configuration stage - it used to be global and one for the while process, but now it can be done at different time for different threads. > > Thanks. > -Yevgeny Rouban > > From: Yevgeny Rouban [mailto:yrouban at gmail.com <mailto:yrouban at gmail.com>] > Sent: Wednesday, October 31, 2018 12:47 PM > To: llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> > Cc: Yevgeny Rouban <yevgeny.rouban at azul.com <mailto:yevgeny.rouban at azul.com>> > Subject: Re: [RFC] Enable thread specific cl::opt values for multi-threaded support > > Here are copies of some key comments from the patch > https://reviews.llvm.org/D53424 <https://reviews.llvm.org/D53424> Enable thread specific cl::opt values for multi-threaded support > So, please continue discussion here. > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181106/4d815e2f/attachment.html>
Yevgeny Rouban via llvm-dev
2018-Nov-08 07:50 UTC
[llvm-dev] [RFC] Enable thread specific cl::opt values for multi-threaded support
Hello Chris.> One of the perennial problems with cl::opt is that ...My RFC does not change the way the options are initialized and registered. So the static initialization problem is not made worse by this RFC.> Personally I very much prefer the idea of having *Context objects beresponsible for options rather than thread-local storage ... This RFC defines a *Context* for options. In the patch the class is called ContextValues. Essentially, it is similar to *Context objects*. With the RFC we will be able to make further changes for specific options to get them from any specific Context, not only from the thread local one. The thread local is proposed as the default context to keep unchanged all places where cl::opts are accessed. If we have a flag: static cl::opt<bool> SomeFlag; // static global Flag access code looks like the following: if ( SomeFlag ) ... // global flag access With the RFC (thread specific cl::opt values) this code being unchanged becomes equivalent to: if ( SomeFlag.getValue(cl::ContextValues::GetThreadOptionContext()) ) ... // thread local flag access with fallback to global flag access Then we can manually change this source to: if ( SomeFlag.getValue(SomeSpecificContext) ) ... // context specific flag access with fallback to thread local flag access In this example SomeSpecificContext can be specific to LLVMContext, Pass, ... Thanks. -Yevgeny Rouban -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181108/2ef99e13/attachment.html>
Neil Henning via llvm-dev
2018-Dec-13 11:06 UTC
[llvm-dev] [RFC] Enable thread specific cl::opt values for multi-threaded support
Hey all, Trying to revive the RFC thread here - we have a need for this functionality (or some similar mechanism that lets us have a per-thread or per-context setting for options). Is there any scope for getting something in that enables our (AMD's) graphics drivers and Azul to make use of it, in the short term, even if it's not the perfect solution that everyone desires? Cheers, -Neil. On 08/11/2018 07:50, Yevgeny Rouban via llvm-dev wrote:> Hello Chris. > > > One of the perennial problems with cl::opt is that ... > My RFC does not change the way the options are initialized and > registered. So the static initialization problem is not made worse by > this RFC. > > > Personally I very much prefer the idea of having *Context objects be > responsible for options rather than thread-local storage ... > This RFC defines a *Context* for options. In the patch the class is > called ContextValues. Essentially, it is similar to *Context objects*. > With the RFC we will be able to make further changes for specific > options to get them from any specific Context, not only from the > thread local one. The thread local is proposed as the default context > to keep unchanged all places where cl::opts are accessed. > > If we have a flag: > static cl::opt<bool> SomeFlag; // static global > > Flag access code looks like the following: > if ( SomeFlag ) ... // global flag access > > With the RFC (thread specific cl::opt values) this code being > unchanged becomes equivalent to: > if ( SomeFlag.getValue(cl::ContextValues::GetThreadOptionContext()) ) > ... // thread local flag access with fallback to global flag access > > Then we can manually change this source to: > if ( SomeFlag.getValue(SomeSpecificContext) ) ... // context specific > flag access with fallback to thread local flag access > > In this example SomeSpecificContext can be specific to LLVMContext, > Pass, ... > > Thanks. > -Yevgeny Rouban > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181213/b598899e/attachment.html>