Chris Lattner via llvm-dev
2018-Dec-31 04:54 UTC
[llvm-dev] RFC: Modernizing our use of auto
On Dec 16, 2018, at 11:44 AM, Stephen Kelly via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > On 25/11/2018 14:43, Stephen Kelly via llvm-dev wrote: >> However this is a proposal for more modern thinking regarding the permissiveness of auto in LLVM codebases. >> Currently the rule on the use of auto is here: > > Hi, > > Thanks for the input on this topic, which has appeared here on the mailing list, on the phab review, and on IRC. > > Almost all respondants generally expressed the claim "The type must be obvious if auto is used", though as I wrote before the guide uses auto in context that the type is not obvious: > > for (const auto &Val : Container) { observe(Val); } > > It seems that the respondants wish for 'almost never auto'. Fair enough, if the existing practice supports that. > > There is one problem though, or rather ~56,000 problems. That's how many uses of auto there are in the entire codebase currently, according to someone on IRC.I find this to be a helpful line of argument. We should, as a community, decide what the right thing is regardless of the existing art. As you say, the current patches going in have been arbitrary, so making an argument based on what is in the code base isn’t particularly informative.> Do those uses conform to the guide? If they don't, then should the guide be updated? Are the types there 'obvious’?If/when we revise the policy, then it would make sense for non-conforming uses of auto to be changed. However, I don’t think that actually making a widespread change would be high priority...> How did all of those uses get into the codebase? Does it indicate that the guide is not followed, or does it indicate that the guide is too subjective, or that maybe the 'the type must be obvios' guide does not reflect the 'reality at the coalface' anymore? Should those uses of auto be changed?My understanding is that there has been no widely understood or accepted policy, so different coders and reviewers are doing different things.> How is a new contributor to react to any of that? What are the real criteria that we can use to determine where auto will cause a patch to be rejected? Does it only depend on who you get as a reviewer?Right now, it is quite arbitrary. This is a bug, not a feature.> Here is a quote from this thread from Chris and supported by Chandler and Quentin at least: > > > Perhaps the rule came be rewritten more generally, as > > in “auto should generally only be used when the inferred > > type is obvious from context, e.g. as a result of a cast > > like dyn_cast or as a result of constructing a value with > > a constructor containing a type name.”? > > Is it reasonable to have that as a rule if there are ~12000 uses which break that rule?Yes. If you’d like to make progress on this, I think you should start by building consensus. It seems like there is widespread enough acknowledgement that the current state of things is broken, but there is no concrete proposal for a coding standards change. Please prepare a patch so we can discuss it. -Chris
Stephen Kelly via llvm-dev
2019-Jan-05 19:36 UTC
[llvm-dev] RFC: Modernizing our use of auto
On 31/12/2018 04:54, Chris Lattner wrote:> On Dec 16, 2018, at 11:44 AM, Stephen Kelly via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> On 25/11/2018 14:43, Stephen Kelly via llvm-dev wrote: >>> However this is a proposal for more modern thinking regarding the permissiveness of auto in LLVM codebases. >>> Currently the rule on the use of auto is here: >> Hi, >> >> Thanks for the input on this topic, which has appeared here on the mailing list, on the phab review, and on IRC. >> >> Almost all respondants generally expressed the claim "The type must be obvious if auto is used", though as I wrote before the guide uses auto in context that the type is not obvious: >> >> for (const auto &Val : Container) { observe(Val); } >> >> It seems that the respondants wish for 'almost never auto'. Fair enough, if the existing practice supports that. >> >> There is one problem though, or rather ~56,000 problems. That's how many uses of auto there are in the entire codebase currently, according to someone on IRC. > I find this to be a helpful line of argument.Given what you wrote below, maybe you are missing a negation somewhere in this sentence?> We should, as a community, decide what the right thing is regardless of the existing art.The existing art is part of 'the community deciding what to do'. And yes, I think it makes sense to 'standardize existing practice' where possible.>> How did all of those uses get into the codebase? Does it indicate that the guide is not followed, or does it indicate that the guide is too subjective, or that maybe the 'the type must be obvios' guide does not reflect the 'reality at the coalface' anymore? Should those uses of auto be changed? > My understanding is that there has been no widely understood or accepted policy, so different coders and reviewers are doing different things.And different contexts within LLVM are fine with auto, but seem to get campaigning from other parts who have a different interpretation of the guideline: https://reviews.llvm.org/D33672#inline-475812>> How is a new contributor to react to any of that? What are the real criteria that we can use to determine where auto will cause a patch to be rejected? Does it only depend on who you get as a reviewer? > Right now, it is quite arbitrary. This is a bug, not a feature.Do we have some idea of who is interested in fixing the bug? It can't be just one person fixing it - this is a community issue. You've suggested that the guideline needs an update, and I've already suggested an update. Is it only the two of us? How can we proceed?>> Here is a quote from this thread from Chris and supported by Chandler and Quentin at least: >> >>> Perhaps the rule came be rewritten more generally, as >>> in “auto should generally only be used when the inferred >>> type is obvious from context, e.g. as a result of a cast >>> like dyn_cast or as a result of constructing a value with >>> a constructor containing a type name.”? >> Is it reasonable to have that as a rule if there are ~12000 uses which break that rule? > If you’d like to make progress on this, I think you should start by building consensus.Well, I'm trying to find out what the positions people have are, but even though there are so many existing usages of auto, this thread is not getting responses from the people who put them there. So, the code says one thing, and the guideline says arguably the same thing, but people have alternative interpretations of the guideline. But at least the responses here are not really representative. For me that means I'm not able to get my clang-query features (https://steveire.wordpress.com/2018/11/11/future-developments-in-clang-query/) upstream because I'm getting reviews which say "remove auto, here and everywhere in this file." in https://reviews.llvm.org/D54408?id=173770#inline-480255 That's a bit of a difficult review comment, given the ways it is already used throughout the code.> It seems like there is widespread enough acknowledgement that the current state of things is broken, but there is no concrete proposal for a coding standards change. Please prepare a patch so we can discuss it.I made a proposal in my initial mail in this thread. See the end of the email: https://lists.llvm.org/pipermail/llvm-dev/2018-November/127953.html Phab is not well-suited to discussion like this, so we should probably keep it on the mailing list for now. But, here's some updated thinking: New guidelines should * Be easy to follow * Have some consistency * Be modern * Be welcoming (or at least non-hostile) to newcomers * Standardize existing practice in LLVM * Achieve a consensus of support about the spirit of the guideline (consensus is not unanimity) and be acceptable to people who dislike auto Can we agree on that much to start? On the Phab review, some people expressed that they liked the examples of when auto is acceptable. Here is an updated attempt at guideline text for that section: ``` Some are advocating a policy of "almost always ``auto``" in C++11, however LLVM uses a more moderate stance. Don't "almost always" use ``auto``, but it may be used where either the Concept or the type is obvious from the context. Here are some cases where use of ``auto`` would make sense: * Where the type appears elsewhere in the line (eg a dyn_cast) * Where the variable name or initializing expression provides enough information (`auto SR = getSourceRange()`) * Where the context makes the *Concept* obvious, even if the type is not obvious (eg, where the instance is only used as an iterator, or in an algorithm as a container-like concept, or only with a validity check, or an AST Matcher). Exceptions may arise, but they should only arise in exceptional cases. If the case is not exceptional, apply the guidelines in review discussion. ``` The most important thing here is that it does not accept your proposal that 'the type must be obvious'. Instead, it recognizes that `auto` is really "an unspecified concept" - unspecified only because we can't specify the concept in C++ yet. However, the point/concern seems to be that readers of code should know the instance may be used in its local scope. That's why these guidelines would allow `auto Ctors` in llvm::Optional<std::pair<std::string, MatcherCtor>> getNodeConstructorType(ASTNodeKind targetType) { const auto &Ctors = RegistryData->nodeConstructors(); auto It = llvm::find_if( Ctors, [targetType](const NodeConstructorMap::value_type &Ctor) { return Ctor.first.isSame(targetType); }); if (It == Ctors.end()) return llvm::None; return It->second; } because `Ctors` is obviously the Container *concept*, and knowing exactly what type it is is not necessary in the local context. However, in the below code it would probably not be ok because we're calling methods on the instance which opens up more possibilities (is it a base interface? etc): void SomeClass::foo(int input) { auto Ctors = getCtors(input); m_widgets = Ctors->calculate(); } Here, `Ctors` is definitely not a Container concept, we don't know what kind of concept it is, so we should know the type by seeing it typed in the code. Another example from earlier in the thread: template <typename BaseT, typename DerivedT> void registerIfNodeMatcher(...) { auto NodeKind = ASTNodeKind::getFromNodeKind<DerivedT>(); if (!NodeKind.isNone()) NodeCtors[NodeKind] = std::make_pair(MatcherName, Descriptor); } Here, `NodeKind` is used as an Optional (or Maybe) *concept*. All we do is a validity check. So, `auto` should be allowed here. This 'the concept must be obvious' guideline is also what allows the use of `auto` for iterators. What do people think of "Either the Concept or the type should be obvious from the context" as a baseline guideline? Thanks, Stephen. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190105/1ee4747b/attachment.html>
Mehdi AMINI via llvm-dev
2019-Jan-05 21:56 UTC
[llvm-dev] [cfe-dev] RFC: Modernizing our use of auto
On Sat, Jan 5, 2019 at 11:37 AM Stephen Kelly via cfe-dev < cfe-dev at lists.llvm.org> wrote:> > On 31/12/2018 04:54, Chris Lattner wrote: > > On Dec 16, 2018, at 11:44 AM, Stephen Kelly via llvm-dev <llvm-dev at lists.llvm.org> <llvm-dev at lists.llvm.org> wrote: > > On 25/11/2018 14:43, Stephen Kelly via llvm-dev wrote: > > However this is a proposal for more modern thinking regarding the permissiveness of auto in LLVM codebases. > Currently the rule on the use of auto is here: > > Hi, > > Thanks for the input on this topic, which has appeared here on the mailing list, on the phab review, and on IRC. > > Almost all respondants generally expressed the claim "The type must be obvious if auto is used", though as I wrote before the guide uses auto in context that the type is not obvious: > > for (const auto &Val : Container) { observe(Val); } > > It seems that the respondants wish for 'almost never auto'. Fair enough, if the existing practice supports that. > > There is one problem though, or rather ~56,000 problems. That's how many uses of auto there are in the entire codebase currently, according to someone on IRC. > > I find this to be a helpful line of argument. > > > Given what you wrote below, maybe you are missing a negation somewhere in > this sentence? > > > We should, as a community, decide what the right thing is regardless of the existing art. > > > The existing art is part of 'the community deciding what to do'. > > > And yes, I think it makes sense to 'standardize existing practice' where > possible. > > > How did all of those uses get into the codebase? Does it indicate that the guide is not followed, or does it indicate that the guide is too subjective, or that maybe the 'the type must be obvios' guide does not reflect the 'reality at the coalface' anymore? Should those uses of auto be changed? > > My understanding is that there has been no widely understood or accepted policy, so different coders and reviewers are doing different things. > > > And different contexts within LLVM are fine with auto, but seem to get > campaigning from other parts who have a different interpretation of the > guideline: > > > https://reviews.llvm.org/D33672#inline-475812 > > > How is a new contributor to react to any of that? What are the real criteria that we can use to determine where auto will cause a patch to be rejected? Does it only depend on who you get as a reviewer? > > Right now, it is quite arbitrary. This is a bug, not a feature. > > > Do we have some idea of who is interested in fixing the bug? It can't be > just one person fixing it - this is a community issue. You've suggested > that the guideline needs an update, and I've already suggested an update. > Is it only the two of us? How can we proceed? > > > Here is a quote from this thread from Chris and supported by Chandler and Quentin at least: > > > Perhaps the rule came be rewritten more generally, as > in “auto should generally only be used when the inferred > type is obvious from context, e.g. as a result of a cast > like dyn_cast or as a result of constructing a value with > a constructor containing a type name.”? > > Is it reasonable to have that as a rule if there are ~12000 uses which break that rule? > > If you’d like to make progress on this, I think you should start by building consensus. > > > Well, I'm trying to find out what the positions people have are, but even > though there are so many existing usages of auto, this thread is not > getting responses from the people who put them there. So, the code says one > thing, and the guideline says arguably the same thing, but people have > alternative interpretations of the guideline. But at least the responses > here are not really representative. >You're discounting some aspects: - Code is written downstream without much consideration for the guideline and then upstreamed - Review process isn't perfect: we let things slip, we don't always want to nitpick on things like `auto`, etc. - Some developers just don't know the guidelines and because of the point above, some cases slip. - People move back and forth between projects (i.e. LLVM is not their main project) and thus aren't focused on every details of the code guidelines (For instance I would review a patch today and not necessarily catch on every style aspect). For the particular case of auto, there is an even more complex effect: since the intention is that "types should be obvious when reading the code", someone very familiar with some libraries will always unconsciously infer the returned type from API calls (from conventions, past experience, etc.), but someone who is less familiar with this area of the codebase would be quickly confused. Now add to this the fact that the reviewers are (in general) familiar with the area they are reviewing, they aren't necessarily in a good position to catch immediately all the uses of auto that makes the types less obvious to other people. The view you mentioned above is a good example of this: one reviewer points very accurately that "if you have to explain that in the variable's name, justify it in review comments" then "This really shouldn't be auto". To me this is the important part of the guideline: make it easy for anyone to read the code and infer the types (ideally without extra steps like clicking in an IDE to debunk it). On the other hand, another reviewer mentions that there is a specific pattern here and that "people get used to it very quickly when they start actively working on the codebase." This is also a valid point, and while I'm very much in favor of the current guideline in general, there are pattern that are so much repetitive that it can be worthwhile to endorse them as the kind of things you need to know for this area. The question in this case is if: const auto ValueToCast = ....getAs<DefinedOrUnknownSVal>(); has an "obvious" type of being an Optional<DefinedOrUnknownSVal>. I claim that this example is less about `auto` itself but rather a question about can you consider that `getAs` is such a "core" pattern of this area of the code base that we can accept as "common knowledge" that it always wrap the returned type in an optional. And if we do, then the current guideline is actually fulfilled: "the type is already obvious from the context" (getAs being part of the context at this point). There are other such example, for example I believe we can assume that all the standard STL algorithm are known and so in this case: llvm::all_of(Container, [] (const auto &Value) { ...}) The use of auto in the lambda should be OK (assuming c++14 and the type of Container is obvious). For me that means I'm not able to get my clang-query features (> https://steveire.wordpress.com/2018/11/11/future-developments-in-clang-query/) > upstream because I'm getting reviews which say "remove auto, here and > everywhere in this file." in > > > https://reviews.llvm.org/D54408?id=173770#inline-480255 > > > That's a bit of a difficult review comment, given the ways it is already > used throughout the code. > > > It seems like there is widespread enough acknowledgement that the current state of things is broken, but there is no concrete proposal for a coding standards change. Please prepare a patch so we can discuss it. > > > I made a proposal in my initial mail in this thread. See the end of the > email: https://lists.llvm.org/pipermail/llvm-dev/2018-November/127953.html >Phab is not well-suited to discussion like this, so we should probably keep> it on the mailing list for now. > > > But, here's some updated thinking: > > > New guidelines should > > * Be easy to follow > * Have some consistency > * Be modern > * Be welcoming (or at least non-hostile) to newcomers > * Standardize existing practice in LLVM > * Achieve a consensus of support about the spirit of the guideline > (consensus is not unanimity) and be acceptable to people who dislike auto > > Can we agree on that much to start? >It depends what you mean by "standardize existing practice in LLVM", this seems like a "guideline" to define the new guideline, more than a rule. I.e. if some area are "abusing" auto for example, this is not automatically a reason to standardize, this may just be an indication that some cleanup is needed there.> > On the Phab review, some people expressed that they liked the examples of > when auto is acceptable. Here is an updated attempt at guideline text for > that section: > > > > ``` > Some are advocating a policy of "almost always ``auto``" in C++11, however > LLVM > uses a more moderate stance. Don't "almost always" use ``auto``, but it > may be used where either the Concept or the type is obvious from the > context. Here are > some cases where use of ``auto`` would make sense: > > * Where the type appears elsewhere in the line (eg a dyn_cast) > * Where the variable name or initializing expression provides enough > information (`auto SR = getSourceRange()`) > * Where the context makes the *Concept* obvious, even if the type is not > obvious (eg, where the instance is only used as an iterator, or in an > algorithm as a container-like concept, or only with a validity check, or an > AST Matcher). >The later point isn't clear to me: even if you're only using the instance as an iterator, I may want to know what types the iterator is actually iterating on. I'm not saying that the idea you have here is not desirable, just that the language used does not help me visualize what is / isn't OK: it does not fit your first criteria "Be easy to follow".> > Exceptions may arise, but they should only arise in exceptional cases. If > the case is not exceptional, apply the guidelines in review discussion. >This last sentence seems general to the full document rather than this section?> ``` > > > The most important thing here is that it does not accept your proposal > that 'the type must be obvious'. Instead, it recognizes that `auto` is > really "an unspecified concept" - unspecified only because we can't specify > the concept in C++ yet. > > However, the point/concern seems to be that readers of code should know > the instance may be used in its local scope. > > That's why these guidelines would allow `auto Ctors` in > > > llvm::Optional<std::pair<std::string, MatcherCtor>> > getNodeConstructorType(ASTNodeKind targetType) { > const auto &Ctors = RegistryData->nodeConstructors(); > auto It = llvm::find_if( > Ctors, [targetType](const NodeConstructorMap::value_type &Ctor) { > return Ctor.first.isSame(targetType); > }); > if (It == Ctors.end()) > return llvm::None; > return It->second; > } > > > because `Ctors` is obviously the Container *concept* >It is only obvious after your read the following, but more importantly: why use auto here? It wouldn't hurt to write: const NodeConstructorMap &Ctors = RegistryData->nodeConstructors(); I.e. auto does not make the code more readable to me in this case. On the other hand, if the type of Ctors is explicit, the lambda argument type could be auto to me (it becomes obvious from the local context and the use of llvm::find_if).> , and knowing exactly what type it is is not necessary in the local > context. >However, in the below code it would probably not be ok because we're> calling methods on the instance which opens up more possibilities (is it a > base interface? etc): > > > void SomeClass::foo(int input) > { > auto Ctors = getCtors(input); > > m_widgets = Ctors->calculate(); > } > > Here, `Ctors` is definitely not a Container concept, we don't know what > kind of concept it is, so we should know the type by seeing it typed in the > code. > > Another example from earlier in the thread: > > > template <typename BaseT, typename DerivedT> > void registerIfNodeMatcher(...) { > auto NodeKind = ASTNodeKind::getFromNodeKind<DerivedT>(); > if (!NodeKind.isNone()) > NodeCtors[NodeKind] = std::make_pair(MatcherName, Descriptor); > } > > Here, `NodeKind` is used as an Optional (or Maybe) *concept*. All we do is > a validity check. So, `auto` should be allowed here. > > This 'the concept must be obvious' guideline is also what allows the use > of `auto` for iterators. > > > What do people think of "Either the Concept or the type should be obvious > from the context" as a baseline guideline? >You forgot to add "and knowing exactly what type it is is not necessary in the local context" after "Concept", it seems that this is necessary for your definition. I'm still fairly unconvinced, because the concept of "concept" seems too fuzzy to be applicable in such a guideline. What is a "Concept" other than a class that honor an API? How is your previous example " m_widgets Ctors->calculate();" not just obvious that Ctors is an instance of a Concept that "can calculate a widget"? Best, -- Mehdi -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190105/db91b8d5/attachment.html>
Stephen Kelly via llvm-dev
2019-Feb-03 14:49 UTC
[llvm-dev] RFC: Modernizing our use of auto
On 31/12/2018 04:54, Chris Lattner via llvm-dev wrote:>> Do those uses conform to the guide? If they don't, then should the guide be updated? Are the types there 'obvious’? > > If/when we revise the policy, then it would make sense for non-conforming uses of auto to be changed. However, I don’t think that actually making a widespread change would be high priority... > >> How did all of those uses get into the codebase? Does it indicate that the guide is not followed, or does it indicate that the guide is too subjective, or that maybe the 'the type must be obvios' guide does not reflect the 'reality at the coalface' anymore? Should those uses of auto be changed? > > My understanding is that there has been no widely understood or accepted policy, so different coders and reviewers are doing different things.One of the things which has no consensus here is whether 'auto' may be used in lambdas (using c++-14). This feature was celebrated as a big feature which gets unlocked by migrating to toolchains which provide that feature: https://groups.google.com/forum/#!msg/llvm-dev/0VkIuhn10nE/QZ5FwYEmHAAJ So, does this need a guideline update? Is there consistency in celbrating that but writing 'remove all use of auto from this file' in reviews? If there's no consensus and no consistency, what does that mean for the code? Is if (const auto *TSI = D->getTypeSourceInfo()) ok? Some reviewers say 'no'. What is the consensus and how is that expressed in the guidelines? Does anyone have any interest in making the guidelines more clear on this? I have made several proposals, and at least Chris agreed that something should be improved, but then he left the discussion. Does anyone else think that something can be improved? Is anyone willing to read and comment on my proposal and get a change to the guidelines committed? The original email in this thread was about how to handle features that become 'unlocked' by updates to our minimum toolchain requirements. That is now upon us. Thanks, Stephen.
Mehdi AMINI via llvm-dev
2019-Feb-03 17:59 UTC
[llvm-dev] RFC: Modernizing our use of auto
On Sun, Feb 3, 2019 at 6:50 AM Stephen Kelly via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On 31/12/2018 04:54, Chris Lattner via llvm-dev wrote: > >> Do those uses conform to the guide? If they don't, then should the > guide be updated? Are the types there 'obvious’? > > > > If/when we revise the policy, then it would make sense for > non-conforming uses of auto to be changed. However, I don’t think that > actually making a widespread change would be high priority... > > > >> How did all of those uses get into the codebase? Does it indicate that > the guide is not followed, or does it indicate that the guide is too > subjective, or that maybe the 'the type must be obvios' guide does not > reflect the 'reality at the coalface' anymore? Should those uses of auto be > changed? > > > > My understanding is that there has been no widely understood or accepted > policy, so different coders and reviewers are doing different things. > > One of the things which has no consensus here is whether 'auto' may be > used in lambdas (using c++-14).Under the current guidelines, my understanding is that nothing prevents to use auto in lambda in order to "make the code more readable" when "the type is already obvious from the context" or "when the type would have been abstracted away anyways, often behind a container’s typedef such as std::vector<T>::iterator". We don't need to update the guideline on auto to be able to use auto in lambda as soon as c++14 is available.> This feature was celebrated as a big > feature which gets unlocked by migrating to toolchains which provide > that feature: > > https://groups.google.com/forum/#!msg/llvm-dev/0VkIuhn10nE/QZ5FwYEmHAAJ > > So, does this need a guideline update? > > Is there consistency in celbrating that but writing 'remove all use of > auto from this file' in reviews? > > If there's no consensus and no consistency, what does that mean for the > code? > > Is > > if (const auto *TSI = D->getTypeSourceInfo()) > > ok? > > Some reviewers say 'no'. What is the consensus and how is that expressed > in the guidelines? > > Does anyone have any interest in making the guidelines more clear on > this? > > I have made several proposals, and at least Chris agreed that something > should be improved, but then he left the discussion. > > Does anyone else think that something can be improved? Is anyone willing > to read and comment on my proposal and get a change to the guidelines > committed? >I think that multiple people read your proposal and gave feedback on Phabricator that mandates a revision (for instance for-range loop). Also in this topic I believe some feedback was given that rewording in order to remove ambiguity is always good, as long as the "spirit" of the current rule as it is written is preserved. So my take on the subject is that we're waiting on a new revision of your patch? Best, -- Mehdi> > The original email in this thread was about how to handle features that > become 'unlocked' by updates to our minimum toolchain requirements. That > is now upon us. > > Thanks, > > Stephen. > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190203/948fe688/attachment.html>