Robinson, Paul via llvm-dev
2017-Jan-11 16:34 UTC
[llvm-dev] Redefining optnone to help LTO
In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record "I was compiled with -O0" in the IR, because it seems like a good idea to remember that fact in an LTO compilation and there is no way to remember that fact currently. A couple of people felt it might be better to have this idea discussed on the dev list, where it might get better exposure, so I'm volunteering to get that discussion started. While 'optnone' does cause lots of optimizations to bypass a function, exactly matching -O0 was not the motivation and never a hard requirement. The implementation makes a distinct effort to get close to the behavior of -O0, but it's not an exact match and for the intended purpose (allowing a given function to be un-optimized to help debugging) it worked fine. Using 'optnone' to convey -O0 to LTO is something of a redefinition, or at least a re-purposing, of the attribute. To get there from here, I think we would need a couple of things to happen, separately from the minor grunt work of adding 'optnone' to function IR at -O0. 1) Update the LangRef definition of 'optnone' to reflect this intent. The current definition doesn't provide a motivation, and the description is (deliberately) a bit vague. If we want 'optnone' to intentionally match -O0, that should be tightened up. 2) Make a concerted effort to teach 'optnone' to targets. Currently I know the X86 target is aware of it, but I'm not so sure about others. 3) Take another look at what 'optnone' currently does *not* turn off, and see if there is something we can do about that. In some cases this will not be practical, and we may just have to live with that. (Okay, we need 3 things to happen.) I won't say this is blocking Mehdi's work, but it would remove a point of contention and allow the review to proceed more smoothly. --paulr
Robinson, Paul via llvm-dev
2017-Jan-11 18:46 UTC
[llvm-dev] Redefining optnone to help LTO
As someone who spent rather a lot of effort getting optnone done in the first place, let me say explicitly that I support this redefinition.> it seems like a good idea to remember that fact in an LTO compilationIn fact the use-case is rather more compelling than I said. One goal is supporting cross-component IR analyses such as those needed for Call-Frame Integrity, while still allowing one or more components to be compiled with -O0 (and having LTO comply with the -O0 request). I don't plan to do any of the actual work, but the idea WFM. --paulr
Philip Reames via llvm-dev
2017-Jan-14 05:43 UTC
[llvm-dev] Redefining optnone to help LTO
I would prefer we introduce a new attribute for this purpose. I regularly use optnone for debugging/reduction purposes or when trying to understand the interaction of our pass pipeline. Having an attribute that tells the optimizer to ignore a function (more or less) is really useful, and I'd rather not loose that functionality. Philip On 01/11/2017 08:34 AM, Robinson, Paul via llvm-dev wrote:> In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record > "I was compiled with -O0" in the IR, because it seems like a good idea to > remember that fact in an LTO compilation and there is no way to remember > that fact currently. A couple of people felt it might be better to have > this idea discussed on the dev list, where it might get better exposure, > so I'm volunteering to get that discussion started. > > While 'optnone' does cause lots of optimizations to bypass a function, > exactly matching -O0 was not the motivation and never a hard requirement. > The implementation makes a distinct effort to get close to the behavior > of -O0, but it's not an exact match and for the intended purpose (allowing > a given function to be un-optimized to help debugging) it worked fine. > > Using 'optnone' to convey -O0 to LTO is something of a redefinition, or > at least a re-purposing, of the attribute. To get there from here, I > think we would need a couple of things to happen, separately from the > minor grunt work of adding 'optnone' to function IR at -O0. > > 1) Update the LangRef definition of 'optnone' to reflect this intent. > The current definition doesn't provide a motivation, and the description > is (deliberately) a bit vague. If we want 'optnone' to intentionally > match -O0, that should be tightened up. > > 2) Make a concerted effort to teach 'optnone' to targets. Currently > I know the X86 target is aware of it, but I'm not so sure about others. > > 3) Take another look at what 'optnone' currently does *not* turn off, > and see if there is something we can do about that. In some cases this > will not be practical, and we may just have to live with that. > > (Okay, we need 3 things to happen.) > > I won't say this is blocking Mehdi's work, but it would remove a > point of contention and allow the review to proceed more smoothly. > --paulr > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Can you clarify what would be the semantic of this new attribute compared to optnone? Thanks, — Mehdi> On Jan 13, 2017, at 9:43 PM, Philip Reames via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > I would prefer we introduce a new attribute for this purpose. I regularly use optnone for debugging/reduction purposes or when trying to understand the interaction of our pass pipeline. Having an attribute that tells the optimizer to ignore a function (more or less) is really useful, and I'd rather not loose that functionality. > > Philip > > > On 01/11/2017 08:34 AM, Robinson, Paul via llvm-dev wrote: >> In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record >> "I was compiled with -O0" in the IR, because it seems like a good idea to >> remember that fact in an LTO compilation and there is no way to remember >> that fact currently. A couple of people felt it might be better to have >> this idea discussed on the dev list, where it might get better exposure, >> so I'm volunteering to get that discussion started. >> >> While 'optnone' does cause lots of optimizations to bypass a function, >> exactly matching -O0 was not the motivation and never a hard requirement. >> The implementation makes a distinct effort to get close to the behavior >> of -O0, but it's not an exact match and for the intended purpose (allowing >> a given function to be un-optimized to help debugging) it worked fine. >> >> Using 'optnone' to convey -O0 to LTO is something of a redefinition, or >> at least a re-purposing, of the attribute. To get there from here, I >> think we would need a couple of things to happen, separately from the >> minor grunt work of adding 'optnone' to function IR at -O0. >> >> 1) Update the LangRef definition of 'optnone' to reflect this intent. >> The current definition doesn't provide a motivation, and the description >> is (deliberately) a bit vague. If we want 'optnone' to intentionally >> match -O0, that should be tightened up. >> >> 2) Make a concerted effort to teach 'optnone' to targets. Currently >> I know the X86 target is aware of it, but I'm not so sure about others. >> >> 3) Take another look at what 'optnone' currently does *not* turn off, >> and see if there is something we can do about that. In some cases this >> will not be practical, and we may just have to live with that. >> >> (Okay, we need 3 things to happen.) >> >> I won't say this is blocking Mehdi's work, but it would remove a >> point of contention and allow the review to proceed more smoothly. >> --paulr >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
What is the end goal? If the goal is roughly "if a user passes -O0 when compiling a TU in LTO mode, their final binary should have functions that look like the result of -O0 noLTO compilation", then the frontend can just emit a normal -O0 object file I think. What is the advantage of passing bitcode all the way to the linker only to jump through hoops to prevent LTO code generation from modifying it? -- Sean Silva On Wed, Jan 11, 2017 at 8:34 AM, Robinson, Paul via llvm-dev < llvm-dev at lists.llvm.org> wrote:> In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record > "I was compiled with -O0" in the IR, because it seems like a good idea to > remember that fact in an LTO compilation and there is no way to remember > that fact currently. A couple of people felt it might be better to have > this idea discussed on the dev list, where it might get better exposure, > so I'm volunteering to get that discussion started. > > While 'optnone' does cause lots of optimizations to bypass a function, > exactly matching -O0 was not the motivation and never a hard requirement. > The implementation makes a distinct effort to get close to the behavior > of -O0, but it's not an exact match and for the intended purpose (allowing > a given function to be un-optimized to help debugging) it worked fine. > > Using 'optnone' to convey -O0 to LTO is something of a redefinition, or > at least a re-purposing, of the attribute. To get there from here, I > think we would need a couple of things to happen, separately from the > minor grunt work of adding 'optnone' to function IR at -O0. > > 1) Update the LangRef definition of 'optnone' to reflect this intent. > The current definition doesn't provide a motivation, and the description > is (deliberately) a bit vague. If we want 'optnone' to intentionally > match -O0, that should be tightened up. > > 2) Make a concerted effort to teach 'optnone' to targets. Currently > I know the X86 target is aware of it, but I'm not so sure about others. > > 3) Take another look at what 'optnone' currently does *not* turn off, > and see if there is something we can do about that. In some cases this > will not be practical, and we may just have to live with that. > > (Okay, we need 3 things to happen.) > > I won't say this is blocking Mehdi's work, but it would remove a > point of contention and allow the review to proceed more smoothly. > --paulr > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/d0b093c5/attachment.html>
> On Jan 16, 2017, at 10:00 AM, Sean Silva via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > What is the end goal? If the goal is roughly "if a user passes -O0 when compiling a TU in LTO mode, their final binary should have functions that look like the result of -O0 noLTO compilation", then the frontend can just emit a normal -O0 object file I think. What is the advantage of passing bitcode all the way to the linker only to jump through hoops to prevent LTO code generation from modifying it?That’s a good point, that addresses the “I want LTO for optimization”, which seems contradictory to use `-flto -O0`, and could even advocate for forbidding (or warning). However LTO has other uses than optimizations: for instance instrumentations that needs full program access. One of them is CFI: http://clang.llvm.org/docs/ControlFlowIntegrity.html <http://clang.llvm.org/docs/ControlFlowIntegrity.html> If you want to debug and rebuild only part of the program, CFI still requires to use LTO, IIUC. — Mehdi> > -- Sean Silva > > On Wed, Jan 11, 2017 at 8:34 AM, Robinson, Paul via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record > "I was compiled with -O0" in the IR, because it seems like a good idea to > remember that fact in an LTO compilation and there is no way to remember > that fact currently. A couple of people felt it might be better to have > this idea discussed on the dev list, where it might get better exposure, > so I'm volunteering to get that discussion started. > > While 'optnone' does cause lots of optimizations to bypass a function, > exactly matching -O0 was not the motivation and never a hard requirement. > The implementation makes a distinct effort to get close to the behavior > of -O0, but it's not an exact match and for the intended purpose (allowing > a given function to be un-optimized to help debugging) it worked fine. > > Using 'optnone' to convey -O0 to LTO is something of a redefinition, or > at least a re-purposing, of the attribute. To get there from here, I > think we would need a couple of things to happen, separately from the > minor grunt work of adding 'optnone' to function IR at -O0. > > 1) Update the LangRef definition of 'optnone' to reflect this intent. > The current definition doesn't provide a motivation, and the description > is (deliberately) a bit vague. If we want 'optnone' to intentionally > match -O0, that should be tightened up. > > 2) Make a concerted effort to teach 'optnone' to targets. Currently > I know the X86 target is aware of it, but I'm not so sure about others. > > 3) Take another look at what 'optnone' currently does *not* turn off, > and see if there is something we can do about that. In some cases this > will not be practical, and we may just have to live with that. > > (Okay, we need 3 things to happen.) > > I won't say this is blocking Mehdi's work, but it would remove a > point of contention and allow the review to proceed more smoothly. > --paulr > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/954421a2/attachment.html>
Robinson, Paul via llvm-dev
2017-Jan-17 02:07 UTC
[llvm-dev] Redefining optnone to help LTO
> -----Original Message----- > From: Philip Reames [mailto:listmail at philipreames.com] > Sent: Friday, January 13, 2017 9:43 PM > To: Robinson, Paul; llvm-dev at lists.llvm.org > Subject: Re: [llvm-dev] Redefining optnone to help LTO > > I would prefer we introduce a new attribute for this purpose. I > regularly use optnone for debugging/reduction purposes or when trying to > understand the interaction of our pass pipeline. Having an attribute > that tells the optimizer to ignore a function (more or less) is really > useful, and I'd rather not loose that functionality. > > PhilipHi Philip, I might have incorrectly given the impression that the attribute would become something used _only_ for LTO. That isn't the intent at all. In *addition* to all the ways you can set 'optnone' now, it would *also* be set on more-or-less all functions at -O0. Then the optimization passes (whether they run immediately in a normal compilation, or later in an LTO compilation) would continue to notice the attribute and do the usual thing. At -O0 in normal compilation, the attribute would have no practical effect because no pass that looks for 'optnone' would actually run in the first place; that is, for -O0 in a normal compilation, adding 'optnone' to everything is really a no-op. The "more-or-less" qualification comes up because optnone is not allowed in combination with certain other attributes. Don't want to be creating invalid IR, after all! But that's the only exceptional case. Does that help? --paulr
Eric Christopher via llvm-dev
2017-Jan-18 01:39 UTC
[llvm-dev] Redefining optnone to help LTO
Honestly instead of optnone I'd prefer to do something else around storing optimization levels, but I think the connotations of that in LTO is going to be painful: 1) What does it mean to merge two modules of different optimization levels? 2) What does it mean to inline two functions of different optimization levels? 3) What code generator should be used? Etc. Honestly I think this is a lot of stuff we don't have any good ideas for and that the current LTO scope doesn't really have. If there's really a need for it though... (Also, if we don't want to store the actual optimization levels then replace "different optimization levels" above with "optnone and not-optnone" :) -eric On Wed, Jan 11, 2017 at 8:34 AM Robinson, Paul via llvm-dev < llvm-dev at lists.llvm.org> wrote:> In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record > "I was compiled with -O0" in the IR, because it seems like a good idea to > remember that fact in an LTO compilation and there is no way to remember > that fact currently. A couple of people felt it might be better to have > this idea discussed on the dev list, where it might get better exposure, > so I'm volunteering to get that discussion started. > > While 'optnone' does cause lots of optimizations to bypass a function, > exactly matching -O0 was not the motivation and never a hard requirement. > The implementation makes a distinct effort to get close to the behavior > of -O0, but it's not an exact match and for the intended purpose (allowing > a given function to be un-optimized to help debugging) it worked fine. > > Using 'optnone' to convey -O0 to LTO is something of a redefinition, or > at least a re-purposing, of the attribute. To get there from here, I > think we would need a couple of things to happen, separately from the > minor grunt work of adding 'optnone' to function IR at -O0. > > 1) Update the LangRef definition of 'optnone' to reflect this intent. > The current definition doesn't provide a motivation, and the description > is (deliberately) a bit vague. If we want 'optnone' to intentionally > match -O0, that should be tightened up. > > 2) Make a concerted effort to teach 'optnone' to targets. Currently > I know the X86 target is aware of it, but I'm not so sure about others. > > 3) Take another look at what 'optnone' currently does *not* turn off, > and see if there is something we can do about that. In some cases this > will not be practical, and we may just have to live with that. > > (Okay, we need 3 things to happen.) > > I won't say this is blocking Mehdi's work, but it would remove a > point of contention and allow the review to proceed more smoothly. > --paulr > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170118/fd8e278c/attachment.html>
Robinson, Paul via llvm-dev
2017-Jan-18 02:12 UTC
[llvm-dev] Redefining optnone to help LTO
If we're not storing actual optimization levels (which is Chandler's expressed preference, in the review), and just looking at optnone/not-optnone, then all those questions have straightforward answers (2 & 3 are already things the existing optnone attribute has to deal with). 1) every function would be marked as optnone or not, so there is no concept of a "module" optimization level. 2) inlining is already well-defined for optnone v. not-optnone: 2a) calls made by an optnone function are never inlined except for calls to always_inline functions 2b) optnone functions are always noinline, therefore calls to an optnone function are never inlined 3) optnone functions use FastISel (assuming that's what you meant by "code generator") --paulr From: Eric Christopher [mailto:echristo at gmail.com] Sent: Tuesday, January 17, 2017 5:39 PM To: Robinson, Paul; llvm-dev at lists.llvm.org; Mehdi Amini Subject: Re: [llvm-dev] Redefining optnone to help LTO Honestly instead of optnone I'd prefer to do something else around storing optimization levels, but I think the connotations of that in LTO is going to be painful: 1) What does it mean to merge two modules of different optimization levels? 2) What does it mean to inline two functions of different optimization levels? 3) What code generator should be used? Etc. Honestly I think this is a lot of stuff we don't have any good ideas for and that the current LTO scope doesn't really have. If there's really a need for it though... (Also, if we don't want to store the actual optimization levels then replace "different optimization levels" above with "optnone and not-optnone" :) -eric On Wed, Jan 11, 2017 at 8:34 AM Robinson, Paul via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record "I was compiled with -O0" in the IR, because it seems like a good idea to remember that fact in an LTO compilation and there is no way to remember that fact currently. A couple of people felt it might be better to have this idea discussed on the dev list, where it might get better exposure, so I'm volunteering to get that discussion started. While 'optnone' does cause lots of optimizations to bypass a function, exactly matching -O0 was not the motivation and never a hard requirement. The implementation makes a distinct effort to get close to the behavior of -O0, but it's not an exact match and for the intended purpose (allowing a given function to be un-optimized to help debugging) it worked fine. Using 'optnone' to convey -O0 to LTO is something of a redefinition, or at least a re-purposing, of the attribute. To get there from here, I think we would need a couple of things to happen, separately from the minor grunt work of adding 'optnone' to function IR at -O0. 1) Update the LangRef definition of 'optnone' to reflect this intent. The current definition doesn't provide a motivation, and the description is (deliberately) a bit vague. If we want 'optnone' to intentionally match -O0, that should be tightened up. 2) Make a concerted effort to teach 'optnone' to targets. Currently I know the X86 target is aware of it, but I'm not so sure about others. 3) Take another look at what 'optnone' currently does *not* turn off, and see if there is something we can do about that. In some cases this will not be practical, and we may just have to live with that. (Okay, we need 3 things to happen.) I won't say this is blocking Mehdi's work, but it would remove a point of contention and allow the review to proceed more smoothly. --paulr _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170118/3ad2296a/attachment-0001.html>
Ultimately pushing a much stuff a possible to function attributes makes it the most natural. Think -ffast-math, -Os, -Oz, or subtargets attributes. Handling all this stuff at the function level has been proven the easiest to get straightfoward during LTO, and more flexible for “advanced” use-case when mixing modes in a single module. The reasons to not address O1 vs O2 vs O3 is IMO that 1) it is complicated and 2) there isn’t much use-case for this, or I didn’t hear about anyone really caring a lot about it. So pragmatically, it makes sense to me to try to address first the low-hanging fruits and to solve what is valuable (I can’t build our kernel with ThinLTO because we don’t encode -ffreestanding correctly, etc.). Now, if someone has a great idea on how to handle O1/O2/O3 consistently in a way that includes LTO, I’m willing to help on the implementation side! — Mehdi> On Jan 17, 2017, at 5:39 PM, Eric Christopher <echristo at gmail.com> wrote: > > Honestly instead of optnone I'd prefer to do something else around storing optimization levels, but I think the connotations of that in LTO is going to be painful: > > 1) What does it mean to merge two modules of different optimization levels? > 2) What does it mean to inline two functions of different optimization levels? > 3) What code generator should be used? > > Etc. > > Honestly I think this is a lot of stuff we don't have any good ideas for and that the current LTO scope doesn't really have. If there's really a need for it though... > > (Also, if we don't want to store the actual optimization levels then replace "different optimization levels" above with "optnone and not-optnone" :) > > -eric > > > On Wed, Jan 11, 2017 at 8:34 AM Robinson, Paul via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record > "I was compiled with -O0" in the IR, because it seems like a good idea to > remember that fact in an LTO compilation and there is no way to remember > that fact currently. A couple of people felt it might be better to have > this idea discussed on the dev list, where it might get better exposure, > so I'm volunteering to get that discussion started. > > While 'optnone' does cause lots of optimizations to bypass a function, > exactly matching -O0 was not the motivation and never a hard requirement. > The implementation makes a distinct effort to get close to the behavior > of -O0, but it's not an exact match and for the intended purpose (allowing > a given function to be un-optimized to help debugging) it worked fine. > > Using 'optnone' to convey -O0 to LTO is something of a redefinition, or > at least a re-purposing, of the attribute. To get there from here, I > think we would need a couple of things to happen, separately from the > minor grunt work of adding 'optnone' to function IR at -O0. > > 1) Update the LangRef definition of 'optnone' to reflect this intent. > The current definition doesn't provide a motivation, and the description > is (deliberately) a bit vague. If we want 'optnone' to intentionally > match -O0, that should be tightened up. > > 2) Make a concerted effort to teach 'optnone' to targets. Currently > I know the X86 target is aware of it, but I'm not so sure about others. > > 3) Take another look at what 'optnone' currently does *not* turn off, > and see if there is something we can do about that. In some cases this > will not be practical, and we may just have to live with that. > > (Okay, we need 3 things to happen.) > > I won't say this is blocking Mehdi's work, but it would remove a > point of contention and allow the review to proceed more smoothly. > --paulr > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170117/7827961d/attachment.html>