llvm dev - May 2005 - [LLVMdev] calling conventions and inlining

On Sat, 7 May 2005, Markus F.X.J. Oberhumer wrote:
> I see that you are objecting explicit inline control.
>
> The main problem is that inlining is absolutely crucial for some 
> "modern" programming styles. E.g. we use a huge collection of
small C++
> template classes and template metaclasses, most of which have very 
> trivial and limited functionality (think of it as some "bytecode"
> expressed in classes). Of course, these method calls of these classes 
> _must_ be inlined, but there are also "traditional" calls to
other
> functions which may or may not be meant for inlining.  If the inliner 
> just guesses one of these calls wrong (and it usually does) then 
> performance will drop by an order of magnitude. That's why all C++ 
> compilers I know support explicit inline control.
I understand where you are coming from.  Here are the reasons that I think 
this is a bogus argument:

I.   If the function is an obvious *must inline* choice, the compiler will
      trivially see this and inline it.  LLVM in particular is very good at
      ripping apart layers of abstraction, and there a couple of known ways
      to improve it further.  This leaves the cases where you *dont* want
      to inline stuff and cases where *want* to inline something but it is
      not obvious.

II.  For cases where you don't want it to get inlined, arrange for it to
      get marked coldcc and you're done.

III. The only remaining case is when you have a function call that is
      sufficiently "non obvious" to inline but you want to force it to
be
      inlined.  Usually this is because your are tuning your application
      and note that you get better performance by inlining.

I assume the III is what you're really worried about, so I will explain 
why I think that a "force_inline" directive on functions is a bad
idea.

1. First, this property is something that varies on a *CALL SITE* basis,
    not on a callee basis.  If you really want this, you should be
    annotating call sites, not functions.
2. These annotations are non-portable across different compilers and even
    across the different versions of the same compiler.  For example, in
    the first case, GCC does no IPO other than inlining, so forcing
    something to be inlined can make a huge difference.  LLVM, OTOH, does a
    large amount of IPO and IPA, such as dead argument elimination, IPSCCP,
    and other things.  Something that is good to inline for GCC is not
    necessarily good for LLVM.
3. Once these annotations are added to a source base, they are almost
    never removed or reevaluated.  This exacerbates #2.

In my mind, the right solution to this problem is to use profile-directed 
inlining.  If you actually care this much about the performance of your 
code, you should be willing to use profile information.  Profile 
information will help inlining, but it can also be used for far more than 
just that.

My personal opinion (which you may disagree with strongly!) is that the 
importance that many people place on "force this to be inlined"
directives
is largely based on experience with compilers that don't do any 
non-trivial IPO.  If the compiler is doing link-time IPO, the function 
call boundary is much less of a big deal.

Finally, if you have a piece of code that the LLVM optimizer is doing a 
poor job on, please please please file a bug so we can fix it!!

-Chris

-- 
http://nondot.org/sabre/
http://llvm.cs.uiuc.edu/

There is one case where inlining/not-inlining affects correctness.  A 
function which uses alloca() will behave differently in the two cases.  
You can argue one shouldn't write code like this, but it is legal.

Chris Lattner wrote:
> On Sat, 7 May 2005, Markus F.X.J. Oberhumer wrote:
>
>> I see that you are objecting explicit inline control.
>>
>> The main problem is that inlining is absolutely crucial for some 
>> "modern" programming styles. E.g. we use a huge collection of
small
>> C++ template classes and template metaclasses, most of which have 
>> very trivial and limited functionality (think of it as some 
>> "bytecode" expressed in classes). Of course, these method
calls of
>> these classes _must_ be inlined, but there are also
"traditional"
>> calls to other functions which may or may not be meant for inlining.  
>> If the inliner just guesses one of these calls wrong (and it usually 
>> does) then performance will drop by an order of magnitude. That's
why
>> all C++ compilers I know support explicit inline control.
>
>
> I understand where you are coming from.  Here are the reasons that I 
> think this is a bogus argument:
>
> I.   If the function is an obvious *must inline* choice, the compiler 
> will
>      trivially see this and inline it.  LLVM in particular is very 
> good at
>      ripping apart layers of abstraction, and there a couple of known 
> ways
>      to improve it further.  This leaves the cases where you *dont* want
>      to inline stuff and cases where *want* to inline something but it is
>      not obvious.
>
> II.  For cases where you don't want it to get inlined, arrange for it
to
>      get marked coldcc and you're done.
>
> III. The only remaining case is when you have a function call that is
>      sufficiently "non obvious" to inline but you want to force
it to be
>      inlined.  Usually this is because your are tuning your application
>      and note that you get better performance by inlining.
>
> I assume the III is what you're really worried about, so I will 
> explain why I think that a "force_inline" directive on functions
is a
> bad idea.
>
> 1. First, this property is something that varies on a *CALL SITE* basis,
>    not on a callee basis.  If you really want this, you should be
>    annotating call sites, not functions.
> 2. These annotations are non-portable across different compilers and even
>    across the different versions of the same compiler.  For example, in
>    the first case, GCC does no IPO other than inlining, so forcing
>    something to be inlined can make a huge difference.  LLVM, OTOH, 
> does a
>    large amount of IPO and IPA, such as dead argument elimination, 
> IPSCCP,
>    and other things.  Something that is good to inline for GCC is not
>    necessarily good for LLVM.
> 3. Once these annotations are added to a source base, they are almost
>    never removed or reevaluated.  This exacerbates #2.
>
> In my mind, the right solution to this problem is to use 
> profile-directed inlining.  If you actually care this much about the 
> performance of your code, you should be willing to use profile 
> information.  Profile information will help inlining, but it can also 
> be used for far more than just that.
>
> My personal opinion (which you may disagree with strongly!) is that 
> the importance that many people place on "force this to be
inlined"
> directives is largely based on experience with compilers that don't do 
> any non-trivial IPO.  If the compiler is doing link-time IPO, the 
> function call boundary is much less of a big deal.
>
> Finally, if you have a piece of code that the LLVM optimizer is doing 
> a poor job on, please please please file a bug so we can fix it!!
>
> -Chris
>

On Sat, 7 May 2005, Jeff Cohen wrote:
> There is one case where inlining/not-inlining affects correctness.  A 
> function which uses alloca() will behave differently in the two cases.  You
> can argue one shouldn't write code like this, but it is legal.
The inliner doesn't inline functions that call alloca, or other cases that 
break correctness.

-Chris
> Chris Lattner wrote:
>
>> On Sat, 7 May 2005, Markus F.X.J. Oberhumer wrote:
>> 
>>> I see that you are objecting explicit inline control.
>>> 
>>> The main problem is that inlining is absolutely crucial for some
"modern"
>>> programming styles. E.g. we use a huge collection of small C++
template
>>> classes and template metaclasses, most of which have very trivial
and
>>> limited functionality (think of it as some "bytecode"
expressed in
>>> classes). Of course, these method calls of these classes _must_ be 
>>> inlined, but there are also "traditional" calls to other
functions which
>>> may or may not be meant for inlining.  If the inliner just guesses
one of
>>> these calls wrong (and it usually does) then performance will drop
by an
>>> order of magnitude. That's why all C++ compilers I know support
explicit
>>> inline control.
>> 
>> 
>> I understand where you are coming from.  Here are the reasons that I
think
>> this is a bogus argument:
>> 
>> I.   If the function is an obvious *must inline* choice, the compiler
will
>>      trivially see this and inline it.  LLVM in particular is very good
at
>>      ripping apart layers of abstraction, and there a couple of known
ways
>>      to improve it further.  This leaves the cases where you *dont*
want
>>      to inline stuff and cases where *want* to inline something but it
is
>>      not obvious.
>> 
>> II.  For cases where you don't want it to get inlined, arrange for
it to
>>      get marked coldcc and you're done.
>> 
>> III. The only remaining case is when you have a function call that is
>>      sufficiently "non obvious" to inline but you want to
force it to be
>>      inlined.  Usually this is because your are tuning your application
>>      and note that you get better performance by inlining.
>> 
>> I assume the III is what you're really worried about, so I will
explain why
>> I think that a "force_inline" directive on functions is a bad
idea.
>> 
>> 1. First, this property is something that varies on a *CALL SITE*
basis,
>>    not on a callee basis.  If you really want this, you should be
>>    annotating call sites, not functions.
>> 2. These annotations are non-portable across different compilers and
even
>>    across the different versions of the same compiler.  For example, in
>>    the first case, GCC does no IPO other than inlining, so forcing
>>    something to be inlined can make a huge difference.  LLVM, OTOH,
does a
>>    large amount of IPO and IPA, such as dead argument elimination,
IPSCCP,
>>    and other things.  Something that is good to inline for GCC is not
>>    necessarily good for LLVM.
>> 3. Once these annotations are added to a source base, they are almost
>>    never removed or reevaluated.  This exacerbates #2.
>> 
>> In my mind, the right solution to this problem is to use
profile-directed
>> inlining.  If you actually care this much about the performance of your
>> code, you should be willing to use profile information.  Profile 
>> information will help inlining, but it can also be used for far more
than
>> just that.
>> 
>> My personal opinion (which you may disagree with strongly!) is that the
>> importance that many people place on "force this to be
inlined" directives
>> is largely based on experience with compilers that don't do any
non-trivial
>> IPO.  If the compiler is doing link-time IPO, the function call
boundary is
>> much less of a big deal.
>> 
>> Finally, if you have a piece of code that the LLVM optimizer is doing a
>> poor job on, please please please file a bug so we can fix it!!
>> 
>> -Chris
>> 
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-Chris

-- 
http://nondot.org/sabre/
http://llvm.cs.uiuc.edu/