llvm dev - Mar 2016 - RFC: A new ABI for virtual calls, and a change to the virtual call representation in the IR

> On Mar 11, 2016, at 8:33 AM, Reid Kleckner <rnk at google.com> wrote:
> On Thu, Mar 10, 2016 at 10:32 PM, John McCall via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> I mean, I’ve never really liked ELF’s stance on symbol interposition, but
taking it as given, I’m not sure I agree that it’s reasonable to carve out
virtual functions as a general exception.
> 
> Given that LLVM does IPO (inlining, funcattrs) on symbols that can be
interposed on ELF, we already don't support interposability very well:
> https://llvm.org/bugs/show_bug.cgi?id=23501
<https://llvm.org/bugs/show_bug.cgi?id=23501>
> 
> Adding another exception for virtual functions, especially under an
off-by-default flag, doesn't seem like a big deal.
Okay, so, it sounds to me like LLVM basically treats strong definitions as
protected, then.  Should we just formalize that?

I guess the proposal here would be:
  1. Remove protected visibility from LLVM.  (Or deprecate it as equivalent to
default.)
  2. Teach the ELF emitter to emit non-hidden strong definitions using ELF
protected visibility.
  3. Teach the ELF emitter to emit non-hidden weak definitions using ELF default
visibility (unless they’re unnamed_addr?).

GCC must do *some* inlining between strong definitions.  Do you know what their
rule is?  Is it just a C++ thing?

John.
>  
> Isn’t there some global flag to make (non-weak?) definitions use protected
visibility by default?  Wouldn’t using that be an overall performance win
anyway?
> 
> Joerg mentioned -Bsymbolic, but I'm surprised that I don't see
people use -Bsymbolic-functions more often. C++ has the ODR, so we can say that
all definitions of the same (C++!) function must be equivalent, and it
doesn't matter which you call. Programs very rarely rely on function pointer
identity across DSOs, and it already isn't preserved on a number of
platforms (MSVC does ICF). Typically only global data needs to be coalesced
across DSOs, so that things like static data members of templates and static
locals in inline functions work.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/114b214d/attachment.html>

On Fri, Mar 11, 2016 at 9:41 AM, John McCall <rjmccall at apple.com>
wrote:
>
> Okay, so, it sounds to me like LLVM basically treats strong definitions as
> protected, then.  Should we just formalize that?
>
> I guess the proposal here would be:
>   1. Remove protected visibility from LLVM.  (Or deprecate it as
> equivalent to default.)
>   2. Teach the ELF emitter to emit non-hidden strong definitions using ELF
> protected visibility.
>   3. Teach the ELF emitter to emit non-hidden weak definitions using ELF
> default visibility (unless they’re unnamed_addr?).
>
LLVM definitely thinks that strong definitions are "ODR", so we could
do 2.
I'm sure it would break somebody, somewhere though. We probably don't
want
to do 1 and 3, so that users can still mark their weak definitions as
protected if they want to.

GCC must do *some* inlining between strong definitions.  Do you know
what
> their rule is?  Is it just a C++ thing?
>
GCC only inlines strong definitions when -fPIC is disabled, meaning the
code will be linked into the main executable. In this case, no
interposition can occur because dynamic symbol search always starts in the
executable. They also have -fno-semantic-interposition, which allows
inlining strong definitions:
https://gcc.gnu.org/onlinedocs/gcc-5.3.0/gcc/Optimize-Options.html#index-fsemantic-interposition-972
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/778443cc/attachment.html>

On 11 March 2016 at 12:41, John McCall via cfe-dev
<cfe-dev at lists.llvm.org> wrote:
> On Mar 11, 2016, at 8:33 AM, Reid Kleckner <rnk at google.com> wrote:
> On Thu, Mar 10, 2016 at 10:32 PM, John McCall via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>
>> I mean, I’ve never really liked ELF’s stance on symbol interposition,
but
>> taking it as given, I’m not sure I agree that it’s reasonable to carve
out
>> virtual functions as a general exception.
>
>
> Given that LLVM does IPO (inlining, funcattrs) on symbols that can be
> interposed on ELF, we already don't support interposability very well:
> https://llvm.org/bugs/show_bug.cgi?id=23501
>
> Adding another exception for virtual functions, especially under an
> off-by-default flag, doesn't seem like a big deal.
>
>
> Okay, so, it sounds to me like LLVM basically treats strong definitions as
> protected, then.  Should we just formalize that?
>
> I guess the proposal here would be:
>   1. Remove protected visibility from LLVM.  (Or deprecate it as equivalent
> to default.)
I wouldn't go that far. At least no for now.

It is true that given llvm's view of what it can inline, producing ELF
files with protected visibility would seem reasonable. The problem is
that they don't work well in combination with copy relocations, so we
cannot just produce protected everywhere.

But, there are cases where they are useful, so we should be able to
propagate __attribute__((visibility,("protected"))) to the object
file.

Cheers,
Rafael

On Fri, Mar 11, 2016 at 9:56 AM, Reid Kleckner <rnk at google.com> wrote:
> On Fri, Mar 11, 2016 at 9:41 AM, John McCall <rjmccall at apple.com>
wrote:
>>
>> Okay, so, it sounds to me like LLVM basically treats strong definitions
>> as protected, then.  Should we just formalize that?
>>
>> I guess the proposal here would be:
>>   1. Remove protected visibility from LLVM.  (Or deprecate it as
>> equivalent to default.)
>>   2. Teach the ELF emitter to emit non-hidden strong definitions using
>> ELF protected visibility.
>>   3. Teach the ELF emitter to emit non-hidden weak definitions using
ELF
>> default visibility (unless they’re unnamed_addr?).
>>
>
> LLVM definitely thinks that strong definitions are "ODR", so we
could do
> 2. I'm sure it would break somebody, somewhere though. We probably
don't
> want to do 1 and 3, so that users can still mark their weak definitions as
> protected if they want to.
>
Yes, but what I think John was getting at with 3 could work (modulo
possible breakage, as you mention): "Teach the ELF emitter to emit
non-hidden weak unnamed_addr definitions using ELF protected visibility".

Thanks,
-- 
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/13c6abef/attachment.html>

> On Mar 11, 2016, at 9:56 AM, Reid Kleckner <rnk at google.com> wrote:
> 
> On Fri, Mar 11, 2016 at 9:41 AM, John McCall <rjmccall at apple.com
<mailto:rjmccall at apple.com>> wrote:
> Okay, so, it sounds to me like LLVM basically treats strong definitions as
protected, then.  Should we just formalize that?
> 
> I guess the proposal here would be:
>   1. Remove protected visibility from LLVM.  (Or deprecate it as equivalent
to default.)
>   2. Teach the ELF emitter to emit non-hidden strong definitions using ELF
protected visibility.
>   3. Teach the ELF emitter to emit non-hidden weak definitions using ELF
default visibility (unless they’re unnamed_addr?).
> 
> LLVM definitely thinks that strong definitions are "ODR", so we
could do 2. I'm sure it would break somebody, somewhere though. We probably
don't want to do 1 and 3, so that users can still mark their weak
definitions as protected if they want to.
Ah, that’s true, protected would still be meaningful for weak definitions.  
> GCC must do *some* inlining between strong definitions.  Do you know what
their rule is?  Is it just a C++ thing?
> 
> GCC only inlines strong definitions when -fPIC is disabled, meaning the
code will be linked into the main executable. In this case, no interposition can
occur because dynamic symbol search always starts in the executable. They also
have -fno-semantic-interposition, which allows inlining strong definitions:
>
https://gcc.gnu.org/onlinedocs/gcc-5.3.0/gcc/Optimize-Options.html#index-fsemantic-interposition-972
<https://gcc.gnu.org/onlinedocs/gcc-5.3.0/gcc/Optimize-Options.html#index-fsemantic-interposition-972>
Okay.  The way we should be thinking about this is that
-fno-semantic-interposition is the default rule on Clang.  Now, if we fully
embrace that by making symbols protected by default, we will break somebody out
there who’s relying on interposition despite our poor support for it. 
Therefore, we need to have a story for how to unbreak them.  For a small number
of interposable symbols, it’s reasonable for us to tell them to use
__attribute__((weak)).  However, some people might have a ton of symbols they
interpose, and other people will complain about us doing things differently from
GCC; for those people, we will need to support an explicit
-fsemantic-interposition option.  So let’s talk about how to support that.

Deciding whether a symbol is dynamically replaceable should not be a target- and
option-specific decision in LLVM IR.  It seems obvious to me that the right
default rule is that symbols cannot be replaced.  We already have several ways
to express replaceability in IR, but they’re all linkage kinds, not visibility
kinds; and in fact, there are some basic APIs in LLVM that assume that this
decision is wholly a matter of the linkage kind.  So it seems reasonable to me
to eliminate protected visibility, and then make sure we can still express
everything we need to using linkage, adding new linkage kinds as necessary.

It seems to me that the basic breakdown here is:
  - Is this definition strong, ODR-weak, or semantically replaceable within the
linkage unit?
  - Is this definition strong, ODR-weak, or semantically replaceable outside the
linkage unit?

When the definition uses the same rules for both, the existing linkages are
fine: external, weak_odr / linkonce_odr, or weak, respectively.  So let’s break
down the cases by their declared visibility and see where it’s possible to use
different rules inside and outside the linkage unit.

Hidden definitions obviously never use different rules outside the linkage unit.

Protected definitions are strong outside the linkage unit.  If the definition
isn’t strong within the linkage unit, this is a difference we don’t have the
ability to express in linkage today.

Default definitions under -fno-semantic-interposition use the same rules outside
and inside the linkage unit.

Default definitions under -fsemantic-interposition are semantically replaceable
outside the linkage unit.  The GCC rule is (thankfully) that the ODR still takes
priority for inline definitions, and weak definitions are semantically
replaceable everywhere, so this only actually makes a difference for strong
definitions.  We can’t just say that the definition is weak, because that
changes linker behavior: we still want the linker to diagnose attempts to
provide multiple strong definitions, and the strong definition still needs to
take priority over any weak definitions within the linkage unit.  So we don’t
have the ability to express “strong internally, weak externally” today in
linkage.

Now, there are a number of things about linkage that are kindof orthogonal, and
it would be nice to model them more orthogonally.  That would be a major change
in representation, though.  Absent the will to do that, I propose that we:
  - remove/deprecate protected visibility, making visibility purely a hidden vs.
non-hidden flag
  - add weak_for_linker, weak_odr_for_linker, linkonce_for_linker, and
linkonce_odr_for_linker (better names highly desired) to model the
weak-with-protected-visibility cases
  - add strong_for_linker (better name highly desired) to model the
strong-but-interposable case

John.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/7d24ed3e/attachment.html>