llvm dev - Mar 2016 - RFC: A new ABI for virtual calls, and a change to the virtual call representation in the IR

> On Mar 11, 2016, at 12:16 PM, Rafael Espíndola <rafael.espindola at
gmail.com> wrote:
> On 11 March 2016 at 14:58, John McCall <rjmccall at apple.com> wrote:
>>> On Mar 11, 2016, at 11:26 AM, Rafael Espíndola <rafael.espindola
at gmail.com> wrote:
>>>> Now, there are a number of things about linkage that are kindof
orthogonal,
>>>> and it would be nice to model them more orthogonally.  That
would be a major
>>>> change in representation, though.  Absent the will to do that,
I propose
>>>> that we:
>>>> - remove/deprecate protected visibility, making visibility
purely a hidden
>>>> vs. non-hidden flag
>>> 
>>> This would prevent us from propagating
>>> __attribute__((visibility("protected")) to a
STV_PROTECTED in the .o,
>>> so I don't think we should do it.
>> 
>> The whole point of this proposal is to get us to a state where we can
use
>> STV_PROTECTED for ordinary external or weak_for_linker symbols.
> 
> And you can't also just produce STV_PROTECTED for every symbol. I
> would love for that to be the case, but while most ELF systems support
> copy relocations and related PLT hacks for functions it is not
> practical to do it.
I’m sorry, I'm not familiar with the technical problems here.

For example, I don’t understand why protected symbols require new relocations.
I would expect that, as far as relocations go, they would be treated the same as
hidden symbols within the linkage unit, and the same as default symbols outside
of it.
>> __attribute__((visibility(“protected”))) on a strong definition would
just map
>> to ordinary non-hidden external linkage, which the backend would turn
into
>> STV_PROTECTED.
>> 
>> __attribute__((visibility(“default”))) on a strong definition would map
the
>> same way unless -fsemantic-interposition was enabled.
>> 
>> The point of removing protected visibility is that we don’t actually
honor
>> default visibility in the ELF sense, so it’s silly to pretend that LLVM
>> visibility is the same as ELF visibility.  ELF protected visibility is
basically
>> the semantics LLVM already assigns to default visibility.
> 
> Except that we still produce STV_DEFAULT. Basically we have a mode
> where we handle GVs as if they were protected but produce a
> STV_DEFAULT.
Yes, I don’t think this is a justifiable position.  We either do or do not
support
interposition of global definitions.
> If we are going to drop it, I think the only workable default solution
> would be for -fsemantic-interposition to be the default. That is, on
> ELF systems we would match GCC: with -fPIC we don't inline non-odr
> symbols that end up as STV_DEFAULT.
> 
> In summary, I think it is important that
> 
> * a __attribute__((visibility("protected"))) maps to STV_PROTECED
> * by default a decl with no attributes maps to STV_DEFAULT
It’s not really my place to decide whether or not -fsemantic-interposition
should be
the default, so I’ll leave this up to others to debate.

My instinct is that interposition is not actually important to very many people,
and that the people who rely on it won’t really mind having to opt in.  We have
also been optimizing as if semantic interposition were disabled for many years,
and I don’t remember hearing widespread complaints about it.  So I would
encourage us to turn it off by default, despite not matching GCC.  But like I
said, that is not ultimately my call, and I’ll leave the decision to others.

John.

>> And you can't also just produce STV_PROTECTED for every symbol. I
>> would love for that to be the case, but while most ELF systems support
>> copy relocations and related PLT hacks for functions it is not
>> practical to do it.
>
> I’m sorry, I'm not familiar with the technical problems here.
>
> For example, I don’t understand why protected symbols require new
relocations.
> I would expect that, as far as relocations go, they would be treated the
same as
> hidden symbols within the linkage unit, and the same as default symbols
outside of it.
It is not that they require. It is the other way, they don't really
work with copy relocations.
The issue is that a non -fPIC program can link with a shared library.
When it does that the only way to keep pointer equality is for the
dynamic linker to
* Copy data from .so to the main program
* Use a PLT entry in the main program as the address of functions

with that a protected visibility symbols ends up living in the main
executable. That is not well supported.
>> Except that we still produce STV_DEFAULT. Basically we have a mode
>> where we handle GVs as if they were protected but produce a
>> STV_DEFAULT.
>
> Yes, I don’t think this is a justifiable position.  We either do or do not
support
> interposition of global definitions.
I don't see why it has to be black and white like that. We have been
producing STV_DEFAULT for some time. I don't know how much we would
drop in performance if we just started not inlining some functions
just to keep them STV_DEFAULT.
>> In summary, I think it is important that
>>
>> * a __attribute__((visibility("protected"))) maps to
STV_PROTECED
>> * by default a decl with no attributes maps to STV_DEFAULT
>
> It’s not really my place to decide whether or not -fsemantic-interposition
should be
> the default, so I’ll leave this up to others to debate.
>
> My instinct is that interposition is not actually important to very many
people,
> and that the people who rely on it won’t really mind having to opt in.  We
have
> also been optimizing as if semantic interposition were disabled for many
years,
> and I don’t remember hearing widespread complaints about it.  So I would
> encourage us to turn it off by default, despite not matching GCC.  But like
I
> said, that is not ultimately my call, and I’ll leave the decision to
others.
The problem is not interposition being on or off by default. The
problem is that you are proposing locking that with producing
STV_DEFAULT or STV_PROTECTED, and I don't think we can just start
producing STV_PROTECTED everywhere.

I am personally fine having clang behave like gcc (STV_DEFAULT, assume
regular decls with -fPIC can be interposed), but would like to know
what other ELF users think of that before forcing them to select
between not inlining with -fPIC or getting STV_PROTECTED.

BTW, note that if are going to produce STV_PROTECTED anyway, we don't
need -fsemantic-interposition, we can just use -fvisibility=protected.

Cheers,
Rafael

> On Mar 11, 2016, at 1:40 PM, Rafael Espíndola <rafael.espindola at
gmail.com> wrote:
>>> And you can't also just produce STV_PROTECTED for every symbol.
I
>>> would love for that to be the case, but while most ELF systems
support
>>> copy relocations and related PLT hacks for functions it is not
>>> practical to do it.
>> 
>> I’m sorry, I'm not familiar with the technical problems here.
>> 
>> For example, I don’t understand why protected symbols require new
relocations.
>> I would expect that, as far as relocations go, they would be treated
the same as
>> hidden symbols within the linkage unit, and the same as default symbols
outside of it.
> 
> It is not that they require. It is the other way, they don't really
> work with copy relocations.
> The issue is that a non -fPIC program can link with a shared library.
> When it does that the only way to keep pointer equality is for the
> dynamic linker to
> * Copy data from .so to the main program
> * Use a PLT entry in the main program as the address of functions
> 
> with that a protected visibility symbols ends up living in the main
> executable. That is not well supported.
I’m sorry, but I didn’t quite get that.  The symbol is defined as protected
by the dynamic library, but the main executable can’t just use an ordinary
external symbol reference to it for some reason?
>>> Except that we still produce STV_DEFAULT. Basically we have a mode
>>> where we handle GVs as if they were protected but produce a
>>> STV_DEFAULT.
>> 
>> Yes, I don’t think this is a justifiable position.  We either do or do
not support
>> interposition of global definitions.
> 
> I don't see why it has to be black and white like that. We have been
> producing STV_DEFAULT for some time. I don't know how much we would
> drop in performance if we just started not inlining some functions
> just to keep them STV_DEFAULT.
I’m worried about the semantic model.  Our current model is that we allow
interposition, but calls within the linkage unit might ignore it, depending on
how much our optimizer sees.  That’s not a coherent position.
>>> In summary, I think it is important that
>>> 
>>> * a __attribute__((visibility("protected"))) maps to
STV_PROTECED
>>> * by default a decl with no attributes maps to STV_DEFAULT
>> 
>> It’s not really my place to decide whether or not
-fsemantic-interposition should be
>> the default, so I’ll leave this up to others to debate.
>> 
>> My instinct is that interposition is not actually important to very
many people,
>> and that the people who rely on it won’t really mind having to opt in. 
We have
>> also been optimizing as if semantic interposition were disabled for
many years,
>> and I don’t remember hearing widespread complaints about it.  So I
would
>> encourage us to turn it off by default, despite not matching GCC.  But
like I
>> said, that is not ultimately my call, and I’ll leave the decision to
others.
> 
> The problem is not interposition being on or off by default. The
> problem is that you are proposing locking that with producing
> STV_DEFAULT or STV_PROTECTED, and I don't think we can just start
> producing STV_PROTECTED everywhere.
I agree that we have to resolve that question before we can make a policy
statement here.

John.
> 
> I am personally fine having clang behave like gcc (STV_DEFAULT, assume
> regular decls with -fPIC can be interposed), but would like to know
> what other ELF users think of that before forcing them to select
> between not inlining with -fPIC or getting STV_PROTECTED.
> 
> BTW, note that if are going to produce STV_PROTECTED anyway, we don't
> need -fsemantic-interposition, we can just use -fvisibility=protected.
> 
> Cheers,
> Rafael

> On Mar 11, 2016, at 1:40 PM, Rafael Espíndola via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
>>> And you can't also just produce STV_PROTECTED for every symbol.
I
>>> would love for that to be the case, but while most ELF systems
support
>>> copy relocations and related PLT hacks for functions it is not
>>> practical to do it.
>> 
>> I’m sorry, I'm not familiar with the technical problems here.
>> 
>> For example, I don’t understand why protected symbols require new
relocations.
>> I would expect that, as far as relocations go, they would be treated
the same as
>> hidden symbols within the linkage unit, and the same as default symbols
outside of it.
> 
> It is not that they require. It is the other way, they don't really
> work with copy relocations.
> The issue is that a non -fPIC program can link with a shared library.
> When it does that the only way to keep pointer equality is for the
> dynamic linker to
> * Copy data from .so to the main program
> * Use a PLT entry in the main program as the address of functions
> 
> with that a protected visibility symbols ends up living in the main
> executable. That is not well supported.
> 
>>> Except that we still produce STV_DEFAULT. Basically we have a mode
>>> where we handle GVs as if they were protected but produce a
>>> STV_DEFAULT.
>> 
>> Yes, I don’t think this is a justifiable position.  We either do or do
not support
>> interposition of global definitions.
> 
> I don't see why it has to be black and white like that. We have been
> producing STV_DEFAULT for some time. I don't know how much we would
> drop in performance if we just started not inlining some functions
> just to keep them STV_DEFAULT.
> 
>>> In summary, I think it is important that
>>> 
>>> * a __attribute__((visibility("protected"))) maps to
STV_PROTECED
>>> * by default a decl with no attributes maps to STV_DEFAULT
>> 
>> It’s not really my place to decide whether or not
-fsemantic-interposition should be
>> the default, so I’ll leave this up to others to debate.
>> 
>> My instinct is that interposition is not actually important to very
many people,
>> and that the people who rely on it won’t really mind having to opt in. 
We have
>> also been optimizing as if semantic interposition were disabled for
many years,
>> and I don’t remember hearing widespread complaints about it.  So I
would
>> encourage us to turn it off by default, despite not matching GCC.  But
like I
>> said, that is not ultimately my call, and I’ll leave the decision to
others.
> 
> The problem is not interposition being on or off by default. The
> problem is that you are proposing locking that with producing
> STV_DEFAULT or STV_PROTECTED, and I don't think we can just start
> producing STV_PROTECTED everywhere.
> 
> I am personally fine having clang behave like gcc (STV_DEFAULT, assume
> regular decls with -fPIC can be interposed), but would like to know
> what other ELF users think of that before forcing them to select
> between not inlining with -fPIC or getting STV_PROTECTED.
> 
> BTW, note that if are going to produce STV_PROTECTED anyway, we don't
> need -fsemantic-interposition, we can just use -fvisibility=protected.
My impression is that it is not nice to have in the IR a "default"
visibility that change semantic in a platform specific way. I'd expect to
have well defined visibility and the front-end making the visibility choice he
wants, possibly with a  different default on each platform. It seems to me it
would be easier to support the different variant in LLVM with such a model.
But I may miss some obvious problem with such an approach as well, and I'm
interested in your opinion on this.

-- 
Mehdi