llvm dev - Feb 2018 - ThinLTO and linkonce_odr + unnamed_addr

Interesting. Now I understand the full picture. For MachO, linker is picking
based on atoms so it picks the atom with more visibility which doesn’t have the
constraint you mentioned. Let me do some more digging. Worst case I will add a
different rule for MachO but it might be fine.

Steven
> On Feb 9, 2018, at 1:40 PM, Rafael Avila de Espindola <rafael.espindola
at gmail.com> wrote:
> 
> Steven Wu <stevenwu at apple.com> writes:
> 
>> Rafael, another question for you. IRLinker currently takes min
visibility for the symbol (lib/Linker/LinkModules.cpp:120). Should it take the
max visibility? At least that is what ld64 is doing and is somewhat related to
this change because I want to make sure the behavior is consistent if we mark
more stuff as hidden.
> 
> No, it should take the min visibility to be consistent with the ELF
> spec. The spec says
> 
> If different visibility attributes are specified for distinct
> references to or definitions of a symbol, the most constraining
> visibility attribute must be propagated to the resolving symbol in the
> linked object.
> 
> The reason the ELF spec mandates the minimum is that a .o having a
> hidden symbol foo means that that file can access foo in a way that is
> only valid under the assumption it is local. It not might change the pic
> register before doing a call for example.
> 
> Cheers,
> Rafael

I did a bit more digging for the auto hide problem. Here is my finding that
prevent us from doing this by default in GlobalOpts

1. When a symbol is linkonce_odr hidden unnamed_addr, it emits both
'.private_extern' and '.weak_def_can_be_hidden' asm directives
on macho platform. There result of that is .private_extern will win so this is
essentially linkonce_odr hidden.
2. ld64 does treat these two type of symbols differently. For example, ld64 will
deduplicate all the can_be_hidden symbols to reduce code size. This can't be
achieved when the symbols is private external.

Steven
> On Feb 9, 2018, at 2:00 PM, Steven Wu via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Interesting. Now I understand the full picture. For MachO, linker is
picking based on atoms so it picks the atom with more visibility which doesn’t
have the constraint you mentioned. Let me do some more digging. Worst case I
will add a different rule for MachO but it might be fine.
> 
> Steven
> 
>> On Feb 9, 2018, at 1:40 PM, Rafael Avila de Espindola
<rafael.espindola at gmail.com> wrote:
>> 
>> Steven Wu <stevenwu at apple.com> writes:
>> 
>>> Rafael, another question for you. IRLinker currently takes min
visibility for the symbol (lib/Linker/LinkModules.cpp:120). Should it take the
max visibility? At least that is what ld64 is doing and is somewhat related to
this change because I want to make sure the behavior is consistent if we mark
more stuff as hidden.
>> 
>> No, it should take the min visibility to be consistent with the ELF
>> spec. The spec says
>> 
>> If different visibility attributes are specified for distinct
>> references to or definitions of a symbol, the most constraining
>> visibility attribute must be propagated to the resolving symbol in the
>> linked object.
>> 
>> The reason the ELF spec mandates the minimum is that a .o having a
>> hidden symbol foo means that that file can access foo in a way that is
>> only valid under the assumption it is local. It not might change the
pic
>> register before doing a call for example.
>> 
>> Cheers,
>> Rafael
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

> On Feb 14, 2018, at 09:55, Steven Wu via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> I did a bit more digging for the auto hide problem. Here is my finding that
prevent us from doing this by default in GlobalOpts
> 
> 1. When a symbol is linkonce_odr hidden unnamed_addr, it emits both
'.private_extern' and '.weak_def_can_be_hidden' asm directives
on macho platform. There result of that is .private_extern will win so this is
essentially linkonce_odr hidden.
> 2. ld64 does treat these two type of symbols differently. For example, ld64
will deduplicate all the can_be_hidden symbols to reduce code size. This
can't be achieved when the symbols is private external.
For others' context, this deduplication in ld64 will effectively merge
(e.g.) std::vector<int>::push_back with
std::vector<unsigned>::push_back based on function content hashes.

It would be a shame to block this optimization because of #1 – it seems like
there should be a way to work around that.  Perhaps there's something we can
change in ld64.
> Steven
> 
>> On Feb 9, 2018, at 2:00 PM, Steven Wu via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>> 
>> Interesting. Now I understand the full picture. For MachO, linker is
picking based on atoms so it picks the atom with more visibility which doesn’t
have the constraint you mentioned. Let me do some more digging. Worst case I
will add a different rule for MachO but it might be fine.
>> 
>> Steven
>> 
>>> On Feb 9, 2018, at 1:40 PM, Rafael Avila de Espindola
<rafael.espindola at gmail.com> wrote:
>>> 
>>> Steven Wu <stevenwu at apple.com> writes:
>>> 
>>>> Rafael, another question for you. IRLinker currently takes min
visibility for the symbol (lib/Linker/LinkModules.cpp:120). Should it take the
max visibility? At least that is what ld64 is doing and is somewhat related to
this change because I want to make sure the behavior is consistent if we mark
more stuff as hidden.
>>> 
>>> No, it should take the min visibility to be consistent with the ELF
>>> spec. The spec says
>>> 
>>> If different visibility attributes are specified for distinct
>>> references to or definitions of a symbol, the most constraining
>>> visibility attribute must be propagated to the resolving symbol in
the
>>> linked object.
>>> 
>>> The reason the ELF spec mandates the minimum is that a .o having a
>>> hidden symbol foo means that that file can access foo in a way that
is
>>> only valid under the assumption it is local. It not might change
the pic
>>> register before doing a call for example.
>>> 
>>> Cheers,
>>> Rafael
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Steven Wu <stevenwu at apple.com> writes:
> I did a bit more digging for the auto hide problem. Here is my finding that
prevent us from doing this by default in GlobalOpts
>
> 1. When a symbol is linkonce_odr hidden unnamed_addr, it emits both
'.private_extern' and '.weak_def_can_be_hidden' asm directives
on macho platform. There result of that is .private_extern will win so this is
essentially linkonce_odr hidden.
What do those directives mean? I assume .weak_def_can_be_hidden is the
"you can drop this from the symbol table", but .private_extern I am
not
sure.
> 2. ld64 does treat these two type of symbols differently. For example, ld64
will deduplicate all the can_be_hidden symbols to reduce code size. This
can't be achieved when the symbols is private external.
If I understand you correctly, ld64 will deduplicate
std::vector<int>::push_back and std::vector<unsigned>::push_back,
but it
will not deduplicate std::vector<HiddenClassA>::push_back and
std::vector<HiddenClassB>::push_back. Is that correct? Do you know why
it has that limitation?

Thanks,
Rafael