llvm dev - Jun 2018 - Proposal for address-significance tables for --icf=safe

On Wed, May 23, 2018 at 3:07 PM, Peter Collingbourne <peter at pcc.me.uk>
wrote:
>
>
> On Wed, May 23, 2018 at 12:16 PM, Peter Collingbourne <peter at
pcc.me.uk>
> wrote:
>
>> On Wed, May 23, 2018 at 3:25 AM, Peter Smith <peter.smith at
linaro.org>
>> wrote:
>>
>>> Hello,
>>>
>>> I think that the approach of using a section to record address
>>> significance is a good one. I'm guessing it will have its own
section
>>> type and format? If it does would it make sense to try and submit
this
>>> to the GABI https://groups.google.com/forum/#!forum/generic-abi as
it
>>> could be potentially useful for other linkers, for example gold?
>>>
>>
>> Yes, there is a new section type (SHT_LLVM_ADDRSIG) and format (a
>> sequence of ULEB128-encoded symbol table indexes that are
>> address-significant).
>>
>> I think it makes sense for this to eventually be part of the generic
ABI,
>> and I will send a proposal to generic-abi. As I mentioned in my reply
to
>> James Knight, I don't think we should block on getting a section
number
>> assignment, but we can at least incorporate any design feedback from
that
>> proposal.
>>
>
> I've sent the proposal: https://groups.google.com/foru
> m/#!topic/generic-abi/MPr8TVtnVn4
>
Based on feedback from the generic-abi thread I looked at the object file
size and link time impact of a few other representations for the
address-significance table. My results are here: https://groups.google.
com/d/msg/generic-abi/MPr8TVtnVn4/30Z0_KHMAQAJ

One of the proposals (a compressed array of symbol attributes) is slightly
more expensive than what I originally proposed (+0.03% object file size,
+1% link time) but it would allow for future expansion and therefore seems
more appropriate for the gABI. That's not really a concern for us though
since the initial implementation will use a platform-specific section
number, and we can always switch to the gABI representation at the same
time as we adopt the gABI section numbers. There's also the possibility
that we will end up doing something completely different in the gABI.

Does anyone have a strong opinion that we should do something more aligned
with the gABI? If not, I'll start upstreaming my original proposal.

Peter
>
> Peter
>
>>
>> Peter
>>
>>
>>
>>> Happy to help out with reviews.
>>>
>>> Peter
>>>
>>>
>>> On 22 May 2018 at 23:06, Peter Collingbourne via llvm-dev
>>> <llvm-dev at lists.llvm.org> wrote:
>>> > Hi all,
>>> >
>>> > Context: ld.gold has an --icf=safe flag which is intended to
apply ICF
>>> only
>>> > to sections which can be safely merged according to the
guarantees
>>> provided
>>> > by the language. It works using a set of heuristics (symbol
name
>>> matching
>>> > and relocation scanning). That's not only imprecise but it
only works
>>> with
>>> > certain languages and is slow due to the need to demangle
symbols and
>>> scan
>>> > relocations. It's also redundant with the
(local_)unnamed_addr analysis
>>> > already performed by LLVM.
>>> >
>>> > I implemented an alternative to this approach in clang and
lld. It
>>> works by
>>> > adding a section to each object file containing the indexes of
the
>>> symbols
>>> > which are address-significant (i.e. not (local_)unnamed_addr
in IR).
>>> >
>>> > I used this implementation to link clang with release+asserts
with
>>> each of
>>> > --icf={none,safe,all}. The binary sizes were:
>>> >
>>> > none: 109407184
>>> > safe: 108534736 (-0.8%)
>>> > all: 107281360 (-2%)
>>> >
>>> > I measured the object file overhead of these sections in my
clang
>>> build at
>>> > 0.08%. That's almost nothing, and I think it's small
enough that we
>>> can turn
>>> > it on by default.
>>> >
>>> > I've uploaded a patch series for this feature here:
>>> > https://github.com/pcc/llvm-project/tree/llvm-addrsig
>>> > I intend to start sending it for review soon.
>>> >
>>> > Thanks,
>>> > --
>>> > --
>>> > Peter
>>> >
>>> > _______________________________________________
>>> > LLVM Developers mailing list
>>> > llvm-dev at lists.llvm.org
>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> >
>>>
>>
>>
>>
>> --
>> --
>> Peter
>>
>
>
>
> --
> --
> Peter
>


-- 
-- 
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180531/35c5ba60/attachment.html>

On Thu, May 31, 2018 at 5:06 PM Peter Collingbourne via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Wed, May 23, 2018 at 3:07 PM, Peter Collingbourne <peter at
pcc.me.uk>
> wrote:
>
>>
>>
>> On Wed, May 23, 2018 at 12:16 PM, Peter Collingbourne <peter at
pcc.me.uk>
>> wrote:
>>
>>> On Wed, May 23, 2018 at 3:25 AM, Peter Smith <peter.smith at
linaro.org>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I think that the approach of using a section to record address
>>>> significance is a good one. I'm guessing it will have its
own section
>>>> type and format? If it does would it make sense to try and
submit this
>>>> to the GABI https://groups.google.com/forum/#!forum/generic-abi
as it
>>>> could be potentially useful for other linkers, for example
gold?
>>>>
>>>
>>> Yes, there is a new section type (SHT_LLVM_ADDRSIG) and format (a
>>> sequence of ULEB128-encoded symbol table indexes that are
>>> address-significant).
>>>
>>> I think it makes sense for this to eventually be part of the
generic
>>> ABI, and I will send a proposal to generic-abi. As I mentioned in
my reply
>>> to James Knight, I don't think we should block on getting a
section number
>>> assignment, but we can at least incorporate any design feedback
from that
>>> proposal.
>>>
>>
>> I've sent the proposal:
>> https://groups.google.com/forum/#!topic/generic-abi/MPr8TVtnVn4
>>
>
> Based on feedback from the generic-abi thread I looked at the object file
> size and link time impact of a few other representations for the
> address-significance table. My results are here:
> https://groups.google.com/d/msg/generic-abi/MPr8TVtnVn4/30Z0_KHMAQAJ
>
> One of the proposals (a compressed array of symbol attributes) is slightly
> more expensive than what I originally proposed (+0.03% object file size,
> +1% link time) but it would allow for future expansion and therefore seems
> more appropriate for the gABI. That's not really a concern for us
though
> since the initial implementation will use a platform-specific section
> number, and we can always switch to the gABI representation at the same
> time as we adopt the gABI section numbers. There's also the possibility
> that we will end up doing something completely different in the gABI.
>
> Does anyone have a strong opinion that we should do something more aligned
> with the gABI? If not, I'll start upstreaming my original proposal.
>
>
I suppose it depends on whether or not you're going to be the one to
reconcile your current implementation and that one in the future or if
it'll wait for someone else?

-eric

> Peter
>
>>
>> Peter
>>
>>>
>>> Peter
>>>
>>>
>>>
>>>> Happy to help out with reviews.
>>>>
>>>> Peter
>>>>
>>>>
>>>> On 22 May 2018 at 23:06, Peter Collingbourne via llvm-dev
>>>> <llvm-dev at lists.llvm.org> wrote:
>>>> > Hi all,
>>>> >
>>>> > Context: ld.gold has an --icf=safe flag which is intended
to apply
>>>> ICF only
>>>> > to sections which can be safely merged according to the
guarantees
>>>> provided
>>>> > by the language. It works using a set of heuristics
(symbol name
>>>> matching
>>>> > and relocation scanning). That's not only imprecise
but it only works
>>>> with
>>>> > certain languages and is slow due to the need to demangle
symbols and
>>>> scan
>>>> > relocations. It's also redundant with the
(local_)unnamed_addr
>>>> analysis
>>>> > already performed by LLVM.
>>>> >
>>>> > I implemented an alternative to this approach in clang and
lld. It
>>>> works by
>>>> > adding a section to each object file containing the
indexes of the
>>>> symbols
>>>> > which are address-significant (i.e. not
(local_)unnamed_addr in IR).
>>>> >
>>>> > I used this implementation to link clang with
release+asserts with
>>>> each of
>>>> > --icf={none,safe,all}. The binary sizes were:
>>>> >
>>>> > none: 109407184
>>>> > safe: 108534736 (-0.8%)
>>>> > all: 107281360 (-2%)
>>>> >
>>>> > I measured the object file overhead of these sections in
my clang
>>>> build at
>>>> > 0.08%. That's almost nothing, and I think it's
small enough that we
>>>> can turn
>>>> > it on by default.
>>>> >
>>>> > I've uploaded a patch series for this feature here:
>>>> > https://github.com/pcc/llvm-project/tree/llvm-addrsig
>>>> > I intend to start sending it for review soon.
>>>> >
>>>> > Thanks,
>>>> > --
>>>> > --
>>>> > Peter
>>>> >
>>>> > _______________________________________________
>>>> > LLVM Developers mailing list
>>>> > llvm-dev at lists.llvm.org
>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Peter
>>>
>>
>>
>>
>> --
>> --
>> Peter
>>
>
>
>
> --
> --
> Peter
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180531/6155f9a5/attachment.html>

On Thu, May 31, 2018 at 5:13 PM, Eric Christopher <echristo at gmail.com>
wrote:
>
>
> On Thu, May 31, 2018 at 5:06 PM Peter Collingbourne via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On Wed, May 23, 2018 at 3:07 PM, Peter Collingbourne <peter at
pcc.me.uk>
>> wrote:
>>
>>>
>>>
>>> On Wed, May 23, 2018 at 12:16 PM, Peter Collingbourne <peter at
pcc.me.uk>
>>> wrote:
>>>
>>>> On Wed, May 23, 2018 at 3:25 AM, Peter Smith <peter.smith at
linaro.org>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I think that the approach of using a section to record
address
>>>>> significance is a good one. I'm guessing it will have
its own section
>>>>> type and format? If it does would it make sense to try and
submit this
>>>>> to the GABI
https://groups.google.com/forum/#!forum/generic-abi as it
>>>>> could be potentially useful for other linkers, for example
gold?
>>>>>
>>>>
>>>> Yes, there is a new section type (SHT_LLVM_ADDRSIG) and format
(a
>>>> sequence of ULEB128-encoded symbol table indexes that are
>>>> address-significant).
>>>>
>>>> I think it makes sense for this to eventually be part of the
generic
>>>> ABI, and I will send a proposal to generic-abi. As I mentioned
in my reply
>>>> to James Knight, I don't think we should block on getting a
section number
>>>> assignment, but we can at least incorporate any design feedback
from that
>>>> proposal.
>>>>
>>>
>>> I've sent the proposal: https://groups.google.com/
>>> forum/#!topic/generic-abi/MPr8TVtnVn4
>>>
>>
>> Based on feedback from the generic-abi thread I looked at the object
file
>> size and link time impact of a few other representations for the
>> address-significance table. My results are here: https://groups.google.
>> com/d/msg/generic-abi/MPr8TVtnVn4/30Z0_KHMAQAJ
>>
>> One of the proposals (a compressed array of symbol attributes) is
>> slightly more expensive than what I originally proposed (+0.03% object
file
>> size, +1% link time) but it would allow for future expansion and
therefore
>> seems more appropriate for the gABI. That's not really a concern
for us
>> though since the initial implementation will use a platform-specific
>> section number, and we can always switch to the gABI representation at
the
>> same time as we adopt the gABI section numbers. There's also the
>> possibility that we will end up doing something completely different in
the
>> gABI.
>>
>> Does anyone have a strong opinion that we should do something more
>> aligned with the gABI? If not, I'll start upstreaming my original
proposal.
>>
>>
> I suppose it depends on whether or not you're going to be the one to
> reconcile your current implementation and that one in the future or if
> it'll wait for someone else?
>
I'm happy to volunteer to work on that if the proposal ever gets accepted
into the gABI and I'm still working on LLVM at that time.

Peter

> -eric
>
>
>> Peter
>>
>>>
>>> Peter
>>>
>>>>
>>>> Peter
>>>>
>>>>
>>>>
>>>>> Happy to help out with reviews.
>>>>>
>>>>> Peter
>>>>>
>>>>>
>>>>> On 22 May 2018 at 23:06, Peter Collingbourne via llvm-dev
>>>>> <llvm-dev at lists.llvm.org> wrote:
>>>>> > Hi all,
>>>>> >
>>>>> > Context: ld.gold has an --icf=safe flag which is
intended to apply
>>>>> ICF only
>>>>> > to sections which can be safely merged according to
the guarantees
>>>>> provided
>>>>> > by the language. It works using a set of heuristics
(symbol name
>>>>> matching
>>>>> > and relocation scanning). That's not only
imprecise but it only
>>>>> works with
>>>>> > certain languages and is slow due to the need to
demangle symbols
>>>>> and scan
>>>>> > relocations. It's also redundant with the
(local_)unnamed_addr
>>>>> analysis
>>>>> > already performed by LLVM.
>>>>> >
>>>>> > I implemented an alternative to this approach in clang
and lld. It
>>>>> works by
>>>>> > adding a section to each object file containing the
indexes of the
>>>>> symbols
>>>>> > which are address-significant (i.e. not
(local_)unnamed_addr in IR).
>>>>> >
>>>>> > I used this implementation to link clang with
release+asserts with
>>>>> each of
>>>>> > --icf={none,safe,all}. The binary sizes were:
>>>>> >
>>>>> > none: 109407184
>>>>> > safe: 108534736 (-0.8%)
>>>>> > all: 107281360 (-2%)
>>>>> >
>>>>> > I measured the object file overhead of these sections
in my clang
>>>>> build at
>>>>> > 0.08%. That's almost nothing, and I think it's
small enough that we
>>>>> can turn
>>>>> > it on by default.
>>>>> >
>>>>> > I've uploaded a patch series for this feature
here:
>>>>> > https://github.com/pcc/llvm-project/tree/llvm-addrsig
>>>>> > I intend to start sending it for review soon.
>>>>> >
>>>>> > Thanks,
>>>>> > --
>>>>> > --
>>>>> > Peter
>>>>> >
>>>>> > _______________________________________________
>>>>> > LLVM Developers mailing list
>>>>> > llvm-dev at lists.llvm.org
>>>>> >
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Peter
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Peter
>>>
>>
>>
>>
>> --
>> --
>> Peter
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>

-- 
-- 
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180531/3282879f/attachment-0001.html>