llvm dev - May 2018 - Proposal for address-significance tables for --icf=safe

Hi Peter,

Thanks for doing this!

On Wed, May 23, 2018 at 3:07 PM Peter Collingbourne via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>
> On Wed, May 23, 2018 at 12:16 PM, Peter Collingbourne <peter at
pcc.me.uk>
> wrote:
>
>> On Wed, May 23, 2018 at 3:25 AM, Peter Smith <peter.smith at
linaro.org>
>> wrote:
>>
>>> Hello,
>>>
>>> I think that the approach of using a section to record address
>>> significance is a good one. I'm guessing it will have its own
section
>>> type and format? If it does would it make sense to try and submit
this
>>> to the GABI https://groups.google.com/forum/#!forum/generic-abi as
it
>>> could be potentially useful for other linkers, for example gold?
>>>
>>
>> Yes, there is a new section type (SHT_LLVM_ADDRSIG) and format (a
>> sequence of ULEB128-encoded symbol table indexes that are
>> address-significant).
>>
>> I think it makes sense for this to eventually be part of the generic
ABI,
>> and I will send a proposal to generic-abi. As I mentioned in my reply
to
>> James Knight, I don't think we should block on getting a section
number
>> assignment, but we can at least incorporate any design feedback from
that
>> proposal.
>>
>
> I've sent the proposal:
> https://groups.google.com/forum/#!topic/generic-abi/MPr8TVtnVn4
>
>
The proposal looks decent. I'll probably comment more once I (like the
others) get a chance to read more deeply. I've also taken a look at the
basic patches - one request is that since this is newly standardizing bits
that you make sure to comment or even put a detailed standards-esque
description of the tables into the code?

-eric



> Peter
>
>>
>> Peter
>>
>>
>>
>>> Happy to help out with reviews.
>>>
>>> Peter
>>>
>>>
>>> On 22 May 2018 at 23:06, Peter Collingbourne via llvm-dev
>>> <llvm-dev at lists.llvm.org> wrote:
>>> > Hi all,
>>> >
>>> > Context: ld.gold has an --icf=safe flag which is intended to
apply ICF
>>> only
>>> > to sections which can be safely merged according to the
guarantees
>>> provided
>>> > by the language. It works using a set of heuristics (symbol
name
>>> matching
>>> > and relocation scanning). That's not only imprecise but it
only works
>>> with
>>> > certain languages and is slow due to the need to demangle
symbols and
>>> scan
>>> > relocations. It's also redundant with the
(local_)unnamed_addr analysis
>>> > already performed by LLVM.
>>> >
>>> > I implemented an alternative to this approach in clang and
lld. It
>>> works by
>>> > adding a section to each object file containing the indexes of
the
>>> symbols
>>> > which are address-significant (i.e. not (local_)unnamed_addr
in IR).
>>> >
>>> > I used this implementation to link clang with release+asserts
with
>>> each of
>>> > --icf={none,safe,all}. The binary sizes were:
>>> >
>>> > none: 109407184
>>> > safe: 108534736 (-0.8%)
>>> > all: 107281360 (-2%)
>>> >
>>> > I measured the object file overhead of these sections in my
clang
>>> build at
>>> > 0.08%. That's almost nothing, and I think it's small
enough that we
>>> can turn
>>> > it on by default.
>>> >
>>> > I've uploaded a patch series for this feature here:
>>> > https://github.com/pcc/llvm-project/tree/llvm-addrsig
>>> > I intend to start sending it for review soon.
>>> >
>>> > Thanks,
>>> > --
>>> > --
>>> > Peter
>>> >
>>> > _______________________________________________
>>> > LLVM Developers mailing list
>>> > llvm-dev at lists.llvm.org
>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> >
>>>
>>
>>
>>
>> --
>> --
>> Peter
>>
>
>
>
> --
> --
> Peter
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180524/e0cc9884/attachment.html>

On Thu, May 24, 2018 at 10:24 AM, Eric Christopher <echristo at gmail.com>
wrote:
> Hi Peter,
>
> Thanks for doing this!
>
> On Wed, May 23, 2018 at 3:07 PM Peter Collingbourne via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>>
>> On Wed, May 23, 2018 at 12:16 PM, Peter Collingbourne <peter at
pcc.me.uk>
>> wrote:
>>
>>> On Wed, May 23, 2018 at 3:25 AM, Peter Smith <peter.smith at
linaro.org>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I think that the approach of using a section to record address
>>>> significance is a good one. I'm guessing it will have its
own section
>>>> type and format? If it does would it make sense to try and
submit this
>>>> to the GABI https://groups.google.com/forum/#!forum/generic-abi
as it
>>>> could be potentially useful for other linkers, for example
gold?
>>>>
>>>
>>> Yes, there is a new section type (SHT_LLVM_ADDRSIG) and format (a
>>> sequence of ULEB128-encoded symbol table indexes that are
>>> address-significant).
>>>
>>> I think it makes sense for this to eventually be part of the
generic
>>> ABI, and I will send a proposal to generic-abi. As I mentioned in
my reply
>>> to James Knight, I don't think we should block on getting a
section number
>>> assignment, but we can at least incorporate any design feedback
from that
>>> proposal.
>>>
>>
>> I've sent the proposal: https://groups.google.com/
>> forum/#!topic/generic-abi/MPr8TVtnVn4
>>
>>
> The proposal looks decent. I'll probably comment more once I (like the
> others) get a chance to read more deeply. I've also taken a look at the
> basic patches - one request is that since this is newly standardizing bits
> that you make sure to comment or even put a detailed standards-esque
> description of the tables into the code?
>
Sure, I'll document the assembly and object file extensions here:
http://llvm.org/docs/Extensions.html
and reference it from the code.

Peter


> -eric
>
>
>
>
>> Peter
>>
>>>
>>> Peter
>>>
>>>
>>>
>>>> Happy to help out with reviews.
>>>>
>>>> Peter
>>>>
>>>>
>>>> On 22 May 2018 at 23:06, Peter Collingbourne via llvm-dev
>>>> <llvm-dev at lists.llvm.org> wrote:
>>>> > Hi all,
>>>> >
>>>> > Context: ld.gold has an --icf=safe flag which is intended
to apply
>>>> ICF only
>>>> > to sections which can be safely merged according to the
guarantees
>>>> provided
>>>> > by the language. It works using a set of heuristics
(symbol name
>>>> matching
>>>> > and relocation scanning). That's not only imprecise
but it only works
>>>> with
>>>> > certain languages and is slow due to the need to demangle
symbols and
>>>> scan
>>>> > relocations. It's also redundant with the
(local_)unnamed_addr
>>>> analysis
>>>> > already performed by LLVM.
>>>> >
>>>> > I implemented an alternative to this approach in clang and
lld. It
>>>> works by
>>>> > adding a section to each object file containing the
indexes of the
>>>> symbols
>>>> > which are address-significant (i.e. not
(local_)unnamed_addr in IR).
>>>> >
>>>> > I used this implementation to link clang with
release+asserts with
>>>> each of
>>>> > --icf={none,safe,all}. The binary sizes were:
>>>> >
>>>> > none: 109407184
>>>> > safe: 108534736 (-0.8%)
>>>> > all: 107281360 (-2%)
>>>> >
>>>> > I measured the object file overhead of these sections in
my clang
>>>> build at
>>>> > 0.08%. That's almost nothing, and I think it's
small enough that we
>>>> can turn
>>>> > it on by default.
>>>> >
>>>> > I've uploaded a patch series for this feature here:
>>>> > https://github.com/pcc/llvm-project/tree/llvm-addrsig
>>>> > I intend to start sending it for review soon.
>>>> >
>>>> > Thanks,
>>>> > --
>>>> > --
>>>> > Peter
>>>> >
>>>> > _______________________________________________
>>>> > LLVM Developers mailing list
>>>> > llvm-dev at lists.llvm.org
>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Peter
>>>
>>
>>
>>
>> --
>> --
>> Peter
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>

-- 
-- 
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180524/df3e83e1/attachment.html>

Great, thanks!

On Thu, May 24, 2018, 11:02 AM Peter Collingbourne <peter at pcc.me.uk>
wrote:
>
>
> On Thu, May 24, 2018 at 10:24 AM, Eric Christopher <echristo at
gmail.com>
> wrote:
>
>> Hi Peter,
>>
>> Thanks for doing this!
>>
>> On Wed, May 23, 2018 at 3:07 PM Peter Collingbourne via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>>
>>>
>>> On Wed, May 23, 2018 at 12:16 PM, Peter Collingbourne <peter at
pcc.me.uk>
>>> wrote:
>>>
>>>> On Wed, May 23, 2018 at 3:25 AM, Peter Smith <peter.smith at
linaro.org>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I think that the approach of using a section to record
address
>>>>> significance is a good one. I'm guessing it will have
its own section
>>>>> type and format? If it does would it make sense to try and
submit this
>>>>> to the GABI
https://groups.google.com/forum/#!forum/generic-abi as it
>>>>> could be potentially useful for other linkers, for example
gold?
>>>>>
>>>>
>>>> Yes, there is a new section type (SHT_LLVM_ADDRSIG) and format
(a
>>>> sequence of ULEB128-encoded symbol table indexes that are
>>>> address-significant).
>>>>
>>>> I think it makes sense for this to eventually be part of the
generic
>>>> ABI, and I will send a proposal to generic-abi. As I mentioned
in my reply
>>>> to James Knight, I don't think we should block on getting a
section number
>>>> assignment, but we can at least incorporate any design feedback
from that
>>>> proposal.
>>>>
>>>
>>> I've sent the proposal:
>>> https://groups.google.com/forum/#!topic/generic-abi/MPr8TVtnVn4
>>>
>>>
>> The proposal looks decent. I'll probably comment more once I (like
the
>> others) get a chance to read more deeply. I've also taken a look at
the
>> basic patches - one request is that since this is newly standardizing
bits
>> that you make sure to comment or even put a detailed standards-esque
>> description of the tables into the code?
>>
>
> Sure, I'll document the assembly and object file extensions here:
> http://llvm.org/docs/Extensions.html
> and reference it from the code.
>
> Peter
>
>
>
>> -eric
>>
>>
>>
>>
>>> Peter
>>>
>>>>
>>>> Peter
>>>>
>>>>
>>>>
>>>>> Happy to help out with reviews.
>>>>>
>>>>> Peter
>>>>>
>>>>>
>>>>> On 22 May 2018 at 23:06, Peter Collingbourne via llvm-dev
>>>>> <llvm-dev at lists.llvm.org> wrote:
>>>>> > Hi all,
>>>>> >
>>>>> > Context: ld.gold has an --icf=safe flag which is
intended to apply
>>>>> ICF only
>>>>> > to sections which can be safely merged according to
the guarantees
>>>>> provided
>>>>> > by the language. It works using a set of heuristics
(symbol name
>>>>> matching
>>>>> > and relocation scanning). That's not only
imprecise but it only
>>>>> works with
>>>>> > certain languages and is slow due to the need to
demangle symbols
>>>>> and scan
>>>>> > relocations. It's also redundant with the
(local_)unnamed_addr
>>>>> analysis
>>>>> > already performed by LLVM.
>>>>> >
>>>>> > I implemented an alternative to this approach in clang
and lld. It
>>>>> works by
>>>>> > adding a section to each object file containing the
indexes of the
>>>>> symbols
>>>>> > which are address-significant (i.e. not
(local_)unnamed_addr in IR).
>>>>> >
>>>>> > I used this implementation to link clang with
release+asserts with
>>>>> each of
>>>>> > --icf={none,safe,all}. The binary sizes were:
>>>>> >
>>>>> > none: 109407184
>>>>> > safe: 108534736 (-0.8%)
>>>>> > all: 107281360 (-2%)
>>>>> >
>>>>> > I measured the object file overhead of these sections
in my clang
>>>>> build at
>>>>> > 0.08%. That's almost nothing, and I think it's
small enough that we
>>>>> can turn
>>>>> > it on by default.
>>>>> >
>>>>> > I've uploaded a patch series for this feature
here:
>>>>> > https://github.com/pcc/llvm-project/tree/llvm-addrsig
>>>>> > I intend to start sending it for review soon.
>>>>> >
>>>>> > Thanks,
>>>>> > --
>>>>> > --
>>>>> > Peter
>>>>> >
>>>>> > _______________________________________________
>>>>> > LLVM Developers mailing list
>>>>> > llvm-dev at lists.llvm.org
>>>>> >
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Peter
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Peter
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>
>
> --
> --
> Peter
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180524/0b3a136b/attachment.html>