llvm dev - Sep 2016 - [lld][ELF] Addends adjustment for relocatable object

Initial issue is related to R_MIPS_GPREL16/32 relocations. When we
calculate such relocations we need to know _gp symbol's value. This
value might come from different sources. Usually we setup it using
constant offset from the .got section. Also it might be defined by a
linker script. And in rare but possible case it comes from .reginfo /
.MIPS.options sections.

Got example, GNU bfd linker has the following code to adjust
_gp-relative relocation addends:

 rel->r_addend += _bfd_get_gp_value (input_bfd);
 rel->r_addend -= _bfd_get_gp_value (output_bfd);

But I think we can escape to implement the same adjustments in LLD if
we do not support non-zero gp-value in the .reginfo / .MIPS.options
sections in input object files. We can get such files if somebody
produce a relocatable object file by bfd/gold linkers and attempt to
link this file using lld. It seems to be an uncommon case. To notify a
user lld might show an error in that case.

BTW it looks like there is another problem with relocation addend
adjustment in case of relocatable object generation. I did not
investigate it thoroughly though. Suppose we have a SHF_MERGE section
with two "equal" entries and a relocation which points to the second
entry using section+addend. Now we merge such sections even if we
generate a relocatable object. In that case we write reduced SHF_MERGE
section with the only entry. But the relocation still points to the
removed second entry and in fact points out of the section border.
Please correct me if I miss something.

I think we can fix that by the following way:
1. Do not try to merge SHF_MERGE sections if Config->Relocatable is true.
2. Do not group such sections together or group sections with the same
SHF_STRINGS / sh_entsize parameters.
3. Keep SHF_STRINGS flag and sh_entsize value when writing the
sections. In that case a linker will be able to merge/reduce such
sections later when it links a relocatable object to get DSO or
executable file.

On Wed, Sep 28, 2016 at 11:41 PM, Rui Ueyama <ruiu at google.com>
wrote:
> Why this is MIPS-specific issue? Do you mean MIPS is the only architecture
> that uses both REL and RELA?
>
> On Tue, Sep 27, 2016 at 1:32 PM, Simon Atanasyan <simon at
atanasyan.com>
> wrote:
>>
>> You are right. LLD does not have this problem. Initially I bumped into
>> the MIPS specific bug related to GP relative relocations calculation.
>> I tried to make reproduction script as general as possible and
>> mistakenly make it too general. So now with your help I realized that
>> the problem solely affects MIPS.
-- 
Simon Atanasyan

On Wed, Sep 28, 2016 at 10:49 PM, Simon Atanasyan <simon at atanasyan.com>
wrote:
> Initial issue is related to R_MIPS_GPREL16/32 relocations. When we
> calculate such relocations we need to know _gp symbol's value. This
> value might come from different sources. Usually we setup it using
> constant offset from the .got section. Also it might be defined by a
> linker script. And in rare but possible case it comes from .reginfo /
> .MIPS.options sections.
>
> Got example, GNU bfd linker has the following code to adjust
> _gp-relative relocation addends:
>
>  rel->r_addend += _bfd_get_gp_value (input_bfd);
>  rel->r_addend -= _bfd_get_gp_value (output_bfd);
>
> But I think we can escape to implement the same adjustments in LLD if
> we do not support non-zero gp-value in the .reginfo / .MIPS.options
> sections in input object files. We can get such files if somebody
> produce a relocatable object file by bfd/gold linkers and attempt to
> link this file using lld. It seems to be an uncommon case. To notify a
> user lld might show an error in that case.
>
> BTW it looks like there is another problem with relocation addend
> adjustment in case of relocatable object generation. I did not
> investigate it thoroughly though. Suppose we have a SHF_MERGE section
> with two "equal" entries and a relocation which points to the
second
> entry using section+addend. Now we merge such sections even if we
> generate a relocatable object. In that case we write reduced SHF_MERGE
> section with the only entry. But the relocation still points to the
> removed second entry and in fact points out of the section border.
> Please correct me if I miss something.
>
> I think we can fix that by the following way:
> 1. Do not try to merge SHF_MERGE sections if Config->Relocatable is
true.
> 2. Do not group such sections together or group sections with the same
> SHF_STRINGS / sh_entsize parameters.
> 3. Keep SHF_STRINGS flag and sh_entsize value when writing the
> sections. In that case a linker will be able to merge/reduce such
> sections later when it links a relocatable object to get DSO or
> executable file.

I'd vote for 1. There might be a corner case that wouldn't work with
that
approach (maybe kernel modules?) though.

On Wed, Sep 28, 2016 at 11:41 PM, Rui Ueyama <ruiu at google.com>
wrote:
> > Why this is MIPS-specific issue? Do you mean MIPS is the only
> architecture
> > that uses both REL and RELA?
> >
> > On Tue, Sep 27, 2016 at 1:32 PM, Simon Atanasyan <simon at
atanasyan.com>
> > wrote:
> >>
> >> You are right. LLD does not have this problem. Initially I bumped
into
> >> the MIPS specific bug related to GP relative relocations
calculation.
> >> I tried to make reproduction script as general as possible and
> >> mistakenly make it too general. So now with your help I realized
that
> >> the problem solely affects MIPS.
>
> --
> Simon Atanasyan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160928/e6d6c2e2/attachment-0001.html>

My  understanding is that SHF_MERGE permits but does not require a
linker to merge identical entries, if they aren't merged they end up
looking like data sections. So I think option 1 should work without
problems.

Peter

On 28 September 2016 at 22:54, Rui Ueyama <ruiu at google.com>
wrote:
> On Wed, Sep 28, 2016 at 10:49 PM, Simon Atanasyan <simon at
atanasyan.com>
> wrote:
>>
>> Initial issue is related to R_MIPS_GPREL16/32 relocations. When we
>> calculate such relocations we need to know _gp symbol's value. This
>> value might come from different sources. Usually we setup it using
>> constant offset from the .got section. Also it might be defined by a
>> linker script. And in rare but possible case it comes from .reginfo /
>> .MIPS.options sections.
>>
>> Got example, GNU bfd linker has the following code to adjust
>> _gp-relative relocation addends:
>>
>>  rel->r_addend += _bfd_get_gp_value (input_bfd);
>>  rel->r_addend -= _bfd_get_gp_value (output_bfd);
>>
>> But I think we can escape to implement the same adjustments in LLD if
>> we do not support non-zero gp-value in the .reginfo / .MIPS.options
>> sections in input object files. We can get such files if somebody
>> produce a relocatable object file by bfd/gold linkers and attempt to
>> link this file using lld. It seems to be an uncommon case. To notify a
>> user lld might show an error in that case.
>>
>> BTW it looks like there is another problem with relocation addend
>> adjustment in case of relocatable object generation. I did not
>> investigate it thoroughly though. Suppose we have a SHF_MERGE section
>> with two "equal" entries and a relocation which points to the
second
>> entry using section+addend. Now we merge such sections even if we
>> generate a relocatable object. In that case we write reduced SHF_MERGE
>> section with the only entry. But the relocation still points to the
>> removed second entry and in fact points out of the section border.
>> Please correct me if I miss something.
>>
>> I think we can fix that by the following way:
>> 1. Do not try to merge SHF_MERGE sections if Config->Relocatable is
true.
>> 2. Do not group such sections together or group sections with the same
>> SHF_STRINGS / sh_entsize parameters.
>> 3. Keep SHF_STRINGS flag and sh_entsize value when writing the
>> sections. In that case a linker will be able to merge/reduce such
>> sections later when it links a relocatable object to get DSO or
>> executable file.
>
>
> I'd vote for 1. There might be a corner case that wouldn't work
with that
> approach (maybe kernel modules?) though.
>
>> On Wed, Sep 28, 2016 at 11:41 PM, Rui Ueyama <ruiu at google.com>
wrote:
>> > Why this is MIPS-specific issue? Do you mean MIPS is the only
>> > architecture
>> > that uses both REL and RELA?
>> >
>> > On Tue, Sep 27, 2016 at 1:32 PM, Simon Atanasyan <simon at
atanasyan.com>
>> > wrote:
>> >>
>> >> You are right. LLD does not have this problem. Initially I
bumped into
>> >> the MIPS specific bug related to GP relative relocations
calculation.
>> >> I tried to make reproduction script as general as possible and
>> >> mistakenly make it too general. So now with your help I
realized that
>> >> the problem solely affects MIPS.
>>
>> --
>> Simon Atanasyan
>
>