llvm dev - Oct 2016 - LLD: creating linker-generated sections as input sections instead of output sections

This idea popped up in the review thread for https://reviews.llvm.org/D25627
.

Problem:

Currently, LLD creates special sections that are not just concatenations of
input sections but need link-time data generation, such as .got, .plt,
interp, .mips.options, etc., as output sections. We have OutputSectionBase
subclasses (e.g. GotSection, PltSection, etc.) to create data. Even though
this scheme works in most cases, there are a few situations that doesn't
work well as you may have noticed. Here are a issues.

 - You cannot mix special sections with other types of sections.

  For example, using linker scripts, you can instruct the linker put
mergeable sections and non-mergeable sections into the same output section.
Such script makes sense. However, LLD cannot handle such script because
string merging is the special mergeable output section's feature. The
output section doesn't know how to handle other types of sections, so you
cannot feed non-mergeable sections to a mergeable output section.

 - It cannot handle linker scripts like this as pointed by Eugene.

  .got { *(.got.plt) *(.got) }

   In our current architecture, .got section is an output section, so it
cannot be added to other output section. There's no clean way to handle
this linker script.

Proposal:

Here's my idea: how about creating all special sections as input sections
instead of output sections?

GotSection, PltSection, etc. will be subclasses of InputSection that don't
have corresponding input files. What they will do remain the same. They
will be added to OutputSections just like other regular sections are added.
I think we could simplify OutputSection a lot -- OutputSection will
probably become a dumb container that just concatenates all input sections.

This approach would solve the problems described above. Now that we create
.got as an special input section with ".got" as a name, so they can
naturally be added to any output section. String merging occurs inside a
special mergeable input section, so they can be added to any section, too.

So, I think by moving the implementations from OutputSection to
InputSection, we can solve many problems. I do not think of any obvious
problem with the approach.

What do you think?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161018/da41a772/attachment.html>

>This idea popped up in the review thread for
https://reviews.llvm.org/D25627.
>
>Problem:
>
>Currently, LLD creates special sections that are not just concatenations of
input sections but need link-time data generation, such as .got, .plt, interp,
.mips.options, >etc., as output sections. We have OutputSectionBase
subclasses (e.g. GotSection, PltSection, etc.) to create data. Even though this
scheme works in most cases, >there are a few situations that doesn't work
well as you may have noticed. Here are a issues.
>
 >- You cannot mix special sections with other types of
sections.
>
>  For example, using linker scripts, you can instruct the linker put
mergeable sections and non-mergeable sections into the same output section. Such
script makes >sense. However, LLD cannot handle such script because string
merging is the special mergeable output section's feature. The output
section doesn't know how to >handle other types of sections, so you
cannot feed non-mergeable sections to a mergeable output section.
>
> - It cannot handle linker scripts like this as pointed by Eugene.
>
>  .got { *(.got.plt) *(.got) }
>
>   In our current architecture, .got section is an output section, so it
cannot be added to other output section. There's no clean way to handle this
linker script.
>
>Proposal:
>
>Here's my idea: how about creating all special sections as input
sections instead of output sections?
>
>GotSection, PltSection, etc. will be subclasses of InputSection that
don't have corresponding input files. What they will do remain the same.
They will be added to >OutputSections just like other regular sections are
added. I think we could simplify OutputSection a lot -- OutputSection will
probably become a dumb container >that just concatenates all input sections.
>
>This approach would solve the problems described above. Now that we create
.got as an special input section with ".got" as a name, so they can
naturally be added >to any output section. String merging occurs inside a
special mergeable input section, so they can be added to any section, too.
>
>So, I think by moving the implementations from OutputSection to
InputSection, we can solve many problems. I do not think of any obvious problem
with the >approach.
>
>What do you think?
For me that sounds as interesting idea. My consern and guess that amount of code
changes can be really large for that.
But generally I so not see real problems with this approach too.

George.

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161019/2c595b98/attachment.html>

I would suggest converting only part of linker generated sections to
input sections to reduce amount of code changes.
For example it's unlikely that SymbolTableSection or
StringTableSection would ever require such treatment, so why
converting them to input sections?


2016-10-19 11:03 GMT+03:00 George Rimar <grimar at
accesssoftek.com>:
>>This idea popped up in the review thread for
>> https://reviews.llvm.org/D25627.
>
>>
>>Problem:
>>
>>Currently, LLD creates special sections that are not just concatenations
of
>> input sections but need link-time data generation, such as .got, .plt,
>> interp, .mips.options, >etc., as output sections. We have
OutputSectionBase
>> subclasses (e.g. GotSection, PltSection, etc.) to create data. Even
though
>> this scheme works in most cases, >there are a few situations that
doesn't
>> work well as you may have noticed. Here are a issues.
>>
>  >- You cannot mix special sections with other types of sections.
>>
>>  For example, using linker scripts, you can instruct the linker put
>> mergeable sections and non-mergeable sections into the same output
section.
>> Such script makes >sense. However, LLD cannot handle such script
because
>> string merging is the special mergeable output section's feature.
The output
>> section doesn't know how to >handle other types of sections, so
you cannot
>> feed non-mergeable sections to a mergeable output section.
>>
>> - It cannot handle linker scripts like this as pointed by Eugene.
>>
>>  .got { *(.got.plt) *(.got) }
>>
>>   In our current architecture, .got section is an output section, so it
>> cannot be added to other output section. There's no clean way to
handle this
>> linker script.
>>
>>Proposal:
>>
>>Here's my idea: how about creating all special sections as input
sections
>> instead of output sections?
>>
>>GotSection, PltSection, etc. will be subclasses of InputSection that
don't
>> have corresponding input files. What they will do remain the same. They
will
>> be added to >OutputSections just like other regular sections are
added. I
>> think we could simplify OutputSection a lot -- OutputSection will
probably
>> become a dumb container >that just concatenates all input sections.
>>
>>This approach would solve the problems described above. Now that we
create
>> .got as an special input section with ".got" as a name, so
they can
>> naturally be added >to any output section. String merging occurs
inside a
>> special mergeable input section, so they can be added to any section,
too.
>>
>>So, I think by moving the implementations from OutputSection to
>> InputSection, we can solve many problems. I do not think of any obvious
>> problem with the >approach.
>>
>>What do you think?
>
> For me that sounds as interesting idea. My consern and guess that amount of
> code changes can be really large for that.
> But generally I so not see real problems with this approach too.
>
> George.
>

It looks like a reasonable idea for some sections.

How exactly would you handle merge sections? In gold there is a 3
level mapping. Input sections are mapped to a "Chunk" (not sure what
exact name is used in gold) that have the same SHF_MERGE, alignment,
sh_entsize, etc. Multiple of these are then mapped to a single .rodata
output section section.

In lld we use a two level mapping and end up with multiple .rodata
sections. It was done this way for simplicity since the output
sections should be almost an implementation detail on elf (outputting
the section table is even optional).

Unfortunately linker scripts make this implementation detail very
visible, so I think we have to consider this in a case by case way.

BTW, a synthetic input section should also allow us to simplify
handling of input binary files.

Cheers,
Rafael




On 19 October 2016 at 04:03, George Rimar <grimar at accesssoftek.com>
wrote:
>>This idea popped up in the review thread for
>> https://reviews.llvm.org/D25627.
>
>>
>>Problem:
>>
>>Currently, LLD creates special sections that are not just concatenations
of
>> input sections but need link-time data generation, such as .got, .plt,
>> interp, .mips.options, >etc., as output sections. We have
OutputSectionBase
>> subclasses (e.g. GotSection, PltSection, etc.) to create data. Even
though
>> this scheme works in most cases, >there are a few situations that
doesn't
>> work well as you may have noticed. Here are a issues.
>>
>  >- You cannot mix special sections with other types of sections.
>>
>>  For example, using linker scripts, you can instruct the linker put
>> mergeable sections and non-mergeable sections into the same output
section.
>> Such script makes >sense. However, LLD cannot handle such script
because
>> string merging is the special mergeable output section's feature.
The output
>> section doesn't know how to >handle other types of sections, so
you cannot
>> feed non-mergeable sections to a mergeable output section.
>>
>> - It cannot handle linker scripts like this as pointed by Eugene.
>>
>>  .got { *(.got.plt) *(.got) }
>>
>>   In our current architecture, .got section is an output section, so it
>> cannot be added to other output section. There's no clean way to
handle this
>> linker script.
>>
>>Proposal:
>>
>>Here's my idea: how about creating all special sections as input
sections
>> instead of output sections?
>>
>>GotSection, PltSection, etc. will be subclasses of InputSection that
don't
>> have corresponding input files. What they will do remain the same. They
will
>> be added to >OutputSections just like other regular sections are
added. I
>> think we could simplify OutputSection a lot -- OutputSection will
probably
>> become a dumb container >that just concatenates all input sections.
>>
>>This approach would solve the problems described above. Now that we
create
>> .got as an special input section with ".got" as a name, so
they can
>> naturally be added >to any output section. String merging occurs
inside a
>> special mergeable input section, so they can be added to any section,
too.
>>
>>So, I think by moving the implementations from OutputSection to
>> InputSection, we can solve many problems. I do not think of any obvious
>> problem with the >approach.
>>
>>What do you think?
>
> For me that sounds as interesting idea. My consern and guess that amount of
> code changes can be really large for that.
> But generally I so not see real problems with this approach too.
>
> George.
>