llvm dev - Nov 2020 - [LLD] Support DWARF64, debug_info "sorting"

On 2020-11-14, Alexander Yermolovich wrote:
>Thanks for doing a diff and asking in other groups.
>
>So if I understand your concern with using first reloc as it relates to
.debug_str.
>
>In DWARF4 for .debug_str is referenced from .debug_info, .debug_type using
DW_FORM_strp. For DWARF32 it's 32bit unsigned, for DWARF64 it's 64bit
unsigned. So in relocation section for some .debug_info section we will have a
relocation entry to patch up DW_FORM_strp. Either R_X86_64_32, or R_X86_64_64,
depending on DWARF format.
>
>A situation we might have is that an input .debug_info section is DWARF32 so
it gets ordered apropriatly within output .debug_info section, but the input
.debug_str section can be put in the output .debug_str section that is above
4GB. In which case we still hit overflow. So we also need to oder the .debug_str
section, except in DWARF4 there is no real clear link, besides looking through
.debug_info relocs.
Yes. For most other .debug_*, we can check whether the section has an 64-bit
absolute relocation to decide whether it is DWARF64. .debug_str is different in
that we need to check relocations referencing it. This makes its behavior
dependent on other sections, which is why I feel lost with the relocation
approach: when we write .debug_str 0 : { *(.debug_str) }, we really want the
output section .debug_str can be made up with just information from the input
section descriptions, not random information from other .debug_*

LLD -O1 (default) and GNU ld -O0 enable constant string merge within
SHF_MERGE&&SHF_STRINGS sections.  We need to pay attention as if a
DWARF32
string gets folded into a DWARF64 string, the section referencing the DWARF32
string can still trigger a relocation overflow.

If we order DWARF32 components before DWARF64 components, with the
llvm::StringTableBuilder usage in LLD, we can make sure DWARF64 strings can get
folded into DWARF32 strings, not the other way around.
>Is that a correct summary?
>
>Also I don't quite understand what the issue is with linker script.
>
>My understanding is that:
>
>.debug_str 0 : { *(.debug_str) }
>
>Just stays that all .debug_str input sections should go in to .debug_str
output section. It doesn't really specify the ordering within the output
.debug_str section.
There is an assumption that linkers concatenate input sections in input order,
which is required by the ELF specification.  There are options which can change
the semantics of `*`: --sort-section, (gold specific) --section-ordering-file,
(LLD specific) --symbol-ordering-file.  Like the other options, we should
justify the `*` ordering by assigning appropriate semantics.
>Thank You
>Alex

> -----Original Message-----
> From: Fāng-ruì Sòng <maskray at google.com>
> Sent: Tuesday, November 17, 2020 1:51 AM
> To: Alexander Yermolovich <ayermolo at fb.com>
> Cc: David Blaikie <dblaikie at gmail.com>; Wenlei He <wenlei at
fb.com>; llvm-
> dev at lists.llvm.org; Robinson, Paul <paul.robinson at sony.com>;
James
> Henderson <jh7370.2008 at my.bristol.ac.uk>; Eric Christopher
> <echristo at gmail.com>; Igor Kudrin <ikudrin at
accesssoftek.com>
> Subject: Re: [llvm-dev] [LLD] Support DWARF64, debug_info
"sorting"
> 
> On 2020-11-14, Alexander Yermolovich wrote:
> >Thanks for doing a diff and asking in other groups.
> >
> >So if I understand your concern with using first reloc as it relates to
> .debug_str.
> >
> >In DWARF4 for .debug_str is referenced from .debug_info, .debug_type
> using DW_FORM_strp. For DWARF32 it's 32bit unsigned, for DWARF64
it's
> 64bit unsigned. So in relocation section for some .debug_info section we
> will have a relocation entry to patch up DW_FORM_strp. Either R_X86_64_32,
> or R_X86_64_64, depending on DWARF format.
> >
> >A situation we might have is that an input .debug_info section is
DWARF32
> so it gets ordered apropriatly within output .debug_info section, but the
> input .debug_str section can be put in the output .debug_str section that
> is above 4GB. In which case we still hit overflow. So we also need to oder
> the .debug_str section, except in DWARF4 there is no real clear link,
> besides looking through .debug_info relocs.
> 
> Yes. For most other .debug_*, we can check whether the section has an 64-
> bit
> absolute relocation to decide whether it is DWARF64. .debug_str is
> different in
> that we need to check relocations referencing it. This makes its behavior
> dependent on other sections, which is why I feel lost with the relocation
> approach: when we write .debug_str 0 : { *(.debug_str) }, we really want
> the
> output section .debug_str can be made up with just information from the
> input
> section descriptions, not random information from other .debug_*
>
> LLD -O1 (default) and GNU ld -O0 enable constant string merge within
> SHF_MERGE&&SHF_STRINGS sections.  We need to pay attention as if a
DWARF32
> string gets folded into a DWARF64 string, the section referencing the
> DWARF32
> string can still trigger a relocation overflow.
This is a problem only if the .debug_str section *by itself* exceeds 4GB;
are we anticipating that will happen IRL?  The section is just a string
section, by itself it has no 32/64 format.

If the .debug_str section *by itself* exceeds 4GB, then yes any string
with a 32-bit reference to it must be in the first 4GB.  Strings that
have only 64-bit references to them can be sorted to the end of the
section, if necessary.  I wouldn't think anyone guarantees or cares
about the order of strings within a string section.

But I think this would be the very last thing to care about, with regard
to DWARF-64 concerns.
--paulr
> 
> If we order DWARF32 components before DWARF64 components, with the
> llvm::StringTableBuilder usage in LLD, we can make sure DWARF64 strings
> can get
> folded into DWARF32 strings, not the other way around.
> 
> >Is that a correct summary?
> >
> >Also I don't quite understand what the issue is with linker script.
> >
> >My understanding is that:
> >
> >.debug_str 0 : { *(.debug_str) }
> >
> >Just stays that all .debug_str input sections should go in to
.debug_str
> output section. It doesn't really specify the ordering within the
output
> .debug_str section.
> 
> There is an assumption that linkers concatenate input sections in input
> order,
> which is required by the ELF specification.  There are options which can
> change
> the semantics of `*`: --sort-section, (gold specific) --section-ordering-
> file,
> (LLD specific) --symbol-ordering-file.  Like the other options, we should
> justify the `*` ordering by assigning appropriate semantics.
> 
> >Thank You
> >Alex

On 18.11.2020 0:20, Robinson, Paul wrote:
> This is a problem only if the .debug_str section *by itself* exceeds 4GB;
> are we anticipating that will happen IRL?  The section is just a string
> section, by itself it has no 32/64 format.
> 
> If the .debug_str section *by itself* exceeds 4GB, then yes any string
> with a 32-bit reference to it must be in the first 4GB.  Strings that
> have only 64-bit references to them can be sorted to the end of the
> section, if necessary.  I wouldn't think anyone guarantees or cares
> about the order of strings within a string section.
> 
> But I think this would be the very last thing to care about, with regard
> to DWARF-64 concerns.
I guess that the relative size of the ".debug_str" section may vary
and depends on the source code, particular build environment, and lots of other
circumstances. I've checked some fresh built samples and always see that the
section is usually close in size to ".debug_info" and sometimes even
bigger. So, this section must be ordered similarly as all other debugging info
sections.

-- 
Best Regards,
Igor Kudrin
C++ Developer, Access Softek, Inc.