Fāng-ruì Sòng via llvm-dev
2020-Nov-13 20:42 UTC
[llvm-dev] [LLD] Support DWARF64, debug_info "sorting"
On Fri, Nov 13, 2020 at 11:29 AM David Blaikie <dblaikie at gmail.com> wrote:> > On Fri, Nov 13, 2020 at 11:24 AM Fāng-ruì Sòng <maskray at google.com> wrote: > > > > On Fri, Nov 13, 2020 at 11:17 AM David Blaikie <dblaikie at gmail.com> wrote: > > > > > > On Fri, Nov 13, 2020 at 11:05 AM Fāng-ruì Sòng <maskray at google.com> wrote: > > > > > > > > I got replies from Nick Clifton and Michael Matz: > > > > https://sourceware.org/pipermail/binutils/2020-November/114116.html > > > > (and its reply). > > > > I have mentioned (a) the difficulty of the > > > > detecting-DWARF64-by-first-relocation approach and (b) the section > > > > type approach in my reply there > > > > https://sourceware.org/pipermail/binutils/2020-November/114125.html > > > > > > > > (a) My prototype has made me feel uneasy with this approach. > > > > > > > > <quote> > > > > In DWARF v4 or if .debug_str_offset is not used, it is a problem. A > > > > heuristic is: if an input section in a file is marked DWARF64, we mark > > > > all other .debug_* DWARF64. This makes me feel a bit uneasy because > > > > for an output section description > > > > > > > > .debug_str 0 : { *(.debug_str) } > > > > > > > > Now the behavior of `*` (or, if we invent a `SORT_*` keyword) is also > > > > dependent on other output sections. > > > > </quote> > > > > > > > > (b) > > > > * It needs a section type (either a gABI one or a SHT_GNU_* in GNU > > > > ABI). Seeking for a gABI one is not that I think this is particularly > > > > related to gABI but that I don't want Solaris (which LLVM also > > > > supports) uses a different section type to unnecessarily cause > > > > friction on our implementation > > > > > > If I'm understawding you correrctly you're suggesting the sorting > > > behavior would only be implemented if the input object file had some > > > new attributes in it designating which sections are debug info > > > sections? > > > > > > I don't think that's a viable solution to the problem at hand, then - > > > if someone is able to update their toolchain and rebuild objects with > > > new attributes, they can probably update the build configuration of > > > those objects to build them with DWARF64 instead, avoiding the mixed > > > 32/64 problem. I think the solution we're looking for would have to > > > work with existing precompiled object files using DWARF32 that are in > > > the wild today, without modification. > > > > I know the "no-modification" requirement:) The first paragraph of > > https://sourceware.org/pipermail/binutils/2020-November/114125.html > > mentioned this. > > OK - thanks for clarifying. I don't really know much/enough about > linker scripts to comment on the rest of the design, and was just > confused/misunderstanding what was being suggested.For .debug_* in object files: DWARF32 -> SHT_PROGBITS (unchanged) DWARF64 -> SHT_DWARF64 or SHT_GNU_DWARF64 In LLD, we will need to allow mixed SHT_PROGBITS and SHT_DWARF64. If all input sections are SHT_DWARF64, the output section type probably should also be SHT_DWARF64. If mixed, SHT_PROGBITS. FWIW I started a generic-abi thread https://groups.google.com/g/generic-abi/c/i2Xio-47QdQ ("Reserve a section type value for DWARF64") to give stakeholders from other ELF operating systems a chance to participate in the design. I have paid attention to my wording: a new section type is **not decided yet** on LLVM/GNU binutils sides. Our discussions on llvm-dev/binutils will benefit from agreement/disagreement from generic-abi.> > > > The section type approach is used this way (in another paragraph): > > > > <quote> > > If we invent a keyword (say, TYPE) to match sections by type, we could use > > > > .debug_info 0 : { *(TYPE (SHT_PROGBITS) > > .debug_info${RELOCATING+ .gnu.linkonce.wi.*}) } > > .debug_info 0 : { *(TYPE (SHT_GNU_DWARF64) .debug_info) } > > > > or > > > > .debug_info 0 : { *(TYPE (SHT_PROGBITS) > > .debug_info${RELOCATING+ .gnu.linkonce.wi.*} TYPE (SHT_GNU_DWARF64) > > .debug_info) } > > </quote> > > > > > > * It needs a clarification on multiple output section descriptions > > > > with the same name. > > > > * It needs a linker script feature to match input sections by type. > > > > > > > > On Fri, Nov 13, 2020 at 9:52 AM David Blaikie <dblaikie at gmail.com> wrote: > > > > > > > > > > On Fri, Nov 13, 2020 at 8:35 AM Wenlei He via llvm-dev > > > > > <llvm-dev at lists.llvm.org> wrote: > > > > > > > > > > > > > Thinking about it, I wouldn't expect an LTO generated object itself to have a mixture of DWARF32/64, although I guess the 32/64 bit state could be encoded in the IR (I am not familiar enough with it to know if it actually is or not). It might be necessary to find ways to configure LTO to generate DWARF64, possibly via a link-time option. > > > > > > > > > > > > > > > > > > > > > > > > I don’t think we need to encode dwarf32/64 in IR as attribute for each module. We’re not going to emit mixed dwarf32/64 for merged LTO module anyways, so allowing each module to express its dwarf setting would only introduce burden for LTO to deal with inconsistency (warning?) among input modules. Having a linker switch to pass the setting from driver to LTO sounds better to me. > > > > > > > > > > Usually the issue there is that existing build systems may be setup > > > > > only to pass such flags to the compilations, and not to the link > > > > > invocations - like DWARF version, we pass that down through IR, emit > > > > > warnings/errors when two IR modules with different DWARF versions are > > > > > linked together, and then emit only one (the higher, I believe) DWARF > > > > > version out the other end. > > > > > > > > > > We aren't 100% consistent on this "anything you could do without LTO, > > > > > you shuold be able to do with LTO/passing the same flags to the same > > > > > actions" kind of strategy (eg: type units and DWARF compression aren't > > > > > passed down through IR - if you want those you have to pass them to > > > > > the link invocation (via the clang driver) yourself). So it's more "is > > > > > there a systemic use of these flags already for the compilation and > > > > > would not supporting it there be a pain"? It's probably not for > > > > > DWARF64, since we haven't had any flag to support it at the moment > > > > > anyway. > > > > > > > > > > - Dave > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> > > > > > > Date: Thursday, November 12, 2020 at 2:21 AM > > > > > > To: Fangrui Song <maskray at google.com> > > > > > > Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org> > > > > > > Subject: Re: [llvm-dev] [LLD] Support DWARF64, debug_info "sorting" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 12 Nov 2020 at 02:10, Fangrui Song <maskray at google.com> wrote: > > > > > > > > > > > > On 2020-11-12, Alexander Yermolovich wrote: > > > > > > >Thanks for feedback. > > > > > > > > > > > > > >I agree with patch and numbers this will be a more concrete discussion, but I wanted to judge overall receptiveness to this approach and see maybe there was a better way. > > > > > > > > > > > > > >"Whilst the majority of objects will only have a single CU in them, there will be exceptions (LTO-generated objects, -r merged objects etc), so we do need to consider this approach." > > > > > > >David can you elaborate under which conditions LTO-generated objects will have a mix of DWARF32/64 in same .debug_info? Looking at how dwarf64 was implemented same flag will be used for the entirety of the dwarf output, even if multiple CUs are included. > > > > > > > > > > > > > > > > > > > > > > > > Thinking about it, I wouldn't expect an LTO generated object itself to have a mixture of DWARF32/64, although I guess the 32/64 bit state could be encoded in the IR (I am not familiar enough with it to know if it actually is or not). It might be necessary to find ways to configure LTO to generate DWARF64, possibly via a link-time option. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >On one hand since this is only applicable for when DWARF64 is used, special option would be the way to go. Although the user will need to be aware of yet another LLD option. Maybe an error when relocations overflow occur can be modified to display this option along with -fdebug-types-section > > > > > > > > > > > > I am quite happy with the relocation approach under a linker option. I'd still > > > > > > want to know generic-abi folks's thoughts, though. James may have prepared something > > > > > > he wants to share with generic-abi:) Let's wait... > > > > > > > > > > > > > > > > > > > > > > > > I hadn't prepared anything if I'm honest (though if there's widespread agreement that this would be useful, I certainly can - it would have other positive improvements too, reducing the need for tools to rely on section names to identify debug data for example). It was more a case of bouncing ideas off of people to see what they thought. Any discussion we have will probably also need circulating on the DWARF mailing list too, since it is more a DWARF issue than a gABI issue (unless the solution is a new section type). Further refinements to this idea that might make it more appealing to the generic group: `SHT_DEBUG` for the section type name, with the first N bytes of the sh_info used to specify the variant of debug data it represents (e.g. 0x1 for DWARF, 0x2 for SOME_OTHER_STANDARD etc), and the remainder for use as flags as defined by the standard (I'm thinking for DWARF you could encode the 64-bit/32-bit state in there, possibly the section variant (info/rnglists/line etc) and the DWARF version too), on the understanding that consumers like the linker wouldn't combine sections in a potentially broken way. This has the advantage that it could be retrofitted to the existing standard versions, but as has been pointed out, this won't help those with linker scripts - that could only be solved with a new DWARF standard and separate names for 64/32 bit sections, at least if we wanted to avoid the linker needing to do anything beyond reading the section header. > > > > > > > > > > > > > > > > > > > > > > > > The relocation approach sounds like a reasonable solution for the current situation - even if we do decide to go the route of changing producers to start emitting a new section type/update the standard etc, it doesn't resolve the problem people may currently face. > > > > > > > > > > > > _______________________________________________ > > > > > > LLVM Developers mailing list > > > > > > llvm-dev at lists.llvm.org > > > > > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > > > > > > > > > > > > > -- > > > > 宋方睿 > > > > > > > > -- > > 宋方睿-- 宋方睿
Alexander Yermolovich via llvm-dev
2020-Nov-14 01:46 UTC
[llvm-dev] [LLD] Support DWARF64, debug_info "sorting"
Thanks for doing a diff and asking in other groups.
So if I understand your concern with using first reloc as it relates to
.debug_str.
In DWARF4 for .debug_str is referenced from .debug_info, .debug_type using
DW_FORM_strp. For DWARF32 it's 32bit unsigned, for DWARF64 it's 64bit
unsigned. So in relocation section for some .debug_info section we will have a
relocation entry to patch up DW_FORM_strp. Either R_X86_64_32, or R_X86_64_64,
depending on DWARF format.
A situation we might have is that an input .debug_info section is DWARF32 so it
gets ordered apropriatly within output .debug_info section, but the input
.debug_str section can be put in the output .debug_str section that is above
4GB. In which case we still hit overflow. So we also need to oder the .debug_str
section, except in DWARF4 there is no real clear link, besides looking through
.debug_info relocs.
Is that a correct summary?
Also I don't quite understand what the issue is with linker script.
My understanding is that:
.debug_str 0 : { *(.debug_str) }
Just stays that all .debug_str input sections should go in to .debug_str output
section. It doesn't really specify the ordering within the output .debug_str
section.
Thank You
Alex
________________________________
From: Fāng-ruì Sòng <maskray at google.com>
Sent: Friday, November 13, 2020 12:42 PM
To: David Blaikie <dblaikie at gmail.com>
Cc: Wenlei He <wenlei at fb.com>; Alexander Yermolovich <ayermolo at
fb.com>; llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>;
Robinson, Paul <paul.robinson at sony.com>; James Henderson
<jh7370.2008 at my.bristol.ac.uk>; Eric Christopher <echristo at
gmail.com>; Igor Kudrin <ikudrin at accesssoftek.com>
Subject: Re: [llvm-dev] [LLD] Support DWARF64, debug_info "sorting"
On Fri, Nov 13, 2020 at 11:29 AM David Blaikie <dblaikie at gmail.com>
wrote:>
> On Fri, Nov 13, 2020 at 11:24 AM Fāng-ruì Sòng <maskray at
google.com> wrote:
> >
> > On Fri, Nov 13, 2020 at 11:17 AM David Blaikie <dblaikie at
gmail.com> wrote:
> > >
> > > On Fri, Nov 13, 2020 at 11:05 AM Fāng-ruì Sòng <maskray at
google.com> wrote:
> > > >
> > > > I got replies from Nick Clifton and Michael Matz:
> > > >
https://sourceware.org/pipermail/binutils/2020-November/114116.html
> > > > (and its reply).
> > > > I have mentioned (a) the difficulty of the
> > > > detecting-DWARF64-by-first-relocation approach and (b) the
section
> > > > type approach in my reply there
> > > >
https://sourceware.org/pipermail/binutils/2020-November/114125.html
> > > >
> > > > (a) My prototype has made me feel uneasy with this approach.
> > > >
> > > > <quote>
> > > > In DWARF v4 or if .debug_str_offset is not used, it is a
problem. A
> > > > heuristic is: if an input section in a file is marked
DWARF64, we mark
> > > > all other .debug_* DWARF64. This makes me feel a bit uneasy
because
> > > > for an output section description
> > > >
> > > > .debug_str 0 : { *(.debug_str) }
> > > >
> > > > Now the behavior of `*` (or, if we invent a `SORT_*`
keyword) is also
> > > > dependent on other output sections.
> > > > </quote>
> > > >
> > > > (b)
> > > > * It needs a section type (either a gABI one or a SHT_GNU_*
in GNU
> > > > ABI). Seeking for a gABI one is not that I think this is
particularly
> > > > related to gABI but that I don't want Solaris (which
LLVM also
> > > > supports) uses a different section type to unnecessarily
cause
> > > > friction on our implementation
> > >
> > > If I'm understawding you correrctly you're suggesting the
sorting
> > > behavior would only be implemented if the input object file had
some
> > > new attributes in it designating which sections are debug info
> > > sections?
> > >
> > > I don't think that's a viable solution to the problem at
hand, then -
> > > if someone is able to update their toolchain and rebuild objects
with
> > > new attributes, they can probably update the build configuration
of
> > > those objects to build them with DWARF64 instead, avoiding the
mixed
> > > 32/64 problem. I think the solution we're looking for would
have to
> > > work with existing precompiled object files using DWARF32 that
are in
> > > the wild today, without modification.
> >
> > I know the "no-modification" requirement:) The first
paragraph of
> > https://sourceware.org/pipermail/binutils/2020-November/114125.html
> > mentioned this.
>
> OK - thanks for clarifying. I don't really know much/enough about
> linker scripts to comment on the rest of the design, and was just
> confused/misunderstanding what was being suggested.
For .debug_* in object files:
DWARF32 -> SHT_PROGBITS (unchanged)
DWARF64 -> SHT_DWARF64 or SHT_GNU_DWARF64
In LLD, we will need to allow mixed SHT_PROGBITS and SHT_DWARF64. If
all input sections are SHT_DWARF64, the output section type probably
should also be SHT_DWARF64.
If mixed, SHT_PROGBITS.
FWIW I started a generic-abi thread
https://groups.google.com/g/generic-abi/c/i2Xio-47QdQ ("Reserve a
section type value for DWARF64") to give stakeholders from other ELF
operating systems a chance to participate in the design. I have paid
attention to my wording: a new section type is **not decided yet** on
LLVM/GNU binutils sides. Our discussions on llvm-dev/binutils will
benefit from agreement/disagreement from generic-abi.
> >
> > The section type approach is used this way (in another paragraph):
> >
> > <quote>
> > If we invent a keyword (say, TYPE) to match sections by type, we could
use
> >
> > .debug_info 0 : { *(TYPE (SHT_PROGBITS)
> > .debug_info${RELOCATING+ .gnu.linkonce.wi.*}) }
> > .debug_info 0 : { *(TYPE (SHT_GNU_DWARF64) .debug_info) }
> >
> > or
> >
> > .debug_info 0 : { *(TYPE (SHT_PROGBITS)
> > .debug_info${RELOCATING+ .gnu.linkonce.wi.*} TYPE (SHT_GNU_DWARF64)
> > .debug_info) }
> > </quote>
> >
> > > > * It needs a clarification on multiple output section
descriptions
> > > > with the same name.
> > > > * It needs a linker script feature to match input sections
by type.
> > > >
> > > > On Fri, Nov 13, 2020 at 9:52 AM David Blaikie <dblaikie
at gmail.com> wrote:
> > > > >
> > > > > On Fri, Nov 13, 2020 at 8:35 AM Wenlei He via llvm-dev
> > > > > <llvm-dev at lists.llvm.org> wrote:
> > > > > >
> > > > > > > Thinking about it, I wouldn't expect an
LTO generated object itself to have a mixture of DWARF32/64, although I guess
the 32/64 bit state could be encoded in the IR (I am not familiar enough with it
to know if it actually is or not). It might be necessary to find ways to
configure LTO to generate DWARF64, possibly via a link-time option.
> > > > > >
> > > > > >
> > > > > >
> > > > > > I don’t think we need to encode dwarf32/64 in IR
as attribute for each module. We’re not going to emit mixed dwarf32/64 for
merged LTO module anyways, so allowing each module to express its dwarf setting
would only introduce burden for LTO to deal with inconsistency (warning?) among
input modules. Having a linker switch to pass the setting from driver to LTO
sounds better to me.
> > > > >
> > > > > Usually the issue there is that existing build systems
may be setup
> > > > > only to pass such flags to the compilations, and not to
the link
> > > > > invocations - like DWARF version, we pass that down
through IR, emit
> > > > > warnings/errors when two IR modules with different
DWARF versions are
> > > > > linked together, and then emit only one (the higher, I
believe) DWARF
> > > > > version out the other end.
> > > > >
> > > > > We aren't 100% consistent on this "anything
you could do without LTO,
> > > > > you shuold be able to do with LTO/passing the same
flags to the same
> > > > > actions" kind of strategy (eg: type units and
DWARF compression aren't
> > > > > passed down through IR - if you want those you have to
pass them to
> > > > > the link invocation (via the clang driver) yourself).
So it's more "is
> > > > > there a systemic use of these flags already for the
compilation and
> > > > > would not supporting it there be a pain"? It's
probably not for
> > > > > DWARF64, since we haven't had any flag to support
it at the moment
> > > > > anyway.
> > > > >
> > > > > - Dave
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > From: llvm-dev <llvm-dev-bounces at
lists.llvm.org>
> > > > > > Date: Thursday, November 12, 2020 at 2:21 AM
> > > > > > To: Fangrui Song <maskray at google.com>
> > > > > > Cc: llvm-dev at lists.llvm.org <llvm-dev at
lists.llvm.org>
> > > > > > Subject: Re: [llvm-dev] [LLD] Support DWARF64,
debug_info "sorting"
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, 12 Nov 2020 at 02:10, Fangrui Song
<maskray at google.com> wrote:
> > > > > >
> > > > > > On 2020-11-12, Alexander Yermolovich wrote:
> > > > > > >Thanks for feedback.
> > > > > > >
> > > > > > >I agree with patch and numbers this will be a
more concrete discussion, but I wanted to judge overall receptiveness to this
approach and see maybe there was a better way.
> > > > > > >
> > > > > > >"Whilst the majority of objects will only
have a single CU in them, there will be exceptions (LTO-generated objects, -r
merged objects etc), so we do need to consider this approach."
> > > > > > >David can you elaborate under which conditions
LTO-generated objects will have a mix of DWARF32/64 in same .debug_info? Looking
at how dwarf64 was implemented same flag will be used for the entirety of the
dwarf output, even if multiple CUs are included.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thinking about it, I wouldn't expect an LTO
generated object itself to have a mixture of DWARF32/64, although I guess the
32/64 bit state could be encoded in the IR (I am not familiar enough with it to
know if it actually is or not). It might be necessary to find ways to configure
LTO to generate DWARF64, possibly via a link-time option.
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > >On one hand since this is only applicable for
when DWARF64 is used, special option would be the way to go. Although the user
will need to be aware of yet another LLD option. Maybe an error when relocations
overflow occur can be modified to display this option along with
-fdebug-types-section
> > > > > >
> > > > > > I am quite happy with the relocation approach
under a linker option. I'd still
> > > > > > want to know generic-abi folks's thoughts,
though. James may have prepared something
> > > > > > he wants to share with generic-abi:) Let's
wait...
> > > > > >
> > > > > >
> > > > > >
> > > > > > I hadn't prepared anything if I'm honest
(though if there's widespread agreement that this would be useful, I
certainly can - it would have other positive improvements too, reducing the need
for tools to rely on section names to identify debug data for example). It was
more a case of bouncing ideas off of people to see what they thought. Any
discussion we have will probably also need circulating on the DWARF mailing list
too, since it is more a DWARF issue than a gABI issue (unless the solution is a
new section type). Further refinements to this idea that might make it more
appealing to the generic group: `SHT_DEBUG` for the section type name, with the
first N bytes of the sh_info used to specify the variant of debug data it
represents (e.g. 0x1 for DWARF, 0x2 for SOME_OTHER_STANDARD etc), and the
remainder for use as flags as defined by the standard (I'm thinking for
DWARF you could encode the 64-bit/32-bit state in there, possibly the section
variant (info/rnglists/line etc) and the DWARF version too), on the
understanding that consumers like the linker wouldn't combine sections in a
potentially broken way. This has the advantage that it could be retrofitted to
the existing standard versions, but as has been pointed out, this won't help
those with linker scripts - that could only be solved with a new DWARF standard
and separate names for 64/32 bit sections, at least if we wanted to avoid the
linker needing to do anything beyond reading the section header.
> > > > > >
> > > > > >
> > > > > >
> > > > > > The relocation approach sounds like a reasonable
solution for the current situation - even if we do decide to go the route of
changing producers to start emitting a new section type/update the standard etc,
it doesn't resolve the problem people may currently face.
> > > > > >
> > > > > > _______________________________________________
> > > > > > LLVM Developers mailing list
> > > > > > llvm-dev at lists.llvm.org
> > > > > >
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > > >
> > > >
> > > >
> > > > --
> > > > 宋方睿
> >
> >
> >
> > --
> > 宋方睿
--
宋方睿
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201114/4d0b0d0e/attachment-0001.html>
Igor Kudrin via llvm-dev
2020-Nov-17 06:42 UTC
[llvm-dev] [EXTERNAL] Re: [LLD] Support DWARF64, debug_info "sorting"
On 14.11.2020 3:42, Fāng-ruì Sòng wrote:> For .debug_* in object files: > > DWARF32 -> SHT_PROGBITS (unchanged) > DWARF64 -> SHT_DWARF64 or SHT_GNU_DWARF64 > > In LLD, we will need to allow mixed SHT_PROGBITS and SHT_DWARF64. If > all input sections are SHT_DWARF64, the output section type probably > should also be SHT_DWARF64. > If mixed, SHT_PROGBITS.I am not really sure that we need a new section type. This gets a small simplification in one part of the linker but adds much more burden of supporting that to all tools and specs around. And that support would be required for a rarely used feature, which certainly results that support to be partial and sometimes erroneous. If we aim for clarity and not ambiguity, I would suggest considering the following. The DWARF specs designed so that the data of DWARF32 and DWARF64 formats can be intermixed together in one section. What section type should be used for that combination? Should it be another new section type, something like SHT_DWARF32_DWARF64? Probably not, that should be a regular SHT_PROGBITS. That means that DWARF64 data can be contained in a SHT_PROGBITS section. Consequently, the section which contains DWARF64 data without DWARF32 data should also be SHT_PROGBITS, because the latter is just a special case of the former.> FWIW I started a generic-abi thread > https://groups.google.com/g/generic-abi/c/i2Xio-47QdQ ("Reserve a > section type value for DWARF64") to give stakeholders from other ELF > operating systems a chance to participate in the design. I have paid > attention to my wording: a new section type is **not decided yet** on > LLVM/GNU binutils sides. Our discussions on llvm-dev/binutils will > benefit from agreement/disagreement from generic-abi.Thank you for conducting all these discussions. I hope we will finally find a way to bring the feature alive. -- Best Regards, Igor Kudrin C++ Developer, Access Softek, Inc.
Fāng-ruì Sòng via llvm-dev
2020-Nov-17 06:51 UTC
[llvm-dev] [LLD] Support DWARF64, debug_info "sorting"
On 2020-11-14, Alexander Yermolovich wrote:>Thanks for doing a diff and asking in other groups. > >So if I understand your concern with using first reloc as it relates to .debug_str. > >In DWARF4 for .debug_str is referenced from .debug_info, .debug_type using DW_FORM_strp. For DWARF32 it's 32bit unsigned, for DWARF64 it's 64bit unsigned. So in relocation section for some .debug_info section we will have a relocation entry to patch up DW_FORM_strp. Either R_X86_64_32, or R_X86_64_64, depending on DWARF format. > >A situation we might have is that an input .debug_info section is DWARF32 so it gets ordered apropriatly within output .debug_info section, but the input .debug_str section can be put in the output .debug_str section that is above 4GB. In which case we still hit overflow. So we also need to oder the .debug_str section, except in DWARF4 there is no real clear link, besides looking through .debug_info relocs.Yes. For most other .debug_*, we can check whether the section has an 64-bit absolute relocation to decide whether it is DWARF64. .debug_str is different in that we need to check relocations referencing it. This makes its behavior dependent on other sections, which is why I feel lost with the relocation approach: when we write .debug_str 0 : { *(.debug_str) }, we really want the output section .debug_str can be made up with just information from the input section descriptions, not random information from other .debug_* LLD -O1 (default) and GNU ld -O0 enable constant string merge within SHF_MERGE&&SHF_STRINGS sections. We need to pay attention as if a DWARF32 string gets folded into a DWARF64 string, the section referencing the DWARF32 string can still trigger a relocation overflow. If we order DWARF32 components before DWARF64 components, with the llvm::StringTableBuilder usage in LLD, we can make sure DWARF64 strings can get folded into DWARF32 strings, not the other way around.>Is that a correct summary? > >Also I don't quite understand what the issue is with linker script. > >My understanding is that: > >.debug_str 0 : { *(.debug_str) } > >Just stays that all .debug_str input sections should go in to .debug_str output section. It doesn't really specify the ordering within the output .debug_str section.There is an assumption that linkers concatenate input sections in input order, which is required by the ELF specification. There are options which can change the semantics of `*`: --sort-section, (gold specific) --section-ordering-file, (LLD specific) --symbol-ordering-file. Like the other options, we should justify the `*` ordering by assigning appropriate semantics.>Thank You >Alex
Fāng-ruì Sòng via llvm-dev
2020-Nov-17 07:05 UTC
[llvm-dev] [LLD] Support DWARF64, debug_info "sorting"
On Mon, Nov 16, 2020 at 10:42 PM Igor Kudrin <ikudrin at accesssoftek.com> wrote:> > On 14.11.2020 3:42, Fāng-ruì Sòng wrote: > > For .debug_* in object files: > > > > DWARF32 -> SHT_PROGBITS (unchanged) > > DWARF64 -> SHT_DWARF64 or SHT_GNU_DWARF64 > > > > In LLD, we will need to allow mixed SHT_PROGBITS and SHT_DWARF64. If > > all input sections are SHT_DWARF64, the output section type probably > > should also be SHT_DWARF64. > > If mixed, SHT_PROGBITS. > > I am not really sure that we need a new section type. This gets a small > simplification in one part of the linker but adds much more burden of > supporting that to all tools and specs around. And that support would be > required for a rarely used feature, which certainly results that support > to be partial and sometimes erroneous.This is not a small simplification. If we consider how we would address .debug_str with the relocation approach, it would be quite contrived. A reply I just made https://lists.llvm.org/pipermail/llvm-dev/2020-November/146673.html detailed why I don't like the conceptual model: the behavior is dependent on other sections. I also felt bad as I had to do string comparison on ".debug_" (https://reviews.llvm.org/D91404) I've chatted with gdb folks: gdb (gdb/dwarf2/read.c) is agnostic about the section type. (They just cannot handle multiple .debug_info sections.) SHT_PROGBITS is a default (or "use this if nothing more specific exists") section type that probably no tool will specifically check the type. For many tools, they don't understand DWARF64 yet - there is little they need to adapt when they add DWARF64 support. Currently the only problem I can think of is readelf -S (from my understanding of https://sourceware.org/pipermail/binutils/2020-November/114116.html , Nick will be happy if I write a GNU readelf patch to make the section header table dump look good:) )> If we aim for clarity and not ambiguity, I would suggest considering the > following. The DWARF specs designed so that the data of DWARF32 and > DWARF64 formats can be intermixed together in one section. What section > type should be used for that combination? Should it be another new > section type, something like SHT_DWARF32_DWARF64? Probably not, that > should be a regular SHT_PROGBITS. That means that DWARF64 data can be > contained in a SHT_PROGBITS section. Consequently, the section which > contains DWARF64 data without DWARF32 data should also be SHT_PROGBITS, > because the latter is just a special case of the former.I suggest we follow the `canMergeToProgbits` logic in LLD: mixed SHT_DWARF64 and SHT_PROGBITS get SHT_PROGBITS. This is the existing rule for many other section types. Conceptually, the combined section should impose the rigid restriction when it is further combined with other sections. This is probably an extra argument that we don't need SHT_DWARF32.> > FWIW I started a generic-abi thread > > https://groups.google.com/g/generic-abi/c/i2Xio-47QdQ ("Reserve a > > section type value for DWARF64") to give stakeholders from other ELF > > operating systems a chance to participate in the design. I have paid > > attention to my wording: a new section type is **not decided yet** on > > LLVM/GNU binutils sides. Our discussions on llvm-dev/binutils will > > benefit from agreement/disagreement from generic-abi. > > Thank you for conducting all these discussions. I hope we will finally > find a way to bring the feature alive.Thanks to James and Pavel for quoting the standard. https://groups.google.com/g/generic-abi/c/i2Xio-47QdQ It seems that we've made progress on the thread and may be able to get a generic value. I am following up with binutils folks https://sourceware.org/pipermail/binutils/2020-November/ I hope we can get to a design which satisfies all parties.> -- > Best Regards, > Igor Kudrin > C++ Developer, Access Softek, Inc.-- 宋方睿