[llvm-dev] lld suspect behavior .group/rela/text input sections

Fāng-ruì Sòng via llvm-dev
2021-05-13T03:46:08Z

On 2021-05-13, Moshtaghi, Alireza wrote:
>Hi
>The file contents and the commands to build and test are below. assuming
that you have clang++, ld.lld, ld, sed, curl and gdb installed, the commands are
at the end of this email.
>Just copy paste the lines below each file name into that file on your
computer, then run the commands at the end. You’ll see that the file output
linked with ld will load into gdb without any error but the output linked with
ld.lld will produce errors when loaded into gdb
>
>[...]

The gdb command does not give me a warning/error. That said, I have simplified
your example to the following:

cat > ./a.cc <<eof
inline int inline0() { return 0; }
inline int inline1() { return 1; }

int use() {
   return inline0() + inline1();
}
eof
cat > ./a.sh <<eof
clang -c a.cc
ld.lld -r -T ldscript.amd64 a.o -o lld.ro
ld.bfd -r -T ldscript.amd64 a.o -o bfd.ro
readelf -g lld.ro
eof
cat > ./ldscript.amd64 <<eof
SECTIONS
{
   .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
   .text1           : { *(.text .stub .text.* .gnu.linkonce.t.*) }
}
eof

zsh a.sh will give a readelf warning.


COMDAT group section [    3] `.group' [_Z7inline0v] contains 1 sections:
    [Index]    Name
    [    2]   .text1

COMDAT group section [    4] `.group' [_Z7inline1v] contains 1 sections:
    [Index]    Name
readelf: Error: section [    2] in group section [    4] already in group
section [    3]



This is a case about what to do when the output would have two section groups
containing the same section.  ELF specification says "A section cannot be a
member of more than one group." but relocatable output is not covered by
the
specification.

It seems that GNU ld does not respect the *(... .text.*) input section
description for SHF_GROUP sections in this case. (It is unclear what exact
conditions it uses to decide whether to obey an input section description)  It
simply doesn't place .text.* in .text, breaking the expectation of the input
section description usage.  I agree with Peter that we are in the
implemention-defined realm. I currently don't see anything which should be
improved on LLD's side. My advice is to avoid such usage.


>-------------------- commands to build and load into gdb
>curl -o ldscript.amd64
https://raw.githubusercontent.com/freebsd/freebsd-src/master/sys/conf/ldscript.amd64
>sed -i 's/.*kern.*//g' ldscript.amd64
>sed -i 's/.*freebsd.*//g' ldscript.amd64
>clang++ -c file1.cpp -o file1.cpp.o
>clang++ -c file2.cpp -o file2.cpp.o
>clang++ -c file3.cpp -o file3.cpp.o
>ld.lld -r -T ldscript.amd64 *.o -o output.ld.lld
>ld -r -T ldscript.amd64 *.o -o output.ld.bfd
>echo "============== done building"
>gdb -batch -ex 'add-symbol-file output.ld.lld 0x1234'
>echo "============== done loading lld ( Notice the errors above)"
>gdb -batch -ex 'add-symbol-file output.ld.bfd 0x1234'
>echo "============== done loading lld (Notice that no error is produced
here)"
>
>
>
>
>From: Fāng-ruì Sòng <maskray at google.com>
>Date: Monday, May 10, 2021 at 4:01 PM
>To: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>
>Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Peter
Smith <Peter.Smith at arm.com>
>Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input sections
>NetApp Security WARNING: This is an external email. Do not click links or
open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
>On Mon, May 10, 2021 at 3:57 PM Moshtaghi, Alireza
><Alireza.Moshtaghi at netapp.com> wrote:
>>
>> sending it again to the list
>
>I am still waiting for a proper reproduce. Your commands have
>file1.cpp but your attachment has just file1.h.
>
>>
>>
>> From: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>
>> Date: Saturday, May 1, 2021 at 10:53 PM
>> To: Fāng-ruì Sòng <maskray at google.com>
>> Cc: Peter Smith <Peter.Smith at arm.com>
>> Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input
sections
>>
>> Oh
>>
>> Im sorry, I had typo in the file names
>>
>> file.h is the only header file; the other 3 are c++ files. (C may also
do but I just use clang++)
>>
>> And of course you only compile the 3 files: file[1,2,3].cpp
>>
>> clang++ -c file1.cpp -o file1.cpp.o
>>
>> clang++ -c file2.cpp -o file2.cpp.o
>>
>> clang++ -c file3.cpp -o file3.cpp.o
>>
>> ld.lld -r -T ldscript.amd64 *.o -o output.ld.lld
>>
>> ld -r -T ldscript.amd64 *.o -o output.ld.bfd
>>
>> gdb -batch ex “add-symbol-file output.ld.lld 0x1234”
>> gdb -batch ex “add-symbol-file output.ld.bfd 0x1234”
>>
>>
>> ________________________________
>>
>> From: Fāng-ruì Sòng <maskray at google.com>
>> Sent: Friday, April 30, 2021 2:54:58 PM
>> To: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>
>> Cc: Peter Smith <Peter.Smith at arm.com>
>> Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input
sections
>>
>>
>>
>> NetApp Security WARNING: This is an external email. Do not click links
or open attachments unless you recognize the sender and know the content is
safe.
>>
>>
>>
>>
>> On 2021-04-30, Moshtaghi, Alireza wrote:
>> >Yes, the objects with COMDAT are good examples to show the problem.
>> >The problem with lld is when we supply a linker script. Without
linker script, lld does the right thing.
>> >
>> >But with a linker script, it only follows the command to combine
.text.* but does not combine respective .rela.text.*
>> >
>> >This effectively leaves loose ends in the output and makes them
problematic with gdb
>> >
>> >Below there are 4 files. compile them with clang++ and then link
with ld.lld as well as ld.bfd and use -r and -T to pass the linker script from
the link from my first email in this thread. Call the outputs output.ld.lld and
output.ld.bfd respectively.
>> >Then load the images into gdb like:
>> >gdb -batch ex “add-symbol-file output.ld.lld 0x1234”
>> >gdb -batch ex “add-symbol-file output.ld.bfd 0x1234”
>> >
>> >you’ll see that the second gdb command succeeds but first one fails
and when I investigated I find out that the problem is that lld has merged the
.text. _Z22second_inline_functionv and .text. _Z21first_inline_functionv into
.text section but the associated .rela.text.* sections still exist in the output
file and gdb fails to recognize these .rela sections because they are part of a
group that is missing the text sections.
>> >
>> >ld.bfd leaves the .text.* sections alone and does not merge them so
the group remains intact and gdb succeeds to load them.
>> >
>> >If I remove the .text.* from the linkerscript, ld.lld does the
right thing but I don’t want to do this because that way the text sections that
are not part of any group will also not being merged and that is not good.
>> >
>> >******** Minimal example :
>> >--------- file.h
>> >const char * get_char_value (void);
>> >const int * get_int_value (void);
>> >inline const char * first_inline_function (void) {
>> >  return get_char_value ();
>> >}
>> >inline const int * second_inline_function (void) {
>> >  return get_int_value ();
>> >}
>> >--------------------  file1.h
>> >#include "file.h"
>> >static int data = 5;
>> >const int *get_int_value (void) {
>> >  return &data;
>> >}
>> >--------------------  file2.h
>> >#include "file.h"
>> >static const char *data = "is this really a meaningful?";
>> >const char *get_char_value (void) {
>> >  return data;
>> >}
>> >--------------------  file3.h
>> >#include "file.h"
>> >const char * get_it (void) {
>> >  const char *c = first_inline_function();
>> >  const int *i = second_inline_function();
>> >  return &c[*i];
>> >}
>>
>> I'm still puzzled. Do you miss some files?
>> clang -c file2.h  produces a .gch file, unless you specify -xc++.
>>
>> If helps, I use the following script to print reproduce instructions.
>> Then users don't need to spend time creating files by themselves.
>>
>> #!/bin/zsh
>> dry_run>> if [[ $1 == --dry-run ]]; then dry_run=1; shift; fi
>>
>> f() {
>>    for f in $1/*(ND.^x); do
>>      [[ ${f##*/} =~ '^\.' ]] && continue
>>      if [[ -n $dry_run ]]; then
>>        echo $f
>>      elif [[ $(wc -l < $f) == 1 ]]; then
>>        echo "echo '$(<$f)' > $f"
>>      else
>>        echo "cat > $f <<eof"
>>        cat $f
>>        echo eof
>>      fi
>>    done
>>    for d in $1/*(ND/); do
>>      [[ $d =~ '^\.' ]] && continue
>>      echo mkdir -p $d
>>      f $d
>>    done
>> }
>>
>> f ${1:-.}
>>
>> >
>> >From: Peter Smith <Peter.Smith at arm.com>
>> >Date: Thursday, April 29, 2021 at 3:38 AM
>> >To: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>,
Fāng-ruì Sòng <maskray at google.com>
>> >Subject: RE: [llvm-dev] lld suspect behavior .group/rela/text input
sections
>> >NetApp Security WARNING: This is an external email. Do not click
links or open attachments unless you recognize the sender and know the content
is safe.
>> >
>> >
>> >I can reproduce part of it using ld -r on an object with a COMDAT
group.
>> >
>> >Given that ld -r produces a relocatable object that can be used as
input to a subsequent link step retaining the group information and the sections
referenced by it uncollated in the ELF file makes sense to me. If the sections
were collated into a single output section and the COMDAT group information lost
then there would be multiple definition errors if the relocatable object were
combined with other relocatable objects that had COMDAT groups with the same
signature.
>> >
>> >Non COMDAT groups would be possible to collate without affecting
subsequent links. Are the groups COMDAT or non COMDAT in your example?
>> >
>> >My limited understanding is that kernel modules use relocatable
objects as kind of dynamically linked object, so there wouldn’t be any
recombination, but the static linker doesn’t know what the output of ld -r will
be used for.
>> >
>> >I don’t think the ELF spec covers combination of relocatable
objects to output a single relocatable object. It is pretty much implementation
defined behaviour from the linker.
>> >
>> >I’m not sure what to suggest as a resolution. It may be that a
relocatable link needs an additional “kernel module” or “relocatable object”
choice to guide the linker as to whether certain types of section should be
combined.
>> >
>> >Peter
>> >
>> >From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf
Of Moshtaghi, Alireza via llvm-dev
>> >Sent: 28 April 2021 23:20
>> >To: Fāng-ruì Sòng <maskray at google.com>
>> >Cc: llvm-dev at lists.llvm.org
>> >Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input
sections
>> >
>> >It is not easy to produce an minimal example. But I will try
>> >I was hoping that you may have solved it before.
>> >I will come back with an example
>> >
>> >Thanks
>> >A
>> >
>> >From: Fāng-ruì Sòng <maskray at google.com<mailto:maskray at
google.com>>
>> >Date: Wednesday, April 28, 2021 at 3:11 PM
>> >To: Moshtaghi, Alireza <Alireza.Moshtaghi at
netapp.com<mailto:Alireza.Moshtaghi at netapp.com>>
>> >Cc: llvm-dev at lists.llvm.org<mailto:llvm-dev at
lists.llvm.org> <llvm-dev at lists.llvm.org<mailto:llvm-dev at
lists.llvm.org>>
>> >Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input
sections
>> >NetApp Security WARNING: This is an external email. Do not click
links or open attachments unless you recognize the sender and know the content
is safe.
>> >
>> >
>> >
>> >
>> >On Wed, Apr 28, 2021 at 11:02 AM Moshtaghi, Alireza via llvm-dev
>> ><llvm-dev at lists.llvm.org<mailto:llvm-dev at
lists.llvm.org>> wrote:
>> >>
>> >> Hi
>> >>
>> >> Can you help me to understand if I’m doing anything wrong or
the problem is with lld?
>> >>
>> >> I need to use linker-script (below link) for linking freebsd
kernel modules and pass -r to ld.lld (version 10) to make these modules
relocatable.
>> >>
>> >>
>> >>
>> >> Some of the modules that I build are very big and also have
.group sections on rela.text.<mangledname> and .text.<mangledname>
>> >>
>> >>
>> >>
>> >> Per ELF specification these sections should be treated as one
and either all merged or none. But I think lld is not respecting that spec and
merges all .text.<mangledname> sections while leaving the .group and
.rela.text.<mangledname> sections in the output, which results in broken
elf (when I try to load it into gdb, I get Bad Value error)
>> >>
>> >>
>> >>
>> >> Alternatively, when I link with ld.bfd, the
.text.<manglenames> remain in the output and I’m able to load the object
into gdb without error.
>> >
>> >Hi, do you have a minimal reproducible example
>> >(https://stackoverflow.com/help/minimal-reproducible-example)?
>> >
>> >Freebsd is not build-friendly when the user is not using FreeBSD
;-)
>> >
>> >>
>> >>
>> >> Linker-script:
>> >>
>> >>
https://github.com/freebsd/freebsd-src/blob/098dbd7ff7f3da9dda03802cdb2d8755f816eada/sys/conf/ldscript.amd64
>> >IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the contents
to any other person, use it for any purpose, or store or copy the information in
any medium. Thank you.
>
>
>
>--
>宋方睿