llvm dev - Aug 2017 - [XRay] Alternatives to relocations in .text section

Hi llvm-dev,

I'm currently looking for alternatives to the synthetic references that XRay
uses to keep some side-tables live, to avoid linker garbage collection from
deleting those sections. Before going any further, let me give a backgrounder on
what XRay does today.

Background
=========
XRay has two side tables we use at runtime to identify the location of the sleds
for the functions that are instrumented. These are named "xray_fn_idx"
and "xray_instr_map". We emit information in these sections that refer
to labels that are emitted in the .text section (or in function-specific
sections, when building with -ffunction-sections). To keep the entries in these
sections alive from the linker's perspective, we emit references in the
.text section (or function-specific sections) that refer to the
"xray_fn_idx" entry for that function, which in turn refers to the
entries in "xray_instr_map" that point back to the instrumentation
points in .text section (or function-specific sections).

Currently, (in X86) we just write out a raw ".quad <label>"
reference after lowering the function, where <label> is defined in the
"xray_fn_idx" section. Unfortunately this causes the linker (gold?) to
emit a relocation in the .text section (TEXTREL in ELF) which interferes
unfavourably with GNU IFUNC relocations.

Questions
========
Is there an alternative to this approach available to us from the LLVM back-end?
I see that there's ways to define associative COMDAT sections, but
that's COFF-specific and only works with COMDAT. We instrument XRay
functions that aren't necessarily defined in COMDAT sections in ELF (or
COMDAT "group" sections), and would like to be able to keep those two
sections live.

Alternatively, is there a way for us to mark the relocation to be resolved
statically (i.e. not a dynamic relocation)? I've tried marking the
label/symbol defined in the xray_fn_idx section as "hidden" and
"local" (according to ELF definitions) but that doesn't seem to
change the relocation emitted in the .text (of function-specific) section. Is
there a way around this from the back-end, so that, for example, when we emit
that reference ".quad <???>", what goes in the "???"
will not be a dynamic relocation?

Or is there a way to instruct the linker to "fix" these relocations?

Cheers

-- Dean

I think you want to use SHF_LINK_ORDER via !associated metadata. That was
the conclusion that the generic-abi mailing list came to when trying to
solve a similar problem for sanitizers:
https://groups.google.com/forum/#!searchin/generic-abi/garbage$20collection%7Csort:relevance/generic-abi/_CbBM6T6WeM/LZnqx1KZAQAJ

The side table would end up in a section with the SHF_LINK_ORDER flag set
and an sh_link pointing at the text section that its associated with.

On Tue, Aug 15, 2017 at 2:37 AM, Dean Michael Berris via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi llvm-dev,
>
> I'm currently looking for alternatives to the synthetic references that
> XRay uses to keep some side-tables live, to avoid linker garbage collection
> from deleting those sections. Before going any further, let me give a
> backgrounder on what XRay does today.
>
> Background
> =========>
> XRay has two side tables we use at runtime to identify the location of the
> sleds for the functions that are instrumented. These are named
> "xray_fn_idx" and "xray_instr_map". We emit information
in these sections
> that refer to labels that are emitted in the .text section (or in
> function-specific sections, when building with -ffunction-sections). To
> keep the entries in these sections alive from the linker's perspective,
we
> emit references in the .text section (or function-specific sections) that
> refer to the "xray_fn_idx" entry for that function, which in turn
refers to
> the entries in "xray_instr_map" that point back to the
instrumentation
> points in .text section (or function-specific sections).
>
> Currently, (in X86) we just write out a raw ".quad <label>"
reference
> after lowering the function, where <label> is defined in the
"xray_fn_idx"
> section. Unfortunately this causes the linker (gold?) to emit a relocation
> in the .text section (TEXTREL in ELF) which interferes unfavourably with
> GNU IFUNC relocations.
>
> Questions
> ========>
> Is there an alternative to this approach available to us from the LLVM
> back-end? I see that there's ways to define associative COMDAT
sections,
> but that's COFF-specific and only works with COMDAT. We instrument XRay
> functions that aren't necessarily defined in COMDAT sections in ELF (or
> COMDAT "group" sections), and would like to be able to keep those
two
> sections live.
>
> Alternatively, is there a way for us to mark the relocation to be resolved
> statically (i.e. not a dynamic relocation)? I've tried marking the
> label/symbol defined in the xray_fn_idx section as "hidden" and
"local"
> (according to ELF definitions) but that doesn't seem to change the
> relocation emitted in the .text (of function-specific) section. Is there a
> way around this from the back-end, so that, for example, when we emit that
> reference ".quad <???>", what goes in the "???"
will not be a dynamic
> relocation?
>
> Or is there a way to instruct the linker to "fix" these
relocations?
>
> Cheers
>
> -- Dean
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170815/66a28fa5/attachment.html>

Yes, but it is super tricky to do this correctly across all linkers.
See how we handle it in ASan. Also, this is quite bad for object file
size, because now each function section has 2 extra sections - one for
the data, and one for section group, and those aren't free. (section
group is necessary to support legacy linkers)

On Tue, Aug 15, 2017 at 9:59 AM, Reid Kleckner <rnk at google.com>
wrote:
> I think you want to use SHF_LINK_ORDER via !associated metadata. That was
> the conclusion that the generic-abi mailing list came to when trying to
> solve a similar problem for sanitizers:
>
https://groups.google.com/forum/#!searchin/generic-abi/garbage$20collection%7Csort:relevance/generic-abi/_CbBM6T6WeM/LZnqx1KZAQAJ
>
> The side table would end up in a section with the SHF_LINK_ORDER flag set
> and an sh_link pointing at the text section that its associated with.
>
> On Tue, Aug 15, 2017 at 2:37 AM, Dean Michael Berris via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>
>> Hi llvm-dev,
>>
>> I'm currently looking for alternatives to the synthetic references
that
>> XRay uses to keep some side-tables live, to avoid linker garbage
collection
>> from deleting those sections. Before going any further, let me give a
>> backgrounder on what XRay does today.
>>
>> Background
>> =========>>
>> XRay has two side tables we use at runtime to identify the location of
the
>> sleds for the functions that are instrumented. These are named
"xray_fn_idx"
>> and "xray_instr_map". We emit information in these sections
that refer to
>> labels that are emitted in the .text section (or in function-specific
>> sections, when building with -ffunction-sections). To keep the entries
in
>> these sections alive from the linker's perspective, we emit
references in
>> the .text section (or function-specific sections) that refer to the
>> "xray_fn_idx" entry for that function, which in turn refers
to the entries
>> in "xray_instr_map" that point back to the instrumentation
points in .text
>> section (or function-specific sections).
>>
>> Currently, (in X86) we just write out a raw ".quad
<label>" reference
>> after lowering the function, where <label> is defined in the
"xray_fn_idx"
>> section. Unfortunately this causes the linker (gold?) to emit a
relocation
>> in the .text section (TEXTREL in ELF) which interferes unfavourably
with GNU
>> IFUNC relocations.
>>
>> Questions
>> ========>>
>> Is there an alternative to this approach available to us from the LLVM
>> back-end? I see that there's ways to define associative COMDAT
sections, but
>> that's COFF-specific and only works with COMDAT. We instrument XRay
>> functions that aren't necessarily defined in COMDAT sections in ELF
(or
>> COMDAT "group" sections), and would like to be able to keep
those two
>> sections live.
>>
>> Alternatively, is there a way for us to mark the relocation to be
resolved
>> statically (i.e. not a dynamic relocation)? I've tried marking the
>> label/symbol defined in the xray_fn_idx section as "hidden"
and "local"
>> (according to ELF definitions) but that doesn't seem to change the
>> relocation emitted in the .text (of function-specific) section. Is
there a
>> way around this from the back-end, so that, for example, when we emit
that
>> reference ".quad <???>", what goes in the
"???" will not be a dynamic
>> relocation?
>>
>> Or is there a way to instruct the linker to "fix" these
relocations?
>>
>> Cheers
>>
>> -- Dean
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>

> On 16 Aug 2017, at 02:59, Reid Kleckner <rnk at google.com> wrote:
> 
> I think you want to use SHF_LINK_ORDER via !associated metadata. That was
the conclusion that the generic-abi mailing list came to when trying to solve a
similar problem for sanitizers:
>
https://groups.google.com/forum/#!searchin/generic-abi/garbage$20collection%7Csort:relevance/generic-abi/_CbBM6T6WeM/LZnqx1KZAQAJ
<https://groups.google.com/forum/#!searchin/generic-abi/garbage$20collection%7Csort:relevance/generic-abi/_CbBM6T6WeM/LZnqx1KZAQAJ>
> 
> The side table would end up in a section with the SHF_LINK_ORDER flag set
and an sh_link pointing at the text section that its associated with.
> 
Sweet -- thanks, Reid!

I'll go have a read through the discussion there.

Cheers
> On Tue, Aug 15, 2017 at 2:37 AM, Dean Michael Berris via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
> Hi llvm-dev,
> 
> I'm currently looking for alternatives to the synthetic references that
XRay uses to keep some side-tables live, to avoid linker garbage collection from
deleting those sections. Before going any further, let me give a backgrounder on
what XRay does today.
> 
> Background
> =========> 
> XRay has two side tables we use at runtime to identify the location of the
sleds for the functions that are instrumented. These are named
"xray_fn_idx" and "xray_instr_map". We emit information in
these sections that refer to labels that are emitted in the .text section (or in
function-specific sections, when building with -ffunction-sections). To keep the
entries in these sections alive from the linker's perspective, we emit
references in the .text section (or function-specific sections) that refer to
the "xray_fn_idx" entry for that function, which in turn refers to the
entries in "xray_instr_map" that point back to the instrumentation
points in .text section (or function-specific sections).
> 
> Currently, (in X86) we just write out a raw ".quad <label>"
reference after lowering the function, where <label> is defined in the
"xray_fn_idx" section. Unfortunately this causes the linker (gold?) to
emit a relocation in the .text section (TEXTREL in ELF) which interferes
unfavourably with GNU IFUNC relocations.
> 
> Questions
> ========> 
> Is there an alternative to this approach available to us from the LLVM
back-end? I see that there's ways to define associative COMDAT sections, but
that's COFF-specific and only works with COMDAT. We instrument XRay
functions that aren't necessarily defined in COMDAT sections in ELF (or
COMDAT "group" sections), and would like to be able to keep those two
sections live.
> 
> Alternatively, is there a way for us to mark the relocation to be resolved
statically (i.e. not a dynamic relocation)? I've tried marking the
label/symbol defined in the xray_fn_idx section as "hidden" and
"local" (according to ELF definitions) but that doesn't seem to
change the relocation emitted in the .text (of function-specific) section. Is
there a way around this from the back-end, so that, for example, when we emit
that reference ".quad <???>", what goes in the "???"
will not be a dynamic relocation?
> 
> Or is there a way to instruct the linker to "fix" these
relocations?
> 
> Cheers
> 
> -- Dean
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
-- Dean

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170816/6c4cdabb/attachment.html>