llvm dev - Aug 2018 - [lld] avoid emitting PLT entries for ifuncs

In the context of support not only IFunc but DTrace, what kind of features
do you want to add to lld, if you have a chance to implement it in the
linker instead of a post-processing tool? I wonder if we can solve both of
your problems.

On Thu, Aug 23, 2018 at 1:33 AM Mark Johnston <markj at freebsd.org>
wrote:
> On Wed, Aug 22, 2018 at 10:11:00AM -0400, Ed Maste wrote:
> > On 22 August 2018 at 04:27, Rui Ueyama via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> > >
> > > However, if you pass the -emit-relocs option to the linker, lld
keeps
> all
> > > relocations that have already been resolved in an output
executable. By
> > > analyzing a relocation table in a resulting executable, you could
find
> all
> > > locations where the ifunc PLT is called. Then, you can construct
a new
> table
> > > for your linker, embed it to the executable using objcopy or
something
> like
> > > that, and then let the kernel loader interpret it.
> > >
> > > Have you considered that?
> >
> > I've thought about alternative ways to achieve the same thing,
> > including something like the above. My concern with that approach is
> > that it's rather cumbersome and can be error-prone, and introduces
a
> > requirement for awkward multi-stage linking.
>
> A couple of years ago I did something very much like this to experiment
> with hot-patched static DTrace probes in the kernel.  My conclusion at
> the time was that even this kind of functionality (selective filtering
> of relocations as a post-processing step) was quite awkward and is much
> more easily implemented in the static linker.[*]  In this case, we need to
> modify the loadable segment descriptions, and we need to do some
> unwinding of the static linker's work when processing non-RELA
> relocations in the kernel.  It is all doable, but requires a significant
> amount of both userland and kernel code and a custom,
> FreeBSD-kernel-specific post-link step.
>
> [*] dtrace -G basically does this already.  See dt_link.c in libdtrace:
> it's 2000 LOC of complicated ELF handling, and it's required
numerous
> tweaks over the years to deal with changing behavioural quirks of GNU
> ld and lld.  r313262 is an example of this:
> https://svnweb.freebsd.org/changeset/base/313262
> The integration of dtrace(1) into the build systems of projects that
> wish to provide static probes is painful and often creates race
> conditions with incremental rebuilds.  Rafael and I discussed this
> several times and his general sentiment was that the functionality
> provided by dtrace -G can be implemented much more sanely in lld.
>
> > On 22 August 2018 at 09:20, Joerg Sonnenberger via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> > >
> > > On the linker side, it is a custom hack for
> > > something that is generally considered very bad nowadays: text
> > > relocations.
> >
> > True, although the argument against .text relocations doesn't hold
for
> > our kernel use case.
>
> Moreover, while being a hack, it is IMHO simply the right place to
> implement this logic from an architectural standpoint.  The static
> linker is pessimizing ifunc calls in a freestanding environment and it
> has all the information required to avoid doing this, and in a robust
> manner.  The small size of the required lld patch supports this view.
> Addressing the problem any other way involves adding much larger and
> more fragile hacks elsewhere.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180823/bf7ae5d0/attachment.html>

On Thu, Aug 23, 2018 at 03:14:33PM +0900, Rui Ueyama
wrote:
> In the context of support not only IFunc but DTrace, what kind of features
> do you want to add to lld, if you have a chance to implement it in the
> linker instead of a post-processing tool? I wonder if we can solve both of
> your problems.
I will try to explain what dtrace -G does first.  It is required to
support USDT, which is a dtrace feature that allows one to define static
tracepoints in a usermode application.  In C, the tracepoints look like
ordinary function calls (with programmer-defined parameters); dtrace
allows one to "tap" those tracepoints by overwriting them with a
breakpoint at runtime.  At build time, dtrace -G processes each
relocatable object file, and outputs an object file which must be linked
into the final output.  When processing the input object files, dtrace
looks for relocations against undefined symbols prefixed by
"__dtrace_";
each such relocation is a tracepoint.  It overwrites the corresponding
call with nops, changes the relocation type to R_[*]_NONE, and records
the tracepoint address, along with other information, in a metadata
section named ".SUNW_dof".  This metadata section is placed in the
output object file, which contains a constructor that registers the DOF
section with the kernel during application startup.

To me this functionality seems closely related to the -zifunc-noplt
option even if the details differ.  In both cases we really just
want the static linker to leave a certain set of relocations alone, and
pass them through to the output file.  (In the case of dtrace, the
symbols referenced by the relocations are undefined, and in the case of
ifunc-noplt the symbols are defined.)  A third item on my wishlist is
a way to dynamically disable retpolines during boot; having the set of
retpoline thunk calls available as relocations would make that
achievable in principle.

I haven't thought very much about how to extend lld to support my dtrace
use-case - the ifunc problem is simpler and more self-contained so I
started there.

Thank you very much for your explanation. That's very helpful.

It looks like even though DTrace and what you are trying to do with your
patch are conceptually similar, they are quite different in details, so I
can't think of a feature that can be used for both without depending on
other post-processing tools. Maybe it is better to create each feature
directly instead of creating something too generic.

I think  my concern about this patch is the cost of maintenance. The
feature will have a very small number of users (perhaps only FreeBSD), so
the cost of maintenance is relatively high even thought the feature is
tiny. I think I'm fine to accept this patch if you explicitly mark this as
an experimental feature that might be removed in a feature release of lld
if we find a better solution.

On Sat, Aug 25, 2018 at 5:05 AM Mark Johnston <markj at freebsd.org>
wrote:
> On Thu, Aug 23, 2018 at 03:14:33PM +0900, Rui Ueyama wrote:
> > In the context of support not only IFunc but DTrace, what kind of
> features
> > do you want to add to lld, if you have a chance to implement it in the
> > linker instead of a post-processing tool? I wonder if we can solve
both
> of
> > your problems.
>
> I will try to explain what dtrace -G does first.  It is required to
> support USDT, which is a dtrace feature that allows one to define static
> tracepoints in a usermode application.  In C, the tracepoints look like
> ordinary function calls (with programmer-defined parameters); dtrace
> allows one to "tap" those tracepoints by overwriting them with a
> breakpoint at runtime.  At build time, dtrace -G processes each
> relocatable object file, and outputs an object file which must be linked
> into the final output.  When processing the input object files, dtrace
> looks for relocations against undefined symbols prefixed by
"__dtrace_";
> each such relocation is a tracepoint.  It overwrites the corresponding
> call with nops, changes the relocation type to R_[*]_NONE, and records
> the tracepoint address, along with other information, in a metadata
> section named ".SUNW_dof".  This metadata section is placed in
the
> output object file, which contains a constructor that registers the DOF
> section with the kernel during application startup.
>
> To me this functionality seems closely related to the -zifunc-noplt
> option even if the details differ.  In both cases we really just
> want the static linker to leave a certain set of relocations alone, and
> pass them through to the output file.  (In the case of dtrace, the
> symbols referenced by the relocations are undefined, and in the case of
> ifunc-noplt the symbols are defined.)  A third item on my wishlist is
> a way to dynamically disable retpolines during boot; having the set of
> retpoline thunk calls available as relocations would make that
> achievable in principle.
>
> I haven't thought very much about how to extend lld to support my
dtrace
> use-case - the ifunc problem is simpler and more self-contained so I
> started there.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180827/cfe63f91/attachment.html>