llvm dev - Jan 2018 - Linker Option support for ELF

> On Jan 6, 2018, at 6:55 PM, Rui Ueyama <ruiu at google.com> wrote:
> 
> On Sat, Jan 6, 2018 at 8:56 AM, Saleem Abdulrasool <compnerd at
compnerd.org <mailto:compnerd at compnerd.org>> wrote:
> 
> On Fri, Jan 5, 2018 at 2:30 AM Rui Ueyama <ruiu at google.com
<mailto:ruiu at google.com>> wrote:
> Thank you for starting the discussion thread.
> 
> In general I'm in favor of the proposal. Defining a generic way to
convey some information from the compiler to the linker is useful, and it looks
like it is just a historical reason that the ELF lacks the feature at the
moment.
> 
> This is a scenario in which the feature is useful: when you include math.h,
a compiler (which is driven by some pragma) could added `-lm` to the note
section so that a linker automatically links libm.
> 
> I think I'm also in favor of the format, which is essentially runs of
null-terminated strings (*1) that are basically opaque to compilers.
> 
> Yes.  However, I think I want to clarify that we want this to be completely
opaque to the backend.  The front end could possibly have some enhancements to
make this better.  But, that will be a separate change, and that discussion
should take place then.  We shouldn’t paint ourselves into a corner.  Basically,
I think that there is some legitimate concerns here, but they would not be
handled at this layer, but above.
> 
> However, you should define as a spec what options are allowed and what
their semantics are. We should not accept arbitrary linker options because
semantics of some linker options cannot be clearly defined when they appear as
embedded options. Just saying "this feature allows you to embed linker
options to object files" is too weak as a specification. You need to
clearly define a list of options that will be supported by linkers with their
clear semantics.
> 
> Personally, I would like to see the ability to add support for additional
options without having to modify the compiler.  That said, I think that there
are options which can be scary (e.g. -nopie).  I think that the linker should
make the decision of what it supports and error out on others.  This allows for
us to enhance the support over time without a huge overhead.  As a starting
point, I think that -l and -L are two that would be interesting.  I can see -u
being useful as well, but the point is that we can slowly grow the support after
consideration by delaying the validation of the options.
> 
> I think no one including me opposed the idea of designing the feature so
that new options can be added without having to modify the compiler. That's
fine to me.
Excellent.  I think that the TLV approach requires that to a certain extent (the
enumeration would be hardcoded).
> What I said is that you still have to specify a list of options that are
allowed in the "linker options" section in your *specification*. Your
specification is incomplete and underspecified if it doesn't explicitly list
linker options with their semantics. If you don't do that, people would
start using arbitrary linker options whose semantics are vague/that are
dangerous/that are simply wrong (e.g. unclosed `-whole-archive` option) after
you would implement the feature. We really shouldn't let that mess happen.
I think we all agree that blindly allowing the linker to honor the options would
be scary.  I agree that we should whitelist the options, and am of the opinion
that we should force validation on the linker side (use of any option which the
linker doesn't support in this form can be fatal).  Starting small is the
best way, with `-l` and `-L` as a starting point.  I want to retain the ability
to add additional options which may not be available in all linkers.  However,
whitelisting obviously requires working with the linker as would adding such
options, so that could be handled at that time.

Furthermore, note that the LLVM side of the implementation does not provide a
way to add anything from the user - that is a change to clang, and is something
that I intend to be a follow up change.  As I mentioned in a previous email, I
have my own preferences for how to handle those, but, I think that when we start
adding support to the frontend for it is the right time to discuss the syntax of
how to support that.  If we go with the approach of tokenizing the options and
doing a C-string per option, I think it leaves sufficient flexibility to permit
the frontend proper control to emit the necessary argument and not have the
linker parse the options having to worry about quoting and space handling which
I believe to be a good compromise on all sides.
> Concretely, which options do you want to use? I could imagine everyone
wants to use `-l`. `-L` is arguable because I do not see an obvious reason to
add a search path from the compiler. You mentioned `-u`. Is there any other flag
that you have in your mind?
I’m thinking about future enhancements.  MachO does actually provide something
like `-L` -`l` in a single go via `-framework`.  But, no such option exists for
ELF since it doesn’t have the concept of framework bundles (but the layout
itself is interesting), and I just want to try to keep the door open for such
features.
> 
> (*1) One of the big annoyances that I noticed when I was implementing the
same feature for COFF is that the COFF's .drctve section that contains
linker options have to be tokenized in the same way as the Windows command line
does. So it needs to interpret double quotes and backslashes correctly
especially when handling space-containing pathnames. This is a design failure
that a COFF file contains just a single string instead of runs of strings that
have already been tokenized.
> 
> I think that there is room for refinement on this.  The best part is that
the refinement for that is delayed!  It would be best done in the front end IMO,
and we can actually  further discuss and design the feature there.  I don’t
think that we need to be completely blind to the issues we have seen, but we
shouldn’t over constrain either.
> 
> On Thu, Jan 4, 2018 at 9:02 AM, Saleem Abdulrasool via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
> Hello all,
> 
> There was some interest from a number of a few people about adding support
for embedded linker options to ELF.  This would be an extension that requires
linker support to actually work, but has significant prior art with PE/COFF as
well as MachO both having support for this.
> 
> The desire here is to actually add support to LLVM to pass along the
necessary information into the object file.  In order to keep this focused on
that, this thread is specifically for the *backend*, we are not discussing how
to get the information to the backend here at all, but assuming that the
information is present in the same LLVM IR encoding (llvm.linker-options module
metadata string).
> 
> In order to have compatibility with existing linkers, I am suggesting the
use of an ELF note.  These are implicitly dropped by the linker so we can be
certain that the options will not end up in the final binary even if the
extension is not supported.  The payload would be a 4-byte version identifier
(to allow future enhancements) and a null-terminated string of options.
> 
> This allows for the backend to be entirely oblivious to the data as the
other backends and allows for extensions in the future without having to teach
the backend anything about the new functionality (again, something which both of
the other file formats support).
> 
> As an example of how this can be useful, it would help with Swift support
on Linux where currently the linker options are pushed into a custom section,
and a secondary program is used to extract the options from this section prior
to the linker being invoked.  This adds a lot of complexity to the driver as
well as additional tools being invoked in the build chain slowing down the
build.
> 
> -- 
> Saleem Abdulrasool
> compnerd (at) compnerd (dot) org
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> -- 
> Saleem Abdulrasool
> compnerd (at) compnerd (dot) org
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180107/a1adeeec/attachment-0001.html>

> I think we all agree that blindly allowing the linker to honor the options
> would be scary.  I agree that we should whitelist the options, and am of
the
> opinion that we should force validation on the linker side (use of any
> option which the linker doesn't support in this form can be fatal).
> Starting small is the best way, with `-l` and `-L` as a starting point.  I
> want to retain the ability to add additional options which may not be
> available in all linkers.  However, whitelisting obviously requires working
> with the linker as would adding such options, so that could be handled at
> that time.
This is actually why I'd prefer a new "language" over just
whitelisting options. With "lib", "file", and
"path", as I suggested,
there's no question whether an option like "-no-pie" is supported,
and
no temptation to even try. The new language should be tailored for
process-to-process communication, rather than user-to-shell
communication.
> I’m thinking about future enhancements.  MachO does actually provide
> something like `-L` -`l` in a single go via `-framework`.  But, no such
> option exists for ELF since it doesn’t have the concept of framework
bundles
> (but the layout itself is interesting), and I just want to try to keep the
> door open for such features.
This is why I also included "path" in my suggestion. I imagine
something very much like -framework, where include files and library
search paths are handled together.

-cary

On Jan 7, 2018 5:02 PM, "Cary Coutant via llvm-dev" <llvm-dev at
lists.llvm.org>
wrote:
> I think we all agree that blindly allowing the linker to honor the options
> would be scary.  I agree that we should whitelist the options, and am of
the
> opinion that we should force validation on the linker side (use of any
> option which the linker doesn't support in this form can be fatal).
> Starting small is the best way, with `-l` and `-L` as a starting point.  I
> want to retain the ability to add additional options which may not be
> available in all linkers.  However, whitelisting obviously requires
working
> with the linker as would adding such options, so that could be handled at
> that time.
This is actually why I'd prefer a new "language" over just
whitelisting options. With "lib", "file", and
"path", as I suggested,
there's no question whether an option like "-no-pie" is supported,
and
no temptation to even try.


That's a very simple and elegant solution.

-- Sean Silva


The new language should be tailored for
process-to-process communication, rather than user-to-shell
communication.
> I’m thinking about future enhancements.  MachO does actually provide
> something like `-L` -`l` in a single go via `-framework`.  But, no such
> option exists for ELF since it doesn’t have the concept of framework
bundles
> (but the layout itself is interesting), and I just want to try to keep the
> door open for such features.
This is why I also included "path" in my suggestion. I imagine
something very much like -framework, where include files and library
search paths are handled together.

-cary
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180107/b1f289a5/attachment.html>

I think it makes sense to allow specifying "-lNAME" in an ELF object
file,
the same way it makes sense on Apple and MS systems.

But I'm very much against allowing any sort of option that specifies full
paths to be embedded, including "-L". Typically, object files are
location-independent -- if you want to copy them to another system and run
the linker there (with an appropriate sysroot path), that's going to work
perfectly fine. But as soon as you start allowing path options to be
embedded, you're in a whole new world, with a whole new set of possible
pain. And, AFAICT, other systems do not allow that, either.

Other issues:

There will need to be a new linker option, "-ignore-auto-link" or
something
like that, for people who want to disable this feature.

Also, what happens with "-r" and "-Ur"? Probably that should
propagate the
options to the output .o file, instead of using them, but I'm not certain.
Maybe that needs a new option as well.

An additional complication that I think nobody has mentioned yet: what
happens with static libraries containing such object files? I'd hope that
linking against a .a which includes .o files with linker options, only adds
the options from a .o files _actually included_ in the link. That is true
of the Apple implementation, but I don't know about the MS one. But that
really needs to be specified.


A short survey of some existing implementations:

1. Apple

ld64 allows precisely two linker options to be embedded: "-lNAME" and
"-framework NAME". Note that -framework is almost identical to
"-l". It
simply looks for a library called "NAME.framework/NAME" in the
framework-search-path, instead of a library called libNAME.{a,so} in the
library-search-path.

Apple has an option called "-ignore_auto_link", to ignore these
embedded
options from object files.

2. MSVC

MS allows only a few options as well:
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp
- /DEFAULTLIB (equivalent to "-l")
- /EXPORT (export a symbol)
- /INCLUDE (equivalent to "-u")
- /MANIFESTDEPENDENCY (add data to manifest)
- /MERGE (merge two sections)
- /SECTION (specify attributes for sections)

MS has a linker option "/NODEFAULTLIB", which ignores any /DEFAULTLIB
options (from object files or the command-line). MS also allows you to
_selectively_ ignore certain libraries, with "/NODEFAULTLIB:name.lib".
Also
note that "/NODEFAULTLIB" doesn't ignore ALL the options specified
in an
object files, only the additional libraries to link against.

On Sun, Jan 7, 2018 at 2:52 PM, Saleem Abdulrasool via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Jan 6, 2018, at 6:55 PM, Rui Ueyama <ruiu at google.com> wrote:
>
> On Sat, Jan 6, 2018 at 8:56 AM, Saleem Abdulrasool <compnerd at
compnerd.org>
>  wrote:
>
>>
>> On Fri, Jan 5, 2018 at 2:30 AM Rui Ueyama <ruiu at google.com>
wrote:
>>
>>> Thank you for starting the discussion thread.
>>>
>>> In general I'm in favor of the proposal. Defining a generic way
to
>>> convey some information from the compiler to the linker is useful,
and it
>>> looks like it is just a historical reason that the ELF lacks the
feature at
>>> the moment.
>>>
>>> This is a scenario in which the feature is useful: when you include
>>> math.h, a compiler (which is driven by some pragma) could added
`-lm` to
>>> the note section so that a linker automatically links libm.
>>>
>>> I think I'm also in favor of the format, which is essentially
runs of
>>> null-terminated strings (*1) that are basically opaque to
compilers.
>>>
>>
>> Yes.  However, I think I want to clarify that we want this to be
>> completely opaque to the backend.  The front end could possibly have
some
>> enhancements to make this better.  But, that will be a separate change,
and
>> that discussion should take place then.  We shouldn’t paint ourselves
into
>> a corner.  Basically, I think that there is some legitimate concerns
here,
>> but they would not be handled at this layer, but above.
>>
>> However, you should define as a spec what options are allowed and what
>>> their semantics are. We should not accept arbitrary linker options
because
>>> semantics of some linker options cannot be clearly defined when
they appear
>>> as embedded options. Just saying "this feature allows you to
embed linker
>>> options to object files" is too weak as a specification. You
need to
>>> clearly define a list of options that will be supported by linkers
with
>>> their clear semantics.
>>>
>>
>> Personally, I would like to see the ability to add support for
additional
>> options without having to modify the compiler.  That said, I think that
>> there are options which can be scary (e.g. -nopie).  I think that the
>> linker should make the decision of what it supports and error out on
>> others.  This allows for us to enhance the support over time without a
huge
>> overhead.  As a starting point, I think that -l and -L are two that
would
>> be interesting.  I can see -u being useful as well, but the point is
that
>> we can slowly grow the support after consideration by delaying the
>> validation of the options.
>>
>
> I think no one including me opposed the idea of designing the feature so
> that new options can be added without having to modify the compiler.
That's
> fine to me.
>
>
> Excellent.  I think that the TLV approach requires that to a certain
> extent (the enumeration would be hardcoded).
>
> What I said is that you still have to specify a list of options that are
> allowed in the "linker options" section in your *specification*.
Your
> specification is incomplete and underspecified if it doesn't explicitly
> list linker options with their semantics. If you don't do that, people
> would start using arbitrary linker options whose semantics are vague/that
> are dangerous/that are simply wrong (e.g. unclosed `-whole-archive` option)
>  after you would implement the feature. We really shouldn't let that
mess
> happen.
>
>
> I think we all agree that blindly allowing the linker to honor the options
> would be scary.  I agree that we should whitelist the options, and am of
> the opinion that we should force validation on the linker side (use of any
> option which the linker doesn't support in this form can be fatal).
> Starting small is the best way, with `-l` and `-L` as a starting point.  I
> want to retain the ability to add additional options which may not be
> available in all linkers.  However, whitelisting obviously requires working
> with the linker as would adding such options, so that could be handled at
> that time.
>
> Furthermore, note that the LLVM side of the implementation does not
> provide a way to add anything from the user - that is a change to clang,
> and is something that I intend to be a follow up change.  As I mentioned in
> a previous email, I have my own preferences for how to handle those, but, I
> think that when we start adding support to the frontend for it is the right
> time to discuss the syntax of how to support that.  If we go with the
> approach of tokenizing the options and doing a C-string per option, I think
> it leaves sufficient flexibility to permit the frontend proper control to
> emit the necessary argument and not have the linker parse the options
> having to worry about quoting and space handling which I believe to be a
> good compromise on all sides.
>
> Concretely, which options do you want to use? I could imagine everyone
> wants to use `-l`. `-L` is arguable because I do not see an obvious reason
> to add a search path from the compiler. You mentioned `-u`. Is there any
> other flag that you have in your mind?
>
>
> I’m thinking about future enhancements.  MachO does actually provide
> something like `-L` -`l` in a single go via `-framework`.  But, no such
> option exists for ELF since it doesn’t have the concept of framework
> bundles (but the layout itself is interesting), and I just want to try to
> keep the door open for such features.
>
>
> (*1) One of the big annoyances that I noticed when I was implementing the
>>> same feature for COFF is that the COFF's .drctve section that
contains
>>> linker options have to be tokenized in the same way as the Windows
command
>>> line does. So it needs to interpret double quotes and backslashes
correctly
>>> especially when handling space-containing pathnames. This is a
design
>>> failure that a COFF file contains just a single string instead of
runs of
>>> strings that have already been tokenized.
>>>
>>
>> I think that there is room for refinement on this.  The best part is
that
>> the refinement for that is delayed!  It would be best done in the front
end
>> IMO, and we can actually  further discuss and design the feature there.
I
>> don’t think that we need to be completely blind to the issues we have
seen,
>> but we shouldn’t over constrain either.
>>
>> On Thu, Jan 4, 2018 at 9:02 AM, Saleem Abdulrasool via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> Hello all,
>>>>
>>>> There was some interest from a number of a few people about
adding
>>>> support for embedded linker options to ELF.  This would be an
extension
>>>> that requires linker support to actually work, but has
significant prior
>>>> art with PE/COFF as well as MachO both having support for this.
>>>>
>>>> The desire here is to actually add support to LLVM to pass
along the
>>>> necessary information into the object file.  In order to keep
this focused
>>>> on that, this thread is specifically for the *backend*, we are
not
>>>> discussing how to get the information to the backend here at
all, but
>>>> assuming that the information is present in the same LLVM IR
encoding
>>>> (llvm.linker-options module metadata string).
>>>>
>>>> In order to have compatibility with existing linkers, I am
suggesting
>>>> the use of an ELF note.  These are implicitly dropped by the
linker so we
>>>> can be certain that the options will not end up in the final
binary even if
>>>> the extension is not supported.  The payload would be a 4-byte
version
>>>> identifier (to allow future enhancements) and a null-terminated
string of
>>>> options.
>>>>
>>>> This allows for the backend to be entirely oblivious to the
data as the
>>>> other backends and allows for extensions in the future without
having to
>>>> teach the backend anything about the new functionality (again,
something
>>>> which both of the other file formats support).
>>>>
>>>> As an example of how this can be useful, it would help with
Swift
>>>> support on Linux where currently the linker options are pushed
into a
>>>> custom section, and a secondary program is used to extract the
options from
>>>> this section prior to the linker being invoked.  This adds a
lot of
>>>> complexity to the driver as well as additional tools being
invoked in the
>>>> build chain slowing down the build.
>>>>
>>>> --
>>>> Saleem Abdulrasool
>>>> compnerd (at) compnerd (dot) org
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>> --
>> Saleem Abdulrasool
>> compnerd (at) compnerd (dot) org
>>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180108/dadea154/attachment.html>

On Mon, Jan 08, 2018 at 12:14:15AM -0500, James Y Knight via llvm-dev
wrote:
> But I'm very much against allowing any sort of option that specifies
full
> paths to be embedded, including "-L". Typically, object files are
> location-independent -- if you want to copy them to another system and run
> the linker there (with an appropriate sysroot path), that's going to
work
> perfectly fine. But as soon as you start allowing path options to be
> embedded, you're in a whole new world, with a whole new set of possible
> pain. And, AFAICT, other systems do not allow that, either.
It can be also very painful for deterministic builds. GCC wants to
include the command line via DWARF nowadays. Including the option to
make the path names deterministic. Guess what happens...

Joerg

Cary Coutant via llvm-dev <llvm-dev at lists.llvm.org> writes:
>> I think we all agree that blindly allowing the linker to honor the
options
>> would be scary.  I agree that we should whitelist the options, and am
of the
>> opinion that we should force validation on the linker side (use of any
>> option which the linker doesn't support in this form can be fatal).
>> Starting small is the best way, with `-l` and `-L` as a starting point.
I
>> want to retain the ability to add additional options which may not be
>> available in all linkers.  However, whitelisting obviously requires
working
>> with the linker as would adding such options, so that could be handled
at
>> that time.
>
> This is actually why I'd prefer a new "language" over just
> whitelisting options. With "lib", "file", and
"path", as I suggested,
> there's no question whether an option like "-no-pie" is
supported, and
> no temptation to even try. The new language should be tailored for
> process-to-process communication, rather than user-to-shell
> communication.
I agree. This seems to have the two main desirable features

* The compiler/llvm can handle the name/value pairs as black boxes.
* There is no risk of blindly adding an option to the whitelist.

Cheers,
Rafael

> On Jan 7, 2018, at 5:02 PM, Cary Coutant <ccoutant at gmail.com>
wrote:
> 
>> I think we all agree that blindly allowing the linker to honor the
options
>> would be scary.  I agree that we should whitelist the options, and am
of the
>> opinion that we should force validation on the linker side (use of any
>> option which the linker doesn't support in this form can be fatal).
>> Starting small is the best way, with `-l` and `-L` as a starting point.
I
>> want to retain the ability to add additional options which may not be
>> available in all linkers.  However, whitelisting obviously requires
working
>> with the linker as would adding such options, so that could be handled
at
>> that time.
> 
> This is actually why I'd prefer a new "language" over just
> whitelisting options. With "lib", "file", and
"path", as I suggested,
> there's no question whether an option like "-no-pie" is
supported, and
> no temptation to even try. The new language should be tailored for
> process-to-process communication, rather than user-to-shell
> communication.
I suppose I am slightly confused about what you are proposing here now.  From
the user side, it would be something to that effect.  Im suggesting that we take
lib and transform it to a different representation in the frontend.  Taking an
explicit example:

The user input
`#pragma comment(lib, “m”)`

would get transformed to `-lm` in the object file encoding.  However, this would
still not permit you from injecting any arbitrary options, but the backend
doesn’t change for any new option, only the frontend and the linker.
>> I’m thinking about future enhancements.  MachO does actually provide
>> something like `-L` -`l` in a single go via `-framework`.  But, no such
>> option exists for ELF since it doesn’t have the concept of framework
bundles
>> (but the layout itself is interesting), and I just want to try to keep
the
>> door open for such features.
> 
> This is why I also included "path" in my suggestion. I imagine
> something very much like -framework, where include files and library
> search paths are handled together.
> 
> -cary

> But I'm very much against allowing any sort of option that specifies
full
> paths to be embedded, including "-L". Typically, object files are
> location-independent -- if you want to copy them to another system and run
> the linker there (with an appropriate sysroot path), that's going to
work
> perfectly fine. But as soon as you start allowing path options to be
> embedded, you're in a whole new world, with a whole new set of possible
> pain. And, AFAICT, other systems do not allow that, either.
While this is true in some development environments, in others, the
ability to embed a path can be very useful. It's common to have
third-party libraries installed in locations that aren't in the
default search path, but installed in the same location on each build
machine.

We already do the same for dynamic linking with RPATH, which reminds
me that we'll probably want to support an "rpath" option as well.

-cary