llvm dev - Jan 2018 - Linker Option support for ELF

> On Jan 7, 2018, at 5:02 PM, Cary Coutant <ccoutant at gmail.com>
wrote:
> 
>> I think we all agree that blindly allowing the linker to honor the
options
>> would be scary.  I agree that we should whitelist the options, and am
of the
>> opinion that we should force validation on the linker side (use of any
>> option which the linker doesn't support in this form can be fatal).
>> Starting small is the best way, with `-l` and `-L` as a starting point.
I
>> want to retain the ability to add additional options which may not be
>> available in all linkers.  However, whitelisting obviously requires
working
>> with the linker as would adding such options, so that could be handled
at
>> that time.
> 
> This is actually why I'd prefer a new "language" over just
> whitelisting options. With "lib", "file", and
"path", as I suggested,
> there's no question whether an option like "-no-pie" is
supported, and
> no temptation to even try. The new language should be tailored for
> process-to-process communication, rather than user-to-shell
> communication.
I suppose I am slightly confused about what you are proposing here now.  From
the user side, it would be something to that effect.  Im suggesting that we take
lib and transform it to a different representation in the frontend.  Taking an
explicit example:

The user input
`#pragma comment(lib, “m”)`

would get transformed to `-lm` in the object file encoding.  However, this would
still not permit you from injecting any arbitrary options, but the backend
doesn’t change for any new option, only the frontend and the linker.
>> I’m thinking about future enhancements.  MachO does actually provide
>> something like `-L` -`l` in a single go via `-framework`.  But, no such
>> option exists for ELF since it doesn’t have the concept of framework
bundles
>> (but the layout itself is interesting), and I just want to try to keep
the
>> door open for such features.
> 
> This is why I also included "path" in my suggestion. I imagine
> something very much like -framework, where include files and library
> search paths are handled together.
> 
> -cary

By my understanding, the proposal is that the user input looks something
like:

#pragma linker_directive("lib", "m")

which is passed essentially as is to the linker, i.e. the directive payload
is a pair of strings, the first being "lib", the second "m".
It is then up
to the linker to decide if it supports "lib" directives, and translate
that
into the corresponding command, or do something else if it doesn't support
it, e.g. emit an error or ignore it. Neither the backend nor the frontend
need to do any conversion, and it makes it much clearer that these aren't
like any other command-line option (due to the limited set that can be
passed through, and the possible differences in behaviour, such as order on
the command-line).

As an aside, this approach would aid portability to linkers that aren't
command-line compatible with one another. If, for example, the switch is
"-lib" on linker A, and "-l" on linker B, the user
doesn't have to have
different source code for each different linker their object is to be
consumed by, nor does the compiler have to know about the different
possible linkers.

James

On 9 January 2018 at 05:15, Saleem Abdulrasool <compnerd at compnerd.org>
wrote:
>
>
> > On Jan 7, 2018, at 5:02 PM, Cary Coutant <ccoutant at gmail.com>
wrote:
> >
> >> I think we all agree that blindly allowing the linker to honor the
> options
> >> would be scary.  I agree that we should whitelist the options, and
am
> of the
> >> opinion that we should force validation on the linker side (use of
any
> >> option which the linker doesn't support in this form can be
fatal).
> >> Starting small is the best way, with `-l` and `-L` as a starting
> point.  I
> >> want to retain the ability to add additional options which may not
be
> >> available in all linkers.  However, whitelisting obviously
requires
> working
> >> with the linker as would adding such options, so that could be
handled
> at
> >> that time.
> >
> > This is actually why I'd prefer a new "language" over
just
> > whitelisting options. With "lib", "file", and
"path", as I suggested,
> > there's no question whether an option like "-no-pie" is
supported, and
> > no temptation to even try. The new language should be tailored for
> > process-to-process communication, rather than user-to-shell
> > communication.
>
> I suppose I am slightly confused about what you are proposing here now.
> From the user side, it would be something to that effect.  Im suggesting
> that we take lib and transform it to a different representation in the
> frontend.  Taking an explicit example:
>
> The user input
> `#pragma comment(lib, “m”)`
>
> would get transformed to `-lm` in the object file encoding.  However, this
> would still not permit you from injecting any arbitrary options, but the
> backend doesn’t change for any new option, only the frontend and the
linker.
>
> >> I’m thinking about future enhancements.  MachO does actually
provide
> >> something like `-L` -`l` in a single go via `-framework`.  But, no
such
> >> option exists for ELF since it doesn’t have the concept of
framework
> bundles
> >> (but the layout itself is interesting), and I just want to try to
keep
> the
> >> door open for such features.
> >
> > This is why I also included "path" in my suggestion. I
imagine
> > something very much like -framework, where include files and library
> > search paths are handled together.
> >
> > -cary
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180109/8c813986/attachment.html>

James Henderson via llvm-dev <llvm-dev at lists.llvm.org> writes:
> By my understanding, the proposal is that the user input looks something
> like:
>
> #pragma linker_directive("lib", "m")
>
> which is passed essentially as is to the linker, i.e. the directive payload
> is a pair of strings, the first being "lib", the second
"m". It is then up
> to the linker to decide if it supports "lib" directives, and
translate that
> into the corresponding command, or do something else if it doesn't
support
> it, e.g. emit an error or ignore it. Neither the backend nor the frontend
> need to do any conversion, and it makes it much clearer that these
aren't
> like any other command-line option (due to the limited set that can be
> passed through, and the possible differences in behaviour, such as order on
> the command-line).
This is exactly my understanding. I really like that the proposal is
transparent to clang and llvm and avoids any confusion with command line
options.

Cheers,
Rafael

> On Jan 9, 2018, at 2:32 AM, James Henderson <jh7370.2008 at
my.bristol.ac.uk> wrote:
> 
> By my understanding, the proposal is that the user input looks something
like:
> 
> #pragma linker_directive("lib", "m")
> 
> which is passed essentially as is to the linker, i.e. the directive payload
is a pair of strings, the first being "lib", the second "m".
It is then up to the linker to decide if it supports "lib" directives,
and translate that into the corresponding command, or do something else if it
doesn't support it, e.g. emit an error or ignore it. Neither the backend nor
the frontend need to do any conversion, and it makes it much clearer that these
aren't like any other command-line option (due to the limited set that can
be passed through, and the possible differences in behaviour, such as order on
the command-line).
> 
> As an aside, this approach would aid portability to linkers that aren't
command-line compatible with one another. If, for example, the switch is
"-lib" on linker A, and "-l" on linker B, the user
doesn't have to have different source code for each different linker their
object is to be consumed by, nor does the compiler have to know about the
different possible linkers.
Okay, that seems reasonable enough.  I think I still like the simplicity of
having it just be like a response file, but, this approach isn’t too bad for
adding custom extensions to.  I do like that it gives greater portability to
different linkers.

The one last bit that still remains unanswered is the encoding.  Do we go
forward with the abuse of the ELF notes or do is there enough confidence that
adding a new section type shouldn’t break compatibility with older linkers?
> James
> 
> On 9 January 2018 at 05:15, Saleem Abdulrasool <compnerd at compnerd.org
<mailto:compnerd at compnerd.org>> wrote:
> 
> 
> > On Jan 7, 2018, at 5:02 PM, Cary Coutant <ccoutant at gmail.com
<mailto:ccoutant at gmail.com>> wrote:
> >
> >> I think we all agree that blindly allowing the linker to honor the
options
> >> would be scary.  I agree that we should whitelist the options, and
am of the
> >> opinion that we should force validation on the linker side (use of
any
> >> option which the linker doesn't support in this form can be
fatal).
> >> Starting small is the best way, with `-l` and `-L` as a starting
point.  I
> >> want to retain the ability to add additional options which may not
be
> >> available in all linkers.  However, whitelisting obviously
requires working
> >> with the linker as would adding such options, so that could be
handled at
> >> that time.
> >
> > This is actually why I'd prefer a new "language" over
just
> > whitelisting options. With "lib", "file", and
"path", as I suggested,
> > there's no question whether an option like "-no-pie" is
supported, and
> > no temptation to even try. The new language should be tailored for
> > process-to-process communication, rather than user-to-shell
> > communication.
> 
> I suppose I am slightly confused about what you are proposing here now. 
From the user side, it would be something to that effect.  Im suggesting that we
take lib and transform it to a different representation in the frontend.  Taking
an explicit example:
> 
> The user input
> `#pragma comment(lib, “m”)`
> 
> would get transformed to `-lm` in the object file encoding.  However, this
would still not permit you from injecting any arbitrary options, but the backend
doesn’t change for any new option, only the frontend and the linker.
> 
> >> I’m thinking about future enhancements.  MachO does actually
provide
> >> something like `-L` -`l` in a single go via `-framework`.  But, no
such
> >> option exists for ELF since it doesn’t have the concept of
framework bundles
> >> (but the layout itself is interesting), and I just want to try to
keep the
> >> door open for such features.
> >
> > This is why I also included "path" in my suggestion. I
imagine
> > something very much like -framework, where include files and library
> > search paths are handled together.
> >
> > -cary
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180109/e831ac75/attachment.html>