llvm dev - Mar 2017 - Please dogfood LLD

Michael Johnson <mpj at rowley.co.uk> writes:
> Hi Rafael,
>> Michael Johnson via llvm-dev <llvm-dev at lists.llvm.org> writes:
>>
>>> Hi Rui,
>>>
>>> Are there any plans to support the -defsym command line option?
>> It doesn't look that hard, it was just never requested. What
project is
>> using it?
> Not sure I understand what the project set is. It's not an uncommon 
> feature for a linker in the embedded world.
>
> It's the first problem I discovered when trying to use lld rather than 
> ld e.g.
>
> -defsym=__vfprintf=__vfprintf_float_long_long
>
> to select a particular implementation of printf.
>
> The second problem was failing to parse the .ld script - I can provide 
> it if required.
If you could open bugs for both that would be awesome.

Thanks,
Rafael

As to the Phoronix benchmark results, I tried linking fftw as an example
and got a different result. In my test, ld.bfd, ld.gold and ld.lld are all
on par, and that is reasonable because compile time is the dominant factor
of building fftw. In the Phoronix tests, LLD was slower than the GNU
linkers when linking fftw, and I don't know why. The Phoronix tests
measured the entire build time (as opposed to the time that the linker
actually consumed), and it seems to me that that is too noisy. (Michael, if
you are watching this, you probably want to measure only the time spent by
the linker?)

Regarding the libtool issue, I'm inclined to accept the "GNU"
hack. That's
not really a bad hack. As Ed explained, that is not going to be fixed
easily as the code generated by libtool is copied to so many projects. If
we can convince libtool-generated script that we are a GNU compatible
linker just by adding "GNU" to somewhere in the --version message, it
sounds like a good deal. We can add a string "LLD is a GNU-compatible
linker" or "LLD accepts the same command line options as the GNU
linker" or
something like that to the --version string. I think we need this in
reality.

On Mon, Mar 20, 2017 at 7:17 AM, Rafael Avila de Espindola via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Michael Johnson <mpj at rowley.co.uk> writes:
>
> > Hi Rafael,
> >> Michael Johnson via llvm-dev <llvm-dev at lists.llvm.org>
writes:
> >>
> >>> Hi Rui,
> >>>
> >>> Are there any plans to support the -defsym command line
option?
> >> It doesn't look that hard, it was just never requested. What
project is
> >> using it?
> > Not sure I understand what the project set is. It's not an
uncommon
> > feature for a linker in the embedded world.
> >
> > It's the first problem I discovered when trying to use lld rather
than
> > ld e.g.
> >
> > -defsym=__vfprintf=__vfprintf_float_long_long
> >
> > to select a particular implementation of printf.
> >
> > The second problem was failing to parse the .ld script - I can provide
> > it if required.
>
> If you could open bugs for both that would be awesome.
>
> Thanks,
> Rafael
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170320/bf8af899/attachment.html>

Rui Ueyama <ruiu at google.com> writes:
> As to the Phoronix benchmark results, I tried linking fftw as an example
> and got a different result. In my test, ld.bfd, ld.gold and ld.lld are all
> on par, and that is reasonable because compile time is the dominant factor
> of building fftw. In the Phoronix tests, LLD was slower than the GNU
> linkers when linking fftw, and I don't know why. The Phoronix tests
> measured the entire build time (as opposed to the time that the linker
> actually consumed), and it seems to me that that is too noisy. (Michael, if
> you are watching this, you probably want to measure only the time spent by
> the linker?)
>
> Regarding the libtool issue, I'm inclined to accept the "GNU"
hack. That's
> not really a bad hack. As Ed explained, that is not going to be fixed
> easily as the code generated by libtool is copied to so many projects. If
> we can convince libtool-generated script that we are a GNU compatible
> linker just by adding "GNU" to somewhere in the --version
message, it
> sounds like a good deal. We can add a string "LLD is a GNU-compatible
> linker" or "LLD accepts the same command line options as the GNU
linker" or
> something like that to the --version string. I think we need this in
> reality.
I used to be against this, but I think I agree now.

My reasoning was that if we did this, it would make it harder to get
libtool changed. The change that libtool really needs is to have sane
defaults: if it doesn't know the linker, there is a good chance it has
gnu like command line options and supports shared libraries.

What changed is that I realized that probably the best way to cleanup a
project that is having problems with libtool is to stop using it. LLVM
did that a long time ago and is not any less portable as a result.

Cheers,
Rafael

On Mon, Mar 20, 2017 at 9:38 AM, Rui Ueyama via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> As to the Phoronix benchmark results, I tried linking fftw as an example
> and got a different result. In my test, ld.bfd, ld.gold and ld.lld are all
> on par, and that is reasonable because compile time is the dominant factor
> of building fftw. In the Phoronix tests, LLD was slower than the GNU
> linkers when linking fftw, and I don't know why. The Phoronix tests
> measured the entire build time (as opposed to the time that the linker
> actually consumed), and it seems to me that that is too noisy. (Michael, if
> you are watching this, you probably want to measure only the time spent by
> the linker?)
>
Unfortunately it is not very easy to test just the link time in a build (or
just the non-link time). I have had to do this in the past and there is no
universally applicable solution, so I wouldn't expect Phoronix (which does
high-level black-box testing) to do this. That being said, like you
mentioned, a small part of the overall (clean / full-rebuild) build time is
spent linking for most projects, so I agree that any significant changes in
the numbers by switching linkers is suspect (especially between LLD and
gold; I've seen BFD be "crazy slow" sometimes).

The main thing I can think of is that, for example, if the projects' build
system detects if the compiler/linker supports LTO (by actually properly
testing it, not like stockfish that pretty much hardcodes it, causing the
LLD build to fail in this batch of tests), then the LLD links might end up
with non-LTO builds and the gold/BFD builds could be doing LTO.

-- Sean Silva

>
> Regarding the libtool issue, I'm inclined to accept the "GNU"
hack. That's
> not really a bad hack. As Ed explained, that is not going to be fixed
> easily as the code generated by libtool is copied to so many projects. If
> we can convince libtool-generated script that we are a GNU compatible
> linker just by adding "GNU" to somewhere in the --version
message, it
> sounds like a good deal. We can add a string "LLD is a GNU-compatible
> linker" or "LLD accepts the same command line options as the GNU
linker" or
> something like that to the --version string. I think we need this in
> reality.
>
> On Mon, Mar 20, 2017 at 7:17 AM, Rafael Avila de Espindola via llvm-dev
<
> llvm-dev at lists.llvm.org> wrote:
>
>> Michael Johnson <mpj at rowley.co.uk> writes:
>>
>> > Hi Rafael,
>> >> Michael Johnson via llvm-dev <llvm-dev at
lists.llvm.org> writes:
>> >>
>> >>> Hi Rui,
>> >>>
>> >>> Are there any plans to support the -defsym command line
option?
>> >> It doesn't look that hard, it was just never requested.
What project is
>> >> using it?
>> > Not sure I understand what the project set is. It's not an
uncommon
>> > feature for a linker in the embedded world.
>> >
>> > It's the first problem I discovered when trying to use lld
rather than
>> > ld e.g.
>> >
>> > -defsym=__vfprintf=__vfprintf_float_long_long
>> >
>> > to select a particular implementation of printf.
>> >
>> > The second problem was failing to parse the .ld script - I can
provide
>> > it if required.
>>
>> If you could open bugs for both that would be awesome.
>>
>> Thanks,
>> Rafael
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170320/751373fd/attachment.html>