llvm dev - Oct 2016 - [cfe-dev] LLD to be the default linker in Clang

On 28 October 2016 at 18:12, Ed Maste <emaste at freebsd.org>
wrote:
> It should be possible to search the path for ld.lld first, then ld,
> but to me it seems like it will just be more confusing.
Hum, for me it would be less confusing. :)

GCC uses bfd by default, LLVM uses LLD. If you want to change, use -fuse-ld.

What would be confusing in this scenario?

> Clang's current approach (ld from $PATH by default, or ld.arg from
> -fuse-ld=arg) suits us in FreeBSD. Today we have:
>     /usr/bin/ld.bfd (GNU ld 2.17.50)
>     /usr/bin/ld (ld.bfd symlink)
>
> We'll soon add
>     /usr/bin/ld.lld
This seems like a simple enough approach. I guess my problem is more
about building toolchains than shipping them.

> The only downside of -fuse-ld= that I'm aware of affects GCC only, not
> Clang: GCC accepts only -fuse-ld=bfd and -fuse-ld=gold, and Davide's
> patch to add -fuse-ld=lld was met with hostility. But that shouldn't
> have any effect on Clang.
Just had a look at the thread. Some silly comments were expected, but
I think Davide has withdrawn his patch too early.

AFAIK, there's absolutely nothing in the GCC license, moto and
copyright statements that forbid the usage of a non-GNU linker, and
the argument "lld has bugs" is pointless.

I would try again.

cheers,
--renato

On Fri, Oct 28, 2016 at 10:38 AM, Renato Golin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 28 October 2016 at 18:12, Ed Maste <emaste at freebsd.org> wrote:
> > It should be possible to search the path for ld.lld first, then ld,
> > but to me it seems like it will just be more confusing.
>
> Hum, for me it would be less confusing. :)
>
> GCC uses bfd by default, LLVM uses LLD. If you want to change, use
> -fuse-ld.
>
> What would be confusing in this scenario?
>
In practice, I think your proposal can be read for most people as a
proposal to do dogfooding LLD more widely in LLVM by making it the default
linker. I've been using LLD as default for a long period of time (probably
a year) and saw no problem. So it should be doable. Do people okay with
that? Probably, as long as it works, no one would really care, maybe?

>
> > Clang's current approach (ld from $PATH by default, or ld.arg from
> > -fuse-ld=arg) suits us in FreeBSD. Today we have:
> >     /usr/bin/ld.bfd (GNU ld 2.17.50)
> >     /usr/bin/ld (ld.bfd symlink)
> >
> > We'll soon add
> >     /usr/bin/ld.lld
>
> This seems like a simple enough approach. I guess my problem is more
> about building toolchains than shipping them.
>
>
> > The only downside of -fuse-ld= that I'm aware of affects GCC only,
not
> > Clang: GCC accepts only -fuse-ld=bfd and -fuse-ld=gold, and
Davide's
> > patch to add -fuse-ld=lld was met with hostility. But that
shouldn't
> > have any effect on Clang.
>
> Just had a look at the thread. Some silly comments were expected, but
> I think Davide has withdrawn his patch too early.
>
> AFAIK, there's absolutely nothing in the GCC license, moto and
> copyright statements that forbid the usage of a non-GNU linker, and
> the argument "lld has bugs" is pointless.
>
> I would try again.
>
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161028/783a6350/attachment.html>

On 28 October 2016 at 18:53, Rui Ueyama <ruiu at google.com>
wrote:
> In practice, I think your proposal can be read for most people as a
proposal
> to do dogfooding LLD more widely in LLVM by making it the default linker.
> I've been using LLD as default for a long period of time (probably a
year)
> and saw no problem. So it should be doable. Do people okay with that?
> Probably, as long as it works, no one would really care, maybe?
Dog-feeding and keeping linkers honest. Yes.

Just like Clang made C++ compilers honest and IAS made assemblers
honest. It's about time we do that to linkers.

With the integrated assembler, as soon as it was good quality enough
in one arch, we turned on by default and asked people to report a bug
and meanwhile use "-no-integrated-as" in their flags.

As we migrated the ARM back-end to use IAS, we decided to change it by
default even though a few programs didn't compile entirely (mostly due
to GNU extensions). We still have some programs that don't compile
with IAS, and we're working on the bugs that we see in bugzilla, but
they're by far the exception. Most of the remaining cases were fixed
in the user's code.

The AArch64 port is finishing its stage 1, aka. bootstrap. The
buildbot I'm setting up it to prove it works as well as make sure we
don't regress. From there, test-suite, then chromium and firefox are
the next targets.

The ARM port is work in progress, but will follow a similar path, and
once it's working well, we'll want to make it the default in Clang.

So, we need to make sure the "does it work" flag is correct. There are
two modes, native and cross, and they can have different values at
different times. We'll have to validate them separately.

To begin with, I would set LLD the default on native x86_64 only,
because that's what gets tested the most nowadays. As soon as AArch64
bootstraps and passes the test-suite, we can do the same for native,
but not cross. Same for ARM.

The good news is that when you "apt-get install clang", you don't
get
LLD. So, unless people are installing LLD in their systems explicitly,
or are building LLD from sources with Clang, they will *not* use LLD.

The only confusion, maybe, is in FreeBSD if they have all linkers
installed but still don't want to use LLD as the default for Clang. It
should be trivial to change that with a one-line patch, in the Clang
driver, though. At least until they have validated enough.

cheers,
--renato

On 28 Oct 2016, at 18:38, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> GCC uses bfd by default, LLVM uses LLD. If you want to change, use
-fuse-ld.
GCC doesn’t use BFD by default, it uses /usr/bin/ld (or ${PREFIX}/bin/ld).  If
that is a symlink to ld.bfd, ld.gold, Apple ld64, or ld.lld, it will use
whichever.  I think it would be very confusing for clang to use a linker that is
not whatever the host system has decided the default linker should be, unless
there is some compelling reason to pick a different one (e.g. you’re doing LTO
with a mechanism that absolutely requires lld and not ld64/gold).

David

On 28 October 2016 at 19:52, David Chisnall <David.Chisnall at
cl.cam.ac.uk> wrote:
> GCC doesn’t use BFD by default, it uses /usr/bin/ld (or ${PREFIX}/bin/ld). 
If that is a symlink to ld.bfd, ld.gold, Apple ld64, or ld.lld, it will use
whichever.
Hum, that's actually a good point.

> I think it would be very confusing for clang to use a linker that is not
whatever the host system has decided the default linker should be, unless there
is some compelling reason to pick a different one (e.g. you’re doing LTO with a
mechanism that absolutely requires lld and not ld64/gold).
If you build GCC, and binutils, "ld" is on the path, and it all
bootstraps without glitches. If we build Clang and LLD, "lld" is on
the path, but Clang still picks the system's "ld", which is weird.

So, let's say we don't make it the default (because of your argument
above), what would be the steps to make bootstrap transparent?

Today, we can:
 1. Use -DLLVM_ENABLE_LLD=True, but that only works for tests and
bootstrap (right?)
 2. Use "--cflags -fuse-ld=lld" for LNT, so we can run the test suite
(I'm assuming this works, too)
 3. Add an "ld" symlink into $BUILD/bin, together with the already
existing "ld.lld" and "lld-link" and put $BUILD/bin before
/usr/bin on
the $PATH

They all seem kosher enough, so I'd be ok with any of them, but
they're all *additional* steps, not the default.

So maybe, a better approach would be to make LLD the default in our
own CMake scripts, via (1) above, not in the Clang driver, IFF the LLD
project is built-in.

Setting (3) above would make the test-suite transparent, too. Again,
IFF LLD is built-in.

Makes sense?

--renato

On Fri, Oct 28, 2016 at 10:38 AM, Renato Golin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 28 October 2016 at 18:12, Ed Maste <emaste at freebsd.org> wrote:
> > It should be possible to search the path for ld.lld first, then ld,
> > but to me it seems like it will just be more confusing.
>
> Hum, for me it would be less confusing. :)
>
> GCC uses bfd by default, LLVM uses LLD. If you want to change, use
> -fuse-ld.
>
I don't think that's a fair comparison. GCC uses the system linker by
default (whatever 'ld' happens to name). If ld is a symlink to lld, then
GCC uses lld by default.

That's actually part of a much larger pattern: Clang is currently set up to
act as a drop-in replacement for the system compiler. That means we use
libstdc++ (not libc++) by default on targets where it's the default C++
standard library, we use libgcc_s (not compiler-rt) by default on targets
where it's the default compiler runtime, and so on. That strategy has
worked well in getting us to where we are today.

But our needs today aren't the same as they were a few years ago; we
don't
need to prove ourselves as much as we used to, and while we should keep
supporting the target defaults for the above components, perhaps it's time
that we start to prefer using LLVM components where available. At the very
least, I don't see a good reason why we would ever want to use libgcc_s in
a situation where compiler-rt (and libunwind) are available -- libgcc_s
does not contain some functions that LLVM implicitly adds calls to.

Perhaps we should have a flag to specify whether we prefer the canonical
tools and components for the target, or whether we prefer LLVM's versions
when available (falling back to the target components if not)?

What would be confusing in this scenario?
>
>
> > Clang's current approach (ld from $PATH by default, or ld.arg from
> > -fuse-ld=arg) suits us in FreeBSD. Today we have:
> >     /usr/bin/ld.bfd (GNU ld 2.17.50)
> >     /usr/bin/ld (ld.bfd symlink)
> >
> > We'll soon add
> >     /usr/bin/ld.lld
>
> This seems like a simple enough approach. I guess my problem is more
> about building toolchains than shipping them.
>
>
> > The only downside of -fuse-ld= that I'm aware of affects GCC only,
not
> > Clang: GCC accepts only -fuse-ld=bfd and -fuse-ld=gold, and
Davide's
> > patch to add -fuse-ld=lld was met with hostility. But that
shouldn't
> > have any effect on Clang.
>
> Just had a look at the thread. Some silly comments were expected, but
> I think Davide has withdrawn his patch too early.
>
> AFAIK, there's absolutely nothing in the GCC license, moto and
> copyright statements that forbid the usage of a non-GNU linker, and
> the argument "lld has bugs" is pointless.
>
> I would try again.
>
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161028/f1d38a99/attachment.html>

On 28 October 2016 at 20:35, Richard Smith <richard at metafoo.co.uk>
wrote:
> That's actually part of a much larger pattern: Clang is currently set
up to
> act as a drop-in replacement for the system compiler. That means we use
> libstdc++ (not libc++) by default on targets where it's the default C++
> standard library, we use libgcc_s (not compiler-rt) by default on targets
> where it's the default compiler runtime, and so on. That strategy has
worked
> well in getting us to where we are today.
Agreed! And I still need this very much to work.

> But our needs today aren't the same as they were a few years ago; we
don't
> need to prove ourselves as much as we used to, and while we should keep
> supporting the target defaults for the above components, perhaps it's
time
> that we start to prefer using LLVM components where available.
My feelings exactly.

> At the very
> least, I don't see a good reason why we would ever want to use libgcc_s
in a
> situation where compiler-rt (and libunwind) are available -- libgcc_s does
> not contain some functions that LLVM implicitly adds calls to.
That's another dead-lock we need to solve, yes. :(

> Perhaps we should have a flag to specify whether we prefer the canonical
> tools and components for the target, or whether we prefer LLVM's
versions
> when available (falling back to the target components if not)?
Hum, I had't thought of this...

Do you mean a flag that would (try) to use as much of LLVM as
possible, and fall back to the system's tools when unavailable?

I like the idea very much, but I wonder the kind of spurious bugs
we'll see if one tool suddenly doesn't get built, or change location,
or $PATH changes.

cheers,
--renato