llvm dev - Dec 2016 - LLD status update and performance chart

Mehdi Amini <mehdi.amini at apple.com> writes:
>> On Dec 13, 2016, at 5:55 AM, Rafael Avila de Espindola via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>> 
>> Sean Silva via llvm-dev <llvm-dev at lists.llvm.org> writes:
>>> This will also greatly facilitate certain measurements I'd like
to do
>>> w.r.t. different strategies for avoiding memory costs for input
files (esp.
>>> minor faults and dTLB costs). I've almost gotten to the point
of
>>> implementing this just to do those measurements.
>> 
>> If you do please keep it local. The bare minimum we have of library
>> support is already disproportionately painful and prevents easier
sharing
>> with COFF. We should really not add more until the linker is done.
>
> This is so much in contrast with the LLVM development, I find it quite hard
to see this as an acceptable position on llvm-dev.
Why? What is wrong with setting priorities and observing that what
library support we already have has had a disproportional cost?

Cheers,
Rafael

----- Original Message -----
> From: "Rafael Avila de Espindola via llvm-dev" <llvm-dev at
lists.llvm.org>
> To: "Mehdi Amini" <mehdi.amini at apple.com>
> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>
> Sent: Tuesday, December 13, 2016 11:40:43 AM
> Subject: Re: [llvm-dev] LLD status update and performance chart
> 
> Mehdi Amini <mehdi.amini at apple.com> writes:
> 
> >> On Dec 13, 2016, at 5:55 AM, Rafael Avila de Espindola via
> >> llvm-dev <llvm-dev at lists.llvm.org> wrote:
> >> 
> >> Sean Silva via llvm-dev <llvm-dev at lists.llvm.org> writes:
> >>> This will also greatly facilitate certain measurements I'd
like
> >>> to do
> >>> w.r.t. different strategies for avoiding memory costs for
input
> >>> files (esp.
> >>> minor faults and dTLB costs). I've almost gotten to the
point of
> >>> implementing this just to do those measurements.
> >> 
> >> If you do please keep it local. The bare minimum we have of
> >> library
> >> support is already disproportionately painful and prevents easier
> >> sharing
> >> with COFF. We should really not add more until the linker is done.
> >
> > This is so much in contrast with the LLVM development, I find it
> > quite hard to see this as an acceptable position on llvm-dev.
> 
> Why? What is wrong with setting priorities and observing that what
> library support we already have has had a disproportional cost?
Can you please elaborate on this disproportional cost? I think it is really
important to be specific about these kinds of things for the benefit of all
potential contributors.

Thanks again,
Hal
> 
> Cheers,
> Rafael
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

On Tue, Dec 13, 2016 at 12:40 PM, Rafael Avila de Espindola via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Mehdi Amini <mehdi.amini at apple.com> writes:
>
> >> On Dec 13, 2016, at 5:55 AM, Rafael Avila de Espindola via
llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >>
> >> Sean Silva via llvm-dev <llvm-dev at lists.llvm.org> writes:
> >>> This will also greatly facilitate certain measurements I'd
like to do
> >>> w.r.t. different strategies for avoiding memory costs for
input files
> (esp.
> >>> minor faults and dTLB costs). I've almost gotten to the
point of
> >>> implementing this just to do those measurements.
> >>
> >> If you do please keep it local. The bare minimum we have of
library
> >> support is already disproportionately painful and prevents easier
> sharing
> >> with COFF. We should really not add more until the linker is done.
> >
> > This is so much in contrast with the LLVM development, I find it quite
> hard to see this as an acceptable position on llvm-dev.
>
> Why? What is wrong with setting priorities and observing that what
> library support we already have has had a disproportional cost?
>
I think the discussion has started to turn into generalities. There is an
apparent disagreement here, but perhaps only in philosophical stance and
not in actual practice.

Perhaps no decision with regards to this philosophy needs to be made in
this mailing list; we can evaluate code contributions on a case-by-case
basis and have a more concrete talking point.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161213/a129ab12/attachment.html>

> On Dec 13, 2016, at 9:40 AM, Rafael Avila de Espindola <rafael.espindola
at gmail.com> wrote:
> 
> Mehdi Amini <mehdi.amini at apple.com> writes:
> 
>>> On Dec 13, 2016, at 5:55 AM, Rafael Avila de Espindola via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>> 
>>> Sean Silva via llvm-dev <llvm-dev at lists.llvm.org> writes:
>>>> This will also greatly facilitate certain measurements I'd
like to do
>>>> w.r.t. different strategies for avoiding memory costs for input
files (esp.
>>>> minor faults and dTLB costs). I've almost gotten to the
point of
>>>> implementing this just to do those measurements.
>>> 
>>> If you do please keep it local. The bare minimum we have of library
>>> support is already disproportionately painful and prevents easier
sharing
>>> with COFF. We should really not add more until the linker is done.
>> 
>> This is so much in contrast with the LLVM development, I find it quite
hard to see this as an acceptable position on llvm-dev.
> 
> Why? What is wrong with setting priorities and observing that what
> library support we already have has had a disproportional cost?
The library-hostile lld development goes against one the core principles that, I
believe, drives the LLVM development: providing libraries and reusable
components.

— 
Mehdi

----- Original Message -----
> From: "Mehdi Amini via llvm-dev" <llvm-dev at
lists.llvm.org>
> To: "Rafael Espíndola" <rafael.espindola at gmail.com>
> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>
> Sent: Tuesday, December 13, 2016 11:47:43 AM
> Subject: Re: [llvm-dev] LLD status update and performance chart
> 
> 
> > On Dec 13, 2016, at 9:40 AM, Rafael Avila de Espindola
> > <rafael.espindola at gmail.com> wrote:
> > 
> > Mehdi Amini <mehdi.amini at apple.com> writes:
> > 
> >>> On Dec 13, 2016, at 5:55 AM, Rafael Avila de Espindola via
> >>> llvm-dev <llvm-dev at lists.llvm.org> wrote:
> >>> 
> >>> Sean Silva via llvm-dev <llvm-dev at lists.llvm.org>
writes:
> >>>> This will also greatly facilitate certain measurements
I'd like
> >>>> to do
> >>>> w.r.t. different strategies for avoiding memory costs for
input
> >>>> files (esp.
> >>>> minor faults and dTLB costs). I've almost gotten to
the point of
> >>>> implementing this just to do those measurements.
> >>> 
> >>> If you do please keep it local. The bare minimum we have of
> >>> library
> >>> support is already disproportionately painful and prevents
easier
> >>> sharing
> >>> with COFF. We should really not add more until the linker is
> >>> done.
> >> 
> >> This is so much in contrast with the LLVM development, I find it
> >> quite hard to see this as an acceptable position on llvm-dev.
> > 
> > Why? What is wrong with setting priorities and observing that what
> > library support we already have has had a disproportional cost?
> 
> The library-hostile lld development goes against one the core
> principles that, I believe, drives the LLVM development: providing
> libraries and reusable components.
I certainly agree with this, but I think that we should listen to any technical
rationale that Rafael and others have.

We have always considered the marginal short-term cost of coding separable,
reusable components worthwhile because of the longer-term benefits (including
the benefit of serving yet-undefined future use cases). I'd like to
understand why the cost/benefit tradeoff is claimed to be different in this
case. We should also understand the relationship here with the JIT runtime
linker functionality with which we probably want to unify this codebase.

 -Hal
> 
> —
> Mehdi
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

Hal Finkel <hfinkel at anl.gov> writes:
>> Why? What is wrong with setting priorities and observing that what
>> library support we already have has had a disproportional cost?
>
> Can you please elaborate on this disproportional cost? I think it is really
important to be specific about these kinds of things for the benefit of all
potential contributors.
Yes. Getting the early shutdown code was way more work than it would
have been if this was just a program. The code is also quite ugly.

It also prevents us from sharing error handling functions with the COFF
linker.

So, please, I *BEG YOU*, let us write a linker. Once we have a
workning, production qualitity linker we can setup a performance
tracking bot and evaluate each librification change for its cost in
performance and code quality. We are just not there yet and not in a
position to take those patches.

Cheers,
Rafael

Mehdi Amini <mehdi.amini at apple.com> writes:
>> On Dec 13, 2016, at 9:40 AM, Rafael Avila de Espindola
<rafael.espindola at gmail.com> wrote:
>> 
>> Mehdi Amini <mehdi.amini at apple.com> writes:
>> 
>>>> On Dec 13, 2016, at 5:55 AM, Rafael Avila de Espindola via
llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>> 
>>>> Sean Silva via llvm-dev <llvm-dev at lists.llvm.org>
writes:
>>>>> This will also greatly facilitate certain measurements
I'd like to do
>>>>> w.r.t. different strategies for avoiding memory costs for
input files (esp.
>>>>> minor faults and dTLB costs). I've almost gotten to the
point of
>>>>> implementing this just to do those measurements.
>>>> 
>>>> If you do please keep it local. The bare minimum we have of
library
>>>> support is already disproportionately painful and prevents
easier sharing
>>>> with COFF. We should really not add more until the linker is
done.
>>> 
>>> This is so much in contrast with the LLVM development, I find it
quite hard to see this as an acceptable position on llvm-dev.
>> 
>> Why? What is wrong with setting priorities and observing that what
>> library support we already have has had a disproportional cost?
>
> The library-hostile lld development goes against one the core principles
that, I believe, drives the LLVM development: providing libraries and reusable
components.
Because it is trying to do something fundamentally different. We are
trying to write a *program*.

Cheers,
Rafael