llvm dev - Jun 2016 - [lld] r271569 - Start adding tlsdesc support for aarch64.

On 3 June 2016 at 17:10, Rafael Espíndola <rafael.espindola at gmail.com>
wrote:
> Do keep in mind you are comparing a 11 year old project and a 11 month
> old one. There is a lot more churn on the 11 month old one.
LLD is at least 5 years old. Every time you re-write it doesn't reset
history.

> Again, I am truly sorry we were unable to come up with a perfect
> design the first time. For the record, I don't think it is perfect
> yet. There will likely be more big changes to lld. That is the cost of
> trying to build as good a linker as we can.
This is not about what you do. It's about *how* you do it.

We have developers trying to create a linker. They are working on LLD
because Chris wanted a true LLVM linker. But it seems that you want a
project that you can do whatever you want, whenever you want.

This is *NOT* open source. Right now, LLD is *nothing more* than Rui's
and Rafael's pet project. I cannot recommend Linaro to collaborate on
those terms, and I sincerely recommend anyone that is listening to
this thread to not do so either.

> Being open source doesn't mean I get to implement what someone else
> wants. You are more than welcome to send patches, but they have avoid
> harming the rest of the linker. In particular, at this early stage
> they cannot harm its development. Once we have a mature project we can
> actually evaluate tradeoff.
We're clearly not welcome to send patches. We did, and you re-wrote it
and committed without asking the original author.

So, the plan is to wait for you to finish the initial implementation
alone? Again? What do we do in the interim? How many times are we
going to go through this?

I have waited 2 years before LLD was barely useful, then Adhemerval
implemented the AArch64 back-end. Then you destroyed and now we have
waited another year, but we're still unable to collaborate. If
anything, now it's even harder than it was last year.

Why can't we help with the design, too? We know about ARM and AArch64,
that's what we do, and we can provide you with the expertise without
having to go on your own doing everything. That is the whole point of
collaborative development, and it seems that you're missing this
point.

> And just like we did, you are more than welcome to try to write
> something better. Please let us know how it goes.
Is this the position of every LLD developer?

Rui, Nick, Chris?

I'm seriously looking for others to chime in and let me know if that
is the final stance on LLD, so that I can finally write it off and go
work on another linker.

If the official position is that LLD is a project that only Rui and
Rafael can design and implement for another 2~3 years, I *cannot*
recommend Linaro and its members to participate.

cheers,
--renato

On Fri, Jun 3, 2016 at 9:29 AM, Renato Golin <renato.golin at linaro.org>
wrote:
> On 3 June 2016 at 17:10, Rafael Espíndola <rafael.espindola at
gmail.com>
> wrote:
> > Do keep in mind you are comparing a 11 year old project and a 11 month
> > old one. There is a lot more churn on the 11 month old one.
>
> LLD is at least 5 years old. Every time you re-write it doesn't reset
> history.
>
>
> > Again, I am truly sorry we were unable to come up with a perfect
> > design the first time. For the record, I don't think it is perfect
> > yet. There will likely be more big changes to lld. That is the cost of
> > trying to build as good a linker as we can.
>
> This is not about what you do. It's about *how* you do it.
>
> We have developers trying to create a linker. They are working on LLD
> because Chris wanted a true LLVM linker. But it seems that you want a
> project that you can do whatever you want, whenever you want.
>
> This is *NOT* open source. Right now, LLD is *nothing more* than Rui's
> and Rafael's pet project. I cannot recommend Linaro to collaborate on
> those terms, and I sincerely recommend anyone that is listening to
> this thread to not do so either.
>
Renato, it is not appropriate to call it my and Rafael's pet project. First
of all, Rafael and me are working on it for different purposes and probably
have different visions (although our taste on good code and good design
seems similar.) I'm interested in using it in my company and on FreeBSD,
for example. Rafael and I have no side channel to communicate, and we are
just collaborating openly on this mailing list.

You overlooked a lot of contributions that has been made to LLD. George
designed and wrote a great deal of relocation handling code and TLS
optimizations besides other contributions. Peter came up with a different
symbol table design, sent a proposal to the mailing list for discussion,
and implemented it (even though Peter and I are working for the same
company, that was not a coordinated effort, and the design change proposal
was out of the blue to me. I was surprised by his improvement.) Simon
implemented all of MIPS code that's pretty tricky. Davide is contributing
to AArch64 and LTO. There are many more contributors. I'm truly respectful
to all the contributors, and attributing these efforts to someone's pet
project is honestly a bit insulting.

I have never declined contributions by saying that "I know better." I
requested design documents (because I don't know better!) when contributors
seem to be starting large design changes, but that was needed for
discussion. If you want to do some large change, you want to convince other
collaborators that your design will work. It is I think a usual process.

Needless to say, I don't think I (and Rafael) should get a special
treatment. Both do not have right to submit without convincing other
collaborators. For example, just last week, I sent a patch which I truly
believed non-controversial and good one for review, and got rejected
unexpectedly by Sean. I had to convince him. That's how it works.

I sincerely request you to retract your recommendation to not collaborate
with us.

I'm not protecting what Rafael did. Rafael, even if you think your patch
had less common with the patch that's under review, they overlapped each
other with a great deal. Reviewing one patch and submitting a conflicting
one at the same time is not an appropriate behavior, and I do understand
why it disappointed Adhemerval and Renato. You needed to send it to review
and explain yours if you believe yours is different and better.
> Being open source doesn't mean I get to implement what someone else
> > wants. You are more than welcome to send patches, but they have avoid
> > harming the rest of the linker. In particular, at this early stage
> > they cannot harm its development. Once we have a mature project we can
> > actually evaluate tradeoff.
>
> We're clearly not welcome to send patches. We did, and you re-wrote it
> and committed without asking the original author.
>
> So, the plan is to wait for you to finish the initial implementation
> alone? Again? What do we do in the interim? How many times are we
> going to go through this?
>
> I have waited 2 years before LLD was barely useful, then Adhemerval
> implemented the AArch64 back-end. Then you destroyed and now we have
> waited another year, but we're still unable to collaborate. If
> anything, now it's even harder than it was last year.
>
> Why can't we help with the design, too? We know about ARM and AArch64,
> that's what we do, and we can provide you with the expertise without
> having to go on your own doing everything. That is the whole point of
> collaborative development, and it seems that you're missing this
> point.
>
>
> > And just like we did, you are more than welcome to try to write
> > something better. Please let us know how it goes.
>
> Is this the position of every LLD developer?
>
I cannot say about every developer, but at least to me, my answer is yes,
of course.

Rui, Nick, Chris?
>
> I'm seriously looking for others to chime in and let me know if that
> is the final stance on LLD, so that I can finally write it off and go
> work on another linker.
>
> If the official position is that LLD is a project that only Rui and
> Rafael can design and implement for another 2~3 years, I *cannot*
> recommend Linaro and its members to participate.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160603/dd0f4e3a/attachment.html>

On Fri, Jun 3, 2016 at 10:47 AM, Rui Ueyama <ruiu at google.com> wrote:
> On Fri, Jun 3, 2016 at 9:29 AM, Renato Golin <renato.golin at
linaro.org>
> wrote:
>
>> On 3 June 2016 at 17:10, Rafael Espíndola <rafael.espindola at
gmail.com>
>> wrote:
>> > Do keep in mind you are comparing a 11 year old project and a 11
month
>> > old one. There is a lot more churn on the 11 month old one.
>>
>> LLD is at least 5 years old. Every time you re-write it doesn't
reset
>> history.
>>
>>
>> > Again, I am truly sorry we were unable to come up with a perfect
>> > design the first time. For the record, I don't think it is
perfect
>> > yet. There will likely be more big changes to lld. That is the
cost of
>> > trying to build as good a linker as we can.
>>
>> This is not about what you do. It's about *how* you do it.
>>
>> We have developers trying to create a linker. They are working on LLD
>> because Chris wanted a true LLVM linker. But it seems that you want a
>> project that you can do whatever you want, whenever you want.
>>
>> This is *NOT* open source. Right now, LLD is *nothing more* than
Rui's
>> and Rafael's pet project. I cannot recommend Linaro to collaborate
on
>> those terms, and I sincerely recommend anyone that is listening to
>> this thread to not do so either.
>>
>
> Renato, it is not appropriate to call it my and Rafael's pet project.
> First of all, Rafael and me are working on it for different purposes and
> probably have different visions (although our taste on good code and good
> design seems similar.) I'm interested in using it in my company and on
> FreeBSD, for example. Rafael and I have no side channel to communicate, and
> we are just collaborating openly on this mailing list.
>
> You overlooked a lot of contributions that has been made to LLD. George
> designed and wrote a great deal of relocation handling code and TLS
> optimizations besides other contributions. Peter came up with a different
> symbol table design, sent a proposal to the mailing list for discussion,
> and implemented it (even though Peter and I are working for the same
> company, that was not a coordinated effort, and the design change proposal
> was out of the blue to me. I was surprised by his improvement.) Simon
> implemented all of MIPS code that's pretty tricky. Davide is
contributing
> to AArch64 and LTO. There are many more contributors. I'm truly
respectful
> to all the contributors, and attributing these efforts to someone's pet
> project is honestly a bit insulting.
>
> I have never declined contributions by saying that "I know
better." I
> requested design documents (because I don't know better!) when
contributors
> seem to be starting large design changes, but that was needed for
> discussion. If you want to do some large change, you want to convince other
> collaborators that your design will work. It is I think a usual process.
>
> Needless to say, I don't think I (and Rafael) should get a special
> treatment. Both do not have right to submit without convincing other
> collaborators. For example, just last week, I sent a patch which I truly
> believed non-controversial and good one for review, and got rejected
> unexpectedly by Sean. I had to convince him. That's how it works.
>
> I sincerely request you to retract your recommendation to not collaborate
> with us.
>
> I'm not protecting what Rafael did. Rafael, even if you think your
patch
> had less common with the patch that's under review, they overlapped
each
> other with a great deal. Reviewing one patch and submitting a conflicting
> one at the same time is not an appropriate behavior, and I do understand
> why it disappointed Adhemerval and Renato. You needed to send it to review
> and explain yours if you believe yours is different and better.
>
> > Being open source doesn't mean I get to implement what someone
else
>> > wants. You are more than welcome to send patches, but they have
avoid
>> > harming the rest of the linker. In particular, at this early stage
>> > they cannot harm its development. Once we have a mature project we
can
>> > actually evaluate tradeoff.
>>
>> We're clearly not welcome to send patches. We did, and you re-wrote
it
>> and committed without asking the original author.
>>
>> So, the plan is to wait for you to finish the initial implementation
>> alone? Again? What do we do in the interim? How many times are we
>> going to go through this?
>>
>> I have waited 2 years before LLD was barely useful, then Adhemerval
>> implemented the AArch64 back-end. Then you destroyed and now we have
>> waited another year, but we're still unable to collaborate. If
>> anything, now it's even harder than it was last year.
>>
>> Why can't we help with the design, too? We know about ARM and
AArch64,
>> that's what we do, and we can provide you with the expertise
without
>> having to go on your own doing everything. That is the whole point of
>> collaborative development, and it seems that you're missing this
>> point.
>>
>>
>> > And just like we did, you are more than welcome to try to write
>> > something better. Please let us know how it goes.
>>
>> Is this the position of every LLD developer?
>>
>
> I cannot say about every developer, but at least to me, my answer is yes,
> of course.
>
Oh, wait! I may have been misread the sentence. I read "write better code
in LLD to replace existing part of LLD". I had totally no intention to say
that you should try something outside LLD. I'm sorry for the confusion.

Rui, Nick, Chris?
>>
>> I'm seriously looking for others to chime in and let me know if
that
>> is the final stance on LLD, so that I can finally write it off and go
>> work on another linker.
>>
>> If the official position is that LLD is a project that only Rui and
>> Rafael can design and implement for another 2~3 years, I *cannot*
>> recommend Linaro and its members to participate.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160603/5f3b38b1/attachment.html>

On 3 June 2016 at 18:47, Rui Ueyama <ruiu at google.com>
wrote:
> Renato, it is not appropriate to call it my and Rafael's pet project.
Hi Rui,

I apologise, that was wrong in all levels.

I know how much other people have contributed, but these people are on
the inside already, so their contributions are more easily accepted.

We have been trying to contribute for more than a year and have yet
failed to get in the loop. We were asked to wait for 2 years until the
core linker was ready, we did. Last year we wasted 3 months on the old
back end, and were told to wait again. Now we wasted another 4 months
this year, and all we did was re-written without warning. I cannot
recommend Linaro to continue wasting any more time.

> I have never declined contributions by saying that "I know
better." I
> requested design documents (because I don't know better!) when
contributors
> seem to be starting large design changes, but that was needed for
> discussion. If you want to do some large change, you want to convince other
> collaborators that your design will work. It is I think a usual process.
Again, I apologise. Your behaviour is by no means comparable to Rafael's.

But he's taking the stance and speaking for the LLD community, at
least that's what it looks like to me.

Linaro is not a drive-by contributor to LLVM, or even to LLD. And if
even long time contributors to LLVM, and professional developers with
decades of experience in linker technology, are getting this
treatment, what does that say about others?

> I sincerely request you to retract your recommendation to not collaborate
> with us.
Those are based on facts. Not about you, but about interacting the the
LLD community, which unfortunately, you're part of.

I don't want to blame people, I want to produce software, and the LLD
community is blocking our progress for 3 years already.

> I had totally no intention to say that you should try something outside
LLD. I'm sorry for the confusion.
That's what I meant, yes. :)

Let me re-write my question to make sure we get over the language barrier:

Is the LLD community in agreement that a few core developers can and
will continue to block contributions, while doing all the work
themselves until such a day that it's deemed ready by the same
developers, in which time that everyone else will then, be allowed to
collaborate?

Or is the community in disagreement with Rafael's behaviour and will
try to not only curb it, but openly and actively promote more
collaboration by allowing other people to participate in the design
and implementation of their own targets?

If the former, than I'll respectfully move to MCLinker. If the latter,
than I expect due process and future collaborations, including letting
Adhemerval complete the AArch64 TLS implementation with proper code
review, not bypass re-writes.

As I said earlier, we'll continue to collaborate on LLD for the time
being on both ARM and AArch64 targets, and the next months will be a
test of how the LLD community will respond to this incident.

Peter Smith has just sent an initial implementation for the ARM target
(D20951), and he's working on improving it to be able to self-host
Clang on ARM in the next few months. I seriously hope that this
doesn't end up as another bad decision.

cheers,
--renato