llvm dev - Jul 2016 - [RFC] Make Lanai backend non-experimental

On Mon, Jul 25, 2016 at 11:46 AM, Chandler Carruth via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Tue, Jul 19, 2016 at 9:42 AM Renato Golin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On 19 July 2016 at 17:04, Martin J. O'Riordan via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>> > Presumably if my out-of-tree backend was to be pushed to LLVM, it
too
>> would be considered experimental.
>>
>> Yes. Though, not all out-of-tree back-ends end up upstream for
>> different reasons.
>>
>> A few basic rules to get accepted are if:
>>  * the target exists and can be easily purchased / emulated for
>> investigating problems,
>>  * there are official documents / specs published by the project /
>> company that maintains the targets,
>>
>
> Most of your list makes a lot of sense to me Renato, but these two items
> don't really.
>
> I think a large number of targets in tree would be *extremely* hard to
> purchase and directly investigate problems. As examples:
>
> - Hexagon
> - SystemZ
> - XCore
>
> I'm sure it is theoretically possible for me to acquire such a system
and
> test on it, but the amount of time and energy it would take make it a
> practical impossibility.
>
> Even for fairly pedestrian backends such as AArch64 and PowerPC, we
> routinely have people ask active developers on those backends to do the
> triaging of issues. I've never seen this be a problem in practice, and
I'm
> not convinced that it is something we should actually require to have an
> upstream backend.
>
As I mentioned in the original thread "[llvm-dev] [RFC] Lanai
backend", it
seems to me that the "can I run the code coming out of the backend?"
property is more about enabling LLVM's users to do interesting things with
the backend, rather than about LLVM's internal maintenance.

http://lists.llvm.org/pipermail/llvm-dev/2016-February/095214.html

I'm not sure how that affects the decision.

My gut (and I'm actually not sure if I exactly consider this a pro or a con
personally) is that is sounds Stallman-esque to require backends to answer
"yes" to "can I run the code coming out of the backend?". If
you squint it
is sort of like an anti-tivoization requirement, which would be a very
ironic requirement for a BSD-licensed project.

One example that personally convinces me that "can I run the code coming
out of the backend?" should not be a requirement is the following: if Apple
is willing to develop their mobile GPU backend(s?) upstream, but still keep
their actual driver stack and hardware locked down, I think it would be a
net win for the community to bring it into tree.

-- Sean Silva

>
>
> I also worry that we have backends for targets that don't have any
(real)
> published ISA spec. I'm not sure there is a formal "spec"
per-se for BPF.
> And perhaps there is a spec for AMDGPU or Hexagon, but even when I've
asked
> developers on those backends about problems, they've not pointed be at
the
> published ISA documentation.
>
>
> And I don't think any of these backends are bad or should be removed. I
> rather like several of them. =] I just think that sufficient community
> involvement makes the availability of hardware/emulator "nice to
have" but
> not essential, and it makes the ISA documentation substantially less
> important.
>
>
> The rest of your list below though, I think is a great list.
>
>
>>  * there is a reasonable community maintaining the rest of the system
>> (firmware, OS, other tools, etc),
>>  * enough people commit themselves to maintain the LLVM back-end to
>> avoid bit-rot,
>>  * the back-end is free of contentious features that would mean
>> breaking every other target.
>>
>> For example:
>>  * a PIC back-end ticks many of the boxes, but it doesn't have a
>> community around it and would very quickly bit-rot,
>>  * the CHERI back-end has an active community and interested
>> developers, but it would disrupt the IR too much,
>>  * the BPF back-end has been added mostly as-is and is actively
>> maintained, so gained the status of non-experimental.
>>
>> The main difference between experimental and official is that they get
>> built by default. So, if you don't specify them at CMake time, they
>> will be built and their tests will execute, which is a big win for the
>> back-end maintainers (as they get free testing), but it's also a
big
>> burden on other developers if their tests are not independent enough
>> and keep breaking due to unrelated changes.
>>
>> Under those conditions, it's as easy to remove a back-end that is
too
>> much trouble and no one cares about, as it to add a new experimental
>> back-end or promote it to official. It all depends on the community
>> around it and how much they value the LLVM back-end.
>>
>>
>> >  But what is involved in making a particular target, feature or
>> back-end experimental other than simply agreement?  Are there
particular
>> build processes or '-Dmacroname's to add, or how does somebody
like me edit
>> my out-of-tree sources so that it would obey the "is
experimental" rules?
>>
>> I'm assuming you already have an implementation, for example, in
>> GitHub. The easiest way to get things started is to point people at
>> the repository and make sure it works on trunk.
>>
>> The second step, then, would be to merge that into the tree and
>> propose to include it as an experimental target. It will take a few
>> months until the process is complete and then you'll be able to
check
>> it out and work straight from LLVM's repository.
>>
>> You'll have to change the CMake to force it to be built
>> (-DLLVM_TARGETS_TO_BUILD) until such time that you have shown that you
>> care about the target and that it hasn't failed (best if you have a
>> buildbot showing the history always green).
>>
>> Once all that is done, you can do as Jacques did and request it to be
>> built by default (aka. non-experimental, aka. official back-end).
>>
>> After that change, though, you're permanently enlisted to maintain
it
>> even more intensely. Not just your buildbot will break, but other
>> people's too, and they're probably a lot faster than yours, so
you
>> need to keep an eye on them, especially if they break due to unrelated
>> commits.
>>
>> That is the contract that you have to sign to have your target
>> supported, and if you break that contract (let it bit-rot, disable
>> tests that should be passing, remove parts of the code, or just
>> neglects bug reports), your back-end *will* be removed.
>>
>> Hope that helps,
>> --renato
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160725/519a2e51/attachment.html>

On Jul 25, 2016, at 4:02 PM, Sean Silva via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> I'm sure it is theoretically possible for me to acquire such a system
and test on it, but the amount of time and energy it would take make it a
practical impossibility.
> 
> Even for fairly pedestrian backends such as AArch64 and PowerPC, we
routinely have people ask active developers on those backends to do the triaging
of issues. I've never seen this be a problem in practice, and I'm not
convinced that it is something we should actually require to have an upstream
backend.
> 
> As I mentioned in the original thread "[llvm-dev] [RFC] Lanai
backend", it seems to me that the "can I run the code coming out of
the backend?" property is more about enabling LLVM's users to do
interesting things with the backend, rather than about LLVM's internal
maintenance.
> 
> http://lists.llvm.org/pipermail/llvm-dev/2016-February/095214.html
<http://lists.llvm.org/pipermail/llvm-dev/2016-February/095214.html>
> 
> I'm not sure how that affects the decision.
MHO, but I almost completely disagree with you.  I agree that the question comes
down to asking how much benefit to the community there is to merging the
backend, balances vs how much cost there is (your "internal
maintenance" point).

It is unquestionably easier for a contributor to land their backend in-tree than
to maintain it out-of-tree.  This is because landing it in tree shifts the
maintenance burden from the *contributor* to the *community*.  If there is low
value to the community, then this is a "bad deal” for the project as a
whole, since there is only so much attention to go around.
> My gut (and I'm actually not sure if I exactly consider this a pro or a
con personally) is that is sounds Stallman-esque to require backends to answer
"yes" to "can I run the code coming out of the backend?". If
you squint it is sort of like an anti-tivoization requirement, which would be a
very ironic requirement for a BSD-licensed project.
This seems like some weird FUD or mud slinging argument, I’m not sure what to
make of it.

-Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160725/2a6ac07e/attachment.html>

On 26 July 2016 at 00:08, Chris Lattner via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> It is unquestionably easier for a contributor to land their backend in-tree
> than to maintain it out-of-tree.  This is because landing it in tree shifts
> the maintenance burden from the *contributor* to the *community*.  If there
> is low value to the community, then this is a "bad deal” for the
project as
> a whole, since there is only so much attention to go around.
This expresses my idea very clearly.

The initial Lanai thread had hints that this could be at play, though
they seem to be releasing emulators and documents, which to mee it
seems like that was just FUD.

However, the fact that people did consider it means they care about
not being tossed a piece of code to baby sit, and this has *nothing*
to do with the license.

I'll start the discussion in a new thread, since it's not appropriate
to steal the Lanai discussion to that topic. Please, let's continue
there (Target Acceptance Policy).

cheers,
--renato

On Mon, Jul 25, 2016 at 4:08 PM, Chris Lattner <clattner at apple.com>
wrote:
> On Jul 25, 2016, at 4:02 PM, Sean Silva via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> I'm sure it is theoretically possible for me to acquire such a system
and
>> test on it, but the amount of time and energy it would take make it a
>> practical impossibility.
>>
>> Even for fairly pedestrian backends such as AArch64 and PowerPC, we
>> routinely have people ask active developers on those backends to do the
>> triaging of issues. I've never seen this be a problem in practice,
and I'm
>> not convinced that it is something we should actually require to have
an
>> upstream backend.
>>
>
> As I mentioned in the original thread "[llvm-dev] [RFC] Lanai
backend", it
> seems to me that the "can I run the code coming out of the
backend?"
> property is more about enabling LLVM's users to do interesting things
with
> the backend, rather than about LLVM's internal maintenance.
>
> http://lists.llvm.org/pipermail/llvm-dev/2016-February/095214.html
>
> I'm not sure how that affects the decision.
>
>
> MHO, but I almost completely disagree with you.  I agree that the question
> comes down to asking how much benefit to the community there is to merging
> the backend, balances vs how much cost there is (your "internal
> maintenance" point).
>
Was there supposed to be a sentence here explaining what you specifically
disagree with? I'm having a hard time understanding what it is you disagree
with. AFAICT, in the post you quoted and the linked post, I made only two
statements that you might disagree with:
1. "can I run the code coming out of the backend?" property is more
about
enabling LLVM's users to do interesting things with the backend, rather
than about LLVM's internal maintenance.
2. As far as maintenance of the LLVM project is concerned, leaning on
active maintainers is the way that issues get solved in practice, so the
"can I run the code" quality isn't relevant for maintenance.

Which one is it? (or was there something else I said in this thread or the
linked thread that you were referring to?)
FWIW, I don't disagree with any of the points you brought up.

> It is unquestionably easier for a contributor to land their backend
> in-tree than to maintain it out-of-tree.  This is because landing it in
> tree shifts the maintenance burden from the *contributor* to the
> *community*.  If there is low value to the community, then this is a
"bad
> deal” for the project as a whole, since there is only so much attention to
> go around.
>
> My gut (and I'm actually not sure if I exactly consider this a pro or a
> con personally) is that is sounds Stallman-esque to require backends to
> answer "yes" to "can I run the code coming out of the
backend?". If you
> squint it is sort of like an anti-tivoization requirement, which would be a
> very ironic requirement for a BSD-licensed project.
>
>
> This seems like some weird FUD or mud slinging argument, I’m not sure what
> to make of it.
>
If I understand what you're saying correctly, you don't see the parallel
with tivoization?

Essentially, both anti-tivoization and requiring that one can run the code
that comes out of an LLVM backend involve a situation where a programmer
has access to code (either the GPL code in the anti-tivoization case, or
LLVM source code in the LLVM backend case) and is free to modify it how
ever they want. However, without the extra requirement (either
anti-tivoization clause or requiring the ability to run the code that comes
from an LLVM backend), hardware manufacturers would be able to restrict
that programmer's ability to use their modified code for its intended
purpose (either powering the tivo device or compiling code that is then run
on the device targeted by the LLVM backend).

Hopefully that makes sense. I'm not trying to spread FUD or sling mud or
make an argument, but rather to draw a parallel.

A lot of us are hacker/tinkerers at heart, and so there is some
attractiveness to knowing that if we (or others like us) are interested, we
can go and play around with a backend, learn about the ISA, compile some
code, and run stuff. I've personally done this e.g. for AMDGPU at home,
purely out of curiosity (I purchased a machine with an AMD GPU specifically
for this purpose). I think this resonates with quite a few in the community
(or maybe it is just me :) [1]). So on the surface it seems like this might
be an interesting consideration. However, I think those people have
converged on putting aside these feelings and seeing that for the project
as a whole, it is not the right thing to look at; and also that there are
existing examples like SystemZ that show that really we didn't feel so
strongly about it after all.

Now, from what I gather from the other thread, your incoming concern is
different, given your different set of experiences:
"Maybe I just carry too many scars here. Remember that I was the guy who
got stuck with rewriting the "new” SPARC port from scratch, because
otherwise we “couldn’t” remove the (barely working) “SparcV9" backend,
which was dependent on a bunch of obsolete infrastructure.  I didn’t even
have a Sparc machine to test on, so I had to SSH into remote machines I got
access to."
That's fine, but not everybody is coming into the discussion from that
background.
Although, notice that your experiences gave you quite a head start over
many of us -- it seems that at this point we all have come to agree that
the internal concerns of LLVM maintenance are the common consideration, as
you insisted from the start.

[1] Although the Apple mobile GPU example has consistently convinced me
that "can I run the code" should not play a role in accepting a
backend.


-- Sean Silva

>
> -Chris
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160725/f55be567/attachment.html>