llvm dev - Sep 2016 - GitHub Survey?

On 8 September 2016 at 19:10, Chris Bieneman <cbieneman at apple.com>
wrote:
> My personal preference would be for the decision makers to be either a
> committee of developers or the LLVM Foundation board, and I would prefer if
> the survey were crafted to provide them with information to inform a
> decision, rather than a dictation of a decision.
Hi Chris,

Those are very good points, and I agree with you.

In line with Chris Lattner's previous comments, that this is a purely
technical decision with far reaching consequences, I think the
foundation should continue its role as helpers and not decision
makers. However, we don't yet have a process of forming technical
committees for such cases, and given the delicate nature of this
decision (illustrated by the number of long and heated threads), it
won't be an easy process either. It's quite possible that the
discussions on how to form a representative committee will be as
heated or more so than the git discussion.

Picking code owners or top committers as a base would put out a lot of
people that work hard on LLVM but never ended up owning a piece of
code and would be telling that committing a lof of patches is somehow
better than a few well done and life changing ones. Maybe we do need
to start a parallel discussion on how would this technical committee
would look like, so we can take such decisions while still being
representative with more ease in the future. But none of that is fast
enough, I think.

The only easy solution I can see right now, is for Chris Lattner to
decide, based on both the survey and the BoF. I personally trust Chris
to take a decision that will be best for the community, but I
obviously don't speak for the whole community.

In that case, a voting-in-survey-clothing may not be a bad idea, as
it'll give us an idea of numbers. But we'll need more specific
questions and they can be slightly less statistically relevant, since
the decision process will not rely on them being representative.

Can you suggest a few additional questions to the survey? We had a lot
of feedback in the beginning, but the last one was several weeks ago.

I've added Chris as an editor, Tanya had access already. I don't want
to be the only editor in this, and it'd be good if they could change
it to suit the kind of information a committee/person would be looking
into.

cheers,
--renato

> On Sep 8, 2016, at 12:45 PM, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On 8 September 2016 at 19:10, Chris Bieneman <cbieneman at apple.com>
wrote:
>> My personal preference would be for the decision makers to be either a
>> committee of developers or the LLVM Foundation board, and I would
prefer if
>> the survey were crafted to provide them with information to inform a
>> decision, rather than a dictation of a decision.
> 
> Hi Chris,
> 
> Those are very good points, and I agree with you.
> 
> In line with Chris Lattner's previous comments, that this is a purely
> technical decision with far reaching consequences, I think the
> foundation should continue its role as helpers and not decision
> makers. However, we don't yet have a process of forming technical
> committees for such cases, and given the delicate nature of this
> decision (illustrated by the number of long and heated threads), it
> won't be an easy process either. It's quite possible that the
> discussions on how to form a representative committee will be as
> heated or more so than the git discussion.
> 
> Picking code owners or top committers as a base would put out a lot of
> people that work hard on LLVM but never ended up owning a piece of
> code and would be telling that committing a lof of patches is somehow
> better than a few well done and life changing ones. Maybe we do need
> to start a parallel discussion on how would this technical committee
> would look like, so we can take such decisions while still being
> representative with more ease in the future. But none of that is fast
> enough, I think.
All of these points are totally valid, and I completely agree.
> 
> The only easy solution I can see right now, is for Chris Lattner to
> decide, based on both the survey and the BoF. I personally trust Chris
> to take a decision that will be best for the community, but I
> obviously don't speak for the whole community.
I also would trust Chris in making a decision that would be best for the
community.
> 
> In that case, a voting-in-survey-clothing may not be a bad idea, as
> it'll give us an idea of numbers. But we'll need more specific
> questions and they can be slightly less statistically relevant, since
> the decision process will not rely on them being representative.
I think having the survey contain a question on which solution the respondent
prefers is good, but I feel it is very limited. As someone who has been trying
very hard to process both proposals and all the email threads I have found the
lack of data to drive the decision troubling. In particular because this is as
you put it “a purely technical decision with far reaching consequences” I very
strongly believe data is our friend in helping make the best long term decision.
> 
> Can you suggest a few additional questions to the survey? We had a lot
> of feedback in the beginning, but the last one was several weeks ago.
Again, to work backwards, rather than starting with questions, let me suggest
information that I think we could generate from a survey that would help inform
the decision.

* How many users are already using Git as their primary way of interacting with
LLVM sources?

In general I believe there are three possible decisions that could come from
this. Either we go with one of the two proposals, or we change nothing. Knowing
how many users are already using Git speaks to the impact of a transition from
SVN to Git regardless of which Git proposal, and is a useful bit of data to
collect.

* Which projects people are using, and which ones they contribute to.

Knowing this allows the decision maker(s) to weight results based on the
opinions and impacts on contributors differently from the opinions and impacts
on down-stream users. While I certainly don't think we should disregard
downstream users, this decision will disproportionately impact contributors, so
we need to take that into account.

In addition to that I believe we should actually provide a section of the survey
specifically for questions that can inform the specific proposals, and improve
them.

For example, the mono-repo proposal currently lacks firm details on a few
things, which might benefit from survey answers. For example, which projects
should be included in the mono-repo, or are per-project git mirrors important.

-Chirs
> 
> I've added Chris as an editor, Tanya had access already. I don't
want
> to be the only editor in this, and it'd be good if they could change
> it to suit the kind of information a committee/person would be looking
> into.
> 
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 9 September 2016 at 19:05, Chris Bieneman <cbieneman at apple.com>
wrote:
> I think having the survey contain a question on which solution the
respondent prefers is good, but I feel it is very limited.
Well, the current survey is more than just one question...

> In general I believe there are three possible decisions that could come
from this. Either we go with one of the two proposals, or we change nothing.
Knowing how many users are already using Git speaks to the impact of a
transition from SVN to Git regardless of which Git proposal, and is a useful bit
of data to collect.
We have three specific questions as to what the impact of the
transition (one for each option short term plus one for long term).
Why doesn't that cover your proposal?

> Knowing this allows the decision maker(s) to weight results based on the
opinions and impacts on contributors differently from the opinions and impacts
on down-stream users. While I certainly don't think we should disregard
downstream users, this decision will disproportionately impact contributors, so
we need to take that into account.
We already have that *exact* question.

It's a multiple choice questions where you check all projects that you
work on. The pie chart means nothing to the GitHub survey per se, but
the association of that information with the repo choice and the
impact will be very valuable.

For example, if most libc++ developers prefer option X while core LLVM
developers prefer Y.

What would you suggest in addition to what's already there?


> In addition to that I believe we should actually provide a section of the
survey specifically for questions that can inform the specific proposals, and
improve them.
Right. I steered away from that on purpose, but I guess we can add one
free text field for each impact question, and mention that people
should laid out why this or that option is not good, and how to make
them good, in the case where it's chosen.

> For example, the mono-repo proposal currently lacks firm details on a few
things, which might benefit from survey answers. For example, which projects
should be included in the mono-repo, or are per-project git mirrors important.
Indeed, this is currently missing. Would that be enough to cover this
in the free-text I mention above?

cheers,
--renato