llvm dev - Aug 2016 - [RFC] GitHub Survey - Please review

Folks,

I've created the survey with the feedback I got on the "Voting"
thread
in the llvm-foundation list, and put it here:

https://goo.gl/forms/k4J7M3N7oLNTOlDq2

Apparently, I can't allow people to comment on the form itself. It's
either full permission or nothing. So, I think the best way to do this
is to do a review on the list, with my most sincere apologies to the
anti-spam folks.

For that reason, I have only sent to llvm-dev, and would encourage
people to share privately with colleagues that didn't get it, via
lists, IRC, etc. Let's leave social media out of this, or we risk
having to filter out a lot of spam / trolls and make the whole
exercise moot.

People that have an interest on this question already subscribe to
this list or the IRC channel.


  The Plan

Today it's the 19th, so about the time I promised to put the survey up
for review. From today to the Sep 1st, we'll be filling the form,
trying out the questions, changing the wording, adding new questions,
etc.

If you guys could fill up with some data, see how it feels, and in the
end I'll try to share the bogus results, to see if that's what people
expected.

Around Sep 1st, The GitHub proposal should be finished (we'll have a
common document with both sub-modules and mono-repo explained), and
the survey should also be finished.

Since the survey has some free-text fields, it's less important how
precise is the writing, but we need to get the multiple-choice
questions right, to have a general idea of a "voting" mechanism.

My hope is that by Sep 1st, we'll have the GitHub proposal done and
the survey online for real, when I'll wipe out all responses and we'll
start fresh again.


  Design Choices

TL;DR, feel free to ignore this section...

Just FYI, the design choices for the survey were:

1. Request name, email and affiliation to de-duplicate the data. There
is no way to prevent people from responding twice without forcing them
to sign up on Google, which I will most certainly not do.

The identification also helps us to group people by their affiliations
and to have an idea of representation. I'm not expecting everyone on
the same group to have the same opinion, but it will be interesting to
see how they change.

Name and email will not be shared, but affiliation will (should it?).
I'm expecting the free-text descriptions to be very telling to that
respect, so there's no point is hiding it.

2. Gathering people's involvement in LLVM is important. We want to
know how much stake people have in LLVM, so we can weight more the
choices of people with more stake, but weight the same the *opinions*
of everyone.

What I mean by this is that, if most of the core developers feel
strongly towards using Git and a few external developers feel strongly
against, the people that will be using the most will have a higher
weight.

But the technical arguments of the minority is still weighted in the
same way as the vast majority, after all, they're *technical*
arguments and not *opinions*.

3. Separating "moving to Git/Github" from "using
mono-repo/sub-modules" is crucial. We may not get a consensus on the
latter, but we should get it for the former. It'll be much simpler for
a second iteration if we know we're going to use Git and GitHub and I
want to make sure we get this right.

If we have an overwhelmingly positive response to using GitHub, but
we're still divided to use sub-modules or mono-repo, we can close the
"move to Git" question now, and just work on the details later.

cheers,
--renato

> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of
> Renato Golin via llvm-dev
> Sent: Friday, August 19, 2016 1:23 PM
> To: LLVM Dev
> Subject: [llvm-dev] [RFC] GitHub Survey - Please review
>
> Folks,
>
> I've created the survey with the feedback I got on the
"Voting" thread in
> the
> llvm-foundation list, and put it here:
>
> https://goo.gl/forms/k4J7M3N7oLNTOlDq2
>
> Apparently, I can't allow people to comment on the form itself.
It's either
> full
> permission or nothing. So, I think the best way to do this is to do a
review
> on
> the list, with my most sincere apologies to the anti-spam folks.
IMO this looks good overall, I think the questions are balanced and don't 
encourage any direction themselves.
One minor point while reading (as a non-native speaker): "production
product"
sounds weird :D

A thing that doesn't hit myself but that may affect larger companies with
e.g.
private build bots:
There may be a difference between "impact on developer workflow"
(long-term)
and "one-time impact" (e.g. reconfiguring build bots).
Maybe we can duplicate the question and add an option "No (one-time)
impact"
to the second one?

Thanks for working on the proposal and taking the ungrateful job of conducting 
the survey!
Jonas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5868 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160819/72045b11/attachment.bin>

Thank you for working on this!

Some minor nits:

"If you chose 2~4 above, please explain your reasons here"

The above set of radio buttons are not numbered, so the use of numbers
here is a bit strange. Perhaps "If you answered above that moving to
Git/GitHub would have some (or greater) impact on your productivity,
please explain your reasons here" or something along those lines?

For the question about single repo vs submodules, you should have a
choice for "I have absolutely no idea what you're talking about."
Not
everyone knows what submodules are or why a single repo vs submodules
would be impactful.

~Aaron

On Fri, Aug 19, 2016 at 7:23 AM, Renato Golin via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Folks,
>
> I've created the survey with the feedback I got on the
"Voting" thread
> in the llvm-foundation list, and put it here:
>
> https://goo.gl/forms/k4J7M3N7oLNTOlDq2
>
> Apparently, I can't allow people to comment on the form itself.
It's
> either full permission or nothing. So, I think the best way to do this
> is to do a review on the list, with my most sincere apologies to the
> anti-spam folks.
>
> For that reason, I have only sent to llvm-dev, and would encourage
> people to share privately with colleagues that didn't get it, via
> lists, IRC, etc. Let's leave social media out of this, or we risk
> having to filter out a lot of spam / trolls and make the whole
> exercise moot.
>
> People that have an interest on this question already subscribe to
> this list or the IRC channel.
>
>
>   The Plan
>
> Today it's the 19th, so about the time I promised to put the survey up
> for review. From today to the Sep 1st, we'll be filling the form,
> trying out the questions, changing the wording, adding new questions,
> etc.
>
> If you guys could fill up with some data, see how it feels, and in the
> end I'll try to share the bogus results, to see if that's what
people
> expected.
>
> Around Sep 1st, The GitHub proposal should be finished (we'll have a
> common document with both sub-modules and mono-repo explained), and
> the survey should also be finished.
>
> Since the survey has some free-text fields, it's less important how
> precise is the writing, but we need to get the multiple-choice
> questions right, to have a general idea of a "voting" mechanism.
>
> My hope is that by Sep 1st, we'll have the GitHub proposal done and
> the survey online for real, when I'll wipe out all responses and
we'll
> start fresh again.
>
>
>   Design Choices
>
> TL;DR, feel free to ignore this section...
>
> Just FYI, the design choices for the survey were:
>
> 1. Request name, email and affiliation to de-duplicate the data. There
> is no way to prevent people from responding twice without forcing them
> to sign up on Google, which I will most certainly not do.
>
> The identification also helps us to group people by their affiliations
> and to have an idea of representation. I'm not expecting everyone on
> the same group to have the same opinion, but it will be interesting to
> see how they change.
>
> Name and email will not be shared, but affiliation will (should it?).
> I'm expecting the free-text descriptions to be very telling to that
> respect, so there's no point is hiding it.
>
> 2. Gathering people's involvement in LLVM is important. We want to
> know how much stake people have in LLVM, so we can weight more the
> choices of people with more stake, but weight the same the *opinions*
> of everyone.
>
> What I mean by this is that, if most of the core developers feel
> strongly towards using Git and a few external developers feel strongly
> against, the people that will be using the most will have a higher
> weight.
>
> But the technical arguments of the minority is still weighted in the
> same way as the vast majority, after all, they're *technical*
> arguments and not *opinions*.
>
> 3. Separating "moving to Git/Github" from "using
> mono-repo/sub-modules" is crucial. We may not get a consensus on the
> latter, but we should get it for the former. It'll be much simpler for
> a second iteration if we know we're going to use Git and GitHub and I
> want to make sure we get this right.
>
> If we have an overwhelmingly positive response to using GitHub, but
> we're still divided to use sub-modules or mono-repo, we can close the
> "move to Git" question now, and just work on the details later.
>
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 19 August 2016 at 13:15, Hahnfeld, Jonas <Hahnfeld at
itc.rwth-aachen.de> wrote:
> IMO this looks good overall, I think the questions are balanced and
don't
> encourage any direction themselves.
> One minor point while reading (as a non-native speaker): "production
product"
> sounds weird :D
It does... Changed...

> There may be a difference between "impact on developer workflow"
(long-term)
> and "one-time impact" (e.g. reconfiguring build bots).
> Maybe we can duplicate the question and add an option "No (one-time)
impact"
> to the second one?
That's a good point. Though, I assume everyone will have "some"
ont-time impact, and the real problem is if this is going to be
impossible (or at least very hard) for people to work with Git in the
long term goal.

Wouldn't hurt asking about the short-term impact, but maybe we
shouldn't add another free text, just a small multiple-choice one.
I've done that.

cheers,
--renato

On 19 August 2016 at 13:28, Aaron Ballman <aaron at aaronballman.com>
wrote:
> The above set of radio buttons are not numbered, so the use of numbers
> here is a bit strange. Perhaps "If you answered above that moving to
> Git/GitHub would have some (or greater) impact on your productivity,
> please explain your reasons here" or something along those lines?
Good point, updated.

> For the question about single repo vs submodules, you should have a
> choice for "I have absolutely no idea what you're talking
about." Not
> everyone knows what submodules are or why a single repo vs submodules
> would be impactful.
Hum, interesting. I guess we need to cater for that. Added.

cheers,
--renato

I think it might be good to draw a clearer line between the
contributor and their organization. I suspect Apple's infrastructure
will be far more affected by the change than I will personally and
there's not really a way to fit that information into the current
survey.

Tim.

Renato Golin via llvm-dev <llvm-dev at lists.llvm.org>
writes:
> 3. Separating "moving to Git/Github" from "using
> mono-repo/sub-modules" is crucial. We may not get a consensus on the
> latter, but we should get it for the former. It'll be much simpler for
> a second iteration if we know we're going to use Git and GitHub and I
> want to make sure we get this right.
>
> If we have an overwhelmingly positive response to using GitHub, but
> we're still divided to use sub-modules or mono-repo, we can close the
> "move to Git" question now, and just work on the details later.
This seems problematic. In fact, it makes some of the survey questions
unanswerable, ie:

    "What will be your one-time cost to moving to Git/GitHub?"

The one time cost of the mono-repo proposal is drastically different
than that of the multi-repo.

    "How would moving to Git/GitHub impact your usage of LLVM in the
     long term?"

I already use git, but depending on how things are organized in the new
world this may completely change how I work with LLVM.

> On Aug 19, 2016, at 10:20 AM, Justin Bogner via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> writes:
>> 3. Separating "moving to Git/Github" from "using
>> mono-repo/sub-modules" is crucial. We may not get a consensus on
the
>> latter, but we should get it for the former. It'll be much simpler
for
>> a second iteration if we know we're going to use Git and GitHub and
I
>> want to make sure we get this right.
>> 
>> If we have an overwhelmingly positive response to using GitHub, but
>> we're still divided to use sub-modules or mono-repo, we can close
the
>> "move to Git" question now, and just work on the details
later.
> 
> This seems problematic. In fact, it makes some of the survey questions
> unanswerable, i.e.:
+1.

— 
Mehdi

> 
>    "What will be your one-time cost to moving to Git/GitHub?"
> 
> The one time cost of the mono-repo proposal is drastically different
> than that of the multi-repo.
> 
>    "How would moving to Git/GitHub impact your usage of LLVM in the
>     long term?"
> 
> I already use git, but depending on how things are organized in the new
> world this may completely change how I work with LLVM.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

> I think it might be good to draw a clearer line between the
> contributor and their organization. I suspect Apple's infrastructure
> will be far more affected by the change than I will personally and
> there's not really a way to fit that information into the current
> survey.
> 
> Tim.
Excellent point.  Sony's infrastructure pain would be significant to
those of us having to implement the conversion, but that change would
be essentially invisible to the rest of the team as our internal branches
aren't going to look any different.  Data about our corporate pain would
be appropriate from a couple of people, but not from the rest of the team.
--paulr

>     "How would moving to Git/GitHub impact your usage of LLVM in the
>      long term?"
> 
> I already use git, but depending on how things are organized in the new
> world this may completely change how I work with LLVM.
Conversely, I'm currently using SVN for my own upstream interactions
but replacing my upstream checkouts with git clones (whatever they look
like) would cause me essentially no pain (probably).  The current phrasing 
of the survey doesn't let me say that.
--paulr

On 19 August 2016 at 18:20, Justin Bogner <mail at justinbogner.com>
wrote:
> The one time cost of the mono-repo proposal is drastically different
> than that of the multi-repo.
True.

But maybe not as different as from one company / project to another.
I'm assuming some people will suffer a lot more than others on either
choice.

> I already use git, but depending on how things are organized in the new
> world this may completely change how I work with LLVM.
It will, but you already work around with Git-SVN, which is a pain.

I don't see "the other option" being more cumbersome than Git-SVN,
whatever is the one option you pick. But this is just my opinion.

If that doesn't work for you, can you suggest a solution that will?

cheers,
--renato

> On Aug 19, 2016, at 4:23 AM, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> 
> Folks,
> 
> I've created the survey with the feedback I got on the
"Voting" thread
> in the llvm-foundation list, and put it here:
> 
>
https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_forms_k4J7M3N7oLNTOlDq2&d=CwIGaQ&c=Hw-EJUFt2_D9PK5csBJ29kRV40HqSDXWTLPyZ6W8u84&r=XndYjVJuvcoEtO9BUlAZk8839TPlVRJeJXMNUFEz-qQ&m=OhW1rKp29KzWPJmzePqaWyyFm8koNMtnNM4xM0DOCLM&s=6u9FpXqnNR5dxnPdXhgM17YsrxuvACOEtpCweWvOffM&e=
> 
> Apparently, I can't allow people to comment on the form itself.
It's
> either full permission or nothing. So, I think the best way to do this
> is to do a review on the list, with my most sincere apologies to the
> anti-spam folks.
> 
> For that reason, I have only sent to llvm-dev, and would encourage
> people to share privately with colleagues that didn't get it, via
> lists, IRC, etc. Let's leave social media out of this, or we risk
> having to filter out a lot of spam / trolls and make the whole
> exercise moot.
> 
> People that have an interest on this question already subscribe to
> this list or the IRC channel.
> 
> 
>  The Plan
> 
> Today it's the 19th, so about the time I promised to put the survey up
> for review. From today to the Sep 1st, we'll be filling the form,
> trying out the questions, changing the wording, adding new questions,
> etc.
> 
> If you guys could fill up with some data, see how it feels, and in the
> end I'll try to share the bogus results, to see if that's what
people
> expected.
> 
> Around Sep 1st, The GitHub proposal should be finished (we'll have a
> common document with both sub-modules and mono-repo explained), and
> the survey should also be finished.
> 
> Since the survey has some free-text fields, it's less important how
> precise is the writing, but we need to get the multiple-choice
> questions right, to have a general idea of a "voting" mechanism.
I’m not sure what value we’ll get from these data without a free text field for
*every* question.
For example, for anyone that select the answer "It'll be a major impact
to our build system, as we'll have to stop most of our current production to
refactor the whole build system to adapt to such a scenario” ; I’d like to have
some explanations about this.
This is an example, but it is valid for almost all the questions: otherwise I
wouldn’t trust that the answers are made with a full understanding of the
proposals.

— 
Mehdi



> 
> My hope is that by Sep 1st, we'll have the GitHub proposal done and
> the survey online for real, when I'll wipe out all responses and
we'll
> start fresh again.
> 
> 
>  Design Choices
> 
> TL;DR, feel free to ignore this section...
> 
> Just FYI, the design choices for the survey were:
> 
> 1. Request name, email and affiliation to de-duplicate the data. There
> is no way to prevent people from responding twice without forcing them
> to sign up on Google, which I will most certainly not do.
> 
> The identification also helps us to group people by their affiliations
> and to have an idea of representation. I'm not expecting everyone on
> the same group to have the same opinion, but it will be interesting to
> see how they change.
> 
> Name and email will not be shared, but affiliation will (should it?).
> I'm expecting the free-text descriptions to be very telling to that
> respect, so there's no point is hiding it.
> 
> 2. Gathering people's involvement in LLVM is important. We want to
> know how much stake people have in LLVM, so we can weight more the
> choices of people with more stake, but weight the same the *opinions*
> of everyone.
> 
> What I mean by this is that, if most of the core developers feel
> strongly towards using Git and a few external developers feel strongly
> against, the people that will be using the most will have a higher
> weight.
> 
> But the technical arguments of the minority is still weighted in the
> same way as the vast majority, after all, they're *technical*
> arguments and not *opinions*.
> 
> 3. Separating "moving to Git/Github" from "using
> mono-repo/sub-modules" is crucial. We may not get a consensus on the
> latter, but we should get it for the former. It'll be much simpler for
> a second iteration if we know we're going to use Git and GitHub and I
> want to make sure we get this right.
> 
> If we have an overwhelmingly positive response to using GitHub, but
> we're still divided to use sub-modules or mono-repo, we can close the
> "move to Git" question now, and just work on the details later.
> 
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

We have free text answers for both groups of answers, usage and impact.
People can write whatever they want there.

I don't see what the problem is...

Cheers,
Renato

On 24 Aug 2016 8:01 p.m., "Mehdi Amini" <mehdi.amini at
apple.com> wrote:
>
> > On Aug 19, 2016, at 4:23 AM, Renato Golin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> >
> > Folks,
> >
> > I've created the survey with the feedback I got on the
"Voting" thread
> > in the llvm-foundation list, and put it here:
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_
> forms_k4J7M3N7oLNTOlDq2&d=CwIGaQ&c=Hw-EJUFt2_
> D9PK5csBJ29kRV40HqSDXWTLPyZ6W8u84&r=XndYjVJuvcoEtO9BUlAZk8839TPlVR
> JeJXMNUFEz-qQ&m=OhW1rKp29KzWPJmzePqaWyyFm8koNMtnNM4xM0DOCLM&s>
6u9FpXqnNR5dxnPdXhgM17YsrxuvACOEtpCweWvOffM&e> >
> > Apparently, I can't allow people to comment on the form itself.
It's
> > either full permission or nothing. So, I think the best way to do this
> > is to do a review on the list, with my most sincere apologies to the
> > anti-spam folks.
> >
> > For that reason, I have only sent to llvm-dev, and would encourage
> > people to share privately with colleagues that didn't get it, via
> > lists, IRC, etc. Let's leave social media out of this, or we risk
> > having to filter out a lot of spam / trolls and make the whole
> > exercise moot.
> >
> > People that have an interest on this question already subscribe to
> > this list or the IRC channel.
> >
> >
> >  The Plan
> >
> > Today it's the 19th, so about the time I promised to put the
survey up
> > for review. From today to the Sep 1st, we'll be filling the form,
> > trying out the questions, changing the wording, adding new questions,
> > etc.
> >
> > If you guys could fill up with some data, see how it feels, and in the
> > end I'll try to share the bogus results, to see if that's what
people
> > expected.
> >
> > Around Sep 1st, The GitHub proposal should be finished (we'll have
a
> > common document with both sub-modules and mono-repo explained), and
> > the survey should also be finished.
> >
> > Since the survey has some free-text fields, it's less important
how
> > precise is the writing, but we need to get the multiple-choice
> > questions right, to have a general idea of a "voting"
mechanism.
>
> I’m not sure what value we’ll get from these data without a free text
> field for *every* question.
> For example, for anyone that select the answer "It'll be a major
impact to
> our build system, as we'll have to stop most of our current production
to
> refactor the whole build system to adapt to such a scenario” ; I’d like to
> have some explanations about this.
> This is an example, but it is valid for almost all the questions:
> otherwise I wouldn’t trust that the answers are made with a full
> understanding of the proposals.
>
> —
> Mehdi
>
>
>
>
> >
> > My hope is that by Sep 1st, we'll have the GitHub proposal done
and
> > the survey online for real, when I'll wipe out all responses and
we'll
> > start fresh again.
> >
> >
> >  Design Choices
> >
> > TL;DR, feel free to ignore this section...
> >
> > Just FYI, the design choices for the survey were:
> >
> > 1. Request name, email and affiliation to de-duplicate the data. There
> > is no way to prevent people from responding twice without forcing them
> > to sign up on Google, which I will most certainly not do.
> >
> > The identification also helps us to group people by their affiliations
> > and to have an idea of representation. I'm not expecting everyone
on
> > the same group to have the same opinion, but it will be interesting to
> > see how they change.
> >
> > Name and email will not be shared, but affiliation will (should it?).
> > I'm expecting the free-text descriptions to be very telling to
that
> > respect, so there's no point is hiding it.
> >
> > 2. Gathering people's involvement in LLVM is important. We want to
> > know how much stake people have in LLVM, so we can weight more the
> > choices of people with more stake, but weight the same the *opinions*
> > of everyone.
> >
> > What I mean by this is that, if most of the core developers feel
> > strongly towards using Git and a few external developers feel strongly
> > against, the people that will be using the most will have a higher
> > weight.
> >
> > But the technical arguments of the minority is still weighted in the
> > same way as the vast majority, after all, they're *technical*
> > arguments and not *opinions*.
> >
> > 3. Separating "moving to Git/Github" from "using
> > mono-repo/sub-modules" is crucial. We may not get a consensus on
the
> > latter, but we should get it for the former. It'll be much simpler
for
> > a second iteration if we know we're going to use Git and GitHub
and I
> > want to make sure we get this right.
> >
> > If we have an overwhelmingly positive response to using GitHub, but
> > we're still divided to use sub-modules or mono-repo, we can close
the
> > "move to Git" question now, and just work on the details
later.
> >
> > cheers,
> > --renato
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160824/a09fb272/attachment.html>