Folks, It's 1st of September, and we don't have the document nor the survey ready. With the US meeting on 3-4 November, that leaves us only 2 months to do everything, and I'm not sure we'll be able to if we delay much more. Being the devil's advocate and hoping this doesn't spiral down (again), there were a few pertinent questions left unanswered from the previous threads... 1. How much vote, how much survey? Most people are in agreement that a balance has to be struck, though there is no clear consensus on how much of each. The concerns raised were: * As it is now, we may not have enough information on why, just what people want. * Without multiple choice questions, it'll be too hard to infer what people want. * With too many questions, statistical relevance will be diluted and spread too thin. FWIW, I'm personally happy with the current format. But this is not about me. 2. How much information do we want? Chris' point on the other thread is that he wants a lot more information, so we can derive all sorts of correlations and drive other decisions, not just GitHub. I think this is a valid point, but it depends on the time scale. The concerns are: * If we want to have enough meaningful results by the US LLVM, we need to get it online ASAP * It'll take months to converge on a long list of questions, publish, get feedback, analyse it all * The result will be more general, so we only do it once, but won't help the GitHub choice * Free text interpretation is fuzzy and easy to bias, concrete decisions need concrete information 3. The current GitHub document We need to update the current document to have both options: sub-modules and mono-repo, with an exact description of what they mean. The sub-modules discussion has finished with a solid proposal, though there aren't many example use cases, which makes it hard to demonstrate. But the mono-repo discussion was still around which repositories belong to the mono-repo and which don't, and what do we do about the others. So, the next steps on this task are: * Rename the file to GitHub.rst (since it'll have both proposals in there) * Update the sub-modules part, adding example use cases * Insert the whole mono-repo proposal, with what's in and out + examples * Update the migration plan to cope with either option (should be similar anyway) == Next Steps I'm attaching the pre-survey dump, with an anonymised CSV file (no names or emails), plus a number of pictures of the pie charts. We'll put it up in a web document to look nicer with the official survey. So, the next steps are: 1. Make sure everyone is happy with the survey, and change it if necessary. 1.a Make sure the results are in a format that people expect / can work with. 2. Make the appropriate changes to the proposal document. 2.a Review, reach consensus, approve, publish. 3. Put the survey online, this time for real, deleting all current responses first. 3.a *Everyone* that has filled it already will have to fill it again, sorry. 4. We'll need a deadline, so people feel compelled to answer sooner. 4.a When the deadline is reached, collate the results in a CSV file+PNGs. 4.b A group of people (volunteers? foundation?) will analyse the data and decide how to present. 4.c I volunteer to write the final document, with their final blessing before publishing. 4.d Hopefully before the US LLVM (4 Nov), we can have both the proposal and survey results. 5. At the US LLVM, hold a BoF / Panel on the GitHub move. 5.a Use the proposal and survey results as a starting point. 5.b Gather the feedback, create a third document (I volunteer again). 6. Someone(s) take(s) the final decision... cheers, --renato -------------- next part -------------- A non-text attachment was scrubbed... Name: GitHubSurvey.zip Type: application/zip Size: 262719 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160901/9e6ffa83/attachment-0001.zip>
Hey Renato, I want to again thank you for all your work herding this process along. If I may for a moment I'd like to work backwards through your "Next Steps". The last step is that someone (or a group of people) will make the final decision based on the information in the survey and the conversation in the BoF. A lot of the earlier steps depend on who and how the decision is to be made. For example, if the intent is that the final decision be dictated by the result of the survey, it is less a survey more a vote. On the other hand if the intent is that a group of people interpret survey results and make a decision based on survey results (although not dictated by them), then the survey should be worded differently. It might make the process of constructing an appropriate survey easier if we first figure out who will be the decision makers, and how they are intended to make the decision. My personal preference would be for the decision makers to be either a committee of developers or the LLVM Foundation board, and I would prefer if the survey were crafted to provide them with information to inform a decision, rather than a dictation of a decision. -Chris> On Sep 1, 2016, at 5:44 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Folks, > > It's 1st of September, and we don't have the document nor the survey > ready. With the US meeting on 3-4 November, that leaves us only 2 > months to do everything, and I'm not sure we'll be able to if we delay > much more. > > Being the devil's advocate and hoping this doesn't spiral down > (again), there were a few pertinent questions left unanswered from the > previous threads... > > 1. How much vote, how much survey? > > Most people are in agreement that a balance has to be struck, though > there is no clear consensus on how much of each. > > The concerns raised were: > * As it is now, we may not have enough information on why, just what > people want. > * Without multiple choice questions, it'll be too hard to infer what > people want. > * With too many questions, statistical relevance will be diluted and > spread too thin. > > FWIW, I'm personally happy with the current format. But this is not about me. > > > 2. How much information do we want? > > Chris' point on the other thread is that he wants a lot more > information, so we can derive all sorts of correlations and drive > other decisions, not just GitHub. I think this is a valid point, but > it depends on the time scale. > > The concerns are: > * If we want to have enough meaningful results by the US LLVM, we > need to get it online ASAP > * It'll take months to converge on a long list of questions, publish, > get feedback, analyse it all > * The result will be more general, so we only do it once, but won't > help the GitHub choice > * Free text interpretation is fuzzy and easy to bias, concrete > decisions need concrete information > > > 3. The current GitHub document > > We need to update the current document to have both options: > sub-modules and mono-repo, with an exact description of what they > mean. > > The sub-modules discussion has finished with a solid proposal, though > there aren't many example use cases, which makes it hard to > demonstrate. But the mono-repo discussion was still around which > repositories belong to the mono-repo and which don't, and what do we > do about the others. > > So, the next steps on this task are: > * Rename the file to GitHub.rst (since it'll have both proposals in there) > * Update the sub-modules part, adding example use cases > * Insert the whole mono-repo proposal, with what's in and out + examples > * Update the migration plan to cope with either option (should be > similar anyway) > > > == Next Steps > > I'm attaching the pre-survey dump, with an anonymised CSV file (no > names or emails), plus a number of pictures of the pie charts. We'll > put it up in a web document to look nicer with the official survey. > > So, the next steps are: > > 1. Make sure everyone is happy with the survey, and change it if necessary. > 1.a Make sure the results are in a format that people expect / can work with. > > 2. Make the appropriate changes to the proposal document. > 2.a Review, reach consensus, approve, publish. > > 3. Put the survey online, this time for real, deleting all current > responses first. > 3.a *Everyone* that has filled it already will have to fill it again, sorry. > > 4. We'll need a deadline, so people feel compelled to answer sooner. > 4.a When the deadline is reached, collate the results in a CSV file+PNGs. > 4.b A group of people (volunteers? foundation?) will analyse the data > and decide how to present. > 4.c I volunteer to write the final document, with their final blessing > before publishing. > 4.d Hopefully before the US LLVM (4 Nov), we can have both the > proposal and survey results. > > 5. At the US LLVM, hold a BoF / Panel on the GitHub move. > 5.a Use the proposal and survey results as a starting point. > 5.b Gather the feedback, create a third document (I volunteer again). > > 6. Someone(s) take(s) the final decision... > > cheers, > --renato > <GitHubSurvey.zip> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160908/432bb42a/attachment.html>
On 8 September 2016 at 19:10, Chris Bieneman <cbieneman at apple.com> wrote:> My personal preference would be for the decision makers to be either a > committee of developers or the LLVM Foundation board, and I would prefer if > the survey were crafted to provide them with information to inform a > decision, rather than a dictation of a decision.Hi Chris, Those are very good points, and I agree with you. In line with Chris Lattner's previous comments, that this is a purely technical decision with far reaching consequences, I think the foundation should continue its role as helpers and not decision makers. However, we don't yet have a process of forming technical committees for such cases, and given the delicate nature of this decision (illustrated by the number of long and heated threads), it won't be an easy process either. It's quite possible that the discussions on how to form a representative committee will be as heated or more so than the git discussion. Picking code owners or top committers as a base would put out a lot of people that work hard on LLVM but never ended up owning a piece of code and would be telling that committing a lof of patches is somehow better than a few well done and life changing ones. Maybe we do need to start a parallel discussion on how would this technical committee would look like, so we can take such decisions while still being representative with more ease in the future. But none of that is fast enough, I think. The only easy solution I can see right now, is for Chris Lattner to decide, based on both the survey and the BoF. I personally trust Chris to take a decision that will be best for the community, but I obviously don't speak for the whole community. In that case, a voting-in-survey-clothing may not be a bad idea, as it'll give us an idea of numbers. But we'll need more specific questions and they can be slightly less statistically relevant, since the decision process will not rely on them being representative. Can you suggest a few additional questions to the survey? We had a lot of feedback in the beginning, but the last one was several weeks ago. I've added Chris as an editor, Tanya had access already. I don't want to be the only editor in this, and it'd be good if they could change it to suit the kind of information a committee/person would be looking into. cheers, --renato
Folks, After feedback from Chris and Mehdi, I have added one long text answer to *each* critical questions (impact on productivity), so that people can extend their reasoning. But I have not made them compulsory, so that people that don't know much about or don't have any problem don't feel compelled to expand on nothing. https://docs.google.com/forms/d/e/1FAIpQLSc2PBeHW-meULpCOpmbGK1yb2qX8yzcQBtT4nqNF05vSv69WA/viewform The bottom line is, answers with more substance will be taken more seriously than those without, but the numbers should also tell us something. For example, if 9/10 of the answers are "we don't care", than the other 1/10 will have less weight than if it was 1/2, but still important to any decision. Can we go live with what we have now? Mehdi, how's the document? Can we get that online so I can change the header of the survey? We only have a month and a bit now... cheers, --renato
> On 2016-Sep-18, at 09:51, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Folks, > > After feedback from Chris and Mehdi, I have added one long text answer > to *each* critical questions (impact on productivity), so that people > can extend their reasoning. > > But I have not made them compulsory, so that people that don't know > much about or don't have any problem don't feel compelled to expand on > nothing. > > https://docs.google.com/forms/d/e/1FAIpQLSc2PBeHW-meULpCOpmbGK1yb2qX8yzcQBtT4nqNF05vSv69WA/viewform > > The bottom line is, answers with more substance will be taken more > seriously than those without, but the numbers should also tell us > something. > > For example, if 9/10 of the answers are "we don't care", than the > other 1/10 will have less weight than if it was 1/2, but still > important to any decision. > > Can we go live with what we have now? > > Mehdi, how's the document? Can we get that online so I can change the > header of the survey?Hi Renato, Thanks very much for putting this together. I think the proposal document is almost finished now. Since I ended up reviewing it pretty thoroughly, I've gained a bit of understanding about the concerns we need input on. The survey is a great start, but the final page isn't quite addressing the concerns in the final proposal. I'm not sure it asks the right questions to focus the conversation at the BoF. Firstly, I'm concerned that the questions focus on: - what's good for the individuals responding, instead of what they think is best for the LLVM project; and - how much pain the transition would cause, instead of what they think the right final state is. Secondly, I'm worried about this question: "How does the choice between a single repository with all projects and the use of sub-modules impact your usage of Git?" I'm not sure we'll good signal from this; it's essentially a vote on the two variants, but it doesn't force the respondent to think about the specific issues. I'd rather find a way to ask about the specific concerns raised in the document. Thirdly, I'm worried that the follow-ups talk about "preferred" and "non-preferred" instead of "multirepo" and "monorepo". This makes data-mining non-trivial (because the meaning depends on previous answers) and increases the chance of respondent confusion. I spent some time today thinking through what set of questions would get us the data we want. - I've focused on the main concerns about (and benefits of) the two variants. - I've referred to the multirepo and monorepo by name (consistently) in questions asking about them. This ensures that people know exactly what they're answering. - I've added specific questions about how people plan to use the multirepo and monorepo, so that we know which tooling is most important (and also to determine how worried we should be about some of the concerns). - I've moved the "vote"-like question to the end to force respondents to think through the issues first. I've also restricted "the vote" to "the ideal project setup", so we can clearly separate that from "transition pain". (I'm still not sure the vote will have much value, but it doesn't hurt.) Here are my suggested questions; feedback welcome: ---- 1. How do you use LLVM? // as is 2. Which projects do you contribute to / use? // as is 3. Use this field to expand on your usage, if necessary // as is 4. How often do you work on a small LLVM sub-project without using a checkout of LLVM itself? - Always. - Most of the time. - Sometimes. - Never. 5. Please categorize how you interact with upstream. - I need read/write access, and I have limited disk space. - I need read/write access, but a 1GB clone doesn't scare me. - I only need read access. 6. How important is cross-project blame, grep, etc.? - Vital. I already use SVN/monorepo/custom-tooling to accomplish this. - Extremely. It should be easy enough that everyone does it by default. - Somewhat. I would use it if it were easy, but it's just nice to have. - Not at all. Anyone who cares can write their own tooling. 7. Single-commit cross-project refactoring designs away a class of build failures and simplifies making API changes. How important is it? - Vital. I already use SVN/monorepo/custom-tooling to accomplish this. - Extremely. It should be easy enough that everyone does it by default. - Somewhat. I would use it if it were easy, but it's just nice to have. - Not at all. Anyone who cares can write their own tooling. 8. The multirepo variant provides read-only umbrella repository to coordinate commits between the split sub-project repositories using Git submodules. Assuming multirepo gets adopted, how do you expect to use the umbrella? // checkboxes: + Actively contribute tooling improvements to improve it. + Integrate it into our downstream fork. + Use it for upstream contributions. + Use it as the primary interface development environment. + Use it for bisection. 9. If multirepo is adopted, how do you plan to contribute to upstream? - Using Git submodules. - Using the Git repos directly. - Using the SVN bridges. - n/a: I don't contribute. 10. The monorepo variant provides read/write access to sub-projects via an SVN bridge and git-svn. Contributors will have the option to continue using repositories split on project boundaries. Assuming monorepo gets adopted, how do you plan to contribute? - I'll use the monorepo as soon as it's possible, even before it's canonical. - I'll use the monorepo as soon as it's canonical. - I'll transition to monorepo eventually. - I'll use the SVN bridge on separated sub-projects forever. - I'll use a Git mirror (and/or git-svn) on separated sub-projects forever. - n/a: I don't contribute. 11. If monorepo is adopted, how do you plan to integrate it downstream? - We already use monorepo. - We'll switch to pulling from monorepo during the transition period. - We'll switch to pulling from monorepo eventually. - We'll integrate from the SVN bridge forever. - We'll integrate from the split sub-project Git mirror forever. - n/a: There is no downstream. 12. The multi/mono hybrid variant merges some sub-projects, but leaves runtimes in separate repositories using the umbrella to tie them together. Is this the best or worst of both worlds? - This is great. Native cross-project refactoring, without penalizing runtime-only developers. - Whatever. I'll deal with it. - This is terrible. All the transition pain of monorepo, without the advantages. 13. If multirepo is adopted, how much pain will there be in your transition? - Nothing consequential. - A little; but it'll be fine. - A lot; but it'll get done somehow. - Too much; I/we may stop contributing to LLVM. 14. If monorepo is adopted, how much pain will there be in your transition? - Nothing consequential. - A little; but it'll be fine. - A lot; but it'll get done somehow. - Too much; I/we may stop contributing to LLVM. 15. If we could go back in time and restart the project with today's technologies, which repository scheme would be best for the LLVM project? - CVS. - Subversion repository with split sub-projects (<sub-project>/trunk), with git-svn. - Subversion repository as a single project (trunk/<sub-project>), with git-svn. - Git: multirepo variant. - Git: monorepo variant. - Git: multi/mono hybrid variant. - Other. 16. Other comments. // Open text field. ---- Thanks again for your hard work on this! Duncan P.S. I think there's a bug on in the survey (or maybe my browser). I'm seeing some stray "If responding for a" text at the top. See the PDF I've attached here. -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2016-10-12 at 16.16.05.png.pdf Type: application/pdf Size: 61771 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20161012/56e72377/attachment-0001.pdf> -------------- next part --------------> > We only have a month and a bit now... > > cheers, > --renato > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev