So here's a straw-man proposal for a github migration: 1. Register an official github project with the llvm foundation. 2. Setup another (read-only) mirror of llvm.org/git at this github project 3. Make sure we have ala llvm-project-submodules setup in the official account. (Optional or necessary for the buildbots?) 4. Update the buildbots to pick up updates and commits from the official git repository 5. Update phabricator to pick up commits from the official git repository 6. Tell people living downstream to pick up commits from the official git repository 7. Make sure bisecting with llvm-project-submodules is a good experience 8. Give things time to settle. We could play some games like disabling the svn repository for a few hours on purpose so that people can test that their infrastructure has really become independent of the svn repository. 9. Review and update llvm documentation. 10. Review website links pointing to viewvc/klaus/phab etc. to point to github instead. ... Until this point nothing has changed for developers, it will just boil down to a lot of work for buildbot and other infrstructure owners ... 11. Collect peoples github account information, give them push access. Ideally while still locking the github repository somehow... 12. Switch svn repository to read-only and allow pushs to the github repository. 13. Disable/remove/archive the svn repository. - Matthias> On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > >> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> On 1 June 2016 at 17:02, John Criswell <jtcriswel at gmail.com> wrote: >>> Do you have a set of volunteers lined up to do such a migration? Getting >>> people willing to do the migration will obviously be key, and that was the >>> one thing I didn't see in the original email. >> >> Hi John, >> >> Well, first we need to know if people are in favour, then if the >> migration won't bring any serious problem, and then we can think of a >> migration plan. :) >> >> So far, it seems people are mostly in favour, with a few that reported >> being locked into SVN. I had anticipated that, and have proposed >> GitHub's SVN integration, which allows read-write access, so it should >> be mostly ok. We need more tests on that side to be sure, though. >> >> The biggest problem we're facing right now is how to sync the repos. >> The existing llvm-repos format with all projects as sub-modules seem >> to be a good candidate, but I still haven't seen a consensus on how >> we'd do that. >> >> About the migration plan, most people seem to agree a step-by-step >> process is necessary. > > I personnally disagree with that, see below. > >> So, first we move to git-only, possibly with >> sub-modules > > If you move to git-only without the rest of the infrastructure/scripts, we're losing the convenience we have today with svn, and the "user experience" will not be so great. We may face resistance to this change. > I advocate to first set it up till it reaches the point of "good enough" in term of usability before actually migrating. > >> , then we move to GitHub/Lab/BB, > > It means we first need to host our git, write the tooling around it, to then migrate to github. I don't see the benefit over migrating directly to github: if people have to change their configuration, better have one single change. > >> then we move Phab to >> GitHub merge requests, etc. > > Phab to GitHub merge requests is a totally separate discussion IMO, and this discussion can happen after a successful move. > > >> >> Regarding volunteers, I haven't yet asked around yet, but I'm sure we >> have enough interested people to help. If everything else fails, I'm >> more than happy to do all of that myself. Though, I'd have to learn a >> lot more about sub-modules than I know today, which is effectively >> nothing. :) > > I'll be happy to throw manpower into tooling/infrastructure to make it happen. > > -- > Mehdi > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> On Jun 1, 2016, at 11:51 AM, Matthias Braun <mbraun at apple.com> wrote: > > So here's a straw-man proposal for a github migration: > > 1. Register an official github project with the llvm foundation. > 2. Setup another (read-only) mirror of llvm.org/git at this github project > 3. Make sure we have ala llvm-project-submodules setup in the official account. (Optional or necessary for the buildbots?) > 4. Update the buildbots to pick up updates and commits from the official git repository > 5. Update phabricator to pick up commits from the official git repository > 6. Tell people living downstream to pick up commits from the official git repository > 7. Make sure bisecting with llvm-project-submodules is a good experience > 8. Give things time to settle. We could play some games like disabling the svn repository for a few hours on purpose so that people can test that their infrastructure has really become independent of the svn repository. > 9. Review and update llvm documentation. > 10. Review website links pointing to viewvc/klaus/phab etc. to point to github instead. > ... Until this point nothing has changed for developers, it will just boil down to a lot of work for buildbot and other infrstructure owners ... > 11. Collect peoples github account information, give them push access. Ideally while still locking the github repository somehow... > 12. Switch svn repository to read-only and allow pushs to the github repository.12.2: mirror git to svn :)> 13. Disable/remove/archive the svn repository.LGTM, this is how I'd approach it as well, thanks for writing it down! -- Mehdi> > - Matthias > >> On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> >>> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: >>> >>> On 1 June 2016 at 17:02, John Criswell <jtcriswel at gmail.com> wrote: >>>> Do you have a set of volunteers lined up to do such a migration? Getting >>>> people willing to do the migration will obviously be key, and that was the >>>> one thing I didn't see in the original email. >>> >>> Hi John, >>> >>> Well, first we need to know if people are in favour, then if the >>> migration won't bring any serious problem, and then we can think of a >>> migration plan. :) >>> >>> So far, it seems people are mostly in favour, with a few that reported >>> being locked into SVN. I had anticipated that, and have proposed >>> GitHub's SVN integration, which allows read-write access, so it should >>> be mostly ok. We need more tests on that side to be sure, though. >>> >>> The biggest problem we're facing right now is how to sync the repos. >>> The existing llvm-repos format with all projects as sub-modules seem >>> to be a good candidate, but I still haven't seen a consensus on how >>> we'd do that. >>> >>> About the migration plan, most people seem to agree a step-by-step >>> process is necessary. >> >> I personnally disagree with that, see below. >> >>> So, first we move to git-only, possibly with >>> sub-modules >> >> If you move to git-only without the rest of the infrastructure/scripts, we're losing the convenience we have today with svn, and the "user experience" will not be so great. We may face resistance to this change. >> I advocate to first set it up till it reaches the point of "good enough" in term of usability before actually migrating. >> >>> , then we move to GitHub/Lab/BB, >> >> It means we first need to host our git, write the tooling around it, to then migrate to github. I don't see the benefit over migrating directly to github: if people have to change their configuration, better have one single change. >> >>> then we move Phab to >>> GitHub merge requests, etc. >> >> Phab to GitHub merge requests is a totally separate discussion IMO, and this discussion can happen after a successful move. >> >> >>> >>> Regarding volunteers, I haven't yet asked around yet, but I'm sure we >>> have enough interested people to help. If everything else fails, I'm >>> more than happy to do all of that myself. Though, I'd have to learn a >>> lot more about sub-modules than I know today, which is effectively >>> nothing. :) >> >> I'll be happy to throw manpower into tooling/infrastructure to make it happen. >> >> -- >> Mehdi >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >
+1 from me. Our company has very restrictive firewalls and the proxy setup doesn't work with SVN and http. We have to use special machines in a DMZ when we push changes to SVN. To mitigate this we are able to use the individual git repos but that poses problems for regression testing when we attempt to use git bisect. We either have to try and find a "next nearest" commit for clang, compiler-rt, etc or use the unified repository llvm-project on github already. Lastly, it's non-trivial from only the Git repos to reconstruct the code that goes into a released compiler. I'd hope with this new submodule layout it'd be as simple as git branch llvm-3.10-release and cmake. On Wed, Jun 1, 2016 at 1:51 PM, Matthias Braun via llvm-dev < llvm-dev at lists.llvm.org> wrote:> So here's a straw-man proposal for a github migration: > > 1. Register an official github project with the llvm foundation. > 2. Setup another (read-only) mirror of llvm.org/git at this github project > 3. Make sure we have ala llvm-project-submodules setup in the official > account. (Optional or necessary for the buildbots?) > 4. Update the buildbots to pick up updates and commits from the official > git repository > 5. Update phabricator to pick up commits from the official git repository > 6. Tell people living downstream to pick up commits from the official git > repository > 7. Make sure bisecting with llvm-project-submodules is a good experience > 8. Give things time to settle. We could play some games like disabling the > svn repository for a few hours on purpose so that people can test that > their infrastructure has really become independent of the svn repository. > 9. Review and update llvm documentation. > 10. Review website links pointing to viewvc/klaus/phab etc. to point to > github instead. > ... Until this point nothing has changed for developers, it will just boil > down to a lot of work for buildbot and other infrstructure owners ... > 11. Collect peoples github account information, give them push access. > Ideally while still locking the github repository somehow... > 12. Switch svn repository to read-only and allow pushs to the github > repository. > 13. Disable/remove/archive the svn repository. > > - Matthias > > > On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > > > > >> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> > >> On 1 June 2016 at 17:02, John Criswell <jtcriswel at gmail.com> wrote: > >>> Do you have a set of volunteers lined up to do such a migration? > Getting > >>> people willing to do the migration will obviously be key, and that was > the > >>> one thing I didn't see in the original email. > >> > >> Hi John, > >> > >> Well, first we need to know if people are in favour, then if the > >> migration won't bring any serious problem, and then we can think of a > >> migration plan. :) > >> > >> So far, it seems people are mostly in favour, with a few that reported > >> being locked into SVN. I had anticipated that, and have proposed > >> GitHub's SVN integration, which allows read-write access, so it should > >> be mostly ok. We need more tests on that side to be sure, though. > >> > >> The biggest problem we're facing right now is how to sync the repos. > >> The existing llvm-repos format with all projects as sub-modules seem > >> to be a good candidate, but I still haven't seen a consensus on how > >> we'd do that. > >> > >> About the migration plan, most people seem to agree a step-by-step > >> process is necessary. > > > > I personnally disagree with that, see below. > > > >> So, first we move to git-only, possibly with > >> sub-modules > > > > If you move to git-only without the rest of the infrastructure/scripts, > we're losing the convenience we have today with svn, and the "user > experience" will not be so great. We may face resistance to this change. > > I advocate to first set it up till it reaches the point of "good enough" > in term of usability before actually migrating. > > > >> , then we move to GitHub/Lab/BB, > > > > It means we first need to host our git, write the tooling around it, to > then migrate to github. I don't see the benefit over migrating directly to > github: if people have to change their configuration, better have one > single change. > > > >> then we move Phab to > >> GitHub merge requests, etc. > > > > Phab to GitHub merge requests is a totally separate discussion IMO, and > this discussion can happen after a successful move. > > > > > >> > >> Regarding volunteers, I haven't yet asked around yet, but I'm sure we > >> have enough interested people to help. If everything else fails, I'm > >> more than happy to do all of that myself. Though, I'd have to learn a > >> lot more about sub-modules than I know today, which is effectively > >> nothing. :) > > > > I'll be happy to throw manpower into tooling/infrastructure to make it > happen. > > > > -- > > Mehdi > > > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160601/1d7bb3e5/attachment.html>
On 1 June 2016 at 19:55, Mehdi Amini <mehdi.amini at apple.com> wrote:> 12.2: mirror git to svn :)either that or use GitHub's SVN interface, which is RW. --renato
+1 We use git exclusively within QC, so this looks like simplification to us. There was mention early in the thread of continuing to enforce linear history; that’s important to our internal integration machinery. We do currently use the git-svn-id as a key for some of our internal processes, but it won’t be a problem to switch to something else. We use google repo to stitch together our build tree, so the submodule discussion is mostly a no-op for us (providing it ends up being the “llvm-project” flat tree implementation). On Jun 1, 2016, at 1:51 PM, Matthias Braun via llvm-dev <llvm-dev at lists.llvm.org> wrote:> So here's a straw-man proposal for a github migration: > > 1. Register an official github project with the llvm foundation. > 2. Setup another (read-only) mirror of llvm.org/git at this github project > 3. Make sure we have ala llvm-project-submodules setup in the official account. (Optional or necessary for the buildbots?) > 4. Update the buildbots to pick up updates and commits from the official git repository > 5. Update phabricator to pick up commits from the official git repository > 6. Tell people living downstream to pick up commits from the official git repository > 7. Make sure bisecting with llvm-project-submodules is a good experience > 8. Give things time to settle. We could play some games like disabling the svn repository for a few hours on purpose so that people can test that their infrastructure has really become independent of the svn repository. > 9. Review and update llvm documentation. > 10. Review website links pointing to viewvc/klaus/phab etc. to point to github instead. > ... Until this point nothing has changed for developers, it will just boil down to a lot of work for buildbot and other infrstructure owners ... > 11. Collect peoples github account information, give them push access. Ideally while still locking the github repository somehow... > 12. Switch svn repository to read-only and allow pushs to the github repository. > 13. Disable/remove/archive the svn repository. > > - Matthias > >> On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> >>> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: >>> >>> On 1 June 2016 at 17:02, John Criswell <jtcriswel at gmail.com> wrote: >>>> Do you have a set of volunteers lined up to do such a migration? Getting >>>> people willing to do the migration will obviously be key, and that was the >>>> one thing I didn't see in the original email. >>> >>> Hi John, >>> >>> Well, first we need to know if people are in favour, then if the >>> migration won't bring any serious problem, and then we can think of a >>> migration plan. :) >>> >>> So far, it seems people are mostly in favour, with a few that reported >>> being locked into SVN. I had anticipated that, and have proposed >>> GitHub's SVN integration, which allows read-write access, so it should >>> be mostly ok. We need more tests on that side to be sure, though. >>> >>> The biggest problem we're facing right now is how to sync the repos. >>> The existing llvm-repos format with all projects as sub-modules seem >>> to be a good candidate, but I still haven't seen a consensus on how >>> we'd do that. >>> >>> About the migration plan, most people seem to agree a step-by-step >>> process is necessary. >> >> I personnally disagree with that, see below. >> >>> So, first we move to git-only, possibly with >>> sub-modules >> >> If you move to git-only without the rest of the infrastructure/scripts, we're losing the convenience we have today with svn, and the "user experience" will not be so great. We may face resistance to this change. >> I advocate to first set it up till it reaches the point of "good enough" in term of usability before actually migrating. >> >>> , then we move to GitHub/Lab/BB, >> >> It means we first need to host our git, write the tooling around it, to then migrate to github. I don't see the benefit over migrating directly to github: if people have to change their configuration, better have one single change. >> >>> then we move Phab to >>> GitHub merge requests, etc. >> >> Phab to GitHub merge requests is a totally separate discussion IMO, and this discussion can happen after a successful move. >> >> >>> >>> Regarding volunteers, I haven't yet asked around yet, but I'm sure we >>> have enough interested people to help. If everything else fails, I'm >>> more than happy to do all of that myself. Though, I'd have to learn a >>> lot more about sub-modules than I know today, which is effectively >>> nothing. :) >> >> I'll be happy to throw manpower into tooling/infrastructure to make it happen. >> >> -- >> Mehdi >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-devJim Rowan jmr at codeaurora.org Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation
Matthias Braun via cfe-dev <cfe-dev at lists.llvm.org> writes:> 3. Make sure we have ala llvm-project-submodules setup in the official > account. (Optional or necessary for the buildbots?)> 7. Make sure bisecting with llvm-project-submodules is a good experienceI would like to have a fuller discussion about how to handle the multiple repositories before going forward. Submodules is not the only solution available and we should take some time to evaluate what's out there. There are definite downsides to submodules (and all the various solutions) and I personally need to better understand the tradeoffs. -David
> On Jun 2, 2016, at 11:22 AM, dag at cray.com wrote: > > Matthias Braun via cfe-dev <cfe-dev at lists.llvm.org> writes: > >> 3. Make sure we have ala llvm-project-submodules setup in the official >> account. (Optional or necessary for the buildbots?) > >> 7. Make sure bisecting with llvm-project-submodules is a good experience > > I would like to have a fuller discussion about how to handle the > multiple repositories before going forward. Submodules is not the only > solution available and we should take some time to evaluate what's out > there. There are definite downsides to submodules (and all the various > solutions) and I personally need to better understand the tradeoffs.My opinion is that submodules or not is an implementation details. For the sake of this high-level discussion we should be able to keep it as "submodules" meaning "some system to integrate and manage different repositories coherently". We should evaluate subtree as well, as you mentioned, but also the tradeoff of other tools like repo (Android). -- Mehdi