Robinson, Paul via llvm-dev
2016-Jul-26 17:42 UTC
[llvm-dev] [RFC] One or many git repositories?
>> 3. For many (most?) developers, changing to a monolithic git repo is a >> *bigger* workflow change than switching to separate git repos. Many >> people (and at least some downstream infrastructure) use the git >> mirrors exclusively, aside from git-svn for committing. >> >> I believe the idea is to continue to maintain the read-only independent >> git repos for each project. The only change is instead of sourcing those >> commits from the official upstream(independent) svn repos, they will be >> sourced from the official upstream monorepo. >> >> Thus downstream developers can continue to use the read-only view of >> the independent projects if that is easier for them; but people hacking >> on llvm/clang itself get the benefits of easier checkout, patching, >> bisection, atomic commits between projects, etc that come from using a >> monorepo as the official repository. > > It is true that downstream should be able continue to work easily based > of these official independent *git* repos ( i.e.: > http://llvm.org/git/llvm.git will continue to exist and be updated > without changing its history). > However for individual developers that are continuously upstreaming their > work it does not seem sustainable, how do you pull from the individual > repos, do you work, commit as today, and push upstream? Today it is > possible with git-svn, while with the monorepo it won’t be possible.Speaking only for myself, the problem you suggest simply does not exist. Or another way of saying it: I already can't work the way you imagine. Our downstream repo is fed by the upstream projects, but it's simply impossible to directly commit work from there into upstream. I create a patch based on local work, apply it to a separate "vanilla" upstream checkout and commit from there. The upstream version of the patch cycles back into our repo and we use it to supersede the local version. All done. Git or svn, makes no difference. Monorepo or submodules, makes no difference. Those are all implementation details. Even if it were possible, I would still keep my upstream checkout separate just as a safety measure, to keep from sending private stuff upstream by accident. --paulr
Mehdi Amini via llvm-dev
2016-Jul-26 17:58 UTC
[llvm-dev] [RFC] One or many git repositories?
> On Jul 26, 2016, at 10:42 AM, Robinson, Paul <paul.robinson at sony.com> wrote: > >>> 3. For many (most?) developers, changing to a monolithic git repo is a >>> *bigger* workflow change than switching to separate git repos. Many >>> people (and at least some downstream infrastructure) use the git >>> mirrors exclusively, aside from git-svn for committing. >>> >>> I believe the idea is to continue to maintain the read-only independent >>> git repos for each project. The only change is instead of sourcing those >>> commits from the official upstream(independent) svn repos, they will be >>> sourced from the official upstream monorepo. >>> >>> Thus downstream developers can continue to use the read-only view of >>> the independent projects if that is easier for them; but people hacking >>> on llvm/clang itself get the benefits of easier checkout, patching, >>> bisection, atomic commits between projects, etc that come from using a >>> monorepo as the official repository. >> >> It is true that downstream should be able continue to work easily based >> of these official independent *git* repos ( i.e.: >> http://llvm.org/git/llvm.git will continue to exist and be updated >> without changing its history). >> However for individual developers that are continuously upstreaming their >> work it does not seem sustainable, how do you pull from the individual >> repos, do you work, commit as today, and push upstream? Today it is >> possible with git-svn, while with the monorepo it won’t be possible. > > Speaking only for myself, the problem you suggest simply does not exist. > Or another way of saying it: I already can't work the way you imagine. > Our downstream repo is fed by the upstream projects, but it's simply > impossible to directly commit work from there into upstream. I create > a patch based on local work, apply it to a separate "vanilla" upstream > checkout and commit from there.Except that you won’t be able to use "git format-patch” and “git am” anymore, you’ll have to use `git diff` and `patch` (Maybe that’s already what you’re doing).> The upstream version of the patch > cycles back into our repo and we use it to supersede the local version. > All done. Git or svn, makes no difference. Monorepo or submodules, > makes no difference. Those are all implementation details.I believe we have a similar setup in the sense that I live on a downstream branch, except that I have a unified git object store (git worktree) and I can *cherry-pick* from my private downstream branch checkout to the upstream checkout, which is even more streamline than creating a patch on disk and applying it. When the workflow is significantly impacted, it becomes more than an “implementation detail”. Maybe you don’t exploit the current possibility of git, but the fact is that some people (and I know this includes some of my colleagues so I’m not alone) will be impacted by this: i.e. having our downstream integration based of the individual repo would be a pain for me when switch back-and-forth my patches. — Mehdi> Even if it were possible, I would still keep my upstream checkout > separate just as a safety measure, to keep from sending private stuff > upstream by accident. > --paulr >
Mehdi Amini via llvm-dev
2016-Jul-26 18:04 UTC
[llvm-dev] [RFC] One or many git repositories?
> On Jul 26, 2016, at 10:58 AM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: > >> >> On Jul 26, 2016, at 10:42 AM, Robinson, Paul <paul.robinson at sony.com> wrote: >> >>>> 3. For many (most?) developers, changing to a monolithic git repo is a >>>> *bigger* workflow change than switching to separate git repos. Many >>>> people (and at least some downstream infrastructure) use the git >>>> mirrors exclusively, aside from git-svn for committing. >>>> >>>> I believe the idea is to continue to maintain the read-only independent >>>> git repos for each project. The only change is instead of sourcing those >>>> commits from the official upstream(independent) svn repos, they will be >>>> sourced from the official upstream monorepo. >>>> >>>> Thus downstream developers can continue to use the read-only view of >>>> the independent projects if that is easier for them; but people hacking >>>> on llvm/clang itself get the benefits of easier checkout, patching, >>>> bisection, atomic commits between projects, etc that come from using a >>>> monorepo as the official repository. >>> >>> It is true that downstream should be able continue to work easily based >>> of these official independent *git* repos ( i.e.: >>> http://llvm.org/git/llvm.git will continue to exist and be updated >>> without changing its history). >>> However for individual developers that are continuously upstreaming their >>> work it does not seem sustainable, how do you pull from the individual >>> repos, do you work, commit as today, and push upstream? Today it is >>> possible with git-svn, while with the monorepo it won’t be possible. >> >> Speaking only for myself, the problem you suggest simply does not exist. >> Or another way of saying it: I already can't work the way you imagine. >> Our downstream repo is fed by the upstream projects, but it's simply >> impossible to directly commit work from there into upstream. I create >> a patch based on local work, apply it to a separate "vanilla" upstream >> checkout and commit from there. > > Except that you won’t be able to use "git format-patch” and “git am” anymore, you’ll have to use `git diff` and `patch` (Maybe that’s already what you’re doing).I just found the `git am` option I was missing: —directory. This should allow to `git format-patch` from the single llvm repo and `git am —directory=llvm` in the monorepo. Less disruptive than I thought in my previous answer. — Mehdi> >> The upstream version of the patch >> cycles back into our repo and we use it to supersede the local version. >> All done. Git or svn, makes no difference. Monorepo or submodules, >> makes no difference. Those are all implementation details. > > I believe we have a similar setup in the sense that I live on a downstream branch, except that I have a unified git object store (git worktree) and I can *cherry-pick* from my private downstream branch checkout to the upstream checkout, which is even more streamline than creating a patch on disk and applying it. > > When the workflow is significantly impacted, it becomes more than an “implementation detail”. Maybe you don’t exploit the current possibility of git, but the fact is that some people (and I know this includes some of my colleagues so I’m not alone) will be impacted by this: i.e. having our downstream integration based of the individual repo would be a pain for me when switch back-and-forth my patches. > > — > Mehdi > > >> Even if it were possible, I would still keep my upstream checkout >> separate just as a safety measure, to keep from sending private stuff >> upstream by accident. >> --paulr >> > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160726/ad58add7/attachment.html>
Sanjoy Das via llvm-dev
2016-Jul-26 18:28 UTC
[llvm-dev] [RFC] One or many git repositories?
Hi all, Robinson, Paul via llvm-dev wrote:> Speaking only for myself, the problem you suggest simply does not exist. > Or another way of saying it: I already can't work the way you imagine. > Our downstream repo is fed by the upstream projects, but it's simply > impossible to directly commit work from there into upstream. I create > a patch based on local work, apply it to a separate "vanilla" upstream > checkout and commit from there. The upstream version of the patch > cycles back into our repo and we use it to supersede the local version. > All done. Git or svn, makes no difference. Monorepo or submodules, > makes no difference. Those are all implementation details. > > Even if it were possible, I would still keep my upstream checkout > separate just as a safety measure, to keep from sending private stuff > upstream by accident.Just FYI, this is our (Azul's) workflow as well, and for similar reasons. -- Sanjoy
Renato Golin via llvm-dev
2016-Jul-26 18:31 UTC
[llvm-dev] [RFC] One or many git repositories?
On 26 July 2016 at 19:28, Sanjoy Das via llvm-dev <llvm-dev at lists.llvm.org> wrote:>> Even if it were possible, I would still keep my upstream checkout >> separate just as a safety measure, to keep from sending private stuff >> upstream by accident. > > Just FYI, this is our (Azul's) workflow as well, and for similar > reasons.Same here. cheers, --renato