Mehdi Amini via llvm-dev
2016-Jul-21 21:32 UTC
[llvm-dev] [RFC] One or many git repositories?
> On Jul 21, 2016, at 2:29 PM, Mehdi Amini <mehdi.amini at apple.com> wrote: > > >> On Jul 21, 2016, at 11:03 AM, Sanjoy Das via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> FWIW, like David Chisnall, we (Azul) have a problem with rewriting >> history. >> Our LLVM fork has O(100) changes diverging from upstream >> (though our branching structure is simple), and keeping all of that >> history is important. > > Nobody downstream has to adopt the new structure, I believe it is possible to extract only the “llvm” commits from the new repo and rebase them on top of the existing llvm repo. > This can be done on the fly by you CI, but it is also a deterministic process, i.e. you can restart from scratch anytime (assuming you have the original llvm.git repo and the new one). > >> >> What do people think of having one (or a set of) merge commit(s) >> merging in the non-llvm projects that will be part of the new >> monorepo? That's the only technique I can think of that will preserve >> history for downstream users by construction. > > I have no idea what you mean here?I think I understand what you mean: 1) checkout the existing clang repo 2) move everything in a subdirectory “clang” 3) commit the move 4) merge this into the new “llvm-project”. 5) repeat for every single project That should preserve the hashes and avoid user to have to “extract” the subproject to merge into their own branch. Annoyingly, it breaks git log path/to/file though. — Mehdi>> >> >> >> On Thu, Jul 21, 2016 at 10:44 AM, Jonathan Roelofs via llvm-dev >> <llvm-dev at lists.llvm.org> wrote: >>> >>> >>> On 7/21/16 11:03 AM, C Bergström via llvm-dev wrote: >>>> >>>> Monolithic is trying to solve the wrong problem - it's that simple. >>>> Any discussion or attempt to coddle those who think it's necessary is >>>> a waste of time. #dictator >>> >>> >>> Christopher, >>> >>> AFAICT, you haven't explained *why* it is the wrong problem. Mind >>> elaborating on that? >>> >>> >>> Jon >>> >>> p.s: edicts, appeals to authority, and ad hominems are not useful for >>> discussion. Doing that, and following up with "#dictator" further solidifies >>> that you know your own argument is b-s.... please stop. >>> >>> -- >>> Jon Roelofs >>> jonathan at codesourcery.com >>> CodeSourcery / Mentor Embedded >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> >> >> -- >> Sanjoy Das >> http://playingwithpointers.com >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >
Justin Bogner via llvm-dev
2016-Jul-21 21:56 UTC
[llvm-dev] [RFC] One or many git repositories?
Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> writes:>>> What do people think of having one (or a set of) merge commit(s) >>> merging in the non-llvm projects that will be part of the new >>> monorepo? That's the only technique I can think of that will preserve >>> history for downstream users by construction. >> >> I have no idea what you mean here? > > I think I understand what you mean: > > 1) checkout the existing clang repo > 2) move everything in a subdirectory “clang” > 3) commit the move > 4) merge this into the new “llvm-project”. > 5) repeat for every single project > > That should preserve the hashes and avoid user to have to “extract” > the subproject to merge into their own branch. > Annoyingly, it breaks git log path/to/file though.Use `git log --follow path/to/file`. It's better ;)
Mehdi Amini via llvm-dev
2016-Jul-21 22:03 UTC
[llvm-dev] [RFC] One or many git repositories?
> On Jul 21, 2016, at 2:56 PM, Justin Bogner <mail at justinbogner.com> wrote: > > Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> writes: >>>> What do people think of having one (or a set of) merge commit(s) >>>> merging in the non-llvm projects that will be part of the new >>>> monorepo? That's the only technique I can think of that will preserve >>>> history for downstream users by construction. >>> >>> I have no idea what you mean here? >> >> I think I understand what you mean: >> >> 1) checkout the existing clang repo >> 2) move everything in a subdirectory “clang” >> 3) commit the move >> 4) merge this into the new “llvm-project”. >> 5) repeat for every single project >> >> That should preserve the hashes and avoid user to have to “extract” >> the subproject to merge into their own branch. >> Annoyingly, it breaks git log path/to/file though. > > Use `git log --follow path/to/file`. It's better ;)I know, it works most of the time for log, but how do blame it at a revision older than the move? — Mehdi
Mehdi Amini via llvm-dev
2016-Jul-21 22:27 UTC
[llvm-dev] [RFC] One or many git repositories?
> On Jul 21, 2016, at 2:32 PM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: > >> >> On Jul 21, 2016, at 2:29 PM, Mehdi Amini <mehdi.amini at apple.com> wrote: >> >> >>> On Jul 21, 2016, at 11:03 AM, Sanjoy Das via llvm-dev <llvm-dev at lists.llvm.org> wrote: >>> >>> FWIW, like David Chisnall, we (Azul) have a problem with rewriting >>> history. >>> Our LLVM fork has O(100) changes diverging from upstream >>> (though our branching structure is simple), and keeping all of that >>> history is important. >> >> Nobody downstream has to adopt the new structure, I believe it is possible to extract only the “llvm” commits from the new repo and rebase them on top of the existing llvm repo. >> This can be done on the fly by you CI, but it is also a deterministic process, i.e. you can restart from scratch anytime (assuming you have the original llvm.git repo and the new one).To clarify, I believe we* can maintain this read-only view of the individual repo like (https://github.com/llvm-mirror/llvm <https://github.com/llvm-mirror/llvm> and siblings) with their current history (hashes…), and adding on top the new commits from the new repo: *: we could be the LLVM foundation or some volunteer or each downstream user/org. — Mehdi>> >>> >>> What do people think of having one (or a set of) merge commit(s) >>> merging in the non-llvm projects that will be part of the new >>> monorepo? That's the only technique I can think of that will preserve >>> history for downstream users by construction. >> >> I have no idea what you mean here? > > I think I understand what you mean: > > 1) checkout the existing clang repo > 2) move everything in a subdirectory “clang” > 3) commit the move > 4) merge this into the new “llvm-project”. > 5) repeat for every single project > > That should preserve the hashes and avoid user to have to “extract” the subproject to merge into their own branch. > Annoyingly, it breaks git log path/to/file though. > > — > Mehdi > > > > >>> >>> >>> >>> On Thu, Jul 21, 2016 at 10:44 AM, Jonathan Roelofs via llvm-dev >>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >>>> >>>> >>>> On 7/21/16 11:03 AM, C Bergström via llvm-dev wrote: >>>>> >>>>> Monolithic is trying to solve the wrong problem - it's that simple. >>>>> Any discussion or attempt to coddle those who think it's necessary is >>>>> a waste of time. #dictator >>>> >>>> >>>> Christopher, >>>> >>>> AFAICT, you haven't explained *why* it is the wrong problem. Mind >>>> elaborating on that? >>>> >>>> >>>> Jon >>>> >>>> p.s: edicts, appeals to authority, and ad hominems are not useful for >>>> discussion. Doing that, and following up with "#dictator" further solidifies >>>> that you know your own argument is b-s.... please stop. >>>> >>>> -- >>>> Jon Roelofs >>>> jonathan at codesourcery.com >>>> CodeSourcery / Mentor Embedded >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> >>> >>> -- >>> Sanjoy Das >>> http://playingwithpointers.com >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160721/5d64d33b/attachment.html>
Mehdi Amini via llvm-dev
2016-Jul-22 06:14 UTC
[llvm-dev] [RFC] One or many git repositories?
> On Jul 21, 2016, at 2:32 PM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote: > >> >> On Jul 21, 2016, at 2:29 PM, Mehdi Amini <mehdi.amini at apple.com> wrote: >> >> >>> On Jul 21, 2016, at 11:03 AM, Sanjoy Das via llvm-dev <llvm-dev at lists.llvm.org> wrote: >>> >>> FWIW, like David Chisnall, we (Azul) have a problem with rewriting >>> history. >>> Our LLVM fork has O(100) changes diverging from upstream >>> (though our branching structure is simple), and keeping all of that >>> history is important. >> >> Nobody downstream has to adopt the new structure, I believe it is possible to extract only the “llvm” commits from the new repo and rebase them on top of the existing llvm repo. >> This can be done on the fly by you CI, but it is also a deterministic process, i.e. you can restart from scratch anytime (assuming you have the original llvm.git repo and the new one). >> >>> >>> What do people think of having one (or a set of) merge commit(s) >>> merging in the non-llvm projects that will be part of the new >>> monorepo? That's the only technique I can think of that will preserve >>> history for downstream users by construction. >> >> I have no idea what you mean here? > > I think I understand what you mean: > > 1) checkout the existing clang repo > 2) move everything in a subdirectory “clang” > 3) commit the move > 4) merge this into the new “llvm-project”. > 5) repeat for every single project > > That should preserve the hashes and avoid user to have to “extract” the subproject to merge into their own branch. > Annoyingly, it breaks git log path/to/file though.Just tried to set it up there: https://github.com/joker-eph/llvm-unified (git log —follow is working fine with this setup). While it preserves the history fine (I.e. the hashes are identical to the current git), it has a drawback: there isn’t anymore a common ancestor for the parents of the merge (this may or may not be an issue in practice, not sure yet, but it is uncommon for git). Now I need to write the script to regenerate the independent “read-only” repos. — Mehdi> > > > >>> >>> >>> >>> On Thu, Jul 21, 2016 at 10:44 AM, Jonathan Roelofs via llvm-dev >>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >>>> >>>> >>>> On 7/21/16 11:03 AM, C Bergström via llvm-dev wrote: >>>>> >>>>> Monolithic is trying to solve the wrong problem - it's that simple. >>>>> Any discussion or attempt to coddle those who think it's necessary is >>>>> a waste of time. #dictator >>>> >>>> >>>> Christopher, >>>> >>>> AFAICT, you haven't explained *why* it is the wrong problem. Mind >>>> elaborating on that? >>>> >>>> >>>> Jon >>>> >>>> p.s: edicts, appeals to authority, and ad hominems are not useful for >>>> discussion. Doing that, and following up with "#dictator" further solidifies >>>> that you know your own argument is b-s.... please stop. >>>> >>>> -- >>>> Jon Roelofs >>>> jonathan at codesourcery.com >>>> CodeSourcery / Mentor Embedded >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> >>> >>> -- >>> Sanjoy Das >>> http://playingwithpointers.com >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160721/81ab58bb/attachment.html>
David Chisnall via llvm-dev
2016-Jul-22 07:31 UTC
[llvm-dev] [RFC] One or many git repositories?
On 22 Jul 2016, at 07:14, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > Just tried to set it up there: https://github.com/joker-eph/llvm-unified > (git log —follow is working fine with this setup). > > While it preserves the history fine (I.e. the hashes are identical to the current git), it has a drawback: there isn’t anymore a common ancestor for the parents of the merge (this may or may not be an issue in practice, not sure yet, but it is uncommon for git).Perhaps I’m missing something, but on GitHub I can’t see any of the history. Assuming that this is just a GitHub UI issue, can you explain the steps for a downstream user who has clones of the llvm and clang repos that were forked at some point in the past, have been periodically merged, and contain a load of commits on top of upstream would migrate to using this? David -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3719 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160722/a19c4213/attachment.bin>