llvm dev - Nov 2018 - RFC: Dealing with out of tree changes and the LLVM git monorepo

Mehdi AMINI <joker.eph at gmail.com> writes:
> Yes, but that's the case for the zipper repo anyway: one merge per
> commit. The point is that the second commit is just a trivial merge,
> it wouldn't show up in a file `git log` for example.
> In the linear rewritten monorepo, adding the history taken from the
> existing git mirror would lead to duplicated commits, as in
> *identical* commit / commit with the same diff but different git
> hashes. I'd expect git log to show us the two commits in the git log
> of a single file.
Would it be valuable to have a tool to take branches from existing git
mirrors and rewrite them in terms of the monorepo so there would be no
duplicate commits and everything would appear to have been done against
the monorepo?

I know Justin is worried about hashes in old e-mails being invalid, but
the tool could include the mapping from old hash to new hash in the
commit message.  Of course that would only be done for local downstream
commits, as the monorepo commits were already rewritten without
including that information.  Would it be helpful to have the monorepo
commits contain that information?

                            -David

I'm about to post exactly this tool -- I've been testing it on the CHERI
forks of llvm/clang/lld (lots of history and merges and stuff there, makes
a pretty nice test case!)

On Mon, Nov 5, 2018 at 1:07 PM David Greene via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Mehdi AMINI <joker.eph at gmail.com> writes:
>
> > Yes, but that's the case for the zipper repo anyway: one merge per
> > commit. The point is that the second commit is just a trivial merge,
> > it wouldn't show up in a file `git log` for example.
> > In the linear rewritten monorepo, adding the history taken from the
> > existing git mirror would lead to duplicated commits, as in
> > *identical* commit / commit with the same diff but different git
> > hashes. I'd expect git log to show us the two commits in the git
log
> > of a single file.
>
> Would it be valuable to have a tool to take branches from existing git
> mirrors and rewrite them in terms of the monorepo so there would be no
> duplicate commits and everything would appear to have been done against
> the monorepo?
>
> I know Justin is worried about hashes in old e-mails being invalid, but
> the tool could include the mapping from old hash to new hash in the
> commit message.  Of course that would only be done for local downstream
> commits, as the monorepo commits were already rewritten without
> including that information.  Would it be helpful to have the monorepo
> commits contain that information?
>
>                             -David
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181105/77742af4/attachment.html>

Well shoot, you beat me to it.  :)

I've been working on a similar tool but it's not ready yet.  Looking
forward to trying yours!

                            -David

James Y Knight via llvm-dev <llvm-dev at lists.llvm.org> writes:
> I'm about to post exactly this tool -- I've been testing it on the
> CHERI forks of llvm/clang/lld (lots of history and merges and stuff
> there, makes a pretty nice test case!)
>
> On Mon, Nov 5, 2018 at 1:07 PM David Greene via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
>     Mehdi AMINI <joker.eph at gmail.com> writes:
>     
>     > Yes, but that's the case for the zipper repo anyway: one merge
>     per
>     > commit. The point is that the second commit is just a trivial
>     merge,
>     > it wouldn't show up in a file `git log` for example.
>     > In the linear rewritten monorepo, adding the history taken from
>     the
>     > existing git mirror would lead to duplicated commits, as in
>     > *identical* commit / commit with the same diff but different git
>     > hashes. I'd expect git log to show us the two commits in the
git
>     log
>     > of a single file.
>     
>     Would it be valuable to have a tool to take branches from existing
>     git
>     mirrors and rewrite them in terms of the monorepo so there would
>     be no
>     duplicate commits and everything would appear to have been done
>     against
>     the monorepo?
>     
>     I know Justin is worried about hashes in old e-mails being
>     invalid, but
>     the tool could include the mapping from old hash to new hash in
>     the
>     commit message. Of course that would only be done for local
>     downstream
>     commits, as the monorepo commits were already rewritten without
>     including that information. Would it be helpful to have the
>     monorepo
>     commits contain that information?
>     
>     -David
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev