llvm dev - Nov 2018 - RFC: Dealing with out of tree changes and the LLVM git monorepo

Mehdi AMINI <joker.eph at gmail.com> writes:
>     > If you want a monorepo view for all of your branches'
histories
>     > too it's more involved, but I'm not sure anyone really
needs
>     > that. In any case, even if someone does want that the nature of
>     > the zipper approach means it could be done later
>     > non-destructively.
>     
>     That's true, but then they would suffer the same duplication of
>     history that motivated you to create the zippered repository in
>     the first place.
>
> It isn't clear to me why it would suffer from the same duplication of
> history?
> The duplication of history is only due to the *rewrite* of the git
> hash between the individual repo and the monorepo.
> If we were starting with a subtree merge and later add a "zipped"
> history, we would only add the merges commit without rewriting
> anything I think.
I'm not sure how this after-the-fact merge would work.  How was the
zippered monorepo history created?  Do a subtree merge of each
subproject and then...?

I was imagining someone taking their existing subproject repository tree
and simply pushing their local branches to a clone of the zippered
monorepo (ZMR).  So their commits would exist as branches off the ZMR's
subproject histories.  To then get them merged into a "proper"
monorepo
history would require...<something>...to merge a subproject commit with
an appropriate zippered commit to create a zippered local branch.
That's effectively two commits for each local downstream subproject
commit.

                          -David

Le lun. 5 nov. 2018 à 07:43, David Greene <dag at cray.com> a écrit :
> Mehdi AMINI <joker.eph at gmail.com> writes:
>
> >     > If you want a monorepo view for all of your branches'
histories
> >     > too it's more involved, but I'm not sure anyone
really needs
> >     > that. In any case, even if someone does want that the nature
of
> >     > the zipper approach means it could be done later
> >     > non-destructively.
> >
> >     That's true, but then they would suffer the same duplication
of
> >     history that motivated you to create the zippered repository in
> >     the first place.
> >
> > It isn't clear to me why it would suffer from the same duplication
of
> > history?
> > The duplication of history is only due to the *rewrite* of the git
> > hash between the individual repo and the monorepo.
> > If we were starting with a subtree merge and later add a
"zipped"
> > history, we would only add the merges commit without rewriting
> > anything I think.
>
> I'm not sure how this after-the-fact merge would work.  How was the
> zippered monorepo history created?  Do a subtree merge of each
> subproject and then...?
>
> I was imagining someone taking their existing subproject repository tree
> and simply pushing their local branches to a clone of the zippered
> monorepo (ZMR).  So their commits would exist as branches off the ZMR's
> subproject histories.  To then get them merged into a "proper"
monorepo
> history would require...<something>...to merge a subproject commit
with
> an appropriate zippered commit to create a zippered local branch.
> That's effectively two commits for each local downstream subproject
> commit.

Yes, but that's the case for the zipper repo anyway: one merge per commit.
The point is that the second commit is just a trivial merge, it wouldn't
show up in a file `git log` for example.
In the linear rewritten monorepo, adding the history taken from the
existing git mirror would lead to duplicated commits, as in *identical*
commit / commit with the same diff but different git hashes. I'd expect git
log to show us the two commits in the git log of a single file.

-- 
Mehdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181105/bad351fa/attachment.html>

Mehdi AMINI <joker.eph at gmail.com> writes:
> Yes, but that's the case for the zipper repo anyway: one merge per
> commit. The point is that the second commit is just a trivial merge,
> it wouldn't show up in a file `git log` for example.
> In the linear rewritten monorepo, adding the history taken from the
> existing git mirror would lead to duplicated commits, as in
> *identical* commit / commit with the same diff but different git
> hashes. I'd expect git log to show us the two commits in the git log
> of a single file.
Would it be valuable to have a tool to take branches from existing git
mirrors and rewrite them in terms of the monorepo so there would be no
duplicate commits and everything would appear to have been done against
the monorepo?

I know Justin is worried about hashes in old e-mails being invalid, but
the tool could include the mapping from old hash to new hash in the
commit message.  Of course that would only be done for local downstream
commits, as the monorepo commits were already rewritten without
including that information.  Would it be helpful to have the monorepo
commits contain that information?

                            -David