llvm dev - Nov 2018 - RFC: Dealing with out of tree changes and the LLVM git monorepo

Justin Bogner via llvm-dev <llvm-dev at lists.llvm.org> writes:
> The layout here is not at all different, only the process by which the
> repo is generated. I strongly believe that a history preserving
> conversion is very important if we want to avoid making porting
> out-of-tree work horribly disruptive.
How would an out-of-tree branch be ported with this new approach?  Do
you have scripts to do it?

                          -David

David Greene <dag at cray.com> writes:
> Justin Bogner via llvm-dev <llvm-dev at lists.llvm.org> writes:
>
>> The layout here is not at all different, only the process by which the
>> repo is generated. I strongly believe that a history preserving
>> conversion is very important if we want to avoid making porting
>> out-of-tree work horribly disruptive.
>
> How would an out-of-tree branch be ported with this new approach?  Do
> you have scripts to do it?
I'll write up some more detailed docs on this, but all you need to do is
a subtree merge to one of the zippered commits. Given an out of tree
branch called $YourBranch, some temporary name for the result
$YourMonorepoBranch, and a remote for the zippered monorepo called
monorepo, this works:

   % merge_base=$(git merge-base $YourBranch monorepo/master)
   % monorepo_commit=$(git log --reverse --ancestry-path --format=%H \
                               $merge_base..monorepo/master | head -1)
   % git checkout $YourMonorepoBranch $monorepo_commit
   % git merge -Xsubtree=llvm $YourMonorepoBranch

Now you have a version of your repo where everything was moved into the
llvm/ folder. You can now update your bots and build scripts, push it to
your internal repo and carry on.

If you want a monorepo view for all of your branches' histories too it's
more involved, but I'm not sure anyone really needs that. In any case,
even if someone does want that the nature of the zipper approach means
it could be done later non-destructively.

Justin Bogner <mail at justinbogner.com> writes:
> I'll write up some more detailed docs on this, but all you need to do
is
> a subtree merge to one of the zippered commits.
This will then prevent git-biisect from working properly, unfortunately.
Maybe most people don't need it be we should be aware of and communicate
the tradeoffs.
> If you want a monorepo view for all of your branches' histories too
it's
> more involved, but I'm not sure anyone really needs that. In any case,
> even if someone does want that the nature of the zipper approach means
> it could be done later non-destructively.
That's true, but then they would suffer the same duplication of history
that motivated you to create the zippered repository in the first place.
Again, we need to make the tradeoffs clear.

I'm not saying the zippered approach is bad, just that it's not quite
the panacea it may seem to be.

                        -David