[llvm-dev] [RFC] One or many git repositories?

Mehdi Amini via llvm-dev
Fri, 22 Jul 2016 23:35:36 -0000

> On Jul 22, 2016, at 1:16 AM, Simon Taylor via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>
> Hi all,
>
> Ill start by saying Ive skimmed this thread and am not actually a user of
LLVM at all, but had some git thoughts that might be worth contributing.
>
>> On 22 Jul 2016, at 01:16, Sanjoy Das via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>
>> @David Chisnall and others with local forks: can you spot any
>> potential issues with Mehdi's plan?  Are there cases where it
won't
>> work?
>
> One potential issue is that a single commit into the monolithic repository
would potentially touch multiple subprojects (thats one of the advantages).
Projecting that into individual repositories would only commit changes to those
files, but the commit message would be maintained and might therefore be
confusing in the context of the individual repository, especially if only a
small part of the commit affects that individual sub-repo.
>
> Essentially if the projects are supposed to be separate modules, then
submodules is the solution to enforce that independence, ensuring commits in
each module only affect that module and have appropriate commit messages for
that context.
>
> If the submodules are in practice more intertwined then that then it does
feel like an ideologically pure solution that in the end just gets in the way of
developer productivity.
>
> Ive got a setup here that uses a hierarchy of submodules, so there is a
combined submodule that just ensures that its children (other submodules) are at
mutually compatible versions. That helped productivity (multiple consumers of
the combined submodule dont need to manually track versions of all the children)
but this discussion is pushing me towards the thought that actually a monorepo
would be a more productive solution anyway, and make more sense for
cross-cutting changes.
>
> And sorry to throw another option into the ring; and one that might already
have been discussed and discounted, but thought it worth sharing.
>
> 1) Create a new llvm-project-mono repo
> 2) Use git subtree instead of git submodule to add all the directories to
match the layout of llvm-project.
> 3) From now on, all commits go to the monorepo
> 4) monorepo commits can be projected to the individual project repos, and
additionally a new commit on llvm-project can be made with the submodule version
updates

This is what I proposed except Im not using subtree but an explicit move commit
in the existing repo before merging them:
https://github.com/joker-eph/llvm-unified

The reason I didnt go with subtree merging is that it breaks `git log --follow
path/to/file`. I suspect not many tools (blame history in a text editor) are
supporting the subtree metadata in the merge commit.

Do you any drawback to what I did instead?


Mehdi



>
> Advantages:
>
> - No change for existing downstream users unless they want to move to the
mono view
> - Easier developer experience for cross-cutting changes
> - Git log by path would work identically on either view of the repository
> - Hashes from before the creation of the mono repo would match in both
views - the mono repo will have multiple roots but thats not unusual with git
subtree
>
> Disadvantages:
>
> - Step 4 from my list would need a script to keep things updated. A
server-side hook would be best. The mapping is deterministic (every mono repo
commit will map to one commit in any affected submodules and one submodule
update commit in the umbrella llvm-project repo), so if the server responsible
falls over the updates might be delayed but can be caught up without losing
anything
> - Less ideologically pure in terms of trying to keep the modules
independent
> - Commit hashes will diverge between the two views from the creation of the
mono repo, making comparisons / merges between clones of the different views
more difficult
>
>
> Simon