llvm dev - Jun 2016 - Git Move: GitHub+modules proposal

On 27 June 2016 at 01:20, Matthias Braun <matze at braunis.de>
wrote:
> I really liked the the solution proposed earlier in this thread: Do nothing
server side, but instead use
> `git rev-list --count master` on the client side (which takes 0.9s on my
machine) to get the number of the commit. So nothing to do on the ID part IMO.
Mehdi replied to this proposal:

"it does not help to solve the cross-repository problem, we need a
"meta integration repository"."

> As for updating the meta repository: We could disable write access for the
normal llvm developer and delegate the submodule bumping to an external
> server. I believe this would be an easy enough job for buildbot or jenkins.
The plan is to disable all write access to this repository (otherwise
we'll create a nightmare). Having an external counter could be
problematic due to synchronisation issues.

If the hook doesn't work, we'll have serious problems.

cheers,
--renato

>> As for updating the meta repository: We could disable write access for
the normal llvm developer and delegate the submodule bumping to an external
>> server. I believe this would be an easy enough job for buildbot or
jenkins.
>
> The plan is to disable all write access to this repository (otherwise
> we'll create a nightmare). Having an external counter could be
> problematic due to synchronisation issues.
>
> If the hook doesn't work, we'll have serious problems.
So, I probably missed something, but what was the main objection to
just using submodules? This would put llvm inside clang instead of the
other way around. When changing an API one currently has to

* Change llvm.
* Quickly change clang and hope no bot picks up a revision with only
the llvm change.

With submodules it becomes

* Change llvm.
* Change clang and in the same atomic commit change what revision the
submodule points to.


Cheers,
Rafael

Two problems:
1) Submodules have some UX problems for developers around updating the
parent project and its effects on the submodule which make them annoying to
use.
2) I find the advantage you claim especially scary and bad. Put another
way: if a developer *doesn't* make a commit to clang with the new submodule
pointer, clang will not actually start using the new version of LLVM until
someone gets around to updating the pointer. Meaning, the next time anyone
*ELSE* goes to update the submodule pointer in clang, they would have to,
effectively, integrate all of the as-yet-untested-with-clang changes from
llvm, and fix any problems that might cause. I really don't think we want
that.

What I do think would be nice is a way to associate commits to llvm and
commits to clang as one unit that updates a parent repository. I've
mentioned before that gerrit seems to have that functionality.

I think it'd be a great idea to do some testing, and make a second proposal
centered around using Gerrit to manage commits to the github repository,
versus committing to github directly. I'm not sure if I'll have time to
do
that properly, though.


On Mon, Jun 27, 2016 at 10:39 AM, Rafael Espíndola <cfe-dev at
lists.llvm.org>
wrote:
> >> As for updating the meta repository: We could disable write access
for
> the normal llvm developer and delegate the submodule bumping to an external
> >> server. I believe this would be an easy enough job for buildbot or
> jenkins.
> >
> > The plan is to disable all write access to this repository (otherwise
> > we'll create a nightmare). Having an external counter could be
> > problematic due to synchronisation issues.
> >
> > If the hook doesn't work, we'll have serious problems.
>
> So, I probably missed something, but what was the main objection to
> just using submodules? This would put llvm inside clang instead of the
> other way around. When changing an API one currently has to
>
> * Change llvm.
> * Quickly change clang and hope no bot picks up a revision with only
> the llvm change.
>
> With submodules it becomes
>
> * Change llvm.
> * Change clang and in the same atomic commit change what revision the
> submodule points to.
>
>
> Cheers,
> Rafael
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160627/aa83e3ed/attachment.html>

On 27 June 2016 at 15:39, Rafael Espíndola <rafael.espindola at gmail.com>
wrote:
> So, I probably missed something, but what was the main objection to
> just using submodules? This would put llvm inside clang instead of the
> other way around. When changing an API one currently has to
I don't think the consensus was to change the order of inclusion (llvm
into clang), but to *not* change anything else at this stage.

That's one of the reasons the umbrella project with sub-modules was
the most accepted solution, because we can later change the inclusion
order (if we all agree, etc), without changing the underlying storage.

> * Change llvm.
> * Change clang and in the same atomic commit change what revision the
> submodule points to.
Having one repository inside another was rejected due to the problems
it brings for development, validation and release. James has just
pointed a few of those problems for development.

An umbrella project with a commit hook and an auto-update would make
sure all commits are synchronised correctly. Though, indeed, this will
mean we'll still have the trouble of buildbots picking up one commit
and not the other, I don't think this is a big enough problem that we
should mess up everyone's workflow.

cheers,
--renato