llvm dev - Jun 2016 - Git Move: GitHub+modules proposal

> On Jun 29, 2016, at 10:03 AM, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi all,
> 
> A short summary: Takumi has done 90% of the work here:
> 
> https://github.com/llvm-project/llvm-project-submodule
> 
> and I've been talking to GitHub, and here are the answers to my
questions:
> 
> 
>> 1. How will the umbrella project's auto-increment hook work?
> 
> Since the umbrella project cannot see the sub-modules' commits without
> some form of update, there are two ways to do this:
> 
> P. Per push: Every push (not commit) on all other repositories will
> trigger a hook that will hit a URL on our server, telling it to
> generate an incremental ID, update some umbrella's SeqID property (or
> even a commit SHA) and update the sub-modules.
> 
> T. Time based: A cron job in our server would frequently pull from all
> repos and update ID/modules.
> 
> Option P is less confusion and more fine grained, but if it misfire,
> we'll lose that push, and its commits will be bundled with the next
> push on that repo.
> 
> Option T will invariably bundle things together, even from different
> repositories. The change that this will "correctly" merge an
> LLVM+Clang double-patch is not worth the trouble for the noise.
> 
> For both of them, we need an external server, as there's no way to
> update a repository's property from another.
That makes it fragile, and that’s why I disagree with your “90% done”
assessment.
What if the service behing the hook is down for a few days? Who will maintain
it?

— 
Mehdi


> 
> Multiple commits eventually getting into a single umbrella revision
> can be innocuous for development, but they can make controlling the
> version for releases a bit more complicated. Though, it would also
> have no effect on back-ports, since they'll be done on Git and get
> their own SeqID.
> 
> All in all, I'm not too worried about this...
> 
> 
>> 2. How do we update the commits mailing lists?
> 
> This is, apparently, trivial in GitHub:
> 
>
https://help.github.com/articles/managing-notifications-for-pushes-to-a-repository/#enabling-email-service-notifications-for-pushes-to-your-repository
> 
> Any more comments before we put this proposal to vote?
> 
> Is anyone going to propose an alternative Git solution?
> 
> Or maybe an external, reliable and trustworthy SVN repository (ie.
> *not* SourceForge)?
> 
> In the interests of brevity and peacefulness, we should aim to only
> have one vote, even if it has multiple choices, so if we have more
> proposals, please bring them up.
> 
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

> That makes it fragile, and that’s why I disagree with your “90% done”
assessment.
> What if the service behing the hook is down for a few days?
In the long-term view, a pretty trivial catch-up script ought to be
able to synthesize a sane history after any amount of down-time.
People could even run it locally for their bisecting needs if it was
that important to them.

In the short term, I don't think it's a critical enough service to
worry about, frankly. What we already have is hopelessly fragile:
right now when LLVM's server plays up it takes out absolutely
everything, in the proposed world it would take out this bisecting
convenience feature. Seems like a strict improvement to me.
> Who will maintain it?
I'm not the best scripter and I'd be happy to cede to someone else,
but I'd be willing if it meant we could make progress.

Tim.

On 30 June 2016 at 05:14, Tim Northover <t.p.northover at gmail.com>
wrote:
>> That makes it fragile, and that’s why I disagree with your “90% done”
assessment.
>> What if the service behing the hook is down for a few days?
>
> In the long-term view, a pretty trivial catch-up script ought to be
> able to synthesize a sane history after any amount of down-time.
> People could even run it locally for their bisecting needs if it was
> that important to them.
Yup. If the script is stable (as in sort stable), anyone running it
locally will get the same results as upstream and each other.

> In the short term, I don't think it's a critical enough service to
> worry about, frankly. What we already have is hopelessly fragile:
> right now when LLVM's server plays up it takes out absolutely
> everything, in the proposed world it would take out this bisecting
> convenience feature. Seems like a strict improvement to me.
I think it's even less important than that. Bisecting will work
*better* when using submodules than it does using SVN (because git
bisect is more powerful, allows me to track all modules' history, and
will rid me of a complicated downstream set of SVN-bisect scripts).

The only thing we *have* to have a sequential number for, are
releases. Even that can be ran manually.

We agreed to have sequential numbering from the start to allow
infrastructure to migrate slowly to a Git model. That can also have an
extra step to run the script if IDs are not populated yet.

>> Who will maintain it?
Whoever maintains the current infrastructure, which is currently the
Foundation. All scripts will be upstream.

So far, they (Tanya, Anton, Galina) have been very responsible to
every downtime and problems I found. I have no doubt that this will
continue to be a trend.

cheers,
--renato