llvm dev - Feb 2019 - [cfe-dev] [Github] RFC: linear history vs merge commits

<paul.robinson at sony.com> writes:
> Systems that I've seen will funnel all submitted PRs into a single
queue
> which *does* guarantee that the trial builds are against HEAD and there
> are no "later commits" that can screw it up. If the trial build
passes,
> the PR goes in and becomes the new HEAD.
The downside of a system like this is that when changes are going in
rapidly, the queue grows very large and it takes forever to get your
change merged.  PR builds are serialized and if a "build" means
"make
sure it builds on all the Buildbots" then it takes a very long time
indeed.

There are ways to parallelize builds by speculating that PRs will build
cleanly but it gets pretty complicated quickly.
> But this would be a radical redesign of the LLVM bot system, and would
> have to wait until we're done with the GitHub migration and can spend
> a couple of years debating the use of PRs. :-)
Heh.  Fully guaranteeing buildability of a branch is not a trivial task
and will take a LOT of thought and discussion.

                              -David

I realise that llvm trunk can move fairly quickly.
So my original, but brief, suggestion was to merge the current set of
approved patches together rather than attempting them one at a time.
Build on a set of fast smoke test bots. If something breaks, it should be
possible to bisect it to reject a PR and make forward progress.
Occasionally bisecting a large set of PR's should still be less bot time
than attempting to build each of them individually.
Blocking the PR's due to target specific and or slow bot failures would be
a different question.
You could probably do this with a linear history, so long as the final tree
is the same as the merge commit, it should still build.
I'm just fond of the idea of trying to prevent bad commits from ever being
merged. Since they sometimes waste everyone's time.

On Fri, 1 Feb 2019 at 04:02, David Greene <dag at cray.com> wrote:
> <paul.robinson at sony.com> writes:
>
> > Systems that I've seen will funnel all submitted PRs into a single
queue
> > which *does* guarantee that the trial builds are against HEAD and
there
> > are no "later commits" that can screw it up. If the trial
build passes,
> > the PR goes in and becomes the new HEAD.
>
> The downside of a system like this is that when changes are going in
> rapidly, the queue grows very large and it takes forever to get your
> change merged.  PR builds are serialized and if a "build" means
"make
> sure it builds on all the Buildbots" then it takes a very long time
> indeed.
>
> There are ways to parallelize builds by speculating that PRs will build
> cleanly but it gets pretty complicated quickly.
>
> > But this would be a radical redesign of the LLVM bot system, and would
> > have to wait until we're done with the GitHub migration and can
spend
> > a couple of years debating the use of PRs. :-)
>
> Heh.  Fully guaranteeing buildability of a branch is not a trivial task
> and will take a LOT of thought and discussion.
>
>                               -David
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190201/7fbb1c74/attachment-0001.html>

Oh, I'm completely in favor of making bad commits much less likely.  I
simply think there is a decent solution between "let everything in"
and
"don't let everything in unless its proven to work everywhere"
that gets
90% of the improvement.  The complexity of guaranteeing a buildable
branch is high.  If someone wants to take that on, great!  I just don't
think we should reasonably expect a group of volunteers to do it.  :)

                        -David

Jeremy Lakeman <Jeremy.Lakeman at gmail.com> writes:
> I realise that llvm trunk can move fairly quickly. 
> So my original, but brief, suggestion was to merge the current set of
> approved patches together rather than attempting them one at a time.
> Build on a set of fast smoke test bots. If something breaks, it should
> be possible to bisect it to reject a PR and make forward progress.
> Occasionally bisecting a large set of PR's should still be less bot
> time than attempting to build each of them individually.
> Blocking the PR's due to target specific and or slow bot failures
> would be a different question.
> You could probably do this with a linear history, so long as the final
> tree is the same as the merge commit, it should still build.
> I'm just fond of the idea of trying to prevent bad commits from ever
> being merged. Since they sometimes waste everyone's time.
>
> On Fri, 1 Feb 2019 at 04:02, David Greene <dag at cray.com> wrote:
>
>     <paul.robinson at sony.com> writes:
>     
>     > Systems that I've seen will funnel all submitted PRs into a
>     single queue
>     > which *does* guarantee that the trial builds are against HEAD
>     and there
>     > are no "later commits" that can screw it up. If the
trial build
>     passes,
>     > the PR goes in and becomes the new HEAD.
>     
>     The downside of a system like this is that when changes are going
>     in
>     rapidly, the queue grows very large and it takes forever to get
>     your
>     change merged. PR builds are serialized and if a "build"
means
>     "make
>     sure it builds on all the Buildbots" then it takes a very long
>     time
>     indeed.
>     
>     There are ways to parallelize builds by speculating that PRs will
>     build
>     cleanly but it gets pretty complicated quickly.
>     
>     > But this would be a radical redesign of the LLVM bot system, and
>     would
>     > have to wait until we're done with the GitHub migration and
can
>     spend
>     > a couple of years debating the use of PRs. :-)
>     
>     Heh. Fully guaranteeing buildability of a branch is not a trivial
>     task
>     and will take a LOT of thought and discussion.
>     
>     -David