llvm dev - Nov 2015 - "Living Downstream Without Drowning" BOF @ Dev Meeting

Mike Edwards and I will be hosting a talk/BOF called "Living Downstream 
Without Drowning" which is for anyone maintaining a bunch of local changes 
to Clang/LLVM/etc.  We will present some procedures and tactics we've
evolved
at Sony, including patch tactics for reducing merge pain, and how we are 
throwing automation at the problem.
 
But we are really curious what YOU have done and what we can learn from each 
other, so we can all manage to keep from drowning under the flood of changes 
from upstream!
--paulr

I find the git imerge script extremely useful for this kind of situation.

https://github.com/mhagger/git-imerge

Logically, it does something similar to rebasing your local branch onto
EVERY commit in the upstream branch, in turn, until it finds conflicts.
There is cleverness to make this efficient, let you stop and restart the
merge, share intermediate state with others, build&test intermediate
results (automatically if you want). At the end of the process you can
choose to keep the intermediate commits that are the same as the result of
a "git merge" or a "git rebase" or a new feature the author
calls "rebase
with history" that keeps the original branch too and makes each rebased
commit have the corresponding commit in the original branch as a parent.


On Mon, Oct 19, 2015 at 8:52 PM, Robinson, Paul via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Mike Edwards and I will be hosting a talk/BOF called "Living
Downstream
> Without Drowning" which is for anyone maintaining a bunch of local
changes
> to Clang/LLVM/etc.  We will present some procedures and tactics we've
> evolved
> at Sony, including patch tactics for reducing merge pain, and how we are
> throwing automation at the problem.
>
> But we are really curious what YOU have done and what we can learn from
> each
> other, so we can all manage to keep from drowning under the flood of
> changes
> from upstream!
> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151019/9b5b53c6/attachment.html>

On 19 Oct 2015, at 19:05, Bruce Hoult via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> I find the git imerge script extremely useful for this kind of situation.
> 
> https://github.com/mhagger/git-imerge
> 
> Logically, it does something similar to rebasing your local branch onto
EVERY commit in the upstream branch, in turn, until it finds conflicts. There is
cleverness to make this efficient, let you stop and restart the merge, share
intermediate state with others, build&test intermediate results
(automatically if you want). At the end of the process you can choose to keep
the intermediate commits that are the same as the result of a "git
merge" or a "git rebase" or a new feature the author calls
"rebase with history" that keeps the original branch too and makes
each rebased commit have the corresponding commit in the original branch as a
parent.
Thank you very much for the recommendation!  The down side of git-imerge is that
it is O(NM), where N is the number of your commits since the last merge and M is
the number of upstream commits.  I just finished a merge of about 6 months with
4000 upstream commits and a bit over 100 local ones - it took about a week of
CPU time.  The huge up side is that it took a lot less human time than previous
merges.

The only irritation is that LLVM has a strong policy of reverting stuff that’s
broken.  The Tuples work was committed and reverted three times in quick
succession.  Each time introduced merge conflicts, though by the third time I’d
had to fix them, I was getting pretty good at it.  I don’t wish to change the
policy with regard to reverts (having a working and stable head is very
valuable), but it would be good if we could have a stronger policy that stuff
that is reverted goes into Phabricator and is not recommitted until the person
who initially reverted it has signed off (or, at least, there’s a strong
consensus that it’s the right thing).

The main lesson for me is that more frequent merges will be less painful. 
That’s a big change, because previously I’d spend a day or two fixing merge
conflicts no matter how frequently or infrequently I merged, so the incentive
was to merge every 3-6 months.

David

Thanks to everyone who attended our talk, and especially the dozen or more
people who talked to me afterward at the reception and all the next day!
PDF of the slides were contributed in r252262.

It's clear there's a pent-up demand for continuing the conversations on
all the related topics, and there will be different solutions that are
appropriate to different situations.  What would be a good way to keep
collaborating properly about all this?

For starters I'm about to head off to the Bay Area Social...
--paulr
> -----Original Message-----
> From: cfe-dev [mailto:cfe-dev-bounces at lists.llvm.org] On Behalf Of
> Robinson, Paul via cfe-dev
> Sent: Monday, October 19, 2015 10:52 AM
> To: llvm-dev at lists.llvm.org; cfe-dev at lists.llvm.org
> Subject: [cfe-dev] "Living Downstream Without Drowning" BOF @ Dev
Meeting
> 
> Mike Edwards and I will be hosting a talk/BOF called "Living
Downstream
> Without Drowning" which is for anyone maintaining a bunch of local
changes
> to Clang/LLVM/etc.  We will present some procedures and tactics we've
> evolved
> at Sony, including patch tactics for reducing merge pain, and how we are
> throwing automation at the problem.
> 
> But we are really curious what YOU have done and what we can learn from
> each
> other, so we can all manage to keep from drowning under the flood of
> changes
> from upstream!
> --paulr
> 
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev