llvm dev - Dec 2019 - Flang landing in the monorepo

I think it would probably make the most sense to land this as a single
merge-commit (from the 2700-commit rewritten history you've created) onto
llvm-project master, rather than as 2700 individual toplevel commits to
master. (Which means: disable the merge-commit prohibition in the github
configuration, temporarily, push this commit, and then enable it again).


On Tue, Dec 17, 2019 at 5:10 PM Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 12/17/2019 01:30 PM, Peter Waller via llvm-dev wrote:
> > Hi All,
> >
> > The flang project (a Fortran compiler) is getting ready to join the
> > monorepo. We intend to preserve the existing history by rewriting the
> > existing commits as a linear series of commits on top of llvm-project.
> >
> > I understand the flang community would like to do this before the LLVM
> > 10 branch in due in mid January, so please speak up soon if you see
> > anything needing fixing in what I write below.
> >
> > I've taken into account the discussion raised during the MLIR
landing
> > discussion found at
> > http://lists.llvm.org/pipermail/llvm-dev/2019-November/136813.html. As
> > with MLIR, we rewrite the commits so that flang's work all appears
to
> > happen in the flang directory, starting with llvm-project master as it
> > appears today. The topology of the f18 history was fairly interesting,
> > which is why I ended up writing a new program to rewrite it rather
than
> > using an existing one.
> >
> > === Key links
> >
> > * Resulting tree of the rewrite:
> >
>
https://github.com/peterwaller-arm/f18/tree/rewritten-history-v2-llvm-project-merge
> >
> > * Rewritten history, with flang commits applied on top of llvm-project
> > master:
> >
>
https://github.com/peterwaller-arm/f18/commits/rewritten-history-v2-llvm-project-merge
> >
> > * The history rewriting program is published here:
> >    https://github.com/flang-compiler/f18/pull/854
> >
> > * Latest mailing list discussion of rewrite on flang-dev:
> > http://lists.llvm.org/pipermail/flang-dev/2019-December/000122.html
> >
> > === Additional considerations
> >
> > * Existing references to pull request and issue numbers are rewritten
so
> > that they point at the originals as, e.g. flang-compiler/f18#123. This
> > prevents those patches from generating bogus references to Issues/PRs
of
> > llvm-project if/when those appear in the llvm-project repository.
> >
> > * Developers using the llvm-project repo, when they pull after this
> > push, will see 2,700ish commits appear on the tip. These will follow
on
> > as normal commits from wherever master is at the time of the push. The
> > fetch takes 40s and I see my ".git" directory grow by
approximately
> > 90MiB when I simulate this.
> >
>
> I think we may want to disable the commit emailer before all these
> new commits are pushed.  Once you are ready to push, can you pick a
> specific time and date for the push and then coordinate with Mike
(cc'd)
> and myself, so we can avoid spamming the mail server.
>
> Thanks,
> Tom
>
> > * Rewriting and validating the rewritten f18 history is sufficiently
> > fast that I don't think it will be necessary to pause commits to
LLVM.
> > The script runs in a few seconds. Before this is done though, I think
> > new commits should no longer be accepted on the original repository.
> >
> > * You can simulate the experience of the fresh merge with `git remote
> > add peterwaller-arm https://github.com/peterwaller-arm/f18 &&
time git
> > fetch peterwaller-arm rewritten-history-v2-llvm-project-merge`, and
then
> > look at the peterwaller-arm/rewritten-history-v2-llvm-project-merge
> > branch with git log.
> >
> > * Remember that you can restrict the "git log" output to
what you are
> > interested in by specifying a directory, e.g. `git log clang/`.
> >
> > That's all for now. Season's greetings!
> >
> > - Peter
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191217/e165d3ad/attachment.html>

On 17/12/2019 22:25, James Y Knight wrote:
> I think it would probably make the most sense to land this as a single 
> merge-commit (from the 2700-commit rewritten history you've created) 
> onto llvm-project master, rather than as 2700 individual toplevel 
> commits to master. (Which means: disable the merge-commit prohibition 
> in the github configuration, temporarily, push this commit, and then 
> enable it again).
I understood that the LLVM project did not accept any merge commit so 
far. It seems a shame to make an exception if we don't have to.

Can you elaborate on the perceived benefits of having it as a merge?

It's of course technically straightforward to push either "follow-on 
commits" or a merge commit.

I think there could be a benefit to having it as a "follow-on
commits",
in that checking out an old flang commit won't have the effect of 
deleting the rest of the llvm monorepo. Granted, this may be a slim 
benefit since it will be a fairly arbitrary revision of the llvm 
monorepo, taken at the point of the initial push.

On Wed, Dec 18, 2019 at 4:56 AM Peter Waller <Peter.Waller at arm.com>
wrote:
> On 17/12/2019 22:25, James Y Knight wrote:
> > I think it would probably make the most sense to land this as a single
> > merge-commit (from the 2700-commit rewritten history you've
created)
> > onto llvm-project master, rather than as 2700 individual toplevel
> > commits to master. (Which means: disable the merge-commit prohibition
> > in the github configuration, temporarily, push this commit, and then
> > enable it again).
>
> I understood that the LLVM project did not accept any merge commit so
> far. It seems a shame to make an exception if we don't have to
>
Can you elaborate on the perceived benefits of having it as a
merge?
>
The rule against merge commits is primarily because we want to encourage
people to make reasonable separate commits to master which are sensible
(reviewable, buildable, etc) in isolation. And, to make it so the history
is more easily understandable to humans. It's not only that we don't
want
merge commits -- we actually don't really want people doing merges, in
general.

But, here we actually *do* have a merge, because flang was developed
externally. This is an exceptional circumstance (and I'm sure we'll have
a
few more). Using a merge commit in this circumstance makes it easier for
humans looking at history to understand what's going on here, rather than
harder -- because it actually marks the merge as being a merge. That's the
main reason why I think we ought to use a merge commit here.

Additionally (and less importantly), I'm going to presume that many/most of
the 2700 commits cannot actually be built. So, having a single merge commit
within which the unbuildable sub-commits are contained will also make
things better for autobuilders.

>  It's of course technically straightforward to push either
"follow-on
> commits" or a merge commit.
>
> I think there could be a benefit to having it as a "follow-on
commits",
> in that checking out an old flang commit won't have the effect of
> deleting the rest of the llvm monorepo. Granted, this may be a slim
> benefit since it will be a fairly arbitrary revision of the llvm
> monorepo, taken at the point of the initial push.
>
I'm suggesting to build the history the exact same way you are currently,
and then simply merging it onto master with a single merge commit ("git
merge --no-ff"), rather than "fast-forwarding" the entire set of
commits.
So this is not a benefit, it'd be the same either way.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191218/585a5757/attachment.html>