llvm dev - Jan 2020 - Flang landing in the monorepo - next Monday!

Hi,

On Wed, Jan 8, 2020 at 6:54 PM Finkel, Hal J. via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> FYI to everyone: If you have things that you would like to see done
> before a merge of Flang, please reply with as many details as you have
> time to provide (and if you have things that you would like to see done
> soon, but you're comfortable with them happening after the merge,
please
> share those items as well). Our Flang community technical call on Monday
> morning will be dedicated to forming a concrete plan from any
> yet-unaddressed items and making sure that we have a checklist to address.
>
What is the latest/current proposed merge commit showing what will be
pushed to the monorepo?
Can you make sure to prune/repack the repo before pushing?
Are the license header updated to be the LLVM license?
The test don't seem to be lit-based testing: is this part of the TODO list?

Looking forward to flang joining the monorepo!

Thanks,

-- 
Mehdi




>
> Thanks again,
>
> Hal
>
> On 1/8/20 1:01 PM, Renato Golin wrote:
> > On Wed, 8 Jan 2020 at 01:48, Eric Christopher via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> >> I am in favor of having a flang front end in tree. I have concerns
> about the design of flang versus other front ends, the lack of llvm based
> library use, and a number of other things that I tried to enumerate in
> previous emails. I don't know if anything has changed and the responses
I
> got back originally were "we're going to do it anyway" so it
didn't leave
> much room for engagement.
> > (Disclaimer: Not taking sides, I have no stake in Flang or Fortran).
> >
> > I think David's comparison to LLDB is interesting. It started very
> > distant, and then with time merged with the codebase and now it's
a
> > full fledged LLVM project.
> >
> > The current Flang (F18) is meant to be much closer to LLVM than the
> > previous one, and the whole mindset was afaik to keep it that way. In
> > the same way that LLDB once was.
> >
> > I do agree with Hal that there is a small but significant overlap of
> > communities, and I do agree with Rick that the longer it stays
> > separate, the harder it will be for that sub-community.
> >
> > But LLDB started at a time where "being an LLVM project" was
mostly
> > about being in our SVN repo. There was no mono-repo, and the cost of
> > being there was lower-ish.
> >
> > I believe that can be solved with build semantics (CMake stuff?), not
> > a big problem, so the main issue is about community: will the project
> > move fast enough towards LLVM integration or will it dangle with
> > downstream implementations and be mostly useless upstream?
> >
> > I think we have enough people that care about Flang publicly putting
> > their names forward. This makes me assume the former, so I'm
> > personally ok with the merge.
> >
> > If it happens before the current branch or not, will depend on how
> > fast they're willing to work towards a working Fortran front-end.
> >
> > I'd assume that, once it's in, the next release (July/20)
would have
> > to have something minimally decent. But that's not a strong
opinion,
> > so, salt & pepper.
> >
> > cheers,
> > --renato
>
> --
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200108/6c796d2f/attachment-0001.html>

On 09/01/2020 03:16, Mehdi AMINI wrote:
What is the latest/current proposed merge commit showing what will be pushed to
the monorepo?

Please see [0] and [1].

If you want a one-liner to obtain the master branch as it would look after the
merge, this will create a directory called llvm-project-post-merge:

git clone --single-branch -b rewritten-history-v4-llvm-project-merge
https://github.com/peterwaller-arm/f18 llvm-project-post-merge

(single-branch is necessary to avoid pulling down all the other branches I have
in there).

Can you make sure to prune/repack the repo before pushing?

My understanding is that when I say "git push", object negotiation
happens between my git client and github, and the objects github needs are sent
there. I will only be pushing one ref (master), so I don't expect prune will
have an effect. I don't see any way to control the repacking on the github
side.

It's not github, but gitlab's docs say they repack after all pushes:
https://docs.gitlab.com/ee/administration/housekeeping.html

I imagine github do the same. I can't find anything in their docs to that
effect that I can link to. I've written to github support to them to ask
what to expect. As a quick experiment I compared the size of the packfiles after
a clone of each:

840M    llvm-project-single-branch/.git
805M    f18-post-merge-single-branch/.git

Hmm, the rewritten history branch, with both llvm and f18, is smaller than f18
alone. This suggests to me that the repositories aren't repacked at the same
rate.

Please correct me if I'm wrong. I can repack for good measure, but I
don't think it will affect anyone other than me.

Are the license header updated to be the LLVM license?
The test don't seem to be lit-based testing: is this part of the TODO list?

Yes on both counts. Please see Richard Barton's email, points 3 and 5 in [2]
where he collects a list of concerns like this.

[0] https://github.com/flang-compiler/f18/pull/854#issuecomment-572578180

[1]
https://github.com/peterwaller-arm/f18/commits/rewritten-history-v4-llvm-project-merge

[2] http://lists.llvm.org/pipermail/llvm-dev/2020-January/138103.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200109/a7399808/attachment-0001.html>

On 09/01/2020 14:40, Peter Waller wrote:
> I imagine github do the same. I can't find anything in their docs to 
> that effect that I can link to. I've written to github support to them 
> to ask what to expect. As a quick experiment I compared the size of 
> the packfiles after a clone of each:
>
> 840M    llvm-project-single-branch/.git
> 805M    f18-post-merge-single-branch/.git
>
> Hmm, the rewritten history branch, with both llvm and f18, is smaller 
> than f18 alone. This suggests to me that the repositories aren't 
> repacked at the same rate.
>
Correction: I was trying to say "LLVM+18" is bigger than "LLVM
alone",
which is surprising, but could be explained if the former was repacked 
by github more recently than the latter :)

(Which I could imagine, because pushing 3,000 commits might happen to 
trigger some packing heuristics)

On Thu, Jan 9, 2020 at 6:40 AM Peter Waller <Peter.Waller at arm.com>
wrote:
> On 09/01/2020 03:16, Mehdi AMINI wrote:
>
> What is the latest/current proposed merge commit showing what will be
> pushed to the monorepo?
>
> Please see [0] and [1].
>
Thanks!

If you want a one-liner to obtain the master branch as it would look
after
> the merge, this will create a directory called llvm-project-post-merge:
>
> git clone --single-branch -b rewritten-history-v4-llvm-project-merge
> https://github.com/peterwaller-arm/f18 llvm-project-post-merge
>
> (single-branch is necessary to avoid pulling down all the other branches I
> have in there).
>
> Can you make sure to prune/repack the repo before pushing?
>
> My understanding is that when I say "git push", object
negotiation happens
> between my git client and github, and the objects github needs are sent
> there. I will only be pushing one ref (master), so I don't expect prune
> will have an effect. I don't see any way to control the repacking on
the
> github side.
>
> It's not github, but gitlab's docs say they repack after all
pushes:
> https://docs.gitlab.com/ee/administration/housekeeping.html
>
GitHub does not. When we migrated to the monorepo, I asked them to do a
one-time repack and the repo shrunk from 1.14GB to 650MB.
Usually with normal git operations, repacking is not necessary / useful,
but with large history rewrite it isn't necessarily the case (in the
monorepo migration, it was svn2git)

I imagine github do the same. I can't find anything in their docs to
that
> effect that I can link to. I've written to github support to them to
ask
> what to expect. As a quick experiment I compared the size of the packfiles
> after a clone of each:
>
> 840M    llvm-project-single-branch/.git
> 805M    f18-post-merge-single-branch/.git
>
> Hmm, the rewritten history branch, with both llvm and f18, is smaller than
> f18 alone. This suggests to me that the repositories aren't repacked at
the
> same rate.
>
> Please correct me if I'm wrong. I can repack for good measure, but I
don't
> think it will affect anyone other than me.
>
It seems that the way you processed the repo stayed fairly well packed
right now, so there does not seem to be a need for anything here.
When I rewrote the MLIR history the impact of repacking was non negligible
(~15% when comparing the size of the .git folder before/after repacking).

> Are the license header updated to be the LLVM license?
> The test don't seem to be lit-based testing: is this part of the TODO
list?
>
> Yes on both counts. Please see Richard Barton's email, points 3 and 5
in
> [2] where he collects a list of concerns like this.
>
> [0] https://github.com/flang-compiler/f18/pull/854#issuecomment-572578180
>
> [1]
>
https://github.com/peterwaller-arm/f18/commits/rewritten-history-v4-llvm-project-merge
>
> [2] http://lists.llvm.org/pipermail/llvm-dev/2020-January/138103.html
>
Looks great!

Thanks,

-- 
Mehdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200109/edbe6493/attachment.html>