llvm dev - Jan 2020 - Flang landing in the monorepo - next Monday!

Hi Hal,

On Tue, Jan 7, 2020 at 3:38 PM Finkel, Hal J. via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
> On 1/7/20 4:38 PM, Doerfert, Johannes via llvm-dev wrote:
>
> On 01/07, David Blaikie via llvm-dev wrote:
>
> Hey Peter - would you be able to link to/describe the history on this
> process/decision. I can find one old thread where this was first proposed
(http://lists.llvm.org/pipermail/llvm-dev/2019-February/130497.html ) with
> some general positive responses and a lot of questions.
>
> I see there's a flang-dev list, though I'm not sure where code
review is
> happening (is there flang-commits or equivalent?) as there's not much
> chatter on the mailing list that I can see.
>
> There is a flang-commits. We have everything set up to use Phabricator
> as MLIR did.
>
>
> & I'm a bit concerned about flang, already a project with no LLVM
API
> usage, it seems, and a community that doesn't currently look like it
has
> much overlap with the LLVM developer community (a very rough glance at
> flang-dev participants, but I could be wrong - the LLVM community is large,
> to be sure) - I'm concerned that this might create a not very well
> integrated community, as we saw in the past with lldb, for instance.
>
> I think I mentioned this before but some people are waiting for the move
> to be able to actually participate in the Flang development. So the
> current github contributions do not necessarily reflect the LLVM
> developers that will be involved.
>
>
> I'll add to this that a number of us who have been involved with LLVM
for
> a long time are involved in guiding the Flang project. While it's
certainly
> true that a few of the people working hard on Flang are new to the LLVM
> community, there is a reasonable overlap with the existing LLVM community
> in the overall effort.
>
>
> On flang-dev [0] we have long term LLVM developers and people that join
> the LLVM developers community with the merge of Flang but even those
> already actively participated at the LLVM-Dev!
>
> The API usage is a valid concern, among others, which will require code
> changes in the near future.
>
> [0] http://lists.llvm.org/pipermail/flang-dev/2019-December/author.html
>
>
> The other thing that I'll add here is that the existing Flang
development
> has been broad and not deep - the development started with the lexical
> analysis (and preprocessor), then the semantic analysis, and so on. It is
> only very recently that any work on actual code generation has begun, and
> there, the initial focus has been on targeting LLVM via MLIR. I'm not
> saying that there aren't more places to use LLVM APIs (e.g., filesystem
> abstractions, ADT, etc.) -- there are -- but that's something we're
> planning to improve over time.
>
> At a high level, however, the whole point of this Flang project is produce
> an LLVM-community-integrated Fortran frontend for LLVM. The current
> developers of the project, which is a growing community, are committed to
> realizing that outcome and are open to feedback on code structure, API
> usage, and other aspects of integrating with the rest of the LLVM project.
>
I added you to the original thread when I was replying because I knew you
were around this effort, it might be nice to see if you can respond to the
other thread too.

A lot of my concerns are echoed by David here and I have additional
concerns as well. I don't think the project is ready for inclusion into the
main llvm tree as I don't see the point right now. There's nothing llvm
about it.

-eric

>  -Hal
>
>
> Thanks,
>   Johannes
>
>
> On Tue, Jan 7, 2020 at 6:04 AM Peter Waller via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>
>
> Hi All,
>
> As discussed before Christmas, this is a reminder that we intend to
> merge flang on the 13th January (next Monday) before the llvm-10 branch.
> At the moment I'm proposing to do it at 10am GMT. I can be flexible on
> this point if it requires close coordination with anyone in another
> timezone, just let me know.
>
> Previous discussion was in [llvm-dev] Flang landing in the
monorepo<http://lists.llvm.org/pipermail/llvm-dev/2019-December/137661.html>
<http://lists.llvm.org/pipermail/llvm-dev/2019-December/137661.html>.
>
> The current plan of action is summarized
at<https://github.com/flang-compiler/f18/issues/876>
<https://github.com/flang-compiler/f18/issues/876>.
>
> The result will look a lot like the recent MLIR merge from 24th
Dec:<https://github.com/llvm/llvm-project/commits/0f0d0ed1c78>
<https://github.com/llvm/llvm-project/commits/0f0d0ed1c78>, commit
> 0f0d0ed1c78 in the llvm-project monorepo.
>
> Once it is done I'll mail the list. If you want to coordinate with me,
> please get in touch.
>
> Regards,
>
> - Peter
> _______________________________________________
> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> --
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200107/ce501a3f/attachment.html>

On Tue, Jan 7, 2020 at 5:29 PM Eric Christopher <echristo at gmail.com>
wrote:
> Hi Hal,
>
> On Tue, Jan 7, 2020 at 3:38 PM Finkel, Hal J. via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>> On 1/7/20 4:38 PM, Doerfert, Johannes via llvm-dev wrote:
>>
>> On 01/07, David Blaikie via llvm-dev wrote:
>>
>> Hey Peter - would you be able to link to/describe the history on this
>> process/decision. I can find one old thread where this was first
proposed (http://lists.llvm.org/pipermail/llvm-dev/2019-February/130497.html )
with
>> some general positive responses and a lot of questions.
>>
>> I see there's a flang-dev list, though I'm not sure where code
review is
>> happening (is there flang-commits or equivalent?) as there's not
much
>> chatter on the mailing list that I can see.
>>
>> There is a flang-commits. We have everything set up to use Phabricator
>> as MLIR did.
>>
>>
>> & I'm a bit concerned about flang, already a project with no
LLVM API
>> usage, it seems, and a community that doesn't currently look like
it has
>> much overlap with the LLVM developer community (a very rough glance at
>> flang-dev participants, but I could be wrong - the LLVM community is
large,
>> to be sure) - I'm concerned that this might create a not very well
>> integrated community, as we saw in the past with lldb, for instance.
>>
>> I think I mentioned this before but some people are waiting for the
move
>> to be able to actually participate in the Flang development. So the
>> current github contributions do not necessarily reflect the LLVM
>> developers that will be involved.
>>
>>
>> I'll add to this that a number of us who have been involved with
LLVM for
>> a long time are involved in guiding the Flang project. While it's
certainly
>> true that a few of the people working hard on Flang are new to the LLVM
>> community, there is a reasonable overlap with the existing LLVM
community
>> in the overall effort.
>>
>>
>> On flang-dev [0] we have long term LLVM developers and people that join
>> the LLVM developers community with the merge of Flang but even those
>> already actively participated at the LLVM-Dev!
>>
>> The API usage is a valid concern, among others, which will require code
>> changes in the near future.
>>
>> [0] http://lists.llvm.org/pipermail/flang-dev/2019-December/author.html
>>
>>
>> The other thing that I'll add here is that the existing Flang
development
>> has been broad and not deep - the development started with the lexical
>> analysis (and preprocessor), then the semantic analysis, and so on. It
is
>> only very recently that any work on actual code generation has begun,
and
>> there, the initial focus has been on targeting LLVM via MLIR. I'm
not
>> saying that there aren't more places to use LLVM APIs (e.g.,
filesystem
>> abstractions, ADT, etc.) -- there are -- but that's something
we're
>> planning to improve over time.
>>
>> At a high level, however, the whole point of this Flang project is
>> produce an LLVM-community-integrated Fortran frontend for LLVM. The
current
>> developers of the project, which is a growing community, are committed
to
>> realizing that outcome and are open to feedback on code structure, API
>> usage, and other aspects of integrating with the rest of the LLVM
project.
>>
> I added you to the original thread when I was replying because I knew you
> were around this effort, it might be nice to see if you can respond to the
> other thread too.
>
> A lot of my concerns are echoed by David here and I have additional
> concerns as well. I don't think the project is ready for inclusion into
the
> main llvm tree as I don't see the point right now. There's nothing
llvm
> about it.
>
>
To elaborate a bit because this almost assuredly comes off poorly (and I
apologize):

I am in favor of having a flang front end in tree. I have concerns about
the design of flang versus other front ends, the lack of llvm based library
use, and a number of other things that I tried to enumerate in previous
emails. I don't know if anything has changed and the responses I got back
originally were "we're going to do it anyway" so it didn't
leave much room
for engagement.

Thanks.

-eric

> -eric
>
>
>>  -Hal
>>
>>
>> Thanks,
>>   Johannes
>>
>>
>> On Tue, Jan 7, 2020 at 6:04 AM Peter Waller via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>
>>
>> Hi All,
>>
>> As discussed before Christmas, this is a reminder that we intend to
>> merge flang on the 13th January (next Monday) before the llvm-10
branch.
>> At the moment I'm proposing to do it at 10am GMT. I can be flexible
on
>> this point if it requires close coordination with anyone in another
>> timezone, just let me know.
>>
>> Previous discussion was in [llvm-dev] Flang landing in the
monorepo<http://lists.llvm.org/pipermail/llvm-dev/2019-December/137661.html>
<http://lists.llvm.org/pipermail/llvm-dev/2019-December/137661.html>.
>>
>> The current plan of action is summarized
at<https://github.com/flang-compiler/f18/issues/876>
<https://github.com/flang-compiler/f18/issues/876>.
>>
>> The result will look a lot like the recent MLIR merge from 24th
Dec:<https://github.com/llvm/llvm-project/commits/0f0d0ed1c78>
<https://github.com/llvm/llvm-project/commits/0f0d0ed1c78>, commit
>> 0f0d0ed1c78 in the llvm-project monorepo.
>>
>> Once it is done I'll mail the list. If you want to coordinate with
me,
>> please get in touch.
>>
>> Regards,
>>
>> - Peter
>> _______________________________________________
>> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>> _______________________________________________
>> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> --
>> Hal Finkel
>> Lead, Compiler Technology and Programming Languages
>> Leadership Computing Facility
>> Argonne National Laboratory
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200107/047510d8/attachment.html>

Hi Eric, David, all

I work with Peter at Arm in a team of 5 engineers working on F18, and will
continue as it becomes llvm-project/flang. We depend on Fortran support in
(PGI/Nvidia) flang for our commercial Fortran compiler and will move over to
llvm-project/flang (I'll call it Flang from now on) when it is ready so we
are committed long-term to the technology and the community. We have a further
12 or so engineers working on LLVM core codegen, mostly for Arm’s SVE
architecture, and we have a long-term involvement and commitment to the LLVM
project as a whole.

I totally agree with your points that Flang as it stands is not fully integrated
into LLVM. Hal has explained the issue about Flang not using LLVM APIs for
codegen so hopefully you will agree that will get better over time. The next
large development in Flang will be to add the lowering code from the syntax tree
to MLIR, which is now an LLVM API. This code is being prototyped in a separate
branch of Flang [1], but Eric at Nvidia will start to land the changes soon.

I think your point on usage of LLVM data structures and support libraries is
spot on and we need to take that on board. Over time we want Flang to use more
of the common LLVM infrastructure and I think that being a part of the monorepo
should help this to happen.

At Arm, we have worked on integrating Flang into Clang's driver [2] [3] [4]
and we are trying to follow the existing structure of the driver and to take
advantage of common functionality as much as possible in that exercise. I should
also say that we are working on a patch to Flang that adds testing using
llvm-lit/FileCheck in the style of llvm projects [5]. We'll be adding all
tests for the new driver as lit tests. We are also working internally on porting
over the existing Flang tests that use bespoke scripts to using lit (this code
has not made it to github yet.)

As a community, we are also trying as much as possible to follow the LLVM coding
style in Flang [6] and other design principles like modularity so each part of
the compiler can be tested in isolation.

Our rationale for contributing Flang to LLVM at this point rather than waiting
is so that we can work on integrating it more closely with LLVM as a community
effort, in tree. The longer we develop Flang as a separate entity, the more its
community ethos, its processes, its code style and structure, etc. will likely
diverge from the LLVM norms. We think that the eventual inclusion Flang would be
harder on that side even though the code itself may make more use of LLVM APIs
and datastructures. I don’t think there is a right or wrong answer but this is
the position we support at Arm and we look forward to working with the community
to improve Flang in the ways you suggest.

Yours
Rich

[1] https://github.com/schweitzpgi/f18/tree/f18
[2] https://reviews.llvm.org/D63607
[3] https://github.com/flang-compiler/f18/pull/759
[4] https://github.com/flang-compiler/f18/pull/762 and
https://github.com/flang-compiler/f18/pull/763
[5] https://github.com/flang-compiler/f18/pull/861
[6]
https://github.com/flang-compiler/f18/blob/master/documentation/C%2B%2Bstyle.md


From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Eric
Christopher via llvm-dev
Sent: 8 January, 2020 01:48
To: Finkel, Hal J. <hfinkel at anl.gov>
Cc: llvm-dev <llvm-dev at lists.llvm.org>; Mike Edwards <medwards at
llvm.org>
Subject: Re: [llvm-dev] Flang landing in the monorepo - next Monday!



On Tue, Jan 7, 2020 at 5:29 PM Eric Christopher <echristo at
gmail.com<mailto:echristo at gmail.com>> wrote:
Hi Hal,

On Tue, Jan 7, 2020 at 3:38 PM Finkel, Hal J. via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:


On 1/7/20 4:38 PM, Doerfert, Johannes via llvm-dev wrote:

On 01/07, David Blaikie via llvm-dev wrote:

Hey Peter - would you be able to link to/describe the history on this

process/decision. I can find one old thread where this was first proposed (

http://lists.llvm.org/pipermail/llvm-dev/2019-February/130497.html ) with

some general positive responses and a lot of questions.



I see there's a flang-dev list, though I'm not sure where code review is

happening (is there flang-commits or equivalent?) as there's not much

chatter on the mailing list that I can see.

There is a flang-commits. We have everything set up to use Phabricator

as MLIR did.



& I'm a bit concerned about flang, already a project with no LLVM API

usage, it seems, and a community that doesn't currently look like it has

much overlap with the LLVM developer community (a very rough glance at

flang-dev participants, but I could be wrong - the LLVM community is large,

to be sure) - I'm concerned that this might create a not very well

integrated community, as we saw in the past with lldb, for instance.

I think I mentioned this before but some people are waiting for the move

to be able to actually participate in the Flang development. So the

current github contributions do not necessarily reflect the LLVM

developers that will be involved.



I'll add to this that a number of us who have been involved with LLVM for a
long time are involved in guiding the Flang project. While it's certainly
true that a few of the people working hard on Flang are new to the LLVM
community, there is a reasonable overlap with the existing LLVM community in the
overall effort.



On flang-dev [0] we have long term LLVM developers and people that join

the LLVM developers community with the merge of Flang but even those

already actively participated at the LLVM-Dev!



The API usage is a valid concern, among others, which will require code

changes in the near future.



[0] http://lists.llvm.org/pipermail/flang-dev/2019-December/author.html



The other thing that I'll add here is that the existing Flang development
has been broad and not deep - the development started with the lexical analysis
(and preprocessor), then the semantic analysis, and so on. It is only very
recently that any work on actual code generation has begun, and there, the
initial focus has been on targeting LLVM via MLIR. I'm not saying that there
aren't more places to use LLVM APIs (e.g., filesystem abstractions, ADT,
etc.) -- there are -- but that's something we're planning to improve
over time.

At a high level, however, the whole point of this Flang project is produce an
LLVM-community-integrated Fortran frontend for LLVM. The current developers of
the project, which is a growing community, are committed to realizing that
outcome and are open to feedback on code structure, API usage, and other aspects
of integrating with the rest of the LLVM project.
I added you to the original thread when I was replying because I knew you were
around this effort, it might be nice to see if you can respond to the other
thread too.

A lot of my concerns are echoed by David here and I have additional concerns as
well. I don't think the project is ready for inclusion into the main llvm
tree as I don't see the point right now. There's nothing llvm about it.


To elaborate a bit because this almost assuredly comes off poorly (and I
apologize):

I am in favor of having a flang front end in tree. I have concerns about the
design of flang versus other front ends, the lack of llvm based library use, and
a number of other things that I tried to enumerate in previous emails. I
don't know if anything has changed and the responses I got back originally
were "we're going to do it anyway" so it didn't leave much
room for engagement.

Thanks.

-eric

-eric


 -Hal



Thanks,

  Johannes



On Tue, Jan 7, 2020 at 6:04 AM Peter Waller via llvm-dev <

llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:



Hi All,



As discussed before Christmas, this is a reminder that we intend to

merge flang on the 13th January (next Monday) before the llvm-10 branch.

At the moment I'm proposing to do it at 10am GMT. I can be flexible on

this point if it requires close coordination with anyone in another

timezone, just let me know.



Previous discussion was in [llvm-dev] Flang landing in the monorepo

<http://lists.llvm.org/pipermail/llvm-dev/2019-December/137661.html><http://lists.llvm.org/pipermail/llvm-dev/2019-December/137661.html>.



The current plan of action is summarized at

<https://github.com/flang-compiler/f18/issues/876><https://github.com/flang-compiler/f18/issues/876>.



The result will look a lot like the recent MLIR merge from 24th Dec:

<https://github.com/llvm/llvm-project/commits/0f0d0ed1c78><https://github.com/llvm/llvm-project/commits/0f0d0ed1c78>,
commit

0f0d0ed1c78 in the llvm-project monorepo.



Once it is done I'll mail the list. If you want to coordinate with me,

please get in touch.



Regards,



- Peter

_______________________________________________

LLVM Developers mailing list

llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>

https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________

LLVM Developers mailing list

llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>

https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________

LLVM Developers mailing list

llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>

https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

--

Hal Finkel

Lead, Compiler Technology and Programming Languages

Leadership Computing Facility

Argonne National Laboratory
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200108/1dbb2d14/attachment.html>

On Wed, 8 Jan 2020 at 01:48, Eric Christopher via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> I am in favor of having a flang front end in tree. I have concerns about
the design of flang versus other front ends, the lack of llvm based library use,
and a number of other things that I tried to enumerate in previous emails. I
don't know if anything has changed and the responses I got back originally
were "we're going to do it anyway" so it didn't leave much
room for engagement.
(Disclaimer: Not taking sides, I have no stake in Flang or Fortran).

I think David's comparison to LLDB is interesting. It started very
distant, and then with time merged with the codebase and now it's a
full fledged LLVM project.

The current Flang (F18) is meant to be much closer to LLVM than the
previous one, and the whole mindset was afaik to keep it that way. In
the same way that LLDB once was.

I do agree with Hal that there is a small but significant overlap of
communities, and I do agree with Rick that the longer it stays
separate, the harder it will be for that sub-community.

But LLDB started at a time where "being an LLVM project" was mostly
about being in our SVN repo. There was no mono-repo, and the cost of
being there was lower-ish.

I believe that can be solved with build semantics (CMake stuff?), not
a big problem, so the main issue is about community: will the project
move fast enough towards LLVM integration or will it dangle with
downstream implementations and be mostly useless upstream?

I think we have enough people that care about Flang publicly putting
their names forward. This makes me assume the former, so I'm
personally ok with the merge.

If it happens before the current branch or not, will depend on how
fast they're willing to work towards a working Fortran front-end.

I'd assume that, once it's in, the next release (July/20) would have
to have something minimally decent. But that's not a strong opinion,
so, salt & pepper.

cheers,
--renato