llvm dev - Feb 2019 - RFC for f18+runtimes in LLVM

Hi, everyone,

As you may know, NVIDIA has developed an open-source Fortran frontend for LLVM
(http://flang-compiler.org), which consists of the flang frontend itself along
with the corresponding Fortran runtime library. The existing frontend's code
is mostly written in C, and while a production-quality implementation, does not
follow modern software-engineering practices.

Our long-standing desire is that our work forms the basis of a Fortran frontend
that is part of the LLVM project and developed within the LLVM community. In
addition, we would like the frontend to be useful in the many ways that Clang is
useful: not just as a frontend, but also for static analysis and tooling.

Recognizing that the current flang's code base will not meet these needs, we
started a ground-up rewrite of the frontend in modern C++. This effort is called
f18.

At this point, we have documented and implemented a healthy subset of the
compiler for symbol tables and scoping, name resolution, USE statements and
module files, constant representation, constant folding and much of declaration,
label and expression semantics. The parser handles all of Fortran 2018 and
OpenMP 4.5 and implements a Fortran-aware preprocessor. The Fortran control flow
graph (CFG) is in review now.  We continue to update other documentation, such
as the style guide and runtime descriptor design.

Before this effort proceeds much farther, we'd like to contribute f18, and
the existing Fortran runtime libraries, to the LLVM project. The Fortran
frontend would become a proper LLVM subproject, design discussions would take
place on an associated LLVM mailing list, code reviews would use phabricator,
and so on.

We're committed to developing LLVM's Fortran frontend for years to come,
and together with other members of the LLVM community (e.g., ARM, US Dept of
Energy) would like to do so as part of the LLVM project.

* The Fortran runtime library will be moved into a subdirectory of this new LLVM
subproject, or compiler-rt, or its own subproject (based on feedback here).

* The current f18 code will be committed to the new LLVM subproject.  The f18
code is a set of libraries that implements the Fortran compiler.  Today, there’s
a rudimentary driver; however, we propose to adapt the clang driver to accept
Fortran source and invoke f18 as appropriate. We would also adapt the f18
libraries to fit into the existing llvm build systems and testing
infrastructure.

The f18 compiler source code complies with most of LLVM's coding guidelines;
however, the code uses several C++17 features.  We've documented our use of
C++17 here:

  https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md

In particular, the parse tree and the lowered forms of expressions and variables
are defined in terms of C++17 std::variant. Most of the compiler uses C++17
std::visit to walk these data structures.

It’s possible to reimplement the most important functionality of std:variant as
a subset class, say llvm:variant; however, variant gets its power from the C++17
features generic lambdas and parameter pack expansion on “using”.  Without these
C++17 features, use of variant would be impractical.

Our thinking when we started was that llvm would adopt C++17 before mid-2020,
which lines up with our projected completion date. If we were to adopt C++11 or
C++14, we would likely create substitutes for these classes, certainly at a cost
of calendar time and perhaps type safety and notational convenience.  One of our
principles is to take advantage of the standard library as much as possible, so
casual readers will better understand our code and so we avoid the time and bugs
associated with writing class libraries.

Our request would be to get a waiver for the C++11 requirement based on the fact
that we're skating to where the puck will be.  In the meantime, because F18
only exists as a stand-alone program, early adopters would still have a useful
parser and analyzer for Fortran.

Also, as part of the ongoing flang effort, NVIDIA has developed a library of
scalar, vector, and masked math functions. This library should be useful for
autovectorization, and OpenMP SIMD, in general. We’ll create another RFC to
discuss this contribution.

Please let us know if you have any feedback or concerns.

- Steve





-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may
contain
confidential information.  Any unauthorized review, use, disclosure or
distribution
is prohibited.  If you are not the intended recipient, please contact the sender
by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190225/77110757/attachment.html>

On Mon, Feb 25, 2019 at 1:06 PM Stephen Scalpone via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Our request would be to get a waiver for the C++11 requirement based on
> the fact that we're skating to where the puck will be.  In the
meantime,
> because F18 only exists as a stand-alone program, early adopters would
> still have a useful parser and analyzer for Fortran.
>
+1. The reasons why we haven't yet switched LLVM to c++17 do not apply to a
brand new frontend, since it's effectively all about not disrupting current
users, and a brand new frontend does not yet have those.

So, IMO, there's no reason f18 cannot have more stringent requirements at
the time of its introduction, and then going forward it should follow along
with the rest of the project's minimum toolchain requirements, after those
eventually meet and surpass f18's initial requirements.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190225/a8f9b3de/attachment.html>

James Y Knight via llvm-dev <llvm-dev at lists.llvm.org> writes:
> On Mon, Feb 25, 2019 at 1:06 PM Stephen Scalpone via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
>     
>         Our request would be to get a waiver for the C++11 requirement
>     based on the fact that we're skating to where the puck will be. In
>     the meantime, because F18 only exists as a stand-alone program,
>     early adopters would still have a useful parser and analyzer for
>     Fortran.
>     
>
> +1. The reasons why we haven't yet switched LLVM to c++17 do not apply
> to a brand new frontend, since it's effectively all about not
> disrupting current users, and a brand new frontend does not yet have
> those.
I generally agree with this, but how will it work in practice?  Our
current buildbots don't have any requirement to support C++17 and they
might not all be able to.

                          -David

On Mon, Feb 25, 2019 at 10:06 AM Stephen Scalpone via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> * The current f18 code will be committed to the new LLVM subproject.  The
> f18 code is a set of libraries that implements the Fortran compiler.
>
Awesome. This is an important aspect of the design of LLVM projects IMO ->
they build their functionality primarily as re-usable libraries, and then
expose that in useful command line utilities.

> The f18 compiler source code complies with most of LLVM's coding
> guidelines; however, the code uses several C++17 features.  We've
> documented our use of C++17 here:
>
>
>
>   https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md
>
>
>
> In particular, the parse tree and the lowered forms of expressions and
> variables are defined in terms of C++17 std::variant. Most of the
> compiler uses C++17 std::visit to walk these data structures.
>
>
>
> It’s possible to reimplement the most important functionality of
> std:variant as a subset class, say llvm:variant; however, variant gets
> its power from the C++17 features generic lambdas and parameter pack
> expansion on “using”.  Without these C++17 features, use of variant would
> be impractical.
>
>
>
> Our thinking when we started was that llvm would adopt C++17 before
> mid-2020, which lines up with our projected completion date. If we were to
> adopt C++11 or C++14, we would likely create substitutes for these classes,
> certainly at a cost of calendar time and perhaps type safety and notational
> convenience.  One of our principles is to take advantage of the standard
> library as much as possible, so casual readers will better understand our
> code and so we avoid the time and bugs associated with writing class
> libraries.
>
>
>
> Our request would be to get a waiver for the C++11 requirement based on
> the fact that we're skating to where the puck will be.  In the
meantime,
> because F18 only exists as a stand-alone program, early adopters would
> still have a useful parser and analyzer for Fortran.
>
Hold on, either it is a collection of libraries or it is a stand-alone
program. It can't really be both?

Generally, I think the idea that diverging from the rest of the project
here is low-cost for a subproject isn't supported by experience with other
projects.

Notably, it has a strong tendancy to create tension. You want some ADT or
support library in LLVM to work well with your C++17 code. But it is C++11.
Every time this has been done in the past, the result has been that
generically useful tools and libraries get added to the subproject rather
than to LLVM as a whole.

So FWIW, I'd be really opposed to this. Instead, I think that F18 should
have rich libraries, and develop them exactly the same way as the rest of
LLVM.

We're getting close to switching to C++14, so maybe due to timing, you
could merge F18 when that happens?

Ultimately, I think you either need to raise the LLVM base language version
or lower the F18 one so that they match when merged IMO. Anything else I
think will hamper integration with the larger project.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190225/5179194d/attachment.html>

FWIW, I think if the LLVM project will not accept C++17 code soon, then f18
should detect C++17 support and if it's not there, use local
implementations.  Cray once released a library back when some compilers did not
yet support C++11, so I detected if C++11 support existed and used my own
implementations when it wasn't present.  For some things, it was trivial,
for others, really annoying....but it worked.  It would be lovely to see f18 in
the new LLVM repo sooner than later, but obviously if f18 merges in after C++14
is allowed, then less work is required to duplicate std:: functionality. 
It's a tradeoff.

-Troy
________________________________
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Chandler
Carruth via llvm-dev <llvm-dev at lists.llvm.org>
Sent: Monday, February 25, 2019 4:45:28 PM
To: Stephen Scalpone
Cc: llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] RFC for f18+runtimes in LLVM

On Mon, Feb 25, 2019 at 10:06 AM Stephen Scalpone via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

* The current f18 code will be committed to the new LLVM subproject.  The f18
code is a set of libraries that implements the Fortran compiler.

Awesome. This is an important aspect of the design of LLVM projects IMO ->
they build their functionality primarily as re-usable libraries, and then expose
that in useful command line utilities.


The f18 compiler source code complies with most of LLVM's coding guidelines;
however, the code uses several C++17 features.  We've documented our use of
C++17 here:



  https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md



In particular, the parse tree and the lowered forms of expressions and variables
are defined in terms of C++17 std::variant. Most of the compiler uses C++17
std::visit to walk these data structures.



It’s possible to reimplement the most important functionality of std:variant as
a subset class, say llvm:variant; however, variant gets its power from the C++17
features generic lambdas and parameter pack expansion on “using”.  Without these
C++17 features, use of variant would be impractical.



Our thinking when we started was that llvm would adopt C++17 before mid-2020,
which lines up with our projected completion date. If we were to adopt C++11 or
C++14, we would likely create substitutes for these classes, certainly at a cost
of calendar time and perhaps type safety and notational convenience.  One of our
principles is to take advantage of the standard library as much as possible, so
casual readers will better understand our code and so we avoid the time and bugs
associated with writing class libraries.



Our request would be to get a waiver for the C++11 requirement based on the fact
that we're skating to where the puck will be.  In the meantime, because F18
only exists as a stand-alone program, early adopters would still have a useful
parser and analyzer for Fortran.

Hold on, either it is a collection of libraries or it is a stand-alone program.
It can't really be both?

Generally, I think the idea that diverging from the rest of the project here is
low-cost for a subproject isn't supported by experience with other projects.

Notably, it has a strong tendancy to create tension. You want some ADT or
support library in LLVM to work well with your C++17 code. But it is C++11.
Every time this has been done in the past, the result has been that generically
useful tools and libraries get added to the subproject rather than to LLVM as a
whole.

So FWIW, I'd be really opposed to this. Instead, I think that F18 should
have rich libraries, and develop them exactly the same way as the rest of LLVM.

We're getting close to switching to C++14, so maybe due to timing, you could
merge F18 when that happens?

Ultimately, I think you either need to raise the LLVM base language version or
lower the F18 one so that they match when merged IMO. Anything else I think will
hamper integration with the larger project.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190226/393db90d/attachment-0001.html>

On Mon, Feb 25, 2019 at 5:46 PM Chandler Carruth via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Mon, Feb 25, 2019 at 10:06 AM Stephen Scalpone via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> * The current f18 code will be committed to the new LLVM subproject. 
The
>> f18 code is a set of libraries that implements the Fortran compiler.
>>
>
> Awesome. This is an important aspect of the design of LLVM projects IMO
->
> they build their functionality primarily as re-usable libraries, and then
> expose that in useful command line utilities.
>
>
>> The f18 compiler source code complies with most of LLVM's coding
>> guidelines; however, the code uses several C++17 features.  We've
>> documented our use of C++17 here:
>>
>>
>>
>>
>>
https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md
>>
>>
>>
>> In particular, the parse tree and the lowered forms of expressions and
>> variables are defined in terms of C++17 std::variant. Most of the
>> compiler uses C++17 std::visit to walk these data structures.
>>
>>
>>
>> It’s possible to reimplement the most important functionality of
>> std:variant as a subset class, say llvm:variant; however, variant gets
>> its power from the C++17 features generic lambdas and parameter pack
>> expansion on “using”.  Without these C++17 features, use of variant
>> would be impractical.
>>
>>
>>
>> Our thinking when we started was that llvm would adopt C++17 before
>> mid-2020, which lines up with our projected completion date. If we were
to
>> adopt C++11 or C++14, we would likely create substitutes for these
classes,
>> certainly at a cost of calendar time and perhaps type safety and
notational
>> convenience.  One of our principles is to take advantage of the
standard
>> library as much as possible, so casual readers will better understand
our
>> code and so we avoid the time and bugs associated with writing class
>> libraries.
>>
>>
>>
>> Our request would be to get a waiver for the C++11 requirement based on
>> the fact that we're skating to where the puck will be.  In the
meantime,
>> because F18 only exists as a stand-alone program, early adopters would
>> still have a useful parser and analyzer for Fortran.
>>
>
> Hold on, either it is a collection of libraries or it is a stand-alone
> program. It can't really be both?
>
> Generally, I think the idea that diverging from the rest of the project
> here is low-cost for a subproject isn't supported by experience with
other
> projects.
>
> Notably, it has a strong tendancy to create tension. You want some ADT or
> support library in LLVM to work well with your C++17 code. But it is C++11.
> Every time this has been done in the past, the result has been that
> generically useful tools and libraries get added to the subproject rather
> than to LLVM as a whole.
>
If there are such features that ought to be added to the support libraries
in LLVM for better C++17 support, then they can indeed be added, with an
appropriate #ifdef on language version, no?

The primary reason that I think it makes sense to allow f18 to require
C++17 is that this will be a _temporary_ divergence -- the rest of the LLVM
project will certainly move to require C++17 as well at some point
relatively soon. Any C++17-specific improvements made to llvm common
libraries will be useful for the rest of LLVM too.

I don't know what the expected timeline of f18 completion is, nor would I
like to predict excatly how long it'll be before LLVM might start requiring
C++17, but it certainly seems possible that LLVM might be ready to require
c++17 before f18 is even finished.

So FWIW, I'd be really opposed to this. Instead, I think that F18
should
> have rich libraries, and develop them exactly the same way as the rest of
> LLVM.
>
> We're getting close to switching to C++14, so maybe due to timing, you
> could merge F18 when that happens?
>
> Ultimately, I think you either need to raise the LLVM base language
> version or lower the F18 one so that they match when merged IMO. Anything
> else I think will hamper integration with the larger project.
>
Even if a decision is made to rewrite parts of the code in order to not
rely on C++17 features, I don't think that the initial import of the
project ought to be tied to that task being completed. More generally, I
think the prerequisite to merging it should be having an agreed-upon target
state and an understood path on how to reach that state, rather than the
code actually being _in_ that state already. Merging sooner is generally
better than waiting and merging later.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190225/500f0ac1/attachment.html>

On 2/25/19 2:45 PM, Chandler Carruth via llvm-dev wrote:
>
> Hold on, either it is a collection of libraries or it is a stand-alone 
> program. It can't really be both?
>
I went and looked at its repo out of curiosity, and I could not tell 
which way it is meant to be. Unless I am missing something, It appears 
that neither f18 sources have any CMake `install` directives, nor the 
docs have any description on whether/how it would be used as a set of 
libraries. Just wondering out loud, are there any plans for documenting 
that?

Cheers,
Petr

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190225/c3aea76d/attachment.html>

Chandler Carruth via llvm-dev <llvm-dev at lists.llvm.org> writes:
> Notably, it has a strong tendancy to create tension. You want some ADT
> or support library in LLVM to work well with your C++17 code. But it
> is C++11. Every time this has been done in the past, the result has
> been that generically useful tools and libraries get added to the
> subproject rather than to LLVM as a whole.
Building on this, there was some discussion months ago on flang-dev
about f18 tooling and whether it could integrate with existing clang
libraries.  Whatever became of that?  I recall some references to
earlier iterations of flang that successfully extended clang's AST to
handle Fortran and thereby leveraged much of the existing great clang
tooling infrastructure.

My sense of f18 is that it is very much a "ground up" implementation
and
doesn't make use of any of the existing clang infrastructure.  It would
be a real shame to have two completely separate projects that provide
many of the same services.  It seems like a good idea to have a
conversation about f18 tooling before we add f18 as a subproject and
have to support two independent tooling efforts.

                               -David

On Mon, Feb 25, 2019 at 2:45 PM Chandler Carruth via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Mon, Feb 25, 2019 at 10:06 AM Stephen Scalpone via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> * The current f18 code will be committed to the new LLVM subproject. 
The
>> f18 code is a set of libraries that implements the Fortran compiler.
>>
>
> Awesome. This is an important aspect of the design of LLVM projects IMO
->
> they build their functionality primarily as re-usable libraries, and then
> expose that in useful command line utilities.
>
>
>> The f18 compiler source code complies with most of LLVM's coding
>> guidelines; however, the code uses several C++17 features.  We've
>> documented our use of C++17 here:
>>
>>
>>
>>
>>
https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md
>>
>>
>>
>> In particular, the parse tree and the lowered forms of expressions and
>> variables are defined in terms of C++17 std::variant. Most of the
>> compiler uses C++17 std::visit to walk these data structures.
>>
>>
>>
>> It’s possible to reimplement the most important functionality of
>> std:variant as a subset class, say llvm:variant; however, variant gets
>> its power from the C++17 features generic lambdas and parameter pack
>> expansion on “using”.  Without these C++17 features, use of variant
>> would be impractical.
>>
>>
>>
>> Our thinking when we started was that llvm would adopt C++17 before
>> mid-2020, which lines up with our projected completion date. If we were
to
>> adopt C++11 or C++14, we would likely create substitutes for these
classes,
>> certainly at a cost of calendar time and perhaps type safety and
notational
>> convenience.  One of our principles is to take advantage of the
standard
>> library as much as possible, so casual readers will better understand
our
>> code and so we avoid the time and bugs associated with writing class
>> libraries.
>>
>>
>>
>> Our request would be to get a waiver for the C++11 requirement based on
>> the fact that we're skating to where the puck will be.  In the
meantime,
>> because F18 only exists as a stand-alone program, early adopters would
>> still have a useful parser and analyzer for Fortran.
>>
>
> Hold on, either it is a collection of libraries or it is a stand-alone
> program. It can't really be both?
>
> Generally, I think the idea that diverging from the rest of the project
> here is low-cost for a subproject isn't supported by experience with
other
> projects.
>
> Notably, it has a strong tendancy to create tension. You want some ADT or
> support library in LLVM to work well with your C++17 code. But it is C++11.
> Every time this has been done in the past, the result has been that
> generically useful tools and libraries get added to the subproject rather
> than to LLVM as a whole.
>
> So FWIW, I'd be really opposed to this. Instead, I think that F18
should
> have rich libraries, and develop them exactly the same way as the rest of
> LLVM.
>
> We're getting close to switching to C++14, so maybe due to timing, you
> could merge F18 when that happens?
>
> Ultimately, I think you either need to raise the LLVM base language
> version or lower the F18 one so that they match when merged IMO. Anything
> else I think will hamper integration with the larger project.
>
>
lld used C++11 before the rest of LLVM switched over without issue.

- Michael Spencer
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190226/b0cdfb6f/attachment.html>

On Feb 25, 2019, at 10:06 AM, Stephen Scalpone via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> We're committed to developing LLVM's Fortran frontend for years to
come, and together with other members of the LLVM community (e.g., ARM, US Dept
of Energy) would like to do so as part of the LLVM project.
This is super exciting Stephen, congratulations to you and everyone working on
f18.  I’m very excited to see this happening and am thrilled about the approach
you are taking.
> The f18 compiler source code complies with most of LLVM's coding
guidelines; however, the code uses several C++17 features.  We've documented
our use of C++17 here:
>  
>   https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md
<https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md>
> 
> Our request would be to get a waiver for the C++11 requirement based on the
fact that we're skating to where the puck will be.  In the meantime, because
F18 only exists as a stand-alone program, early adopters would still have a
useful parser and analyzer for Fortran.
I personally see no problem or concerns with this at all.  This is a new project
and the worst case is that f18 comes up with some really cool stuff that the
rest of the LLVM project would love to share, but that can’t be done until it is
refactored to not use c++11.  If/when that comes up, we can deal with it on
demand.  I don’t see any particular reason to block f18 from joining the project
because of that speculative concern.

-Chris


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190227/5b6b9d17/attachment.html>

Following up on my earlier email. If there is a commitment to checking 
in f18 already, feel free to disregard it. I went and took a little bit 
closer look at the sources and want to share some of the findings in 
case if anyone is interested. Disclosure: I contribute to Fort 
<http://fort-compiler.org/> (fort-compiler.org), which is the fork of 
the front-end David Greene mentioned.

 From Stephen's announcement:
> At this point, we have documented and implemented a healthy subset of 
> the compiler for symbol tables and scoping, name resolution, USE 
> statements and module files, constant representation, constant folding 
> and much of declaration, label and expression semantics. The parser 
> handles all of Fortran 2018 and OpenMP 4.5 and implements a 
> Fortran-aware preprocessor. The Fortran control flow graph (CFG) is in 
> review now.  We continue to update other documentation, such as the 
> style guide and runtime descriptor design.
>
Currently it looks like only the parser is /partially/ implemented in 
f18, there is no code generator (via LLVM or otherwise) and, obviously, 
no object output. For that reason and due to the condition of its test 
suite it is impossible to reliably assess the state of Fortran 18 
support (thought it does look like a fair amount of effort went into 
it). State of OpenMP support actually got me a bit puzzled, more on that 
below.

As I understand the announcement, f18 is intended to be used or merged 
with Flang sources at some point, but that still does not explain how it 
would integrate with LLVM, since Flang does not seem invoke LLVM 
directly either (it used to produce LLVM IR as text files). Because of 
this, it is likely that its code generator component would have to be 
written from scratch. It is also unclear if and how it would provide the 
library API which has been announced.

A bit about the test suite -- I looked at the Fortran (regression) part 
of it (as opposed to unit tests, which hopefully are a relatively simple 
affair). Maybe nitpicking, but despite "handles all of OpenMP 4.5" 
statement in the announcement there seem to be only two references to 
OpenMP in tests. Most of the regression tests are challenging to 
understand -- some list all expected output upfront, some of the 
expected output is not particularly human-friendly. Maybe I am used to 
Clang's test suite, but it is unclear to me what each file is testing. 
Also, regression testing relies on a set of shell scripts to do some of 
the output checking.

My worry here that it would actually take years to develop f18 into an a 
working compiler, in which case there might be other options worth 
considering for a Fortran front-end. In my opinion (and this /may/ be a 
matter of personal preference) a healthier subset of the compiler would 
be more of an end-to-end subset of it -- something that can be tested as 
a full product while it is being developed. And then there is also the 
argument for reusing Clang tooling, which David Greene keeps making, 
though that idea does not seem to get a lot of interest.


Best,

Petr

On 2/27/19 1:55 PM, Chris Lattner via llvm-dev wrote:
> On Feb 25, 2019, at 10:06 AM, Stephen Scalpone via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>> We're committed to developing LLVM's Fortran frontend for years
to
>> come, and together with other members of the LLVM community (e.g., 
>> ARM, US Dept of Energy) would like to do so as part of the LLVM
project.
>
> This is super exciting Stephen, congratulations to you and everyone 
> working on f18.  I’m very excited to see this happening and am 
> thrilled about the approach you are taking.
>
>> The f18 compiler source code complies with most of LLVM's coding 
>> guidelines; however, the code uses several C++17 features.  We've 
>> documented our use of C++17 here:
>>
https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md
>>
>> Our request would be to get a waiver for the C++11 requirement based 
>> on the fact that we're skating to where the puck will be.  In the 
>> meantime, because F18 only exists as a stand-alone program, early 
>> adopters would still have a useful parser and analyzer for Fortran.
>
> I personally see no problem or concerns with this at all.  This is a 
> new project and the worst case is that f18 comes up with some really 
> cool stuff that the rest of the LLVM project would love to share, but 
> that can’t be done until it is refactored to not use c++11.  If/when 
> that comes up, we can deal with it on demand.  I don’t see any 
> particular reason to block f18 from joining the project because of 
> that speculative concern.
>
> -Chris
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190301/bd83168e/attachment.html>

Reply inline

From: Chandler Carruth <chandlerc at gmail.com>
Date: Monday, February 25, 2019 at 2:46 PM
To: Stephen Scalpone <sscalpone at nvidia.com>
Cc: "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] RFC for f18+runtimes in LLVM

On Mon, Feb 25, 2019 at 10:06 AM Stephen Scalpone via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
* The current f18 code will be committed to the new LLVM subproject.  The f18
code is a set of libraries that implements the Fortran compiler.

Awesome. This is an important aspect of the design of LLVM projects IMO ->
they build their functionality primarily as re-usable libraries, and then expose
that in useful command line utilities.

The f18 compiler source code complies with most of LLVM's coding guidelines;
however, the code uses several C++17 features.  We've documented our use of
C++17 here:

  https://github.com/flang-compiler/f18/blob/master/documentation/C++17.md

In particular, the parse tree and the lowered forms of expressions and variables
are defined in terms of C++17 std::variant. Most of the compiler uses C++17
std::visit to walk these data structures.

It’s possible to reimplement the most important functionality of std:variant as
a subset class, say llvm:variant; however, variant gets its power from the C++17
features generic lambdas and parameter pack expansion on “using”.  Without these
C++17 features, use of variant would be impractical.

Our thinking when we started was that llvm would adopt C++17 before mid-2020,
which lines up with our projected completion date. If we were to adopt C++11 or
C++14, we would likely create substitutes for these classes, certainly at a cost
of calendar time and perhaps type safety and notational convenience.  One of our
principles is to take advantage of the standard library as much as possible, so
casual readers will better understand our code and so we avoid the time and bugs
associated with writing class libraries.

Our request would be to get a waiver for the C++11 requirement based on the fact
that we're skating to where the puck will be.  In the meantime, because F18
only exists as a stand-alone program, early adopters would still have a useful
parser and analyzer for Fortran.

Hold on, either it is a collection of libraries or it is a stand-alone program.
It can't really be both?

I misspoke.

f18 comprises several libraries, as well as a simple executable compiler driver
program for testing purposes.

The libraries correspond to the subdirectories in lib/.  They can be built as
shared or static binary libraries.

lib/common contains utilities and Fortran language representations used
throughout the project.

lib/parser contains the Fortran preprocessor and source normalization code,
source provenance tracking, parser combinator sublibrary, Fortran and OpenMP
grammars, parse tree definitions, an “unparser”, and an error message handling
sublibrary.  It depends only on lib/common.

lib/evaluate contains a strongly typed representation for Fortran expressions,
constants, and constant folding.  It uses the parser’s error message facility
when folding but is otherwise fairly independent code.

lib/semantics contains the symbol table, declaration processing, name
resolution, expression typing, and constraint checking.  It depends on
everything above.  It remains under active development.

lib/FIR contains the beginnings of the representation for executable code and
its translation to LLVM IR.  It depends on everything above as well as on LLVM
libraries.

Both the parser and the semantic analysis libraries comprise definitions of “top
level” objects that compose and orchestrate the contents of their respective
libraries into fairly narrow parameterized interfaces that hide much of their
implementations, so that one may easily parse a Fortran source file within a
context of macro definitions and enabled/disabled language extensions and get
back a parse tree and error message collection with minimal fuss.  The temporary
scaffolding driver in tools/f18 drives and connects these top-level interfaces
for testing purposes.

There will of course also be a large Fortran runtime support library for use by
compiled code.



Generally, I think the idea that diverging from the rest of the project here is
low-cost for a subproject isn't supported by experience with other projects.

Notably, it has a strong tendancy to create tension. You want some ADT or
support library in LLVM to work well with your C++17 code. But it is C++11.
Every time this has been done in the past, the result has been that generically
useful tools and libraries get added to the subproject rather than to LLVM as a
whole.

So FWIW, I'd be really opposed to this. Instead, I think that F18 should
have rich libraries, and develop them exactly the same way as the rest of LLVM.

We're getting close to switching to C++14, so maybe due to timing, you could
merge F18 when that happens?

Ultimately, I think you either need to raise the LLVM base language version or
lower the F18 one so that they match when merged IMO. Anything else I think will
hamper integration with the larger project.

As for the day-to-day usage of C++17, other replies suggest that building f18
should be disabled by default or that f18 builds detect that the host compiler
doesn’t support C++17 and (silently) refuse to build. (I haven’t looked at
whether this is possible or not.)  Either of these options allows build-bots and
older projects to continue as is without changing for the sake of f18 &
C++17.

Nearly all of the support routines we developed so far are very specific to
Fortran.  There are some idea around constant folding and reuse of the runtime
library for intrinsic folding that we’re developing; perhaps these ideas will be
generally useful in the future.

- Steve

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may
contain
confidential information.  Any unauthorized review, use, disclosure or
distribution
is prohibited.  If you are not the intended recipient, please contact the sender
by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190301/fd704ae9/attachment.html>