llvm dev - Oct 2012 - [LLVMdev] [cfe-dev] OpenMP support in CLANG: A proposal

On Wed, Oct 10, 2012 at 5:40 AM, Eli Friedman <eli.friedman at
gmail.com>wrote:
> On Tue, Oct 9, 2012 at 4:37 AM, Mahesha HS <mahesha.llvm at
gmail.com> wrote:
> > Hello All,
> >
> > We would like to make a proposal to support OpenMP in CLANG. The goal
of
> > this effort is to provide support for syntax
> >
> > analysis (parsing), semantic analysis and AST implementation for
OpenMP
> > constructs in CLANG.
> >
> >
> >
> > We would like to defer the design for *lowering* based on the outcome
of
> the
> > discussion happening in the LLVM dev list.
> >
> > Our design is aimed at implementing the necessary support in CLANG
> > irrespective of how it is finally *lowered*.
> >
> >
> >
> > Please find the details of the proposal and the current status in the
> > document attached.
>
> Please don't add a separate clangOMP.a; you're implementing new
> parsing and semantic analysis, but it isn't conceptually separate from
> the existing parsing/semantic analysis.
>
First, I would like to clarify that clangOMP.a implements only, *and only*
the "class OmpPragmaHandler" and nothing else. Parsing and Semantic
Analysis is as usual will be done by clangParse.a and clangSema.a components
respectively.

Second, the reason for adding a separate clangOMP.a. is as follows. I
looked into the
Clang code base, where few *pragmas* already being supported. I noticed that
few pragmas like "\#pragma once" are implemented in
"clangLex.a", and some
other
pragmas like "\#pragma align" are implemented in
"clangParse.a". Also, I
thought,
OpenMP includes many pragma directives and clauses, so better to separate
it out in
clangOMP.a.

However, your feedback is well considered. I will remove clangOMP.a and
move the
implementation of "class OmpPragmaHandler" to "clangLex.a".

>
> I'm not entirely sure what sort of feedback you're expecting;
ignoring
> the lowering, the part of implementing OpenMP that's likely to attract
> discussion is the AST representation, and you haven't described that
> in any detail.
>
As we mentioned in the previous mail, we are currently, deferred the design
for
*lowering* for the time being based on the outcome of the discussion
happening
in LLVM dev list on the topic of supporting OpenMP in LLVM.

However, AST representation and implementation is completed for all OpenMP
directives
and clauses, except for "critical" directive statement.

In the document, that we attached in our previous mail, we gave an idea of
how the AST looks
for an OpenMP directive statement as we did not want to bloat-up the
proposal document
with too many details. However, we are ready to share code base, which will
give more
information to provide comments.

In summary, we have completed all the required *basic* infrastructure
design and implementation
to implement OpenMP in Clang. We would like to get feedback on it before
proceeding for
further support.


> If you have patches that implement useful functionality; please submit
> sooner rather than later.  Doing a bunch of work in a private branch
> will mean more work for you in the long run because you won't get any
> feedback.
>
We have all the functionality implemented to provide the basic
infrastructure along with the
implementation of AST classes to represent different OpenMP directives and
clauses,
and the implementation of the "class OmpPrgamaHandler" as described in
the
doc.

We can share this code very soon, probably by tomorrow.

--
mahesha

>
> -Eli\
>


-- 
mahesha
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121010/80db7c02/attachment.html>

Hi Eli and Others

In response to your feedback, I have taken care of all your review comments
- I removed clangOMP.a
and moved the implementation of "class OmpPragmaHandler" to
"clangLex.a".

The attached zipped file - namely - 'OpenMP_support_in_Clang.tar.gz'
contains all the implemented
"patches" along with *newly* added source files.

Another attached text file - namely -
'OpenMP_support_in_Clang_Read_Me.txt'
briefly describes the
implementation.

You can also refer to the initial proposal document - namely -
'OpenMP_Support_in_Clang.pdf'.

Looking forward for your review comments.

--
mahesha






On Wed, Oct 10, 2012 at 10:30 AM, Mahesha HS <mahesha.llvm at gmail.com>
wrote:
>
>
> On Wed, Oct 10, 2012 at 5:40 AM, Eli Friedman <eli.friedman at
gmail.com>wrote:
>
>> On Tue, Oct 9, 2012 at 4:37 AM, Mahesha HS <mahesha.llvm at
gmail.com>
>> wrote:
>> > Hello All,
>> >
>> > We would like to make a proposal to support OpenMP in CLANG. The
goal of
>> > this effort is to provide support for syntax
>> >
>> > analysis (parsing), semantic analysis and AST implementation for
OpenMP
>> > constructs in CLANG.
>> >
>> >
>> >
>> > We would like to defer the design for *lowering* based on the
outcome
>> of the
>> > discussion happening in the LLVM dev list.
>> >
>> > Our design is aimed at implementing the necessary support in CLANG
>> > irrespective of how it is finally *lowered*.
>> >
>> >
>> >
>> > Please find the details of the proposal and the current status in
the
>> > document attached.
>>
>> Please don't add a separate clangOMP.a; you're implementing new
>> parsing and semantic analysis, but it isn't conceptually separate
from
>> the existing parsing/semantic analysis.
>>
>
> First, I would like to clarify that clangOMP.a implements only, *and only*
> the "class OmpPragmaHandler" and nothing else. Parsing and
Semantic
> Analysis is as usual will be done by clangParse.a and clangSema.a
> components
> respectively.
>
> Second, the reason for adding a separate clangOMP.a. is as follows. I
> looked into the
> Clang code base, where few *pragmas* already being supported. I noticed
> that
> few pragmas like "\#pragma once" are implemented in
"clangLex.a", and some
> other
> pragmas like "\#pragma align" are implemented in
"clangParse.a". Also, I
> thought,
> OpenMP includes many pragma directives and clauses, so better to separate
> it out in
> clangOMP.a.
>
> However, your feedback is well considered. I will remove clangOMP.a and
> move the
> implementation of "class OmpPragmaHandler" to
"clangLex.a".
>
>
>>
>> I'm not entirely sure what sort of feedback you're expecting;
ignoring
>> the lowering, the part of implementing OpenMP that's likely to
attract
>> discussion is the AST representation, and you haven't described
that
>> in any detail.
>>
>
> As we mentioned in the previous mail, we are currently, deferred the
> design for
> *lowering* for the time being based on the outcome of the discussion
> happening
> in LLVM dev list on the topic of supporting OpenMP in LLVM.
>
> However, AST representation and implementation is completed for all OpenMP
> directives
> and clauses, except for "critical" directive statement.
>
> In the document, that we attached in our previous mail, we gave an idea of
> how the AST looks
> for an OpenMP directive statement as we did not want to bloat-up the
> proposal document
> with too many details. However, we are ready to share code base, which
> will give more
> information to provide comments.
>
> In summary, we have completed all the required *basic* infrastructure
> design and implementation
> to implement OpenMP in Clang. We would like to get feedback on it before
> proceeding for
> further support.
>
>
>
>> If you have patches that implement useful functionality; please submit
>> sooner rather than later.  Doing a bunch of work in a private branch
>> will mean more work for you in the long run because you won't get
any
>> feedback.
>>
>
> We have all the functionality implemented to provide the basic
> infrastructure along with the
> implementation of AST classes to represent different OpenMP directives and
> clauses,
> and the implementation of the "class OmpPrgamaHandler" as
described in the
> doc.
>
> We can share this code very soon, probably by tomorrow.
>
> --
> mahesha
>
>
>>
>> -Eli\
>>
>
>
>
> --
> mahesha
>
>

-- 
mahesha
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121010/7ded735e/attachment.html>
-------------- next part --------------
==================================================OpenMP Support in Clang:  A
proposal implementation
==================================================
=========================1. Implementation Details:
=========================

-. We extended the existing Clang classes like Parser, Sema,  etc in order to
handle OpenMP
   in Clang.

-. We added a new class called "class OmpPragmaHandler" to Clang. The
main job of
   OmpPragmaHandler is to assist other Clang components like Parser, Sema, etc
while
   handling OpenMP constructs.

-. Tokens are introduced to represent only OpenMP pragma directives, and these
tokens are
   inserted by OmpPragmaHandler to token streams, upon seeing the OpenMP pragma
directive
   statements. Lexer is completely not aware of these tokens. Additionally,
OpenMP pragma
   clause names are parsed based on the context with string comparison approach.
This
   strategy avoids conflicts with ?Identifier? tokens that would happen when we
introduce
   separate tokens for each and every OpenMP construct names.
 
-. As in the case of parsing any other C/C++ constructs, Parser starts parsing
the OpenMP
   directive statements when it encounters with OpenMP directive tokens, which
are inserted
   by OmpPragmaHandler. Once the parsing is done, Parser calls Semantic Analyzer
to perform
   semantic analysis, and to build AST tree which represents parsed OpenMP
directive
   statement. Finally, CodeGen component of Clang is called to lower OpenMP
ASTs.

-. Clang Preprocessor is responsible for registering all the OpenMP pragma 
   directives, and further processing them when OpenMP pragma directives
statements
   are encountered. Clang Parser is responsible for all syntax analysis and
syntax
   errors checking and reporting (if any). Clang Semantic Analyzer is
responsible
   for all semantic analysis and semantic errors checking and reporting (if
any).
   All the ASTs classes are defined and implemented within Clang AST manager
component
   (clangAST.a) with all the additional supporting implementation for AST
visiting,
   AST writing, AST reading, etc. Finally, CodeGen component of Clang
(clangCodeGen.a)
   implements all the AST lowering routines.

-. During the initialization of Clang compiler proper before processing a 
   translation unit, a PragmaOmpHandler object is constructed before both Parser
   and Sema objects are constructed. Upon creation of PragmaOmpHandler, it is
made
   to initialize all its members, and asked to call Preprocessor to get register
all
   the OpenMP pragma directive?s names. Later, both Parser and Sema objects are
made to
   reference PragmaOmpHandler object when they are constructed.
  
-. During the parsing of a translation unit, Clang Lexer, upon lexing the token
?#?,
   calls Preprocessor to handle possibly encountered pragma statement. Clang
Preprocessor,
   after realizing that the encountered pragma is an OpenMP pragma statement, it
calls
   PragmaOmpHandler to handle it. The PragmaOmpHandler, upon called to handle
OpenMP pragma
   statement, does one of the following two tasks depending on whether the
?-fopenmp?
   option is enabled or not.

    #. When the ?-fopenmp? option is enabled, it inserts a token which
represents the
       encountered OpenMP pragma statement into token stream so that parser can
recognize
       it and can parse the respective OpenMP pragma directive statement. Parser
will later
       take care of parsing the encountered OpenMP pragma directive statement.

    #. When the ?-fopenmp? option is not enabled, it simply eats-up (ignores)
the OpenMP
       pragma directive statement line, so that, Parser can proceed with the
sequential
       compilation as usual. However, in this case, a warning message is emitted
saying
       that the current OpenMP directive statement will be ignored. Note that
Clang emits
       only one warning message per translation unit irrespective of the number
of OpenMP
       constructs encountered within the translation unit being parsed. This
avoids message
       bloating on the output device.

-. OpenMP ASTs are implemented as below:

    #. A separate AST is introduced for every OpenMP directive and clause.  More
than *thirty*
       AST classes are introduced for OpenMP.

    #. All the AST classes which represent OpenMP directives or clauses publicly
derive from
       the statement AST base class namely ?class Stmt?.

    #. There will be a separate AST class implementation for each OpenMP
directive and clause.

    #. There will be an additional AST class node implementation, called ?class
OmpClause?.
       OmpClause AST node holds other AST clause nodes as its children. However,
note that the
       same clauses which appear more than once in an OpenMP directive statement
are represented
       as a linked list. This representation makes tree traversal easier for few
particular
       semantic checks.

    #. Variables in the list clauses like ?private? clause are represented as
?DeclRefExpr? node.

    #. ASTs are designed and implemented in such a way that *no* OpenMP
information is lost
       including source location information.

-. Skeleton routines (with few partially implemented) for parsing, semantic
analysis are
   introduced.

-. The option ?-fopenmp? is implemented. Clang driver now understands the above
option, and pass
   it onto the Clang compiler proper.



==========================1. Source File Information:
==========================

-. The option ?-fopenmp? support

    #. clang/include/clang/Driver/Options.td
    #. clang/include/clang/Basic/LangOptions.def
    #. clang/include/clang/Basic/DiagnosticLexKinds.td
    #. clang/lib/Driver/Tools.cpp
    #. clang/lib/Frontend/CompilerInvocation.cpp


-. class OmpPragmaHandler implementation
  
    #. clang/include/clang/Lex/PragmaOmpHandler.h  -----> newly added
    #. clang/lib/Lex/PragmaOmpHandler.cpp  -----> newly added


-. OmpPragmaHandler 'object' creation
    
    #. clang/include/clang/Frontend/CompilerInstance.h
    #. clang/include/clang/Frontend/ASTUnit.h
    #. clang/include/clang/Parse/ParseAST.h
    #. clang/lib/Frontend/ChainedIncludesSource.cpp
    #. clang/lib/Frontend/FrontendAction.cpp
    #. clang/lib/Frontend/ASTUnit.cpp
    #. clang/lib/Frontend/CompilerInstance.cpp
    #. clang/lib/Parse/ParseAST.cpp
    #. clang/lib/Sema/Sema.cpp
    #. clang/lib/Parse/Parser.cpp
    

-. OpenMP tokens
  
    #. clang/include/clang/Basic/TokenKinds.def


-. OpenMP parsing

    #. clang/include/clang/Parse/Parser.h
    #. clang/lib/Parse/ParseStmtOpenMP.cpp  ------> newly added
    #. clang/lib/Parse/ParseStmt.cpp


-. OpenMP semantic analyis

    #. clang/include/clang/Sema/Sema.h
    #. clang/lib/Sema/SemaStmtOpenMP.cpp  -----> newly added


-. OpenMP AST implementation

    #. clang/include/clang/AST/StmtOpenMP.h  -----> newly added 
    #  clang/include/clang/AST/Stmt.h  
    #. clang/include/clang/Basic/StmtNodes.td
    #. clang/include/clang/AST/RecursiveASTVisitor.h
    #. clang/include/clang/AST/StmtVisitor.h
    #. clang/lib/StaticAnalyzer/Core/ExprEngine.cpp
    #. clang/lib/Serialization/ASTReaderStmt.cpp
    #. clang/lib/Serialization/ASTWriterStmt.cpp
    #. clang/lib/Sema/TreeTransform.h
    #. clang/lib/CodeGen/CGStmt.cpp
    #. clang/lib/AST/StmtOpenMP.cpp  ------> newly added
    #. clang/lib/AST/StmtProfile.cpp
    #. clang/lib/AST/StmtPrinter.cpp
    #. clang/lib/AST/Stmt.cpp
    #. clang/tools/libclang/RecursiveASTVisitor.h
    #. clang/tools/libclang/CXCursor.cpp


-. Makefile changes

    #. clang/lib/AST/CMakeLists.txt
    #. clang/lib/Sema/CMakeLists.txt
    #. clang/lib/Parse/CMakeLists.txt
    #. clang/lib/Lex/CMakeLists.txt


===============================1. New Source File Information:
===============================
-.  PragmaOmpHandler.h and PragmaOmpHandler.cpp:  Implements OmpPragmahandler
class
-.  ParseStmtOpenMP.cpp: Implements all the OpenMP parsing routines
-.  SemaStmtOpenMP.cpp: Implements all the OpenMP semantics routines
-.  StmtOpenMP.h and StmtOpenMP.cpp: Implements all OpenMP AST classes
    
  
	
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenMP_support_in_Clang.tar.gz
Type: application/x-gzip
Size: 35089 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121010/7ded735e/attachment.bin>

On Wed, Oct 10, 2012 at 6:15 AM, Mahesha HS <mahesha.llvm at gmail.com>
wrote:
> Hi Eli and Others
>
> In response to your feedback, I have taken care of all your review comments
> - I removed clangOMP.a
> and moved the implementation of "class OmpPragmaHandler" to
"clangLex.a".
>
> The attached zipped file - namely -
'OpenMP_support_in_Clang.tar.gz'
> contains all the implemented
> "patches" along with *newly* added source files.
>
> Another attached text file - namely -
'OpenMP_support_in_Clang_Read_Me.txt'
> briefly describes the
> implementation.
>
> You can also refer to the initial proposal document - namely -
> 'OpenMP_Support_in_Clang.pdf'.
>
> Looking forward for your review comments.
Can you please split this patch somehow?  This is way too big to
easily review.  I'd like to see separate patches for the command-line
option, the pragma handler itself, parsing each pragma, each new AST
node, and semantic analysis for each kind of pragma, with testcases
for every patch.  And the more you can split them, the better to get
through reviews faster.

Please include new files in patches using "svn add"; don't send
new
files separately.

Please look over patches before submitting to make sure you don't
include unnecessary changes to whitespace etc.

-Eli