llvm dev - Apr 2016 - [Openmp-dev] [cfe-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries

I respect Hal's more tactful approach and response..

Let me play devils advocate for a minute

1) Yet another programming model - Is the advantage spelled out
somewhere? (I know there are reasons, but I'd like to see a FAQ or
this clearly documented. Examples pretty please.. More for long term
than my own selfish benefit)
2) Is this an "open standard" - If I wanted to propose a major change
to SE - how would I or someone else go about it? OpenMP/ACC have more
or less clearly defined paths for new features.. What's the governing
policy here.. Bug fixes are easy to deal with, but does Google have
final say on the roadmap..
3) When the project is created - will it include lots of good tests?
4) It's probably used internally @google - who else will be using
this? Is the target HPC, Android.. etc

Lastly - sorry, but I don't like this kick-the-can approach to what
should be proper engineering and planning upfront. Can someone @google
gentleman's promise to actively work on playing nice with other
projects, specifically OpenMP and Intel. From my perspective nothing
stops Google from tossing it up on github or google code and letting
it stay there until all the pieces are in the correct place. Why it
*must* be an llvm project now doesn't make sense to me. When the shoe
was on the other foot (OpenMP) there was all sorts of shit and redtape
Intel (and others) had to jump around to get it included. Google has a
lot of good karma in the llvm community and maybe that's the
difference..

Andrey,

I may have misunderstood Chandler, but I think it would be good to house
libomptarget together with SE in the new project. I suspect he means to
leave the option open for the libomptarget developers to choose the best
option as they see fit.

In terms of organization, I would expect the project initially to contain a
StreamExecutor directory and a libomptarget directory where each project
could work separately.

When it comes to the difference in review process between the two projects,
I don't know the answer. I would expect the new project to handle reviews
similar to the way they are handled in the LLVM project itself, so I don't
know if there would be any difference compared to what is happening now
with libomptarget.

On Wed, Apr 27, 2016 at 9:32 AM C Bergström <openmp-dev at lists.llvm.org>
wrote:
> I respect Hal's more tactful approach and response..
>
> Let me play devils advocate for a minute
>
> 1) Yet another programming model - Is the advantage spelled out
> somewhere? (I know there are reasons, but I'd like to see a FAQ or
> this clearly documented. Examples pretty please.. More for long term
> than my own selfish benefit)
> 2) Is this an "open standard" - If I wanted to propose a major
change
> to SE - how would I or someone else go about it? OpenMP/ACC have more
> or less clearly defined paths for new features.. What's the governing
> policy here.. Bug fixes are easy to deal with, but does Google have
> final say on the roadmap..
> 3) When the project is created - will it include lots of good tests?
> 4) It's probably used internally @google - who else will be using
> this? Is the target HPC, Android.. etc
>
> Lastly - sorry, but I don't like this kick-the-can approach to what
> should be proper engineering and planning upfront. Can someone @google
> gentleman's promise to actively work on playing nice with other
> projects, specifically OpenMP and Intel. From my perspective nothing
> stops Google from tossing it up on github or google code and letting
> it stay there until all the pieces are in the correct place. Why it
> *must* be an llvm project now doesn't make sense to me. When the shoe
> was on the other foot (OpenMP) there was all sorts of shit and redtape
> Intel (and others) had to jump around to get it included. Google has a
> lot of good karma in the llvm community and maybe that's the
> difference..
> _______________________________________________
> Openmp-dev mailing list
> Openmp-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160427/3d18baf2/attachment.html>

I've put together a proposed "charter" for this new project, which
I am
calling parallel_utils (although I'm very open to suggestions for a better
name). The text of my charter is below, and I welcome any input on how it
can be improved.


====================================================LLVM Parallel Utils
Subproject Charter
====================================================
----------------------------------------------
Description
----------------------------------------------
The LLVM open source project will contain a subproject named
`parallel_utils` which will host the development of libraries which are
aimed at enabling parallelism in code and which are also closely tied to
compiler technology.  Examples of libraries suitable for hosting within the
`parallel_utils` subproject are runtime libraries and parallel math
libraries. The initial candidates for inclusion in this subproject are
StreamExecutor and libomptarget which would live in the `streamexecutor`
and `libomptarget` subdirectories of `parallel_utils`, respectively.

The `parallel_utils` project will host a collection of libraries where each
library may be dependent on other libraries from the project or may be
completely independent of any other libraries in the project. The rationale
for hosting independent libraries within the same subproject is that all
libraries in the project are providing related functionality that lives at
the intersection of parallelism and compiler technology. It is expected
that some libraries which initially began as independent will develop
dependencies over time either between existing libraries or by extracting
common code that can be used by each. One of the purposes of this
subproject is to provide a working space where such refactoring and code
sharing can take place.

Libraries in the `parallel_utils` subproject may also depend on the LLVM
core libraries. This will be useful for avoiding duplication of code within
the LLVM project for common utilities such as those found in the LLVM
support library.


----------------------------------------------
Requirements
----------------------------------------------
Libraries included in the `parallel_utils` subproject must strive to
achieve the following requirements:

1. Adhere to the LLVM coding standards.
2. Use the LLVM build and test infrastructure.
3. Be released under LLVM's license.


Coding standards
----------------
Libraries in `parallel_utils` will match the LLVM coding standards. For
existing projects being checked into the subproject as-is, an exception
will be made during the initial check-in, with the understanding that the
code will be promptly updated to follow the standards. Therefore, a three
month grace period will be allowed for new libraries to meet the LLVM
coding standards.

Additional exceptions to strict adherence to the LLVM coding standards may
be allowed in certain other cases, but the reasons for such exceptions must
be discussed and documented on a case-by-case basis.


LLVM build and test infrastructure
----------------------------------
Using the LLVM build and test infrastructure currently means using `cmake`
for building, `lit` for testing, and `buildbot` for automating build and
testing. This project will follow the main LLVM project conventions here
and track them as they evolve.


LLVM license
------------
For simplicity, the `parallel_utils` project will use the normal LLVM
license. While some runtime libraries use a dual license scheme in LLVM, we
anticipate the project removing the need for this eventually and in the
interim follow the simpler but still permissive license. Among other
things, this makes it straightforward for these libraries to re-use core
LLVM libraries where appropriate.

On Wed, Apr 27, 2016 at 9:52 AM Jason Henline <jhen at google.com> wrote:
> Andrey,
>
> I may have misunderstood Chandler, but I think it would be good to house
> libomptarget together with SE in the new project. I suspect he means to
> leave the option open for the libomptarget developers to choose the best
> option as they see fit.
>
> In terms of organization, I would expect the project initially to contain
> a StreamExecutor directory and a libomptarget directory where each project
> could work separately.
>
> When it comes to the difference in review process between the two
> projects, I don't know the answer. I would expect the new project to
handle
> reviews similar to the way they are handled in the LLVM project itself, so
> I don't know if there would be any difference compared to what is
happening
> now with libomptarget.
>
> On Wed, Apr 27, 2016 at 9:32 AM C Bergström <openmp-dev at
lists.llvm.org>
> wrote:
>
>> I respect Hal's more tactful approach and response..
>>
>> Let me play devils advocate for a minute
>>
>> 1) Yet another programming model - Is the advantage spelled out
>> somewhere? (I know there are reasons, but I'd like to see a FAQ or
>> this clearly documented. Examples pretty please.. More for long term
>> than my own selfish benefit)
>> 2) Is this an "open standard" - If I wanted to propose a
major change
>> to SE - how would I or someone else go about it? OpenMP/ACC have more
>> or less clearly defined paths for new features.. What's the
governing
>> policy here.. Bug fixes are easy to deal with, but does Google have
>> final say on the roadmap..
>> 3) When the project is created - will it include lots of good tests?
>> 4) It's probably used internally @google - who else will be using
>> this? Is the target HPC, Android.. etc
>>
>> Lastly - sorry, but I don't like this kick-the-can approach to what
>> should be proper engineering and planning upfront. Can someone @google
>> gentleman's promise to actively work on playing nice with other
>> projects, specifically OpenMP and Intel. From my perspective nothing
>> stops Google from tossing it up on github or google code and letting
>> it stay there until all the pieces are in the correct place. Why it
>> *must* be an llvm project now doesn't make sense to me. When the
shoe
>> was on the other foot (OpenMP) there was all sorts of shit and redtape
>> Intel (and others) had to jump around to get it included. Google has a
>> lot of good karma in the llvm community and maybe that's the
>> difference..
>> _______________________________________________
>> Openmp-dev mailing list
>> Openmp-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160427/f33f0974/attachment.html>

On Wed, Apr 27, 2016 at 12:52 PM Jason Henline via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> Andrey,
>
> I may have misunderstood Chandler, but I think it would be good to house
> libomptarget together with SE in the new project. I suspect he means to
> leave the option open for the libomptarget developers to choose the best
> option as they see fit.
>
Sorry, this was exactly my intent. =D
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160427/af61fd25/attachment.html>

Jason,

Thank you for the clarification!

On Wed, Apr 27, 2016 at 7:52 PM, Jason Henline <jhen at google.com> wrote:
> When it comes to the difference in review process between the two
> projects, I don't know the answer. I would expect the new project to
handle
> reviews similar to the way they are handled in the LLVM project itself, so
> I don't know if there would be any difference compared to what is
happening
> now with libomptarget.
>
To clarify, I'm not speaking on review of new patches, but on review of
existing code. I suppose SE is a significant project, with code already
reviewed internally @google. Do we plan to require this existing code pass
through llvm community review process as well? If not, same standards
should be applied to libomptarget -- this is my point.

Yours,
Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160428/dd702772/attachment-0001.html>