llvm dev - Jun 2016 - [Openmp-dev] [cfe-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries

On Wed, Jun 1, 2016 at 8:52 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
> ----- Original Message -----
>> From: "C Bergström" <cbergstrom at pathscale.com>
>> To: "Hal Finkel" <hfinkel at anl.gov>
>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev" <cfe-dev at lists.llvm.org>, "openmp-dev"
>> <openmp-dev at lists.llvm.org>, "Chandler Carruth"
<chandlerc at gmail.com>, "Carlo Bertolli" <cbertol at
us.ibm.com>,
>> "Andrey Bokhanko" <andreybokhanko at gmail.com>
>> Sent: Wednesday, June 1, 2016 6:46:57 AM
>> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing an LLVM
subproject for parallelism runtime and support
>> libraries
>>
>> On Wed, Jun 1, 2016 at 7:22 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
>> >
>> > ________________________________
>> >
>> > From: "C Bergström" <cbergstrom at pathscale.com>
>> > To: "Hal Finkel" <hfinkel at anl.gov>
>> > Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev"
>> > <cfe-dev at lists.llvm.org>, "openmp-dev"
<openmp-dev at lists.llvm.org>,
>> > "Chandler Carruth" <chandlerc at gmail.com>,
"Carlo Bertolli"
>> > <cbertol at us.ibm.com>, "Andrey Bokhanko"
<andreybokhanko at gmail.com>
>> > Sent: Wednesday, June 1, 2016 12:19:19 AM
>> > Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing an
>> > LLVM
>> > subproject for parallelism runtime and support libraries
>> >
>> > The thread has lost focus and cherry picking replies..
>> >
>> > To restate things since maybe you missed my points
>> > ----
>> > 1) SE is a programming model and needs a home of it's own.
Having a
>> > programming model with it's headers and all other stuff glued
into
>> > a runtime
>> > project which intends to be universal and PM agnostic doesn't
make
>> > sense.
>> >
>> > They'd start in separate subdirectories.
>> >
>> >
>> > 1.1) The more I look, the most it seems SE is just a step-child
>> > project and
>> > stuffing it in llvm while still not having any users or strong
>> > backing
>> > doesn't make sense. We have enough PM already and my gut
feeling is
>> > this
>> > isn't going in a direction to bring in other stakeholders.
>>
>> Point by point because I don't agree with what you write below
>> entirely
>>
>> >
>> > I think this is the core of my reply. OpenMP has a strong user
>> > community,
>>
>> I'd agree here
>>
>> > but OpenMP 4 offloading is still young.
>>
>> ditto
>>
>> > OpenMP 4 offloading does not yet
>> > have a real user community
>>
>> agreement
>>
>> > yet because the first implementations just
>> > started shipping very recently.
>>
>> Partially strongly disagree - Maybe you meant on Power8?
>> Intel has had their support for OMP4 for quite some time. It may have
>> been buggy and focused on simd directive, but as best I can tell they
>> have worked quite hard to fix bugs and improve it.
>
> I meant across many platforms. Intel had preliminary support for OpenMP 4
offloading directives even before OpenMP 4 was standardized (in 2013). This did
not include all of what ended up in the standard, and the offloading part of the
standard needed to be fixed in a breaking way for OpenMP 4.1/4.5 regardless, so
this is certainly the exception rather than the rule.
>
>>
>> (Shameless self promotion) We have had some degree of OMP4 offloading
>> for like 1.5 years now.. It's mostly targeting the GPU and x86, but
>> also more recently working on Power8/AArch64.
>>
>> I'm quite certain these companies all have varying degrees of OMP4
>> done: Cray, IBM, PGI
>
> Yes, many are working on implementations, and some have shipped.
There's a list here: http://openmp.org/wp/openmp-compilers/ - no one here
really lists any OpenMP 4 offloading implementations before 2015. PGI does not
currently list OpenMP 4 at all (although they've certainly done a lot of
work on OpenACC).
>
>>
>> > Furthermore, our implementation is certainly
>> > quite new, and OpenMP 4 offloading is really quite akin to SE in
>> > that
>> > regard.
>>
>> Strongly disagree - Intel has been working with the LLVM community on
>> the parsing/sema and other parts of OMP4 for like a year or more..
>> This has been a long and well vetted process backed back a well
>> defined standard.
>
> Yes, both Intel and IBM have contributed significantly to Clang in this
regard.
>
>> Compared to SE which is just some thing that popped
>> up out of nowhere and
>
> It popped up in TensorFlow, which itself popped up out of nowhere, but is
already a popular open-source project for machine learning.
>
>> has a couple people from Google throwing their
>> weight around.
>
> Google is a trusted member of the LLVM community and a major contributor.
So I agree with you in the following sense: When Google promises to update the
code to follow LLVM's coding conventions and to manage its future
development in accordance with our community norms, I believe them. I think
they've earned a place in the "trust but verify" category in this
regard, and I think this is a positive thing.
>
yes, but not every project that is started is finished and who would
be the maintainer if the person leaves Google? I still don't see why
they can't fork it on github or create a project to let it bake and
get some users or traction.
>> > I view them both as experimental projects, and both have strong
>> > backing with significant investment, so I expect both to mature
>> > over time.
>>
>> I'd agree both are experimental and probably need some work.
I'd
>> strongly disagree the amount of investment of both projects is equal.
>> Again - OMP4 may have zero real world users, but there's a lot of
>> stubborn people trying to make it work. Compared to SE which the
>> people involved have yet to even answer basic things - like what's
>> the
>> future of SE and how to contribute to it.
>
> They might not know what the future is. They have code that is useful to
them, and they feel would be useful to others. There's strong potential for
compiler integration in the future, but it is just a library now. They've
talked about extending it to cover different backends (OpenCL, etc.), and
investigating host-based-task dependencies, etc. My impression is that, in
general, they'd like the code's users, present and future, to shape its
future. All of this seems perfectly healthy. If it becomes part of LLVM, how to
contribute to it will become clear.
I love your optimistic attitude, but the reality is that successful
programming models take a significant amount of work. I'm well aware
that they can handle at least part of that, but so far I'm not sure
who else will invest in it..

The industry has : CUDA, OpenCL, OpenMP4, OpenACC, C++AMP, Parallel
STL(??), some research projects from DOE and dead stuff like HMPP...
Are all these so broken that they can't be fixed to fill the same
necessity as SE?

How similar is it to the failed AMD Stream Computing (sorry AMD)
http://www.ele.uri.edu/courses/ele408/StreamGPU.pdf

Intel's work had to live and grow outside for quite a while - Google
may have more good karma, but I don't see this as being fair. Nothing
stops them from posting the code online somewhere.

> On Jun 1, 2016, at 7:42 AM, C Bergström via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On Wed, Jun 1, 2016 at 8:52 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
>> ----- Original Message -----
>>> From: "C Bergström" <cbergstrom at pathscale.com>
>>> To: "Hal Finkel" <hfinkel at anl.gov>
>>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev" <cfe-dev at lists.llvm.org>, "openmp-dev"
>>> <openmp-dev at lists.llvm.org>, "Chandler Carruth"
<chandlerc at gmail.com>, "Carlo Bertolli" <cbertol at
us.ibm.com>,
>>> "Andrey Bokhanko" <andreybokhanko at gmail.com>
>>> Sent: Wednesday, June 1, 2016 6:46:57 AM
>>> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing an
LLVM subproject for parallelism runtime and support
>>> libraries
>>> 
>>> On Wed, Jun 1, 2016 at 7:22 PM, Hal Finkel <hfinkel at
anl.gov> wrote:
>>>> 
>>>> ________________________________
>>>> 
>>>> From: "C Bergström" <cbergstrom at
pathscale.com>
>>>> To: "Hal Finkel" <hfinkel at anl.gov>
>>>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev"
>>>> <cfe-dev at lists.llvm.org>, "openmp-dev"
<openmp-dev at lists.llvm.org>,
>>>> "Chandler Carruth" <chandlerc at gmail.com>,
"Carlo Bertolli"
>>>> <cbertol at us.ibm.com>, "Andrey Bokhanko"
<andreybokhanko at gmail.com>
>>>> Sent: Wednesday, June 1, 2016 12:19:19 AM
>>>> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing
an
>>>> LLVM
>>>> subproject for parallelism runtime and support libraries
>>>> 
>>>> The thread has lost focus and cherry picking replies..
>>>> 
>>>> To restate things since maybe you missed my points
>>>> ----
>>>> 1) SE is a programming model and needs a home of it's own.
Having a
>>>> programming model with it's headers and all other stuff
glued into
>>>> a runtime
>>>> project which intends to be universal and PM agnostic
doesn't make
>>>> sense.
>>>> 
>>>> They'd start in separate subdirectories.
>>>> 
>>>> 
>>>> 1.1) The more I look, the most it seems SE is just a step-child
>>>> project and
>>>> stuffing it in llvm while still not having any users or strong
>>>> backing
>>>> doesn't make sense. We have enough PM already and my gut
feeling is
>>>> this
>>>> isn't going in a direction to bring in other stakeholders.
>>> 
>>> Point by point because I don't agree with what you write below
>>> entirely
>>> 
>>>> 
>>>> I think this is the core of my reply. OpenMP has a strong user
>>>> community,
>>> 
>>> I'd agree here
>>> 
>>>> but OpenMP 4 offloading is still young.
>>> 
>>> ditto
>>> 
>>>> OpenMP 4 offloading does not yet
>>>> have a real user community
>>> 
>>> agreement
>>> 
>>>> yet because the first implementations just
>>>> started shipping very recently.
>>> 
>>> Partially strongly disagree - Maybe you meant on Power8?
>>> Intel has had their support for OMP4 for quite some time. It may
have
>>> been buggy and focused on simd directive, but as best I can tell
they
>>> have worked quite hard to fix bugs and improve it.
>> 
>> I meant across many platforms. Intel had preliminary support for OpenMP
4 offloading directives even before OpenMP 4 was standardized (in 2013). This
did not include all of what ended up in the standard, and the offloading part of
the standard needed to be fixed in a breaking way for OpenMP 4.1/4.5 regardless,
so this is certainly the exception rather than the rule.
>> 
>>> 
>>> (Shameless self promotion) We have had some degree of OMP4
offloading
>>> for like 1.5 years now.. It's mostly targeting the GPU and x86,
but
>>> also more recently working on Power8/AArch64.
>>> 
>>> I'm quite certain these companies all have varying degrees of
OMP4
>>> done: Cray, IBM, PGI
>> 
>> Yes, many are working on implementations, and some have shipped.
There's a list here: http://openmp.org/wp/openmp-compilers/ - no one here
really lists any OpenMP 4 offloading implementations before 2015. PGI does not
currently list OpenMP 4 at all (although they've certainly done a lot of
work on OpenACC).
>> 
>>> 
>>>> Furthermore, our implementation is certainly
>>>> quite new, and OpenMP 4 offloading is really quite akin to SE
in
>>>> that
>>>> regard.
>>> 
>>> Strongly disagree - Intel has been working with the LLVM community
on
>>> the parsing/sema and other parts of OMP4 for like a year or more..
>>> This has been a long and well vetted process backed back a well
>>> defined standard.
>> 
>> Yes, both Intel and IBM have contributed significantly to Clang in this
regard.
>> 
>>> Compared to SE which is just some thing that popped
>>> up out of nowhere and
>> 
>> It popped up in TensorFlow, which itself popped up out of nowhere, but
is already a popular open-source project for machine learning.
>> 
>>> has a couple people from Google throwing their
>>> weight around.
>> 
>> Google is a trusted member of the LLVM community and a major
contributor. So I agree with you in the following sense: When Google promises to
update the code to follow LLVM's coding conventions and to manage its future
development in accordance with our community norms, I believe them. I think
they've earned a place in the "trust but verify" category in this
regard, and I think this is a positive thing.
>> 
> 
> yes, but not every project that is started is finished and who would
> be the maintainer if the person leaves Google?
Like anything that bitrot and is no longer maintained in LLVM: it gets deleted
(or the sources stay there but it is no longer updated/maintained).
There are precedents: http://llvm.org/svn/llvm-project/

> I still don't see why
> they can't fork it on github or create a project to let it bake and
> get some users or traction.
The intent may be that instead of creating a separate community of users,
it'd create its community inside the llvm projects community. That looks
like a nice "feature" long term.

Cost/benefit: cost is not really existent, potential benefits are interesting.

-- 
Mehdi

> 
>>>> I view them both as experimental projects, and both have strong
>>>> backing with significant investment, so I expect both to mature
>>>> over time.
>>> 
>>> I'd agree both are experimental and probably need some work.
I'd
>>> strongly disagree the amount of investment of both projects is
equal.
>>> Again - OMP4 may have zero real world users, but there's a lot
of
>>> stubborn people trying to make it work. Compared to SE which the
>>> people involved have yet to even answer basic things - like
what's
>>> the
>>> future of SE and how to contribute to it.
>> 
>> They might not know what the future is. They have code that is useful
to them, and they feel would be useful to others. There's strong potential
for compiler integration in the future, but it is just a library now.
They've talked about extending it to cover different backends (OpenCL,
etc.), and investigating host-based-task dependencies, etc. My impression is
that, in general, they'd like the code's users, present and future, to
shape its future. All of this seems perfectly healthy. If it becomes part of
LLVM, how to contribute to it will become clear.
> 
> I love your optimistic attitude, but the reality is that successful
> programming models take a significant amount of work. I'm well aware
> that they can handle at least part of that, but so far I'm not sure
> who else will invest in it..
> 
> The industry has : CUDA, OpenCL, OpenMP4, OpenACC, C++AMP, Parallel
> STL(??), some research projects from DOE and dead stuff like HMPP...
> Are all these so broken that they can't be fixed to fill the same
> necessity as SE?
> 
> How similar is it to the failed AMD Stream Computing (sorry AMD)
> http://www.ele.uri.edu/courses/ele408/StreamGPU.pdf
> 
> Intel's work had to live and grow outside for quite a while - Google
> may have more good karma, but I don't see this as being fair. Nothing
> stops them from posting the code online somewhere.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Wed, Jun 1, 2016 at 11:21 PM, Mehdi Amini <mehdi.amini at apple.com>
wrote:
>
>> On Jun 1, 2016, at 7:42 AM, C Bergström via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>
>> I still don't see why
>> they can't fork it on github or create a project to let it bake and
>> get some users or traction.
>
> The intent may be that instead of creating a separate community of users,
it'd create its community inside the llvm projects community. That looks
like a nice "feature" long term.
>
> Cost/benefit: cost is not really existent, potential benefits are
interesting.
Not true - the cost isn't zero.

Of the top of my head...

#1 The project will mix and blend with other programming models - The
shared runtime should cover all the popular stuff and "this"

#2 User confusion - Promoting this over other more established
projects and models is just another turd in the pot

Lots of things are interesting.. lets take Legion as an example.. that
little heard of project at *least* has a bunch of examples and more
docs than SE..
https://github.com/StanfordLegion/legion/tree/stable/examples

Why do I feel so strongly - because I've been dealing with
here-today-gone-tomorrow programming models for 8 years now... It's
boring and wastes a lot of time. Research projects should incubate
outside the tree before being included with production parts.

----- Original Message -----
> From: "C Bergström" <cbergstrom at pathscale.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev" <cfe-dev at lists.llvm.org>, "openmp-dev"
> <openmp-dev at lists.llvm.org>, "Chandler Carruth"
<chandlerc at gmail.com>, "Carlo Bertolli" <cbertol at
us.ibm.com>,
> "Andrey Bokhanko" <andreybokhanko at gmail.com>
> Sent: Wednesday, June 1, 2016 9:42:34 AM
> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing an LLVM
subproject for parallelism runtime and support
> libraries
> 
> On Wed, Jun 1, 2016 at 8:52 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
> > ----- Original Message -----
> >> From: "C Bergström" <cbergstrom at pathscale.com>
> >> To: "Hal Finkel" <hfinkel at anl.gov>
> >> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev"
> >> <cfe-dev at lists.llvm.org>, "openmp-dev"
> >> <openmp-dev at lists.llvm.org>, "Chandler Carruth"
> >> <chandlerc at gmail.com>, "Carlo Bertolli"
<cbertol at us.ibm.com>,
> >> "Andrey Bokhanko" <andreybokhanko at gmail.com>
> >> Sent: Wednesday, June 1, 2016 6:46:57 AM
> >> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing an
> >> LLVM subproject for parallelism runtime and support
> >> libraries
> >>
> >> On Wed, Jun 1, 2016 at 7:22 PM, Hal Finkel <hfinkel at
anl.gov>
> >> wrote:
> >> >
> >> > ________________________________
> >> >
> >> > From: "C Bergström" <cbergstrom at
pathscale.com>
> >> > To: "Hal Finkel" <hfinkel at anl.gov>
> >> > Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev"
> >> > <cfe-dev at lists.llvm.org>, "openmp-dev"
> >> > <openmp-dev at lists.llvm.org>,
> >> > "Chandler Carruth" <chandlerc at gmail.com>,
"Carlo Bertolli"
> >> > <cbertol at us.ibm.com>, "Andrey Bokhanko"
> >> > <andreybokhanko at gmail.com>
> >> > Sent: Wednesday, June 1, 2016 12:19:19 AM
> >> > Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing
an
> >> > LLVM
> >> > subproject for parallelism runtime and support libraries
> >> >
> >> > The thread has lost focus and cherry picking replies..
> >> >
> >> > To restate things since maybe you missed my points
> >> > ----
> >> > 1) SE is a programming model and needs a home of it's
own.
> >> > Having a
> >> > programming model with it's headers and all other stuff
glued
> >> > into
> >> > a runtime
> >> > project which intends to be universal and PM agnostic
doesn't
> >> > make
> >> > sense.
> >> >
> >> > They'd start in separate subdirectories.
> >> >
> >> >
> >> > 1.1) The more I look, the most it seems SE is just a
step-child
> >> > project and
> >> > stuffing it in llvm while still not having any users or
strong
> >> > backing
> >> > doesn't make sense. We have enough PM already and my gut
feeling
> >> > is
> >> > this
> >> > isn't going in a direction to bring in other
stakeholders.
> >>
> >> Point by point because I don't agree with what you write below
> >> entirely
> >>
> >> >
> >> > I think this is the core of my reply. OpenMP has a strong
user
> >> > community,
> >>
> >> I'd agree here
> >>
> >> > but OpenMP 4 offloading is still young.
> >>
> >> ditto
> >>
> >> > OpenMP 4 offloading does not yet
> >> > have a real user community
> >>
> >> agreement
> >>
> >> > yet because the first implementations just
> >> > started shipping very recently.
> >>
> >> Partially strongly disagree - Maybe you meant on Power8?
> >> Intel has had their support for OMP4 for quite some time. It may
> >> have
> >> been buggy and focused on simd directive, but as best I can tell
> >> they
> >> have worked quite hard to fix bugs and improve it.
> >
> > I meant across many platforms. Intel had preliminary support for
> > OpenMP 4 offloading directives even before OpenMP 4 was
> > standardized (in 2013). This did not include all of what ended up
> > in the standard, and the offloading part of the standard needed to
> > be fixed in a breaking way for OpenMP 4.1/4.5 regardless, so this
> > is certainly the exception rather than the rule.
> >
> >>
> >> (Shameless self promotion) We have had some degree of OMP4
> >> offloading
> >> for like 1.5 years now.. It's mostly targeting the GPU and
x86,
> >> but
> >> also more recently working on Power8/AArch64.
> >>
> >> I'm quite certain these companies all have varying degrees of
OMP4
> >> done: Cray, IBM, PGI
> >
> > Yes, many are working on implementations, and some have shipped.
> > There's a list here: http://openmp.org/wp/openmp-compilers/ - no
> > one here really lists any OpenMP 4 offloading implementations
> > before 2015. PGI does not currently list OpenMP 4 at all (although
> > they've certainly done a lot of work on OpenACC).
> >
> >>
> >> > Furthermore, our implementation is certainly
> >> > quite new, and OpenMP 4 offloading is really quite akin to SE
in
> >> > that
> >> > regard.
> >>
> >> Strongly disagree - Intel has been working with the LLVM community
> >> on
> >> the parsing/sema and other parts of OMP4 for like a year or more..
> >> This has been a long and well vetted process backed back a well
> >> defined standard.
> >
> > Yes, both Intel and IBM have contributed significantly to Clang in
> > this regard.
> >
> >> Compared to SE which is just some thing that popped
> >> up out of nowhere and
> >
> > It popped up in TensorFlow, which itself popped up out of nowhere,
> > but is already a popular open-source project for machine learning.
> >
> >> has a couple people from Google throwing their
> >> weight around.
> >
> > Google is a trusted member of the LLVM community and a major
> > contributor. So I agree with you in the following sense: When
> > Google promises to update the code to follow LLVM's coding
> > conventions and to manage its future development in accordance
> > with our community norms, I believe them. I think they've earned a
> > place in the "trust but verify" category in this regard, and
I
> > think this is a positive thing.
> >
> 
> yes, but not every project that is started is finished and who would
> be the maintainer if the person leaves Google?
Good question, although I'm under the impression that there are multiple
developers.
> I still don't see why
> they can't fork it on github or create a project to let it bake and
> get some users or traction.
I think they could, but... SE has users, even if they're just inside of
Google currently, and some of the relevant code is in a newly-popular
open-source project (TensorFlow), and so I infer a certain amount of experience
in SE's development team. Will my users want to use SE? I have no idea. I do
care about OpenMP's offloading model because I'm certain I'll have
users who care deeply about it within a couple of years. So regarding my
optimistic attitude you describe below, I'm interested here is getting
another set of eyes on the OpenMP offloading implementation, some sharing of
expertise, voicing of alternate viewpoints, etc. -- hopefully, in the end, this
will benefit both projects. That might influence, for the better, future
versions of OpenMP. It might also end up providing a model that better serves
(perhaps though another abstraction) my C++-based users for which OpenMP
currently does not work well. None of this might happen, but it definitely
won't happen unless we push for it. I don't think we get any of these
other benefits it they just stick the code on github and develop it on their
own. If they're willing to do this as part of the LLVM community, I think we
should take advantage of that.
 
> >> > I view them both as experimental projects, and both have
strong
> >> > backing with significant investment, so I expect both to
mature
> >> > over time.
> >>
> >> I'd agree both are experimental and probably need some work.
I'd
> >> strongly disagree the amount of investment of both projects is
> >> equal.
> >> Again - OMP4 may have zero real world users, but there's a lot
of
> >> stubborn people trying to make it work. Compared to SE which the
> >> people involved have yet to even answer basic things - like
what's
> >> the
> >> future of SE and how to contribute to it.
> >
> > They might not know what the future is. They have code that is
> > useful to them, and they feel would be useful to others. There's
> > strong potential for compiler integration in the future, but it is
> > just a library now. They've talked about extending it to cover
> > different backends (OpenCL, etc.), and investigating
> > host-based-task dependencies, etc. My impression is that, in
> > general, they'd like the code's users, present and future, to
> > shape its future. All of this seems perfectly healthy. If it
> > becomes part of LLVM, how to contribute to it will become clear.
> 
> I love your optimistic attitude, but the reality is that successful
> programming models take a significant amount of work. I'm well aware
> that they can handle at least part of that, but so far I'm not sure
> who else will invest in it..
> 
> The industry has : CUDA, OpenCL, OpenMP4, OpenACC, C++AMP, Parallel
> STL(??), some research projects from DOE and dead stuff like HMPP...
> Are all these so broken that they can't be fixed to fill the same
> necessity as SE?
You're right. SE is certainly different from all of these, higher-level than
some and lower-level than others. And, of course, I'm reminded of this:
https://xkcd.com/927/
> 
> How similar is it to the failed AMD Stream Computing (sorry AMD)
> http://www.ele.uri.edu/courses/ele408/StreamGPU.pdf
> 
> Intel's work had to live and grow outside for quite a while - Google
> may have more good karma,
What they have is a potentially-valuable contribution to our project. For me,
this is about the strategic health of parts of the project I care about. I have
reasons (that I have not been subtle about), that I think there are potential
benefits to having both SE and libompoffload together in some sense.

To be clear, I am an optimist. I think that different groups can collaborate, or
at least share technical discussions, and mutually improve their work. It does
not always happen, but we can encourage it by creating an environment in which
it is more likely to occur.

Thanks again,
Hal
> but I don't see this as being fair. Nothing
> stops them from posting the code online somewhere. 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

On Wed, Jun 1, 2016 at 11:38 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
> ----- Original Message -----
>> From: "C Bergström" <cbergstrom at pathscale.com>
>> To: "Hal Finkel" <hfinkel at anl.gov>
>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev" <cfe-dev at lists.llvm.org>, "openmp-dev"
>> <openmp-dev at lists.llvm.org>, "Chandler Carruth"
<chandlerc at gmail.com>, "Carlo Bertolli" <cbertol at
us.ibm.com>,
>> "Andrey Bokhanko" <andreybokhanko at gmail.com>
>> Sent: Wednesday, June 1, 2016 9:42:34 AM
>> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing an LLVM
subproject for parallelism runtime and support
>> libraries
>>
>> On Wed, Jun 1, 2016 at 8:52 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
>> > ----- Original Message -----
>> >> From: "C Bergström" <cbergstrom at
pathscale.com>
>> >> To: "Hal Finkel" <hfinkel at anl.gov>
>> >> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev"
>> >> <cfe-dev at lists.llvm.org>, "openmp-dev"
>> >> <openmp-dev at lists.llvm.org>, "Chandler
Carruth"
>> >> <chandlerc at gmail.com>, "Carlo Bertolli"
<cbertol at us.ibm.com>,
>> >> "Andrey Bokhanko" <andreybokhanko at
gmail.com>
>> >> Sent: Wednesday, June 1, 2016 6:46:57 AM
>> >> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing
an
>> >> LLVM subproject for parallelism runtime and support
>> >> libraries
>> >>
>> >> On Wed, Jun 1, 2016 at 7:22 PM, Hal Finkel <hfinkel at
anl.gov>
>> >> wrote:
>> >> >
>> >> > ________________________________
>> >> >
>> >> > From: "C Bergström" <cbergstrom at
pathscale.com>
>> >> > To: "Hal Finkel" <hfinkel at anl.gov>
>> >> > Cc: "llvm-dev" <llvm-dev at
lists.llvm.org>, "cfe-dev"
>> >> > <cfe-dev at lists.llvm.org>, "openmp-dev"
>> >> > <openmp-dev at lists.llvm.org>,
>> >> > "Chandler Carruth" <chandlerc at
gmail.com>, "Carlo Bertolli"
>> >> > <cbertol at us.ibm.com>, "Andrey
Bokhanko"
>> >> > <andreybokhanko at gmail.com>
>> >> > Sent: Wednesday, June 1, 2016 12:19:19 AM
>> >> > Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC:
Proposing an
>> >> > LLVM
>> >> > subproject for parallelism runtime and support libraries
>> >> >
>> >> > The thread has lost focus and cherry picking replies..
>> >> >
>> >> > To restate things since maybe you missed my points
>> >> > ----
>> >> > 1) SE is a programming model and needs a home of it's
own.
>> >> > Having a
>> >> > programming model with it's headers and all other
stuff glued
>> >> > into
>> >> > a runtime
>> >> > project which intends to be universal and PM agnostic
doesn't
>> >> > make
>> >> > sense.
>> >> >
>> >> > They'd start in separate subdirectories.
>> >> >
>> >> >
>> >> > 1.1) The more I look, the most it seems SE is just a
step-child
>> >> > project and
>> >> > stuffing it in llvm while still not having any users or
strong
>> >> > backing
>> >> > doesn't make sense. We have enough PM already and my
gut feeling
>> >> > is
>> >> > this
>> >> > isn't going in a direction to bring in other
stakeholders.
>> >>
>> >> Point by point because I don't agree with what you write
below
>> >> entirely
>> >>
>> >> >
>> >> > I think this is the core of my reply. OpenMP has a strong
user
>> >> > community,
>> >>
>> >> I'd agree here
>> >>
>> >> > but OpenMP 4 offloading is still young.
>> >>
>> >> ditto
>> >>
>> >> > OpenMP 4 offloading does not yet
>> >> > have a real user community
>> >>
>> >> agreement
>> >>
>> >> > yet because the first implementations just
>> >> > started shipping very recently.
>> >>
>> >> Partially strongly disagree - Maybe you meant on Power8?
>> >> Intel has had their support for OMP4 for quite some time. It
may
>> >> have
>> >> been buggy and focused on simd directive, but as best I can
tell
>> >> they
>> >> have worked quite hard to fix bugs and improve it.
>> >
>> > I meant across many platforms. Intel had preliminary support for
>> > OpenMP 4 offloading directives even before OpenMP 4 was
>> > standardized (in 2013). This did not include all of what ended up
>> > in the standard, and the offloading part of the standard needed to
>> > be fixed in a breaking way for OpenMP 4.1/4.5 regardless, so this
>> > is certainly the exception rather than the rule.
>> >
>> >>
>> >> (Shameless self promotion) We have had some degree of OMP4
>> >> offloading
>> >> for like 1.5 years now.. It's mostly targeting the GPU and
x86,
>> >> but
>> >> also more recently working on Power8/AArch64.
>> >>
>> >> I'm quite certain these companies all have varying degrees
of OMP4
>> >> done: Cray, IBM, PGI
>> >
>> > Yes, many are working on implementations, and some have shipped.
>> > There's a list here: http://openmp.org/wp/openmp-compilers/ -
no
>> > one here really lists any OpenMP 4 offloading implementations
>> > before 2015. PGI does not currently list OpenMP 4 at all (although
>> > they've certainly done a lot of work on OpenACC).
>> >
>> >>
>> >> > Furthermore, our implementation is certainly
>> >> > quite new, and OpenMP 4 offloading is really quite akin
to SE in
>> >> > that
>> >> > regard.
>> >>
>> >> Strongly disagree - Intel has been working with the LLVM
community
>> >> on
>> >> the parsing/sema and other parts of OMP4 for like a year or
more..
>> >> This has been a long and well vetted process backed back a
well
>> >> defined standard.
>> >
>> > Yes, both Intel and IBM have contributed significantly to Clang in
>> > this regard.
>> >
>> >> Compared to SE which is just some thing that popped
>> >> up out of nowhere and
>> >
>> > It popped up in TensorFlow, which itself popped up out of nowhere,
>> > but is already a popular open-source project for machine learning.
>> >
>> >> has a couple people from Google throwing their
>> >> weight around.
>> >
>> > Google is a trusted member of the LLVM community and a major
>> > contributor. So I agree with you in the following sense: When
>> > Google promises to update the code to follow LLVM's coding
>> > conventions and to manage its future development in accordance
>> > with our community norms, I believe them. I think they've
earned a
>> > place in the "trust but verify" category in this regard,
and I
>> > think this is a positive thing.
>> >
>>
>> yes, but not every project that is started is finished and who would
>> be the maintainer if the person leaves Google?
>
> Good question, although I'm under the impression that there are
multiple developers.
>
>> I still don't see why
>> they can't fork it on github or create a project to let it bake and
>> get some users or traction.
>
> I think they could, but... SE has users, even if they're just inside of
Google currently, and some of the relevant code is in a newly-popular
open-source project (TensorFlow), and so I infer a certain amount of experience
in SE's development team. Will my users want to use SE? I have no idea. I do
care about OpenMP's offloading model because I'm certain I'll have
users who care deeply about it within a couple of years. So regarding my
optimistic attitude you describe below, I'm interested here is getting
another set of eyes on the OpenMP offloading implementation, some sharing of
expertise, voicing of alternate viewpoints, etc. -- hopefully, in the end, this
will benefit both projects. That might influence, for the better, future
versions of OpenMP. It might also end up providing a model that better serves
(perhaps though another abstraction) my C++-based users for which OpenMP
currently does not work well. None of this might happen, but it definitely
won't happen unless we push for it. I don't think we get any of these
other benefits it they just stick the code on github and develop it on their
own. If they're willing to do this as part of the LLVM community, I think we
should take advantage of that.
>
>> >> > I view them both as experimental projects, and both have
strong
>> >> > backing with significant investment, so I expect both to
mature
>> >> > over time.
>> >>
>> >> I'd agree both are experimental and probably need some
work. I'd
>> >> strongly disagree the amount of investment of both projects is
>> >> equal.
>> >> Again - OMP4 may have zero real world users, but there's a
lot of
>> >> stubborn people trying to make it work. Compared to SE which
the
>> >> people involved have yet to even answer basic things - like
what's
>> >> the
>> >> future of SE and how to contribute to it.
>> >
>> > They might not know what the future is. They have code that is
>> > useful to them, and they feel would be useful to others.
There's
>> > strong potential for compiler integration in the future, but it is
>> > just a library now. They've talked about extending it to cover
>> > different backends (OpenCL, etc.), and investigating
>> > host-based-task dependencies, etc. My impression is that, in
>> > general, they'd like the code's users, present and future,
to
>> > shape its future. All of this seems perfectly healthy. If it
>> > becomes part of LLVM, how to contribute to it will become clear.
>>
>> I love your optimistic attitude, but the reality is that successful
>> programming models take a significant amount of work. I'm well
aware
>> that they can handle at least part of that, but so far I'm not sure
>> who else will invest in it..
>>
>> The industry has : CUDA, OpenCL, OpenMP4, OpenACC, C++AMP, Parallel
>> STL(??), some research projects from DOE and dead stuff like HMPP...
>> Are all these so broken that they can't be fixed to fill the same
>> necessity as SE?
>
> You're right. SE is certainly different from all of these, higher-level
than some and lower-level than others. And, of course, I'm reminded of this:
https://xkcd.com/927/
>
>>
>> How similar is it to the failed AMD Stream Computing (sorry AMD)
>> http://www.ele.uri.edu/courses/ele408/StreamGPU.pdf
>>
>> Intel's work had to live and grow outside for quite a while -
Google
>> may have more good karma,
>
> What they have is a potentially-valuable contribution to our project. For
me, this is about the strategic health of parts of the project I care about. I
have reasons (that I have not been subtle about), that I think there are
potential benefits to having both SE and libompoffload together in some sense.
>
> To be clear, I am an optimist. I think that different groups can
collaborate, or at least share technical discussions, and mutually improve their
work. It does not always happen, but we can encourage it by creating an
environment in which it is more likely to occur.
Sorry, but the interaction so far doesn't leave me warm and fuzzy.. I
see them wanting to push their code as-is instead of working more
closely with OMP and Intel.

Nothing stops them from reviewing the OpenMP implementation stuff
today. Since you're friends with them - maybe ask them to help out
there. If they had a strong interest in this area, I suspect that
would have already happened.
>From what you're saying above.. it seems like SE won't succeed
unless
it becomes a project? If that's the case then maybe it should die...
You have to be _____ this tall to go on the ride..

All the positive interactions and feedback shouldn't be exclusive with
SE being a project. Tying them together doesn't seem like the right
approach.

It seems like "I want it MY way or I won't play"...

How about we revisit this in a month? If in 1 month other codes are
using SE we can discuss the benefits...

On Wed, Jun 1, 2016 at 8:21 AM Mehdi Amini via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
>
> > On Jun 1, 2016, at 7:42 AM, C Bergström via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > On Wed, Jun 1, 2016 at 8:52 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
> >> ----- Original Message -----
> >>> From: "C Bergström" <cbergstrom at
pathscale.com>
> >>> To: "Hal Finkel" <hfinkel at anl.gov>
> >>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>,
"cfe-dev" <
> cfe-dev at lists.llvm.org>, "openmp-dev"
> >>> <openmp-dev at lists.llvm.org>, "Chandler
Carruth" <chandlerc at gmail.com>,
> "Carlo Bertolli" <cbertol at us.ibm.com>,
> >>> "Andrey Bokhanko" <andreybokhanko at
gmail.com>
> >>> Sent: Wednesday, June 1, 2016 6:46:57 AM
> >>> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC: Proposing
an LLVM
> subproject for parallelism runtime and support
> >>> libraries
> >>>
> >>> On Wed, Jun 1, 2016 at 7:22 PM, Hal Finkel <hfinkel at
anl.gov> wrote:
> >>>>
> >>>> ________________________________
> >>>>
> >>>> From: "C Bergström" <cbergstrom at
pathscale.com>
> >>>> To: "Hal Finkel" <hfinkel at anl.gov>
> >>>> Cc: "llvm-dev" <llvm-dev at
lists.llvm.org>, "cfe-dev"
> >>>> <cfe-dev at lists.llvm.org>, "openmp-dev"
<openmp-dev at lists.llvm.org>,
> >>>> "Chandler Carruth" <chandlerc at
gmail.com>, "Carlo Bertolli"
> >>>> <cbertol at us.ibm.com>, "Andrey Bokhanko"
<andreybokhanko at gmail.com>
> >>>> Sent: Wednesday, June 1, 2016 12:19:19 AM
> >>>> Subject: Re: [Openmp-dev] [llvm-dev] [cfe-dev] RFC:
Proposing an
> >>>> LLVM
> >>>> subproject for parallelism runtime and support libraries
> >>>>
> >>>> The thread has lost focus and cherry picking replies..
> >>>>
> >>>> To restate things since maybe you missed my points
> >>>> ----
> >>>> 1) SE is a programming model and needs a home of it's
own. Having a
> >>>> programming model with it's headers and all other
stuff glued into
> >>>> a runtime
> >>>> project which intends to be universal and PM agnostic
doesn't make
> >>>> sense.
> >>>>
> >>>> They'd start in separate subdirectories.
> >>>>
> >>>>
> >>>> 1.1) The more I look, the most it seems SE is just a
step-child
> >>>> project and
> >>>> stuffing it in llvm while still not having any users or
strong
> >>>> backing
> >>>> doesn't make sense. We have enough PM already and my
gut feeling is
> >>>> this
> >>>> isn't going in a direction to bring in other
stakeholders.
> >>>
> >>> Point by point because I don't agree with what you write
below
> >>> entirely
> >>>
> >>>>
> >>>> I think this is the core of my reply. OpenMP has a strong
user
> >>>> community,
> >>>
> >>> I'd agree here
> >>>
> >>>> but OpenMP 4 offloading is still young.
> >>>
> >>> ditto
> >>>
> >>>> OpenMP 4 offloading does not yet
> >>>> have a real user community
> >>>
> >>> agreement
> >>>
> >>>> yet because the first implementations just
> >>>> started shipping very recently.
> >>>
> >>> Partially strongly disagree - Maybe you meant on Power8?
> >>> Intel has had their support for OMP4 for quite some time. It
may have
> >>> been buggy and focused on simd directive, but as best I can
tell they
> >>> have worked quite hard to fix bugs and improve it.
> >>
> >> I meant across many platforms. Intel had preliminary support for
OpenMP
> 4 offloading directives even before OpenMP 4 was standardized (in 2013).
> This did not include all of what ended up in the standard, and the
> offloading part of the standard needed to be fixed in a breaking way for
> OpenMP 4.1/4.5 regardless, so this is certainly the exception rather than
> the rule.
> >>
> >>>
> >>> (Shameless self promotion) We have had some degree of OMP4
offloading
> >>> for like 1.5 years now.. It's mostly targeting the GPU and
x86, but
> >>> also more recently working on Power8/AArch64.
> >>>
> >>> I'm quite certain these companies all have varying degrees
of OMP4
> >>> done: Cray, IBM, PGI
> >>
> >> Yes, many are working on implementations, and some have shipped.
> There's a list here: http://openmp.org/wp/openmp-compilers/ - no one
here
> really lists any OpenMP 4 offloading implementations before 2015. PGI does
> not currently list OpenMP 4 at all (although they've certainly done a
lot
> of work on OpenACC).
> >>
> >>>
> >>>> Furthermore, our implementation is certainly
> >>>> quite new, and OpenMP 4 offloading is really quite akin to
SE in
> >>>> that
> >>>> regard.
> >>>
> >>> Strongly disagree - Intel has been working with the LLVM
community on
> >>> the parsing/sema and other parts of OMP4 for like a year or
more..
> >>> This has been a long and well vetted process backed back a
well
> >>> defined standard.
> >>
> >> Yes, both Intel and IBM have contributed significantly to Clang in
this
> regard.
> >>
> >>> Compared to SE which is just some thing that popped
> >>> up out of nowhere and
> >>
> >> It popped up in TensorFlow, which itself popped up out of nowhere,
but
> is already a popular open-source project for machine learning.
> >>
> >>> has a couple people from Google throwing their
> >>> weight around.
> >>
> >> Google is a trusted member of the LLVM community and a major
> contributor. So I agree with you in the following sense: When Google
> promises to update the code to follow LLVM's coding conventions and to
> manage its future development in accordance with our community norms, I
> believe them. I think they've earned a place in the "trust but
verify"
> category in this regard, and I think this is a positive thing.
> >>
> >
> > yes, but not every project that is started is finished and who would
> > be the maintainer if the person leaves Google?
>
> Like anything that bitrot and is no longer maintained in LLVM: it gets
> deleted (or the sources stay there but it is no longer updated/maintained).
> There are precedents: http://llvm.org/svn/llvm-project/

Strong +1.

This is no different than any of other significant contributions, by Google
or other significant contributor -- we're on the hook to maintain it. If we
stop doing that, it gets nuked.

>
>
>
> > I still don't see why
> > they can't fork it on github or create a project to let it bake
and
> > get some users or traction.
>
> The intent may be that instead of creating a separate community of users,
> it'd create its community inside the llvm projects community. That
looks
> like a nice "feature" long term.
>
> Cost/benefit: cost is not really existent, potential benefits are
> interesting.
>
Agreed. That is exactly why we were interested in contributing it to LLVM.
We think it makes both LLVM and SE stronger from a community perspective to
have the diversity here.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160602/7221d6d6/attachment.html>