llvm dev - Apr 2016 - [Openmp-dev] [cfe-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries

Hi Jason,

The long discussion made me wondering where this was going, but re-reading you
original email [0], there was an acknowledgment of a potential future merge
between the projects, and I can of make sense of the current picture.
So you can forget about my question below!

[0]: http://lists.llvm.org/pipermail/llvm-dev/2016-March/096576.html
<http://lists.llvm.org/pipermail/llvm-dev/2016-March/096576.html>

Best,

--
Mehdi


> On Mar 28, 2016, at 10:12 AM, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi Jason,
> 
> This is probably because I'm not aware of the details, but it was
claimed in this thread that liboffload can target Xeon Phi and Nvidia GPUs.
Adding a new library that the compiler has to be aware of has to bring
significant benefit.
> So it is not clear to me yet why the compiler should target two different
runtime libraries that seems to have large chunk of overlapping functionalities.
> On a high-level view, these libraries seems to have the same goals with
respect to what they provide to the compiler.
> 
> Can you elaborate?
> 
> Thanks,
> 
> -- 
> Mehdi
> 
> 
>> On Mar 28, 2016, at 9:37 AM, Jason Henline via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> I did a more thorough read through liboffload and wrote up a more
detailed doc describing how StreamExecutor platforms relate to libomptarget RTL
interfaces. The doc also describes why the lack of support for streams in
libomptarget makes it impossible to implement some of the most important
StreamExecutor platforms in terms of libomptarget
(https://github.com/henline/streamexecutordoc/blob/master/se_and_openmp.rst
<https://github.com/henline/streamexecutordoc/blob/master/se_and_openmp.rst>).
When I was originally optimistic about using liboffload to implement
StreamExecutor platforms, I was not aware of this issue with streams. Thanks to
Carlo Bertolli for bringing this to my attention.
>> 
>> After having looked in detail at the liboffload code, it sounds like
the best thing to do at this point is to keep StreamExecutor and liboffload
separate, but to leave the door open to implement future StreamExecutor
platforms in terms of liboffload. From the recent messages on this subject from
Carlo and Andrey it seems like there is a general consensus on this, so I would
like to move forward with the StreamExecutor project in this spirit.
>> 
>> On Tue, Mar 15, 2016 at 5:09 PM Jason Henline <jhen at google.com
<mailto:jhen at google.com>> wrote:
>> I created a GitHub repo that contains the documentation I have been
creating for StreamExecutor. https://github.com/henline/streamexecutordoc
<https://github.com/henline/streamexecutordoc>
>> 
>> It contains the design docs from the original email in this thread, and
it contains a new doc I just made that gives a more detailed sketch of the
StreamExecutor platform plugin interface. This shows which methods must be
implemented to support a new platform in StreamExecutor, or to provide a new
implementation for an existing platform (e.g. using liboffload to implement the
CUDA platform).
>> 
>> I wrote up this doc in response to a lot of good questions I am getting
about the details of how StreamExecutor might work with the code OpenMP already
has in place.
>> 
>> Best Regards,
>> -Jason
>> 
>> On Tue, Mar 15, 2016 at 12:28 PM Andrey Bokhanko <andreybokhanko at
gmail.com <mailto:andreybokhanko at gmail.com>> wrote:
>> Hola Chandler,
>> 
>> On Tue, Mar 15, 2016 at 1:44 PM, Chandler Carruth via Openmp-dev
<openmp-dev at lists.llvm.org <mailto:openmp-dev at lists.llvm.org>>
wrote:
>> It seems like if the OpenMP folks want to add a liboffload plugin to
StreamExecutor, that would be an awesome additional platform, but I don't
see why we need to force the coupling here.
>> 
>> 
>> Let me give you a reason: while user-facing sides of StreamExecutor and
OpenMP are quite different (and each warrants its place under the sun!),
internal SE's offloading interface and liboffload are doing exactly the same
thing. Why we want to duplicate code? As previous replies demonstrated, SE
can't serve OpenMP's needs, while liboffload API seems to be general
enough to serve SE well (though this has to be verified, of course -- as I
understand, Jason is going to do this).
>> 
>> Sure, there is no "must have need" to couple SE and
liboffload, but this sounds like a solid software engineering decision to me.
Or, quoting Jason, who said this much better than me:
>> 
>> > Although OpenMP and StreamExecutor support different programming
models,
>> > some of the work they perform under the hood will likely be very
similar.
>> > By sharing code and domain expertise, both projects will be
improved and
>> > strengthened as their capabilities are expanded. The
StreamExecutor
>> > community looks forward to much collaboration and discussion with
OpenMP
>> > about the best places and ways to cooperate.
>> 
>> Espere veure't demà!
>> 
>> Yours,
>> Andrey
>> ====>> Enginyer de Software
>> Intel Compiler Team
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160328/a16abf00/attachment.html>

I feel like this thread got a bit stalled. I'd like to pick it up and try
to suggest a path forward.

I don't hear any real objections to the overall idea of having an LLVM
subproject for parallelism runtimes and support libraries. I think we
should get that created.

I don't actually see any real objections to StreamExecutor being one of the
runtimes. There are some interesting questions however:
- Is there common code in the OpenMP runtime that could be unified with
this?
- Could OpenMP end up using SE or some common shared library between them
as a basis for offloading?
- Would instead it make more sense to have the OpenMP offload library be a
plugin for StreamExecutor?

I don't know the answer to any of these really, but I also don't think
that
they should prevent us from making progress here. And I think if anything,
they'll become easier to answer if we do.

So my suggestion would be:
1) Create the broader scoped LLVM subproject, including writing up its
charter, scope, plans, etc.

2) Add stream executor to it

3) Initially, leave the OpenMP offloading stuff targeted at OpenMP. Then,
as it evolves, consider moving it to be another runtime in the broad
project if and when it makes sense.

4) As both OpenMP and SE evolve and are used some in the project, evaluate
whether there is a common core that makes sense to extract. If so, do it
and rebase them appropriately.


Does this make sense? Are there objections to moving forward here?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160422/f66f69ae/attachment.html>

> On Apr 22, 2016, at 3:01 PM, Chandler Carruth <chandlerc at
gmail.com> wrote:
> 
> I feel like this thread got a bit stalled. I'd like to pick it up and
try to suggest a path forward.
> 
> I don't hear any real objections to the overall idea of having an LLVM
subproject for parallelism runtimes and support libraries. I think we should get
that created.
I think it should be clarified if "parallelism runtimes and support
libraries" are intended to expose user-level APIs or if these are intended
to expose APIs for the compiler generated code (this may be part of your point
about "writing up its charter, scope" but I also think it
shouldn't be underestimated as a task so I called it out).

Otherwise you plan sounds good to me.

-- 
Mehdi


> 
> I don't actually see any real objections to StreamExecutor being one of
the runtimes. There are some interesting questions however:
> - Is there common code in the OpenMP runtime that could be unified with
this?
> - Could OpenMP end up using SE or some common shared library between them
as a basis for offloading?
> - Would instead it make more sense to have the OpenMP offload library be a
plugin for StreamExecutor?
> 
> I don't know the answer to any of these really, but I also don't
think that they should prevent us from making progress here. And I think if
anything, they'll become easier to answer if we do.
> 
> So my suggestion would be:
> 1) Create the broader scoped LLVM subproject, including writing up its
charter, scope, plans, etc.
> 
> 2) Add stream executor to it
> 
> 3) Initially, leave the OpenMP offloading stuff targeted at OpenMP. Then,
as it evolves, consider moving it to be another runtime in the broad project if
and when it makes sense.
> 
> 4) As both OpenMP and SE evolve and are used some in the project, evaluate
whether there is a common core that makes sense to extract. If so, do it and
rebase them appropriately.
> 
> 
> Does this make sense? Are there objections to moving forward here?

Hi Chandler

About your points:
> 3) Initially, leave the OpenMP offloading stuff targeted at OpenMP. Then,
as it evolves, consider moving it to be another runtime in the broad
project if and when it makes sense.
> 4) As both OpenMP and SE evolve and are used some in the project,
evaluate whether there is a common core that makes sense to extract. If so,
do it and rebase them appropriately.

I agree with you.

I had some discussion within my group and there seems to be some agreement
that OpenMP offloading, if properly extended, could be used as a plugin for
SE. As time progresses and we learn more about SE we will be able to
re-evaluate this.

Thanks

-- Carlo



From:	Chandler Carruth via cfe-dev <cfe-dev at lists.llvm.org>
To:	Mehdi Amini <mehdi.amini at apple.com>, Jason Henline
            <jhen at google.com>, Hal Finkel <hfinkel at anl.gov>
Cc:	llvm-dev <llvm-dev at lists.llvm.org>, cfe-dev
            <cfe-dev at lists.llvm.org>, "openmp-dev at
lists.llvm.org"
            <openmp-dev at lists.llvm.org>
Date:	04/22/2016 06:02 PM
Subject:	Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM
            subproject for parallelism runtime and support libraries
Sent by:	"cfe-dev" <cfe-dev-bounces at lists.llvm.org>



I feel like this thread got a bit stalled. I'd like to pick it up and try
to suggest a path forward.

I don't hear any real objections to the overall idea of having an LLVM
subproject for parallelism runtimes and support libraries. I think we
should get that created.

I don't actually see any real objections to StreamExecutor being one of the
runtimes. There are some interesting questions however:
- Is there common code in the OpenMP runtime that could be unified with
this?
- Could OpenMP end up using SE or some common shared library between them
as a basis for offloading?
- Would instead it make more sense to have the OpenMP offload library be a
plugin for StreamExecutor?

I don't know the answer to any of these really, but I also don't think
that
they should prevent us from making progress here. And I think if anything,
they'll become easier to answer if we do.

So my suggestion would be:
1) Create the broader scoped LLVM subproject, including writing up its
charter, scope, plans, etc.

2) Add stream executor to it

3) Initially, leave the OpenMP offloading stuff targeted at OpenMP. Then,
as it evolves, consider moving it to be another runtime in the broad
project if and when it makes sense.

4) As both OpenMP and SE evolve and are used some in the project, evaluate
whether there is a common core that makes sense to extract. If so, do it
and rebase them appropriately.


Does this make sense? Are there objections to moving forward here?
_______________________________________________
cfe-dev mailing list
cfe-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160422/12fe234c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160422/12fe234c/attachment.gif>

Please ensure that you have more than just Carlo's feedback from the OMP
side on this. If Intel isn't in agreement then you're leaving out the
most
important stakeholder. (I'd also ask that Hal weigh in since he has quite a
bit of insight into the politics for this)

Pushing forward quickly (steamroller) with something just because the
thread "stalled" doesn't make any sense. Please work with those
who have
experience and expertise.

I'll leave technical feedback for later after rereading the meat of
proposal.

On Sat, Apr 23, 2016 at 9:50 AM, Carlo Bertolli via cfe-dev <
cfe-dev at lists.llvm.org> wrote:
> Hi Chandler
>
> About your points:
>
> > 3) Initially, leave the OpenMP offloading stuff targeted at OpenMP.
> Then, as it evolves, consider moving it to be another runtime in the broad
> project if and when it makes sense.
> > 4) As both OpenMP and SE evolve and are used some in the project,
> evaluate whether there is a common core that makes sense to extract. If so,
> do it and rebase them appropriately.
>
> I agree with you.
>
> I had some discussion within my group and there seems to be some agreement
> that OpenMP offloading, if properly extended, could be used as a plugin for
> SE. As time progresses and we learn more about SE we will be able to
> re-evaluate this.
>
> Thanks
>
> -- Carlo
>
> [image: Inactive hide details for Chandler Carruth via cfe-dev
> ---04/22/2016 06:02:24 PM---I feel like this thread got a bit stalled.
I]Chandler
> Carruth via cfe-dev ---04/22/2016 06:02:24 PM---I feel like this thread got
> a bit stalled. I'd like to pick it up and try to suggest a path
forward.
>
> From: Chandler Carruth via cfe-dev <cfe-dev at lists.llvm.org>
> To: Mehdi Amini <mehdi.amini at apple.com>, Jason Henline <jhen at
google.com>,
> Hal Finkel <hfinkel at anl.gov>
> Cc: llvm-dev <llvm-dev at lists.llvm.org>, cfe-dev <cfe-dev at
lists.llvm.org>,
> "openmp-dev at lists.llvm.org" <openmp-dev at
lists.llvm.org>
> Date: 04/22/2016 06:02 PM
> Subject: Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM
> subproject for parallelism runtime and support libraries
> Sent by: "cfe-dev" <cfe-dev-bounces at lists.llvm.org>
> ------------------------------
>
>
>
> I feel like this thread got a bit stalled. I'd like to pick it up and
try
> to suggest a path forward.
>
> I don't hear any real objections to the overall idea of having an LLVM
> subproject for parallelism runtimes and support libraries. I think we
> should get that created.
>
> I don't actually see any real objections to StreamExecutor being one of
> the runtimes. There are some interesting questions however:
> - Is there common code in the OpenMP runtime that could be unified with
> this?
> - Could OpenMP end up using SE or some common shared library between them
> as a basis for offloading?
> - Would instead it make more sense to have the OpenMP offload library be a
> plugin for StreamExecutor?
>
> I don't know the answer to any of these really, but I also don't
think
> that they should prevent us from making progress here. And I think if
> anything, they'll become easier to answer if we do.
>
> So my suggestion would be:
> 1) Create the broader scoped LLVM subproject, including writing up its
> charter, scope, plans, etc.
>
> 2) Add stream executor to it
>
> 3) Initially, leave the OpenMP offloading stuff targeted at OpenMP. Then,
> as it evolves, consider moving it to be another runtime in the broad
> project if and when it makes sense.
>
> 4) As both OpenMP and SE evolve and are used some in the project, evaluate
> whether there is a common core that makes sense to extract. If so, do it
> and rebase them appropriately.
>
>
> Does this make sense? Are there objections to moving forward here?
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160423/66c1514f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160423/66c1514f/attachment.gif>

No one is pushing forward *without* agreement. I'm trying to unstall the
thread. =D

On Fri, Apr 22, 2016 at 10:21 PM C Bergström <openmp-dev at
lists.llvm.org>
wrote:
>
> Please ensure that you have more than just Carlo's feedback from the
OMP
> side on this. If Intel isn't in agreement then you're leaving out
the most
> important stakeholder. (I'd also ask that Hal weigh in since he has
quite a
> bit of insight into the politics for this)
>
> Pushing forward quickly (steamroller) with something just because the
> thread "stalled" doesn't make any sense. Please work with
those who have
> experience and expertise.
>
> I'll leave technical feedback for later after rereading the meat of
> proposal.
>
> On Sat, Apr 23, 2016 at 9:50 AM, Carlo Bertolli via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> Hi Chandler
>>
>> About your points:
>>
>> > 3) Initially, leave the OpenMP offloading stuff targeted at
OpenMP.
>> Then, as it evolves, consider moving it to be another runtime in the
broad
>> project if and when it makes sense.
>> > 4) As both OpenMP and SE evolve and are used some in the project,
>> evaluate whether there is a common core that makes sense to extract. If
so,
>> do it and rebase them appropriately.
>>
>> I agree with you.
>>
>> I had some discussion within my group and there seems to be some
>> agreement that OpenMP offloading, if properly extended, could be used
as a
>> plugin for SE. As time progresses and we learn more about SE we will be
>> able to re-evaluate this.
>>
>> Thanks
>>
>> -- Carlo
>>
>> [image: Inactive hide details for Chandler Carruth via cfe-dev
>> ---04/22/2016 06:02:24 PM---I feel like this thread got a bit stalled.
I]Chandler
>> Carruth via cfe-dev ---04/22/2016 06:02:24 PM---I feel like this thread
got
>> a bit stalled. I'd like to pick it up and try to suggest a path
forward.
>>
>> From: Chandler Carruth via cfe-dev <cfe-dev at lists.llvm.org>
>> To: Mehdi Amini <mehdi.amini at apple.com>, Jason Henline
<jhen at google.com>,
>> Hal Finkel <hfinkel at anl.gov>
>> Cc: llvm-dev <llvm-dev at lists.llvm.org>, cfe-dev <cfe-dev at
lists.llvm.org>,
>> "openmp-dev at lists.llvm.org" <openmp-dev at
lists.llvm.org>
>> Date: 04/22/2016 06:02 PM
>> Subject: Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM
>> subproject for parallelism runtime and support libraries
>> Sent by: "cfe-dev" <cfe-dev-bounces at lists.llvm.org>
>> ------------------------------
>>
>>
>>
>> I feel like this thread got a bit stalled. I'd like to pick it up
and try
>> to suggest a path forward.
>>
>> I don't hear any real objections to the overall idea of having an
LLVM
>> subproject for parallelism runtimes and support libraries. I think we
>> should get that created.
>>
>> I don't actually see any real objections to StreamExecutor being
one of
>> the runtimes. There are some interesting questions however:
>> - Is there common code in the OpenMP runtime that could be unified with
>> this?
>> - Could OpenMP end up using SE or some common shared library between
them
>> as a basis for offloading?
>> - Would instead it make more sense to have the OpenMP offload library
be
>> a plugin for StreamExecutor?
>>
>> I don't know the answer to any of these really, but I also
don't think
>> that they should prevent us from making progress here. And I think if
>> anything, they'll become easier to answer if we do.
>>
>> So my suggestion would be:
>> 1) Create the broader scoped LLVM subproject, including writing up its
>> charter, scope, plans, etc.
>>
>> 2) Add stream executor to it
>>
>> 3) Initially, leave the OpenMP offloading stuff targeted at OpenMP.
Then,
>> as it evolves, consider moving it to be another runtime in the broad
>> project if and when it makes sense.
>>
>> 4) As both OpenMP and SE evolve and are used some in the project,
>> evaluate whether there is a common core that makes sense to extract. If
so,
>> do it and rebase them appropriately.
>>
>>
>> Does this make sense? Are there objections to moving forward here?
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
> _______________________________________________
> Openmp-dev mailing list
> Openmp-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160423/a4734bef/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160423/a4734bef/attachment.gif>