thr3ads.net - llvm dev - [llvm-dev] SPIRV-LLVM as an external tool [Feb 2018]

If this information is useful, please help other people find it:
Share via:

Pierre Moreau via llvm-dev

2018-Feb-21 23:34 UTC

[llvm-dev] SPIRV-LLVM as an external tool

On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote:> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote:
> > Hi,
> > 
> > for a few months already I have been asking around for opinions on how
> > people could best work together on Khronos' SPIR-V <->
LLVM-IR converter
> > and some consensus seems to have formed.
> > 
> > Most of the people I talked to favored having the converter become an
> > external tool within the LLVM project, similar to libclc. I think that
> > the LLVM project's processes, infrastructure and community form
the best
> > place for this collaboration to happen.
> > 
> > I hope that having the converter as part of LLVM can help expand
LLVM's
> > value proposition in heterogeneous computing, in part by complementing
> > the OpenCL C backend in clang.
> > 
> > Thus I would like to ask what the LLVM community needs to see from us
> > before SPIRV-LLVM can be accepted as an external tool, if at all.
> > 
> 
> Does the external tool exist already?  I recommend starting by posting
> the code publicly and then ensuring that going forward all development
> and discussions about the tool are happening in a public space.
The code is publicly available at https://github.com/pierremoreau/llvm-spirv.
It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and
https://github.com/thewilsonator/llvm-target-spirv/, stripped off of everything
not related to SPIR-V and updated to work with llvm HEAD.
(Some of the commits were lost on the way, so we are in the process of redoing
the stripping and updating.)

One user of that external tool will be Mesa [1] (for its OpenCL implementation
named clover) for
1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by
   translating SPIR-V to LLVM IR; (patches under review [2])
2. allowing additional drivers (nouveau, freedreno, etnativ) to be supported by
   clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then translating
   from SPIR-V to NIR).

Regards,
Pierre

[1]: https://www.mesa3d.org/
[2]: https://patchwork.freedesktop.org/series/38728/
> It's important to demonstrate that there is a developer community
backing
> the tool and that it won't be abandoned if it is added as an llvm
> sub-project.
> 
> -Tom
> 
> > Thanks,
> > 
> > Tomeu
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180222/691a2847/attachment.sig>

Anastasia Stulova via llvm-dev

2018-Feb-22 18:38 UTC

head link

[llvm-dev] SPIRV-LLVM as an external tool

> It's important to demonstrate that there is a developer community
backing
> the tool and that it won't be abandoned if it is added as an llvm
> sub-project.
>
> -Tom

Just to complete the picture, this is an initiative from within Khronos Group

to unify our effort to make this available as an LLVM component. A number of

companies have been involved in the original development of this converter and

there are more that have adopted this design in their propriety or open source

toolchains. Mesa and AMD Vulkan driver
(https://github.com/GPUOpen-Drivers/AMDVLK)

are just some examples of the open source ones. There were a number of threads

regarding putting this work upstream in the past years. And due to several

conceptual differences it, unfortunately, took us a while to get an agreement

on the best integration approach. During this time a number of forks were
created

that rebased this work to the top of trunk and also committed several
improvements.

Some of them are just duplicated in various forks. It would help us if we
finally

have it integrated with LLVM so we can keep it up to date and contribute common

changes. So there are a number of companies that want to contribute to this

currently.


@Tom, if you have any valuable experience to share with us on how to get this

accepted/integrated as a tool, it would be very helpful.

Thanks,
Anastasia

________________________________
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Pierre
Moreau via llvm-dev <llvm-dev at lists.llvm.org>
Sent: 21 February 2018 23:34
To: tstellar at redhat.com
Cc: LLVM Developers; Tomeu Vizoso
Subject: Re: [llvm-dev] SPIRV-LLVM as an external tool

On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote:> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote:
> > Hi,
> >
> > for a few months already I have been asking around for opinions on how
> > people could best work together on Khronos' SPIR-V <->
LLVM-IR converter
> > and some consensus seems to have formed.
> >
> > Most of the people I talked to favored having the converter become an
> > external tool within the LLVM project, similar to libclc. I think that
> > the LLVM project's processes, infrastructure and community form
the best
> > place for this collaboration to happen.
> >
> > I hope that having the converter as part of LLVM can help expand
LLVM's
> > value proposition in heterogeneous computing, in part by complementing
> > the OpenCL C backend in clang.
> >
> > Thus I would like to ask what the LLVM community needs to see from us
> > before SPIRV-LLVM can be accepted as an external tool, if at all.
> >
>
> Does the external tool exist already?  I recommend starting by posting
> the code publicly and then ensuring that going forward all development
> and discussions about the tool are happening in a public space.
The code is publicly available at https://github.com/pierremoreau/llvm-spirv.
It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and
https://github.com/thewilsonator/llvm-target-spirv/, stripped off of everything
not related to SPIR-V and updated to work with llvm HEAD.
(Some of the commits were lost on the way, so we are in the process of redoing
the stripping and updating.)

One user of that external tool will be Mesa [1] (for its OpenCL implementation
named clover) for
1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by
   translating SPIR-V to LLVM IR; (patches under review [2])
2. allowing additional drivers (nouveau, freedreno, etnativ) to be supported by
   clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then translating
   from SPIR-V to NIR).

Regards,
Pierre

[1]: https://www.mesa3d.org/
[2]: https://patchwork.freedesktop.org/series/38728/
> It's important to demonstrate that there is a developer community
backing
> the tool and that it won't be abandoned if it is added as an llvm
> sub-project.
>
> -Tom
>
> > Thanks,
> >
> > Tomeu
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180222/f7992304/attachment.html>

Tom Stellard via llvm-dev

2018-Feb-23 20:23 UTC

head link

[llvm-dev] SPIRV-LLVM as an external tool

On 02/22/2018 10:38 AM, Anastasia Stulova wrote:> 
>> It's important to demonstrate that there is a developer community
backing
>> the tool and that it won't be abandoned if it is added as an llvm
>> sub-project.
>> 
>> -Tom
> 
> 
> Just to complete the picture, this is an initiative from within Khronos
Group
> 
> to unify our effort to make this available as an LLVM component. A number
of
> 
> companies have been involved in the original development of this converter
and
> 
> there are more that have adopted this design in their propriety or open
source
> 
> toolchains. Mesa and AMD Vulkan driver
(https://github.com/GPUOpen-Drivers/AMDVLK)
> 
> are just some examples of the open source ones. There were a number of
threads
> 
> regarding putting this work upstream in the past years. And due to several
> 
> conceptual differences it, unfortunately, took us a while to get an
agreement
> 
> on the best integration approach. During this time a number of forks were
created
> 
> that rebased this work to the top of trunk and also committed several
improvements.
> 
> Some of them are just duplicated in various forks. It would help us if we
finally
> 
> have it integrated with LLVM so we can keep it up to date and contribute
common
> 
> changes. So there are a number of companies that want to contribute to this
> 
> currently.
> 
> 
> @Tom, if you have any valuable experience to share with us on how to get
this
> 
> accepted/integrated as a tool, it would be very helpful.
> 
> 
It's pretty much what I mentioned before.  Put the code for the tool
somewhere
public and start using that as the focal point for all development.  Once that
is done then you can come back to the community to start a discussion about
adding
it as a sub-project.  It's much easier to have a discussion when there is
actual
code to talk about.

-Tom

 
> Thanks,
> Anastasia
> 
>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of
Pierre Moreau via llvm-dev <llvm-dev at lists.llvm.org>
> *Sent:* 21 February 2018 23:34
> *To:* tstellar at redhat.com
> *Cc:* LLVM Developers; Tomeu Vizoso
> *Subject:* Re: [llvm-dev] SPIRV-LLVM as an external tool
>  
> On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote:
>> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote:
>> > Hi,
>> > 
>> > for a few months already I have been asking around for opinions on
how
>> > people could best work together on Khronos' SPIR-V <->
LLVM-IR converter
>> > and some consensus seems to have formed.
>> > 
>> > Most of the people I talked to favored having the converter become
an
>> > external tool within the LLVM project, similar to libclc. I think
that
>> > the LLVM project's processes, infrastructure and community
form the best
>> > place for this collaboration to happen.
>> > 
>> > I hope that having the converter as part of LLVM can help expand
LLVM's
>> > value proposition in heterogeneous computing, in part by
complementing
>> > the OpenCL C backend in clang.
>> > 
>> > Thus I would like to ask what the LLVM community needs to see from
us
>> > before SPIRV-LLVM can be accepted as an external tool, if at all.
>> > 
>> 
>> Does the external tool exist already?  I recommend starting by posting
>> the code publicly and then ensuring that going forward all development
>> and discussions about the tool are happening in a public space.
> 
> The code is publicly available at
https://github.com/pierremoreau/llvm-spirv.
> It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and
> https://github.com/thewilsonator/llvm-target-spirv/, stripped off of
everything
> not related to SPIR-V and updated to work with llvm HEAD.
> (Some of the commits were lost on the way, so we are in the process of
redoing
> the stripping and updating.)
> 
> One user of that external tool will be Mesa [1] (for its OpenCL
implementation
> named clover) for
> 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by
>    translating SPIR-V to LLVM IR; (patches under review [2])
> 2. allowing additional drivers (nouveau, freedreno, etnativ) to be
supported by
>    clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then
translating
>    from SPIR-V to NIR).
> 
> Regards,
> Pierre
> 
> [1]: https://www.mesa3d.org/
> [2]: https://patchwork.freedesktop.org/series/38728/
> 
>> It's important to demonstrate that there is a developer community
backing
>> the tool and that it won't be abandoned if it is added as an llvm
>> sub-project.
>> 
>> -Tom
>> 
>> > Thanks,
>> > 
>> > Tomeu
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > llvm-dev at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Tom Stellard via llvm-dev

2018-Feb-23 20:28 UTC

head link

[llvm-dev] SPIRV-LLVM as an external tool

On 02/21/2018 03:34 PM, Pierre Moreau wrote:> On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote:
>> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote:
>>> Hi,
>>>
>>> for a few months already I have been asking around for opinions on
how
>>> people could best work together on Khronos' SPIR-V <->
LLVM-IR converter
>>> and some consensus seems to have formed.
>>>
>>> Most of the people I talked to favored having the converter become
an
>>> external tool within the LLVM project, similar to libclc. I think
that
>>> the LLVM project's processes, infrastructure and community form
the best
>>> place for this collaboration to happen.
>>>
>>> I hope that having the converter as part of LLVM can help expand
LLVM's
>>> value proposition in heterogeneous computing, in part by
complementing
>>> the OpenCL C backend in clang.
>>>
>>> Thus I would like to ask what the LLVM community needs to see from
us
>>> before SPIRV-LLVM can be accepted as an external tool, if at all.
>>>
>>
>> Does the external tool exist already?  I recommend starting by posting
>> the code publicly and then ensuring that going forward all development
>> and discussions about the tool are happening in a public space.
> 
> The code is publicly available at
https://github.com/pierremoreau/llvm-spirv.
> It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and
> https://github.com/thewilsonator/llvm-target-spirv/, stripped off of
everything
> not related to SPIR-V and updated to work with llvm HEAD.
> (Some of the commits were lost on the way, so we are in the process of
redoing
> the stripping and updating.)
This is great to see.  Is this code the basis of the forks that Anastasia
talked about or did those come from somewhere else?

-Tom
> 
> One user of that external tool will be Mesa [1] (for its OpenCL
implementation
> named clover) for
> 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by
>    translating SPIR-V to LLVM IR; (patches under review [2])
> 2. allowing additional drivers (nouveau, freedreno, etnativ) to be
supported by
>    clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then
translating
>    from SPIR-V to NIR).
> 
> Regards,
> Pierre
> 
> [1]: https://www.mesa3d.org/
> [2]: https://patchwork.freedesktop.org/series/38728/
> 
>> It's important to demonstrate that there is a developer community
backing
>> the tool and that it won't be abandoned if it is added as an llvm
>> sub-project.
>>
>> -Tom
>>
>>> Thanks,
>>>
>>> Tomeu
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Anastasia Stulova via llvm-dev

2018-Feb-26 17:25 UTC

head link

[llvm-dev] SPIRV-LLVM as an external tool

> This is great to see.  Is this code the basis of the forks that Anastasiatalked about or did those come from somewhere else?

Yes, indeed the base is https://github.com/KhronosGroup/SPIRV-LLVM/ and then
there are multiple forks that include some rework as well (some of which were
announced on the LLVM channels). I think the biggest problems we are trying to
solve is:

1. Keeping up to date with the LLVM code base development.

2. Providing unified common place for everyone to contribute their changes.

Now there are two extra goals that are more related to Clang:

3. Missing targets that can be used as an end binary (except for AMD GPU).
Addind SPIRV to LLVM would allow Clang open source users to generate this
portable binary format without going to any other toolchains that just integrate
Clang. This is also a reason why we keep maintaining Clang in all the
toolchains. Hopefully, people will start improving LLVM for GPU targets upstream
because they will be able to use SPIRV available directly in LLVM to demonstate
their issues/improvements.

4. There is a plan for adding OpenCL C++ support that mandates generation of
SPIRV. I started a thread on this last week:
http://lists.llvm.org/pipermail/cfe-dev/2018-February/056972.html

So if for the goals (1) and (2) above, we could setup separate repository
outside of LLVM and try to spread the information about it well enough that
everyone starts using it. This won't help us with item (3) at all, and item
(4) will be incomplete.

Cheers,
Anastasia



________________________________
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Tom
Stellard via llvm-dev <llvm-dev at lists.llvm.org>
Sent: 23 February 2018 20:28
To: Tomeu Vizoso; LLVM Developers
Subject: Re: [llvm-dev] SPIRV-LLVM as an external tool

On 02/21/2018 03:34 PM, Pierre Moreau wrote:> On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote:
>> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote:
>>> Hi,
>>>
>>> for a few months already I have been asking around for opinions on
how
>>> people could best work together on Khronos' SPIR-V <->
LLVM-IR converter
>>> and some consensus seems to have formed.
>>>
>>> Most of the people I talked to favored having the converter become
an
>>> external tool within the LLVM project, similar to libclc. I think
that
>>> the LLVM project's processes, infrastructure and community form
the best
>>> place for this collaboration to happen.
>>>
>>> I hope that having the converter as part of LLVM can help expand
LLVM's
>>> value proposition in heterogeneous computing, in part by
complementing
>>> the OpenCL C backend in clang.
>>>
>>> Thus I would like to ask what the LLVM community needs to see from
us
>>> before SPIRV-LLVM can be accepted as an external tool, if at all.
>>>
>>
>> Does the external tool exist already?  I recommend starting by posting
>> the code publicly and then ensuring that going forward all development
>> and discussions about the tool are happening in a public space.
>
> The code is publicly available at
https://github.com/pierremoreau/llvm-spirv.
> It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and
> https://github.com/thewilsonator/llvm-target-spirv/, stripped off of
everything
> not related to SPIR-V and updated to work with llvm HEAD.
> (Some of the commits were lost on the way, so we are in the process of
redoing
> the stripping and updating.)
This is great to see.  Is this code the basis of the forks that Anastasia
talked about or did those come from somewhere else?

-Tom
>
> One user of that external tool will be Mesa [1] (for its OpenCL
implementation
> named clover) for
> 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by
>    translating SPIR-V to LLVM IR; (patches under review [2])
> 2. allowing additional drivers (nouveau, freedreno, etnativ) to be
supported by
>    clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then
translating
>    from SPIR-V to NIR).
>
> Regards,
> Pierre
>
> [1]: https://www.mesa3d.org/
> [2]: https://patchwork.freedesktop.org/series/38728/
>
>> It's important to demonstrate that there is a developer community
backing
>> the tool and that it won't be abandoned if it is added as an llvm
>> sub-project.
>>
>> -Tom
>>
>>> Thanks,
>>>
>>> Tomeu
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180226/e254f2b9/attachment.html>

Seemingly Similar Threads

Search for more seemingly similar threads

llvm dev - Feb 2018 - SPIRV-LLVM as an external tool

[llvm-dev] SPIRV-LLVM as an external tool

[llvm-dev] SPIRV-LLVM as an external tool

[llvm-dev] SPIRV-LLVM as an external tool

[llvm-dev] SPIRV-LLVM as an external tool

[llvm-dev] SPIRV-LLVM as an external tool

Seemingly Similar Threads