Pierre Moreau via llvm-dev
2018-Feb-21 23:34 UTC
[llvm-dev] SPIRV-LLVM as an external tool
On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote:> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote: > > Hi, > > > > for a few months already I have been asking around for opinions on how > > people could best work together on Khronos' SPIR-V <-> LLVM-IR converter > > and some consensus seems to have formed. > > > > Most of the people I talked to favored having the converter become an > > external tool within the LLVM project, similar to libclc. I think that > > the LLVM project's processes, infrastructure and community form the best > > place for this collaboration to happen. > > > > I hope that having the converter as part of LLVM can help expand LLVM's > > value proposition in heterogeneous computing, in part by complementing > > the OpenCL C backend in clang. > > > > Thus I would like to ask what the LLVM community needs to see from us > > before SPIRV-LLVM can be accepted as an external tool, if at all. > > > > Does the external tool exist already? I recommend starting by posting > the code publicly and then ensuring that going forward all development > and discussions about the tool are happening in a public space.The code is publicly available at https://github.com/pierremoreau/llvm-spirv. It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and https://github.com/thewilsonator/llvm-target-spirv/, stripped off of everything not related to SPIR-V and updated to work with llvm HEAD. (Some of the commits were lost on the way, so we are in the process of redoing the stripping and updating.) One user of that external tool will be Mesa [1] (for its OpenCL implementation named clover) for 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by translating SPIR-V to LLVM IR; (patches under review [2]) 2. allowing additional drivers (nouveau, freedreno, etnativ) to be supported by clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then translating from SPIR-V to NIR). Regards, Pierre [1]: https://www.mesa3d.org/ [2]: https://patchwork.freedesktop.org/series/38728/> It's important to demonstrate that there is a developer community backing > the tool and that it won't be abandoned if it is added as an llvm > sub-project. > > -Tom > > > Thanks, > > > > Tomeu > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180222/691a2847/attachment.sig>
Anastasia Stulova via llvm-dev
2018-Feb-22 18:38 UTC
[llvm-dev] SPIRV-LLVM as an external tool
> It's important to demonstrate that there is a developer community backing > the tool and that it won't be abandoned if it is added as an llvm > sub-project. > > -TomJust to complete the picture, this is an initiative from within Khronos Group to unify our effort to make this available as an LLVM component. A number of companies have been involved in the original development of this converter and there are more that have adopted this design in their propriety or open source toolchains. Mesa and AMD Vulkan driver (https://github.com/GPUOpen-Drivers/AMDVLK) are just some examples of the open source ones. There were a number of threads regarding putting this work upstream in the past years. And due to several conceptual differences it, unfortunately, took us a while to get an agreement on the best integration approach. During this time a number of forks were created that rebased this work to the top of trunk and also committed several improvements. Some of them are just duplicated in various forks. It would help us if we finally have it integrated with LLVM so we can keep it up to date and contribute common changes. So there are a number of companies that want to contribute to this currently. @Tom, if you have any valuable experience to share with us on how to get this accepted/integrated as a tool, it would be very helpful. Thanks, Anastasia ________________________________ From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Pierre Moreau via llvm-dev <llvm-dev at lists.llvm.org> Sent: 21 February 2018 23:34 To: tstellar at redhat.com Cc: LLVM Developers; Tomeu Vizoso Subject: Re: [llvm-dev] SPIRV-LLVM as an external tool On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote:> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote: > > Hi, > > > > for a few months already I have been asking around for opinions on how > > people could best work together on Khronos' SPIR-V <-> LLVM-IR converter > > and some consensus seems to have formed. > > > > Most of the people I talked to favored having the converter become an > > external tool within the LLVM project, similar to libclc. I think that > > the LLVM project's processes, infrastructure and community form the best > > place for this collaboration to happen. > > > > I hope that having the converter as part of LLVM can help expand LLVM's > > value proposition in heterogeneous computing, in part by complementing > > the OpenCL C backend in clang. > > > > Thus I would like to ask what the LLVM community needs to see from us > > before SPIRV-LLVM can be accepted as an external tool, if at all. > > > > Does the external tool exist already? I recommend starting by posting > the code publicly and then ensuring that going forward all development > and discussions about the tool are happening in a public space.The code is publicly available at https://github.com/pierremoreau/llvm-spirv. It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and https://github.com/thewilsonator/llvm-target-spirv/, stripped off of everything not related to SPIR-V and updated to work with llvm HEAD. (Some of the commits were lost on the way, so we are in the process of redoing the stripping and updating.) One user of that external tool will be Mesa [1] (for its OpenCL implementation named clover) for 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by translating SPIR-V to LLVM IR; (patches under review [2]) 2. allowing additional drivers (nouveau, freedreno, etnativ) to be supported by clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then translating from SPIR-V to NIR). Regards, Pierre [1]: https://www.mesa3d.org/ [2]: https://patchwork.freedesktop.org/series/38728/> It's important to demonstrate that there is a developer community backing > the tool and that it won't be abandoned if it is added as an llvm > sub-project. > > -Tom > > > Thanks, > > > > Tomeu > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180222/f7992304/attachment.html>
On 02/22/2018 10:38 AM, Anastasia Stulova wrote:> >> It's important to demonstrate that there is a developer community backing >> the tool and that it won't be abandoned if it is added as an llvm >> sub-project. >> >> -Tom > > > Just to complete the picture, this is an initiative from within Khronos Group > > to unify our effort to make this available as an LLVM component. A number of > > companies have been involved in the original development of this converter and > > there are more that have adopted this design in their propriety or open source > > toolchains. Mesa and AMD Vulkan driver (https://github.com/GPUOpen-Drivers/AMDVLK) > > are just some examples of the open source ones. There were a number of threads > > regarding putting this work upstream in the past years. And due to several > > conceptual differences it, unfortunately, took us a while to get an agreement > > on the best integration approach. During this time a number of forks were created > > that rebased this work to the top of trunk and also committed several improvements. > > Some of them are just duplicated in various forks. It would help us if we finally > > have it integrated with LLVM so we can keep it up to date and contribute common > > changes. So there are a number of companies that want to contribute to this > > currently. > > > @Tom, if you have any valuable experience to share with us on how to get this > > accepted/integrated as a tool, it would be very helpful. > >It's pretty much what I mentioned before. Put the code for the tool somewhere public and start using that as the focal point for all development. Once that is done then you can come back to the community to start a discussion about adding it as a sub-project. It's much easier to have a discussion when there is actual code to talk about. -Tom> Thanks, > Anastasia > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Pierre Moreau via llvm-dev <llvm-dev at lists.llvm.org> > *Sent:* 21 February 2018 23:34 > *To:* tstellar at redhat.com > *Cc:* LLVM Developers; Tomeu Vizoso > *Subject:* Re: [llvm-dev] SPIRV-LLVM as an external tool > > On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote: >> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote: >> > Hi, >> > >> > for a few months already I have been asking around for opinions on how >> > people could best work together on Khronos' SPIR-V <-> LLVM-IR converter >> > and some consensus seems to have formed. >> > >> > Most of the people I talked to favored having the converter become an >> > external tool within the LLVM project, similar to libclc. I think that >> > the LLVM project's processes, infrastructure and community form the best >> > place for this collaboration to happen. >> > >> > I hope that having the converter as part of LLVM can help expand LLVM's >> > value proposition in heterogeneous computing, in part by complementing >> > the OpenCL C backend in clang. >> > >> > Thus I would like to ask what the LLVM community needs to see from us >> > before SPIRV-LLVM can be accepted as an external tool, if at all. >> > >> >> Does the external tool exist already? I recommend starting by posting >> the code publicly and then ensuring that going forward all development >> and discussions about the tool are happening in a public space. > > The code is publicly available at https://github.com/pierremoreau/llvm-spirv. > It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and > https://github.com/thewilsonator/llvm-target-spirv/, stripped off of everything > not related to SPIR-V and updated to work with llvm HEAD. > (Some of the commits were lost on the way, so we are in the process of redoing > the stripping and updating.) > > One user of that external tool will be Mesa [1] (for its OpenCL implementation > named clover) for > 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by > translating SPIR-V to LLVM IR; (patches under review [2]) > 2. allowing additional drivers (nouveau, freedreno, etnativ) to be supported by > clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then translating > from SPIR-V to NIR). > > Regards, > Pierre > > [1]: https://www.mesa3d.org/ > [2]: https://patchwork.freedesktop.org/series/38728/ > >> It's important to demonstrate that there is a developer community backing >> the tool and that it won't be abandoned if it is added as an llvm >> sub-project. >> >> -Tom >> >> > Thanks, >> > >> > Tomeu >> > _______________________________________________ >> > LLVM Developers mailing list >> > llvm-dev at lists.llvm.org >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
On 02/21/2018 03:34 PM, Pierre Moreau wrote:> On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote: >> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote: >>> Hi, >>> >>> for a few months already I have been asking around for opinions on how >>> people could best work together on Khronos' SPIR-V <-> LLVM-IR converter >>> and some consensus seems to have formed. >>> >>> Most of the people I talked to favored having the converter become an >>> external tool within the LLVM project, similar to libclc. I think that >>> the LLVM project's processes, infrastructure and community form the best >>> place for this collaboration to happen. >>> >>> I hope that having the converter as part of LLVM can help expand LLVM's >>> value proposition in heterogeneous computing, in part by complementing >>> the OpenCL C backend in clang. >>> >>> Thus I would like to ask what the LLVM community needs to see from us >>> before SPIRV-LLVM can be accepted as an external tool, if at all. >>> >> >> Does the external tool exist already? I recommend starting by posting >> the code publicly and then ensuring that going forward all development >> and discussions about the tool are happening in a public space. > > The code is publicly available at https://github.com/pierremoreau/llvm-spirv. > It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and > https://github.com/thewilsonator/llvm-target-spirv/, stripped off of everything > not related to SPIR-V and updated to work with llvm HEAD. > (Some of the commits were lost on the way, so we are in the process of redoing > the stripping and updating.)This is great to see. Is this code the basis of the forks that Anastasia talked about or did those come from somewhere else? -Tom> > One user of that external tool will be Mesa [1] (for its OpenCL implementation > named clover) for > 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by > translating SPIR-V to LLVM IR; (patches under review [2]) > 2. allowing additional drivers (nouveau, freedreno, etnativ) to be supported by > clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then translating > from SPIR-V to NIR). > > Regards, > Pierre > > [1]: https://www.mesa3d.org/ > [2]: https://patchwork.freedesktop.org/series/38728/ > >> It's important to demonstrate that there is a developer community backing >> the tool and that it won't be abandoned if it is added as an llvm >> sub-project. >> >> -Tom >> >>> Thanks, >>> >>> Tomeu >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Anastasia Stulova via llvm-dev
2018-Feb-26 17:25 UTC
[llvm-dev] SPIRV-LLVM as an external tool
> This is great to see. Is this code the basis of the forks that Anastasiatalked about or did those come from somewhere else? Yes, indeed the base is https://github.com/KhronosGroup/SPIRV-LLVM/ and then there are multiple forks that include some rework as well (some of which were announced on the LLVM channels). I think the biggest problems we are trying to solve is: 1. Keeping up to date with the LLVM code base development. 2. Providing unified common place for everyone to contribute their changes. Now there are two extra goals that are more related to Clang: 3. Missing targets that can be used as an end binary (except for AMD GPU). Addind SPIRV to LLVM would allow Clang open source users to generate this portable binary format without going to any other toolchains that just integrate Clang. This is also a reason why we keep maintaining Clang in all the toolchains. Hopefully, people will start improving LLVM for GPU targets upstream because they will be able to use SPIRV available directly in LLVM to demonstate their issues/improvements. 4. There is a plan for adding OpenCL C++ support that mandates generation of SPIRV. I started a thread on this last week: http://lists.llvm.org/pipermail/cfe-dev/2018-February/056972.html So if for the goals (1) and (2) above, we could setup separate repository outside of LLVM and try to spread the information about it well enough that everyone starts using it. This won't help us with item (3) at all, and item (4) will be incomplete. Cheers, Anastasia ________________________________ From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Tom Stellard via llvm-dev <llvm-dev at lists.llvm.org> Sent: 23 February 2018 20:28 To: Tomeu Vizoso; LLVM Developers Subject: Re: [llvm-dev] SPIRV-LLVM as an external tool On 02/21/2018 03:34 PM, Pierre Moreau wrote:> On 2018-02-21 — 14:55, Tom Stellard via llvm-dev wrote: >> On 02/21/2018 12:15 AM, Tomeu Vizoso via llvm-dev wrote: >>> Hi, >>> >>> for a few months already I have been asking around for opinions on how >>> people could best work together on Khronos' SPIR-V <-> LLVM-IR converter >>> and some consensus seems to have formed. >>> >>> Most of the people I talked to favored having the converter become an >>> external tool within the LLVM project, similar to libclc. I think that >>> the LLVM project's processes, infrastructure and community form the best >>> place for this collaboration to happen. >>> >>> I hope that having the converter as part of LLVM can help expand LLVM's >>> value proposition in heterogeneous computing, in part by complementing >>> the OpenCL C backend in clang. >>> >>> Thus I would like to ask what the LLVM community needs to see from us >>> before SPIRV-LLVM can be accepted as an external tool, if at all. >>> >> >> Does the external tool exist already? I recommend starting by posting >> the code publicly and then ensuring that going forward all development >> and discussions about the tool are happening in a public space. > > The code is publicly available at https://github.com/pierremoreau/llvm-spirv. > It is based on https://github.com/KhronosGroup/SPIRV-LLVM/ and > https://github.com/thewilsonator/llvm-target-spirv/, stripped off of everything > not related to SPIR-V and updated to work with llvm HEAD. > (Some of the commits were lost on the way, so we are in the process of redoing > the stripping and updating.)This is great to see. Is this code the basis of the forks that Anastasia talked about or did those come from somewhere else? -Tom> > One user of that external tool will be Mesa [1] (for its OpenCL implementation > named clover) for > 1. enabling clCreateProgramWithIL(KHR)? on AMD cards through clover, by > translating SPIR-V to LLVM IR; (patches under review [2]) > 2. allowing additional drivers (nouveau, freedreno, etnativ) to be supported by > clover, by compiling OpenCL C (via LLVM IR) to SPIR-V (and then translating > from SPIR-V to NIR). > > Regards, > Pierre > > [1]: https://www.mesa3d.org/ > [2]: https://patchwork.freedesktop.org/series/38728/ > >> It's important to demonstrate that there is a developer community backing >> the tool and that it won't be abandoned if it is added as an llvm >> sub-project. >> >> -Tom >> >>> Thanks, >>> >>> Tomeu >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev_______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180226/e254f2b9/attachment.html>