Trifunovic, Konrad via llvm-dev
2021-Mar-02 11:07 UTC
[llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
Hi, A very good question. I was actually expecting it π So, at the moment, it does not integrate into MLIR SPIRV backend and we have not thought about it. I guess You are referring to having a SPV dialect in MLIR and using a 'serialize' option to produce a SPIR-V binary? I agree that developing two backends in parallel is a bit redundant. If SPIR-V LLVM backend becomes a production quality it means actually it could consume any LLVM IR (provided it does conform to some SPIR-V restrictions). By any LLVM IR input I mean: it should be irrelevant whether it is produced by a clang, MLIR to LLVM IR lowering or just some other front-end that produces LLVM IR. The biggest 'impedance mismatch' that I currently see is that SPV MLIR dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets compute. Besides instruction set, the fundamental difference is a memory model. So if we want to unify those, we should actually make SPIR-V LLVM backend able to produce Vulkan dialect of SPIR-V as well. My answer is a bit elusive, but I totally agree with Your proposal: we should work towards having a one solution, and, LLVM SPIR-V backend seems like a more universal one (since it sits lower in the compiler stack). My proposal would be to include some MLIR -> LLVM-IR translated code in the testing so to have this final goal in mind. PS: one more thought: SPIR-V does come with a set of builtin/intrinsic functions that expose the full capabilities of target architecture (mostly GPU). This set of intrinsics is actually a dialect in its own. So this is LLVM IR + SPIR-V specific intrinsics and their semantics that fully define the SPIR-V dialect at LLVM IR level. I believe this idea could be used in MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it a LLVM IR SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the idea of 'SPIR-V dialect' still exists, it is just now expressed at the LLVM IR level. regards, konrad> From: Renato Golin <rengolin at gmail.com> > Sent: Tuesday, March 2, 2021 11:12 AM > To: Trifunovic, Konrad <konrad.trifunovic at intel.com> > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal <michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <andrey1.tretyakov at intel.com> > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend > > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev <mailto:llvm-dev at lists.llvm.org> wrote: > Hi all, > > We would like to propose this RFC for upstreaming a proper SPIR-V backend to LLVM: > > Hi, > > Perhaps a parallel question: how does that integrate with MLIR's SPIRV back-end? > > If this proposal goes through and we have a production-quality SPIRV back-end in LLVM, do we remove MLIR's own version and lower to LLVM, then to SPIRV? Or do we still need the MLIR version? > > In a perfect world, translating to LLVM IR then to SPIRV shouldn't make a difference, but there could be some impedance mismatch between MLIR->LLVM lowering that isn't compatible with SPIRV? > > But as a final goal, if SPIRV becomes an official LLVM target, it would be better if we could iron out the impedance problems and keep only one SPIRV backend. > > cheers, > --renato >
Aleksandr Bezzubikov via llvm-dev
2021-Mar-02 12:15 UTC
[llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
Hi, Perhaps some obvious addition to Konrad's answer - a proper LLVM backend for SPIR-V can make it much easier for people who're already using LLVM for codegen purposes (targeting e.g. AArch64 or x86 CPUs) to simply retarget their flow with one (ideally) option changed in cmdline. Thanks, Alex Π²Ρ, 2 ΠΌΠ°Ρ. 2021 Π³. Π² 14:07, Trifunovic, Konrad via llvm-dev < llvm-dev at lists.llvm.org>:> Hi, > > A very good question. I was actually expecting it π > > So, at the moment, it does not integrate into MLIR SPIRV backend and we > have not thought about it. I guess You are referring to having a SPV > dialect in MLIR and using a 'serialize' option to produce a SPIR-V binary? > > I agree that developing two backends in parallel is a bit redundant. If > SPIR-V LLVM backend becomes a production quality it means actually it could > consume any LLVM IR (provided it does conform to some SPIR-V restrictions). > By any LLVM IR input I mean: it should be irrelevant whether it is > produced by a clang, MLIR to LLVM IR lowering or just some other front-end > that produces LLVM IR. > > The biggest 'impedance mismatch' that I currently see is that SPV MLIR > dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets > compute. Besides instruction set, the fundamental difference is a memory > model. > So if we want to unify those, we should actually make SPIR-V LLVM backend > able to produce Vulkan dialect of SPIR-V as well. > > My answer is a bit elusive, but I totally agree with Your proposal: we > should work towards having a one solution, and, LLVM SPIR-V backend seems > like a more universal one (since it sits lower in the compiler stack). > My proposal would be to include some MLIR -> LLVM-IR translated code in > the testing so to have this final goal in mind. > > PS: one more thought: SPIR-V does come with a set of builtin/intrinsic > functions that expose the full capabilities of target architecture (mostly > GPU). This set of intrinsics is actually a dialect in its own. So this is > LLVM IR + SPIR-V specific intrinsics and their semantics that fully define > the SPIR-V dialect at LLVM IR level. I believe this idea could be used in > MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it a LLVM IR > SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the idea of > 'SPIR-V dialect' still exists, it is just now expressed at the LLVM IR > level. > > regards, > konrad > > > From: Renato Golin <rengolin at gmail.com> > > Sent: Tuesday, March 2, 2021 11:12 AM > > To: Trifunovic, Konrad <konrad.trifunovic at intel.com> > > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal < > michal.paszkowski at intel.com>; Bezzubikov, Aleksandr < > aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 < > andrey1.tretyakov at intel.com> > > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend > > > > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev <mailto: > llvm-dev at lists.llvm.org> wrote: > > Hi all, > > > > We would like to propose this RFC for upstreaming a proper SPIR-V > backend to LLVM: > > > > Hi, > > > > Perhaps a parallel question: how does that integrate with MLIR's SPIRV > back-end? > > > > If this proposal goes through and we have a production-quality SPIRV > back-end in LLVM, do we remove MLIR's own version and lower to LLVM, then > to SPIRV? Or do we still need the MLIR version? > > > > In a perfect world, translating to LLVM IR then to SPIRV shouldn't make > a difference, but there could be some impedance mismatch between MLIR->LLVM > lowering that isn't compatible with SPIRV? > > > > But as a final goal, if SPIRV becomes an official LLVM target, it would > be better if we could iron out the impedance problems and keep only one > SPIRV backend. > > > > cheers, > > --renato > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-- Aleksandr Bezzubikov -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/30a59e40/attachment.html>
Renato Golin via llvm-dev
2021-Mar-02 12:57 UTC
[llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
On Tue, 2 Mar 2021 at 11:07, Trifunovic, Konrad <konrad.trifunovic at intel.com> wrote:> I agree that developing two backends in parallel is a bit redundant. If > SPIR-V LLVM backend becomes a production quality it means actually it could > consume any LLVM IR (provided it does conform to some SPIR-V restrictions). > By any LLVM IR input I mean: it should be irrelevant whether it is > produced by a clang, MLIR to LLVM IR lowering or just some other front-end > that produces LLVM IR. >Excellent. The biggest 'impedance mismatch' that I currently see is that SPV MLIR> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets > compute. Besides instruction set, the fundamental difference is a memory > model. > So if we want to unify those, we should actually make SPIR-V LLVM backend > able to produce Vulkan dialect of SPIR-V as well. >I see. It is unfortunate that we have a shader focused backend in one side and a compute focused in another. As you say, it means we can only move the SPV MLIR lowering to LLVM once the LLVM side also supports it. I'm guessing the support is not a trivial addition to compute and that it will probably take place after the current proposal is mostly done. It is unfortunate, but not altogether bad. I think it would be fine for them to co-exist until the time we unify.> So the idea of 'SPIR-V dialect' still exists, it is just now expressed at > the LLVM IR level. >Indeed, that's what I meant. Thanks! --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/6b11ded4/attachment.html>
Lei Zhang via llvm-dev
2021-Mar-02 19:02 UTC
[llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
Chiming in mostly from the perspective of MLIR SPIR-V support. More comments inlined, but first some general comments. :) As I understand it, SPIR-V is actually a mix of multiple things. It is first and foremost 1) a binary format for encoding GPU executables that cross the toolchain and hardware driver boundaries. Then it's 2) an intermediate level language for expressing such GPU executables. It is also 3) a flexible and extensible spec with all sorts of capability and extension mechanisms in order to support the needs of multiple APIs and hardware features. It's unclear to me what a production-quality SPIR-V LLVM backend would entail; but to actually support various use cases SPIR-V can support (OpenCL, OpenGL, Vulkan; shader/kernel; various levels of extensions; etc.), it looks to me that we need a story for all the above points, where the IR aspect (2) is actually just facet. My understanding over LLVM is it's mostly focusing on 2): we have a very coherent single IR threading through the majority layers of the compiler stack and the IR focuses very much as a means for compiler transformations (i.e., no instruction versioning etc.). There isn't much native modelling for most points for 1) and 3) (which makes sense as LLVM IR is a compiler IR). So to make it work, one would need to shorehore through existing LLVM mechanisms (e.g., using intrinsics for various GPU related builtins, using metadata for SPIR-V decorations?, etc.), unless we want to evolve LLVM infrastructure to have native support for the missing SPIR-V mechanisms, which I think might be too much to take on. This is just general mechanisms, not mentioning the different semantics between different SPIR-V consumers (e.g., shader vs. kernel and what that means over memory/execution model, etc.) that needs to be sorted out too.. Just supporting a certain use case of what SPIR-V supports is certainly simpler though as we can bake in assumptions and avoid some infrastructure needs for the full generality. That's why I think using MLIR as the infrastructure to build general support for SPIR-V is more preferable as we control everything there and can feel free to model all SPIR-V concepts in the most native way. For example we can feel free to define all SPIR-V ops natively, including all ops introduced by SPIR-V extensions and extended instruction sets. We can support versions/extensions/capabilities natively and integrate it with the target environment to automatically filter out <https://github.com/llvm/llvm-project/blob/37eca08e5bcfbe926176412a4a0acf5b963da7e6/mlir/lib/Dialect/SPIRV/Transforms/SPIRVConversion.cpp#L690-L755> CodeGen patterns generating ops not available on the target, etc. To me, MLIR's open dialect/op/type/etc. system is a perfect fit for the open SPIR-V spec with many capabilities/extensions/etc. For example we can even make the SPIR-V dialect itself open to allow out-of-tree extensions and development and such. With that said, I understand that software development has many reality concerns (like existing codebase, familiarity with different components, etc.) and we have many different use cases, which may mean that different paths make sense. So please don't take this as a negative feedback in general. It's just that to me it's unclear how we can unify here right now. Even when the time arrives for unification, I'd believe going through MLIR is better to have general SPIR-V support. :) On Tue, Mar 2, 2021 at 6:07 AM Trifunovic, Konrad via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hi, > > A very good question. I was actually expecting it π > > So, at the moment, it does not integrate into MLIR SPIRV backend and we > have not thought about it. I guess You are referring to having a SPV > dialect in MLIR and using a 'serialize' option to produce a SPIR-V binary? > > I agree that developing two backends in parallel is a bit redundant. If > SPIR-V LLVM backend becomes a production quality it means actually it could > consume any LLVM IR (provided it does conform to some SPIR-V restrictions, > e.g. in Vulkan we have logical adressing mode). > By any LLVM IR input I mean: it should be irrelevant whether it is > produced by a clang, MLIR to LLVM IR lowering or just some other front-end > that produces LLVM IR. >I think we have an assumption here: LLVM itself should support all mechanisms and use cases SPIR-V can support, if to make LLVM a layer before SPIR-V. I think there is a huge gap here.> > The biggest 'impedance mismatch' that I currently see is that SPV MLIR > dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets > compute.To be specific, Vulkan compute is the most well supported use case right now. But there are interests from the community to push on Vulkan graphics and OpenCL.> Besides instruction set, the fundamental difference is a memory model. > So if we want to unify those, we should actually make SPIR-V LLVM backend > able to produce Vulkan dialect of SPIR-V as well. > > My answer is a bit elusive, but I totally agree with Your proposal: we > should work towards having a one solution, and, LLVM SPIR-V backend seems > like a more universal one (since it sits lower in the compiler stack). >I actually believe the opposite, because of the reasons I listed at the very beginning. To me SPIR-V also stays at a higher level than LLVM. (But again, depending on what subset we are talking about.) My proposal would be to include some MLIR -> LLVM-IR translated code in the> testing so to have this final goal in mind. > > PS: one more thought: SPIR-V does come with a set of builtin/intrinsic > functions that expose the full capabilities of target architecture (mostly > GPU). This set of intrinsics is actually a dialect in its own. So this is > LLVM IR + SPIR-V specific intrinsics and their semantics that fully define > the SPIR-V dialect at LLVM IR level. I believe this idea could be used in > MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it a LLVM IR > SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the idea of > 'SPIR-V dialect' still exists, it is just now expressed at the LLVM IR > level.Not sure this is the prefered way, given that we can define SPIR-V ops easily in MLIR in its own dialect with native support for various aspects.> regards, > konrad > > > From: Renato Golin <rengolin at gmail.com> > > Sent: Tuesday, March 2, 2021 11:12 AM > > To: Trifunovic, Konrad <konrad.trifunovic at intel.com> > > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal < > michal.paszkowski at intel.com>; Bezzubikov, Aleksandr < > aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 < > andrey1.tretyakov at intel.com> > > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend > > > > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev <mailto: > llvm-dev at lists.llvm.org> wrote: > > Hi all, > > > > We would like to propose this RFC for upstreaming a proper SPIR-V > backend to LLVM: > > > > Hi, > > > > Perhaps a parallel question: how does that integrate with MLIR's SPIRV > back-end? > > > > If this proposal goes through and we have a production-quality SPIRV > back-end in LLVM, do we remove MLIR's own version and lower to LLVM, then > to SPIRV? Or do we still need the MLIR version? > > > > In a perfect world, translating to LLVM IR then to SPIRV shouldn't make > a difference, but there could be some impedance mismatch between MLIR->LLVM > lowering that isn't compatible with SPIRV? > > > > But as a final goal, if SPIRV becomes an official LLVM target, it would > be better if we could iron out the impedance problems and keep only one > SPIRV backend. > > > > cheers, > > --renato > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/f2504a49/attachment-0001.html>
Mehdi AMINI via llvm-dev
2021-Mar-02 21:39 UTC
[llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
On Tue, Mar 2, 2021 at 3:07 AM Trifunovic, Konrad via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hi, > > A very good question. I was actually expecting it π > > So, at the moment, it does not integrate into MLIR SPIRV backend and we > have not thought about it. I guess You are referring to having a SPV > dialect in MLIR and using a 'serialize' option to produce a SPIR-V binary? > > I agree that developing two backends in parallel is a bit redundant. If > SPIR-V LLVM backend becomes a production quality it means actually it could > consume any LLVM IR (provided it does conform to some SPIR-V restrictions). > By any LLVM IR input I mean: it should be irrelevant whether it is > produced by a clang, MLIR to LLVM IR lowering or just some other front-end > that produces LLVM IR.The biggest 'impedance mismatch' that I currently see is that SPV MLIR> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets > compute. Besides instruction set, the fundamental difference is a memory > model. > So if we want to unify those, we should actually make SPIR-V LLVM backend > able to produce Vulkan dialect of SPIR-V as well. > > My answer is a bit elusive, but I totally agree with Your proposal: we > should work towards having a one solution, and, LLVM SPIR-V backend seems > like a more universal one (since it sits lower in the compiler stack). > My proposal would be to include some MLIR -> LLVM-IR translated code in > the testing so to have this final goal in mind. >Something you're missing here, and maybe Lei clarified but I'll reiterate: the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will produce. It can actually round-trip to/from the SPIRV binary format. So it is sitting lower than your backend in my view. I can't figure out a situation where it would make sense to go from MLIR SPIRV dialect to LLVM to use this new backend, but I may miss something here... It would be really great to find a common path here before duplicating a lot of the same thing in the lllvm-project monorepo, for example being able to target the MLIR dialect from GlobalISel, or alternatively converting the MIR to it right after would be an interesting thing to explore. I haven't seen it, but there was a talk last Sunday on this topic: https://llvm.org/devmtg/2021-02-28/#vm1> > PS: one more thought: SPIR-V does come with a set of builtin/intrinsic > functions that expose the full capabilities of target architecture (mostly > GPU). This set of intrinsics is actually a dialect in its own. So this is > LLVM IR + SPIR-V specific intrinsics and their semantics that fully define > the SPIR-V dialect at LLVM IR level. I believe this idea could be used in > MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it a LLVM IR > SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the idea of > 'SPIR-V dialect' still exists, it is just now expressed at the LLVM IR > level.> regards, > konrad > > > From: Renato Golin <rengolin at gmail.com> > > Sent: Tuesday, March 2, 2021 11:12 AM > > To: Trifunovic, Konrad <konrad.trifunovic at intel.com> > > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal < > michal.paszkowski at intel.com>; Bezzubikov, Aleksandr < > aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 < > andrey1.tretyakov at intel.com> > > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend > > > > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev <mailto: > llvm-dev at lists.llvm.org> wrote: > > Hi all, > > > > We would like to propose this RFC for upstreaming a proper SPIR-V > backend to LLVM: > > > > Hi, > > > > Perhaps a parallel question: how does that integrate with MLIR's SPIRV > back-end? > > > > If this proposal goes through and we have a production-quality SPIRV > back-end in LLVM, do we remove MLIR's own version and lower to LLVM, then > to SPIRV? Or do we still need the MLIR version? > > > > In a perfect world, translating to LLVM IR then to SPIRV shouldn't make > a difference, but there could be some impedance mismatch between MLIR->LLVM > lowering that isn't compatible with SPIRV? > > > > But as a final goal, if SPIRV becomes an official LLVM target, it would > be better if we could iron out the impedance problems and keep only one > SPIRV backend. > > > > cheers, > > --renato > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/e2c53208/attachment.html>