llvm dev - Mar 2021 - [RFC] Upstreaming a proper SPIR-V backend

On Tue, Mar 2, 2021 at 3:07 AM Trifunovic, Konrad via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> A very good question. I was actually expecting it 😊
>
> So, at the moment, it does not integrate into MLIR SPIRV backend and we
> have not thought about it. I guess You are referring to having a SPV
> dialect in MLIR and using a 'serialize' option to produce a SPIR-V
binary?
>
> I agree that developing two backends in parallel is a bit redundant. If
> SPIR-V LLVM backend becomes a production quality it means actually it could
> consume any LLVM IR (provided it does conform to some SPIR-V restrictions).
> By any LLVM IR input I mean: it should be irrelevant whether it is
> produced by a clang, MLIR to LLVM IR lowering or just some other front-end
> that produces LLVM IR.
The biggest 'impedance mismatch' that I currently see is that SPV
MLIR
> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets
> compute. Besides instruction set, the fundamental difference is a memory
> model.
> So if we want to unify those, we should actually make SPIR-V LLVM backend
> able to produce Vulkan dialect of SPIR-V as well.
>
> My answer is a bit elusive, but I totally agree with Your proposal: we
> should work towards having a one solution, and, LLVM SPIR-V backend seems
> like a more universal one (since it sits lower in the compiler stack).
> My proposal would be to include some MLIR -> LLVM-IR translated code in
> the testing so to have this final goal in mind.
>
Something you're missing here, and maybe Lei clarified but I'll
reiterate:
the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will
produce. It can actually round-trip to/from the SPIRV binary format. So it
is sitting lower than your backend in my view.
I can't figure out a situation where it would make sense to go from MLIR
SPIRV dialect to LLVM to use this new backend, but I may miss something
here...

It would be really great to find a common path here before duplicating a
lot of the same thing in the lllvm-project monorepo, for example being able
to target the MLIR dialect from GlobalISel, or alternatively converting the
MIR to it right after would be an interesting thing to explore.
I haven't seen it, but there was a talk last Sunday on this topic:
https://llvm.org/devmtg/2021-02-28/#vm1


>
> PS: one more thought: SPIR-V does come with a set of builtin/intrinsic
> functions that expose the full capabilities of target architecture (mostly
> GPU). This set of intrinsics is actually a dialect in its own. So this is
> LLVM IR + SPIR-V specific intrinsics and their semantics that fully define
> the SPIR-V dialect at LLVM IR level. I believe this idea could be used in
> MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it a
LLVM IR
> SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the idea
of
> 'SPIR-V dialect' still exists, it is just now expressed at the LLVM
IR
> level.
> regards,
> konrad
>
> > From: Renato Golin <rengolin at gmail.com>
> > Sent: Tuesday, March 2, 2021 11:12 AM
> > To: Trifunovic, Konrad <konrad.trifunovic at intel.com>
> > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal <
> michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <
> aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <
> andrey1.tretyakov at intel.com>
> > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
> >
> > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:
> llvm-dev at lists.llvm.org> wrote:
> > Hi all,
> >
> > We would like to propose this RFC for upstreaming a proper SPIR-V
> backend to LLVM:
> >
> > Hi,
> >
> > Perhaps a parallel question: how does that integrate with MLIR's
SPIRV
> back-end?
> >
> > If this proposal goes through and we have a production-quality SPIRV
> back-end in LLVM, do we remove MLIR's own version and lower to LLVM,
then
> to SPIRV? Or do we still need the MLIR version?
> >
> > In a perfect world, translating to LLVM IR then to SPIRV shouldn't
make
> a difference, but there could be some impedance mismatch between
MLIR->LLVM
> lowering that isn't compatible with SPIRV?
> >
> > But as a final goal, if SPIRV becomes an official LLVM target, it
would
> be better if we could iron out the impedance problems and keep only one
> SPIRV backend.
> >
> > cheers,
> > --renato
> >
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/e2c53208/attachment.html>

On Tue, Mar 2, 2021 at 4:40 PM Mehdi AMINI via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Tue, Mar 2, 2021 at 3:07 AM Trifunovic, Konrad via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> A very good question. I was actually expecting it 😊
>>
>> So, at the moment, it does not integrate into MLIR SPIRV backend and we
>> have not thought about it. I guess You are referring to having a SPV
>> dialect in MLIR and using a 'serialize' option to produce a
SPIR-V binary?
>>
>> I agree that developing two backends in parallel is a bit redundant. If
>> SPIR-V LLVM backend becomes a production quality it means actually it
could
>> consume any LLVM IR (provided it does conform to some SPIR-V
restrictions).
>> By any LLVM IR input I mean: it should be irrelevant whether it is
>> produced by a clang, MLIR to LLVM IR lowering or just some other
front-end
>> that produces LLVM IR.
>
> The biggest 'impedance mismatch' that I currently see is that SPV
MLIR
>> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend
targets
>> compute. Besides instruction set, the fundamental difference is a
memory
>> model.
>> So if we want to unify those, we should actually make SPIR-V LLVM
backend
>> able to produce Vulkan dialect of SPIR-V as well.
>>
>> My answer is a bit elusive, but I totally agree with Your proposal: we
>> should work towards having a one solution, and, LLVM SPIR-V backend
seems
>> like a more universal one (since it sits lower in the compiler stack).
>> My proposal would be to include some MLIR -> LLVM-IR translated code
in
>> the testing so to have this final goal in mind.
>>
>
> Something you're missing here, and maybe Lei clarified but I'll
reiterate:
> the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will
> produce. It can actually round-trip to/from the SPIRV binary format. So it
> is sitting lower than your backend in my view.
> I can't figure out a situation where it would make sense to go from
MLIR
> SPIRV dialect to LLVM to use this new backend, but I may miss something
> here...
>
> It would be really great to find a common path here before duplicating a
> lot of the same thing in the lllvm-project monorepo, for example being able
> to target the MLIR dialect from GlobalISel, or alternatively converting the
> MIR to it right after would be an interesting thing to explore.
> I haven't seen it, but there was a talk last Sunday on this topic:
> https://llvm.org/devmtg/2021-02-28/#vm1
>
This sort of problem seems like just one of those unfortunate consequences
of MLIR being effectively an "LLVM IR 2.0 -- Generic Edition", but not
yet
actually layered underneath LLVM where it really wants to be. I think it
doesn't really make sense to tie *this* project to those long-term goals of
layering MLIR under LLVM-IR, given the extremely long timescale that is
likely to occur in. The "proper" solution probably won't be
possible any
time soon.

So, in the meantime, we could implement a special-case hack just for SPIRV,
to enable lowering it to MLIR-SPIRV dialect. But, what's the purpose? It
wouldn't really help move towards the longer term goal, I don't think?
And
if someone does need that at the moment, they can just feed the SPIRV
binary format back into the existing MLIR SPIRV dialect, right?


PS: one more thought: SPIR-V does come with a set of
builtin/intrinsic
>> functions that expose the full capabilities of target architecture
(mostly
>> GPU). This set of intrinsics is actually a dialect in its own. So this
is
>> LLVM IR + SPIR-V specific intrinsics and their semantics that fully
define
>> the SPIR-V dialect at LLVM IR level. I believe this idea could be used
in
>> MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it
a LLVM IR
>> SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the
idea of
>> 'SPIR-V dialect' still exists, it is just now expressed at the
LLVM IR
>> level.
>
>
>> regards,
>> konrad
>>
>> > From: Renato Golin <rengolin at gmail.com>
>> > Sent: Tuesday, March 2, 2021 11:12 AM
>> > To: Trifunovic, Konrad <konrad.trifunovic at intel.com>
>> > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal <
>> michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <
>> aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <
>> andrey1.tretyakov at intel.com>
>> > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
>> >
>> > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:
>> llvm-dev at lists.llvm.org> wrote:
>> > Hi all,
>> >
>> > We would like to propose this RFC for upstreaming a proper SPIR-V
>> backend to LLVM:
>> >
>> > Hi,
>> >
>> > Perhaps a parallel question: how does that integrate with
MLIR's SPIRV
>> back-end?
>> >
>> > If this proposal goes through and we have a production-quality
SPIRV
>> back-end in LLVM, do we remove MLIR's own version and lower to
LLVM, then
>> to SPIRV? Or do we still need the MLIR version?
>> >
>> > In a perfect world, translating to LLVM IR then to SPIRV
shouldn't make
>> a difference, but there could be some impedance mismatch between
MLIR->LLVM
>> lowering that isn't compatible with SPIRV?
>> >
>> > But as a final goal, if SPIRV becomes an official LLVM target, it
would
>> be better if we could iron out the impedance problems and keep only one
>> SPIRV backend.
>> >
>> > cheers,
>> > --renato
>> >
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/b355cefd/attachment.html>

>> So, at the moment, it does not integrate into MLIR SPIRV backend and we
have not thought about it. I guess You are referring to having a SPV dialect in
MLIR and using a 'serialize' option to produce a SPIR-V binary?
>> 
>> I agree that developing two backends in parallel is a bit redundant. If
SPIR-V LLVM backend becomes a production quality it means actually it could
consume any LLVM IR (provided it does conform to some SPIR-V restrictions).
>> By any LLVM IR input I mean: it should be irrelevant whether it is
produced by a clang, MLIR to LLVM IR lowering or just some other front-end that
produces LLVM IR.
>> The biggest 'impedance mismatch' that I currently see is that
SPV MLIR dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend
targets compute. Besides instruction set, the fundamental difference is a memory
model.
>> So if we want to unify those, we should actually make SPIR-V LLVM
backend able to produce Vulkan dialect of SPIR-V as well.
>> 
>> My answer is a bit elusive, but I totally agree with Your proposal: we
should work towards having a one solution, and, LLVM SPIR-V backend seems like a
more universal one (since it sits lower in the compiler stack).
>> My proposal would be to include some MLIR -> LLVM-IR translated code
in the testing so to have this final goal in mind.
>> 
> Something you're missing here, and maybe Lei clarified but I'll
reiterate: the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass
will produce. It can actually round-trip to/from the SPIRV binary format. So it
is sitting lower than your backend in my view.
> I can't figure out a situation where it would make sense to go from
MLIR SPIRV dialect to LLVM to use this new backend, but I may miss something
here...
By 'lower' I was referring to the place of backend in a typical compiler
flow that I could imagine: MLIR -> LLVM-IR (opt) -> Bakcend (llc).
And yes, I agree, if we treat MLIR SPV dialect as a final result of what this
backend would produce, then MLIR SPV could be the lowest-level representation
(before streaming into SPIR-V binary).
> It would be really great to find a common path here before duplicating a
lot of the same thing in the lllvm-project monorepo, for example being able to
target the MLIR dialect from GlobalISel, or alternatively converting the MIR to
it right after would be an interesting thing to explore.
> I haven't seen it, but there was a talk last Sunday on this topic:
https://llvm.org/devmtg/2021-02-28/#vm1
We should investigate that. I believe though that GlobalISel is not really that
flexible to produce MLIR (or dialects) - but that is something we might want to
change 😊 That path would open us a door to have a great deal of unification:
We can support two 'entry points' : 
1) Directly through MLIR. It gets translated to SPV dialect, and then streamed
to SPIR-V binary. (without even going into LLVM-IR)
2) Start with LLVM-IR (with some augmented metadata and intrinsics). Feed that
into proposed SPIR-V backend. Backend will produce MLIR SPV dialect and make use
of whatever legalization/binary emission/etc. it provides.
This way, SPIR-V LLVM backend will be (a probably tiny) wrapper around MLIR SPV.
Then, the majority of work would focus on MLIR SPV (e.g. adding support for
OpenCL environment in addition to existing Vulkan compute).

From the implementation point of view, that would bring us huge re-use. Still,
from the design point of view, we need to maintain two 'GPU centric'
representations': LLVM-IR with SPIR-V intrinsics/metadata/attributes + MLIR
SPV dialect.
Still that would be a much better situation from the community point of view.

--
konrad

I would prefer the non-MLIR route first, as proposed by the original
RFC. MLIR is great and all but it certainly opens up the possibility
for unrelated problems to slow down and derail the effort. My 2c.

~ Johannes


On 3/2/21 3:39 PM, Mehdi AMINI via llvm-dev wrote:
> On Tue, Mar 2, 2021 at 3:07 AM Trifunovic, Konrad via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> A very good question. I was actually expecting it 😊
>>
>> So, at the moment, it does not integrate into MLIR SPIRV backend and we
>> have not thought about it. I guess You are referring to having a SPV
>> dialect in MLIR and using a 'serialize' option to produce a
SPIR-V binary?
>>
>> I agree that developing two backends in parallel is a bit redundant. If
>> SPIR-V LLVM backend becomes a production quality it means actually it
could
>> consume any LLVM IR (provided it does conform to some SPIR-V
restrictions).
>> By any LLVM IR input I mean: it should be irrelevant whether it is
>> produced by a clang, MLIR to LLVM IR lowering or just some other
front-end
>> that produces LLVM IR.
> The biggest 'impedance mismatch' that I currently see is that SPV
MLIR
>> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend
targets
>> compute. Besides instruction set, the fundamental difference is a
memory
>> model.
>> So if we want to unify those, we should actually make SPIR-V LLVM
backend
>> able to produce Vulkan dialect of SPIR-V as well.
>>
>> My answer is a bit elusive, but I totally agree with Your proposal: we
>> should work towards having a one solution, and, LLVM SPIR-V backend
seems
>> like a more universal one (since it sits lower in the compiler stack).
>> My proposal would be to include some MLIR -> LLVM-IR translated code
in
>> the testing so to have this final goal in mind.
>>
> Something you're missing here, and maybe Lei clarified but I'll
reiterate:
> the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will
> produce. It can actually round-trip to/from the SPIRV binary format. So it
> is sitting lower than your backend in my view.
> I can't figure out a situation where it would make sense to go from
MLIR
> SPIRV dialect to LLVM to use this new backend, but I may miss something
> here...
>
> It would be really great to find a common path here before duplicating a
> lot of the same thing in the lllvm-project monorepo, for example being able
> to target the MLIR dialect from GlobalISel, or alternatively converting the
> MIR to it right after would be an interesting thing to explore.
> I haven't seen it, but there was a talk last Sunday on this topic:
> https://llvm.org/devmtg/2021-02-28/#vm1
>
>
>
>> PS: one more thought: SPIR-V does come with a set of builtin/intrinsic
>> functions that expose the full capabilities of target architecture
(mostly
>> GPU). This set of intrinsics is actually a dialect in its own. So this
is
>> LLVM IR + SPIR-V specific intrinsics and their semantics that fully
define
>> the SPIR-V dialect at LLVM IR level. I believe this idea could be used
in
>> MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it
a LLVM IR
>> SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the
idea of
>> 'SPIR-V dialect' still exists, it is just now expressed at the
LLVM IR
>> level.
>
>> regards,
>> konrad
>>
>>> From: Renato Golin <rengolin at gmail.com>
>>> Sent: Tuesday, March 2, 2021 11:12 AM
>>> To: Trifunovic, Konrad <konrad.trifunovic at intel.com>
>>> Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal <
>> michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <
>> aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <
>> andrey1.tretyakov at intel.com>
>>> Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
>>>
>>> On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:
>> llvm-dev at lists.llvm.org> wrote:
>>> Hi all,
>>>
>>> We would like to propose this RFC for upstreaming a proper SPIR-V
>> backend to LLVM:
>>> Hi,
>>>
>>> Perhaps a parallel question: how does that integrate with
MLIR's SPIRV
>> back-end?
>>> If this proposal goes through and we have a production-quality
SPIRV
>> back-end in LLVM, do we remove MLIR's own version and lower to
LLVM, then
>> to SPIRV? Or do we still need the MLIR version?
>>> In a perfect world, translating to LLVM IR then to SPIRV
shouldn't make
>> a difference, but there could be some impedance mismatch between
MLIR->LLVM
>> lowering that isn't compatible with SPIRV?
>>> But as a final goal, if SPIRV becomes an official LLVM target, it
would
>> be better if we could iron out the impedance problems and keep only one
>> SPIRV backend.
>>> cheers,
>>> --renato
>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev