llvm dev - Mar 2021 - [RFC] Upstreaming a proper SPIR-V backend

On Tue, Mar 2, 2021 at 4:40 PM Mehdi AMINI via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Tue, Mar 2, 2021 at 3:07 AM Trifunovic, Konrad via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> A very good question. I was actually expecting it 😊
>>
>> So, at the moment, it does not integrate into MLIR SPIRV backend and we
>> have not thought about it. I guess You are referring to having a SPV
>> dialect in MLIR and using a 'serialize' option to produce a
SPIR-V binary?
>>
>> I agree that developing two backends in parallel is a bit redundant. If
>> SPIR-V LLVM backend becomes a production quality it means actually it
could
>> consume any LLVM IR (provided it does conform to some SPIR-V
restrictions).
>> By any LLVM IR input I mean: it should be irrelevant whether it is
>> produced by a clang, MLIR to LLVM IR lowering or just some other
front-end
>> that produces LLVM IR.
>
> The biggest 'impedance mismatch' that I currently see is that SPV
MLIR
>> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend
targets
>> compute. Besides instruction set, the fundamental difference is a
memory
>> model.
>> So if we want to unify those, we should actually make SPIR-V LLVM
backend
>> able to produce Vulkan dialect of SPIR-V as well.
>>
>> My answer is a bit elusive, but I totally agree with Your proposal: we
>> should work towards having a one solution, and, LLVM SPIR-V backend
seems
>> like a more universal one (since it sits lower in the compiler stack).
>> My proposal would be to include some MLIR -> LLVM-IR translated code
in
>> the testing so to have this final goal in mind.
>>
>
> Something you're missing here, and maybe Lei clarified but I'll
reiterate:
> the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will
> produce. It can actually round-trip to/from the SPIRV binary format. So it
> is sitting lower than your backend in my view.
> I can't figure out a situation where it would make sense to go from
MLIR
> SPIRV dialect to LLVM to use this new backend, but I may miss something
> here...
>
> It would be really great to find a common path here before duplicating a
> lot of the same thing in the lllvm-project monorepo, for example being able
> to target the MLIR dialect from GlobalISel, or alternatively converting the
> MIR to it right after would be an interesting thing to explore.
> I haven't seen it, but there was a talk last Sunday on this topic:
> https://llvm.org/devmtg/2021-02-28/#vm1
>
This sort of problem seems like just one of those unfortunate consequences
of MLIR being effectively an "LLVM IR 2.0 -- Generic Edition", but not
yet
actually layered underneath LLVM where it really wants to be. I think it
doesn't really make sense to tie *this* project to those long-term goals of
layering MLIR under LLVM-IR, given the extremely long timescale that is
likely to occur in. The "proper" solution probably won't be
possible any
time soon.

So, in the meantime, we could implement a special-case hack just for SPIRV,
to enable lowering it to MLIR-SPIRV dialect. But, what's the purpose? It
wouldn't really help move towards the longer term goal, I don't think?
And
if someone does need that at the moment, they can just feed the SPIRV
binary format back into the existing MLIR SPIRV dialect, right?


PS: one more thought: SPIR-V does come with a set of
builtin/intrinsic
>> functions that expose the full capabilities of target architecture
(mostly
>> GPU). This set of intrinsics is actually a dialect in its own. So this
is
>> LLVM IR + SPIR-V specific intrinsics and their semantics that fully
define
>> the SPIR-V dialect at LLVM IR level. I believe this idea could be used
in
>> MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it
a LLVM IR
>> SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the
idea of
>> 'SPIR-V dialect' still exists, it is just now expressed at the
LLVM IR
>> level.
>
>
>> regards,
>> konrad
>>
>> > From: Renato Golin <rengolin at gmail.com>
>> > Sent: Tuesday, March 2, 2021 11:12 AM
>> > To: Trifunovic, Konrad <konrad.trifunovic at intel.com>
>> > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal <
>> michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <
>> aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <
>> andrey1.tretyakov at intel.com>
>> > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
>> >
>> > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:
>> llvm-dev at lists.llvm.org> wrote:
>> > Hi all,
>> >
>> > We would like to propose this RFC for upstreaming a proper SPIR-V
>> backend to LLVM:
>> >
>> > Hi,
>> >
>> > Perhaps a parallel question: how does that integrate with
MLIR's SPIRV
>> back-end?
>> >
>> > If this proposal goes through and we have a production-quality
SPIRV
>> back-end in LLVM, do we remove MLIR's own version and lower to
LLVM, then
>> to SPIRV? Or do we still need the MLIR version?
>> >
>> > In a perfect world, translating to LLVM IR then to SPIRV
shouldn't make
>> a difference, but there could be some impedance mismatch between
MLIR->LLVM
>> lowering that isn't compatible with SPIRV?
>> >
>> > But as a final goal, if SPIRV becomes an official LLVM target, it
would
>> be better if we could iron out the impedance problems and keep only one
>> SPIRV backend.
>> >
>> > cheers,
>> > --renato
>> >
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/b355cefd/attachment.html>

Please some some comments inlined

On Wed, Mar 3, 2021 at 2:19 AM James Y Knight via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Tue, Mar 2, 2021 at 4:40 PM Mehdi AMINI via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On Tue, Mar 2, 2021 at 3:07 AM Trifunovic, Konrad via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi,
>>>
>>> A very good question. I was actually expecting it 😊
>>>
>>> So, at the moment, it does not integrate into MLIR SPIRV backend
and we
>>> have not thought about it. I guess You are referring to having a
SPV
>>> dialect in MLIR and using a 'serialize' option to produce a
SPIR-V binary?
>>>
>>> I agree that developing two backends in parallel is a bit
redundant. If
>>> SPIR-V LLVM backend becomes a production quality it means actually
it could
>>> consume any LLVM IR (provided it does conform to some SPIR-V
restrictions).
>>> By any LLVM IR input I mean: it should be irrelevant whether it is
>>> produced by a clang, MLIR to LLVM IR lowering or just some other
front-end
>>> that produces LLVM IR.
>>
>> The biggest 'impedance mismatch' that I currently see is that
SPV MLIR
>>> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend
targets
>>> compute. Besides instruction set, the fundamental difference is a
memory
>>> model.
>>> So if we want to unify those, we should actually make SPIR-V LLVM
>>> backend able to produce Vulkan dialect of SPIR-V as well.
>>>
>>> My answer is a bit elusive, but I totally agree with Your proposal:
we
>>> should work towards having a one solution, and, LLVM SPIR-V backend
seems
>>> like a more universal one (since it sits lower in the compiler
stack).
>>> My proposal would be to include some MLIR -> LLVM-IR translated
code in
>>> the testing so to have this final goal in mind.
>>>
>>
>> Something you're missing here, and maybe Lei clarified but I'll
>> reiterate: the SPIRV dialect in MLIR is equivalent to what your
GlobalISel
>> pass will produce. It can actually round-trip to/from the SPIRV binary
>> format. So it is sitting lower than your backend in my view.
>>
>
What do you mean by lower? I'm not that familiar with the way MLIR deals
with SPIR-V binaries, but isn't it still necessary to convert SPIR-V
dialect to LLVM and then use some hardware-tied codegen to be able to _run_
a SPIR-V binary?

> I can't figure out a situation where it would make sense to go from
MLIR
>> SPIRV dialect to LLVM to use this new backend, but I may miss something
>> here...
>>
>> It would be really great to find a common path here before duplicating
a
>> lot of the same thing in the lllvm-project monorepo, for example being
able
>> to target the MLIR dialect from GlobalISel, or alternatively converting
the
>> MIR to it right after would be an interesting thing to explore.
>> I haven't seen it, but there was a talk last Sunday on this topic:
>> https://llvm.org/devmtg/2021-02-28/#vm1
>>
>
Oh, this sounds interesting actually. Would be nice if someone has any
materials or code to share on the topic.

>
> This sort of problem seems like just one of those unfortunate consequences
> of MLIR being effectively an "LLVM IR 2.0 -- Generic Edition",
but not yet
> actually layered underneath LLVM where it really wants to be. I think it
> doesn't really make sense to tie *this* project to those long-term
goals
> of layering MLIR under LLVM-IR, given the extremely long timescale that is
> likely to occur in. The "proper" solution probably won't be
possible any
> time soon.
>
>
There's definitely some consensus or even roadmap/timeline on this
transition missing IMO :)  And pls forgive me my possibly stupid question,
but is there any way now to _conveniently_ incorporate MLIR flow for
projects which are based on a good old clang->llvm->mir->machinecode
way? I
understand we have 'llvm' dialect and may recall last year there was a
talk
about the common C/C++ dialect, but it isn't public yet, is it?

> So, in the meantime, we could implement a special-case hack just for
> SPIRV, to enable lowering it to MLIR-SPIRV dialect. But, what's the
> purpose? It wouldn't really help move towards the longer term goal, I
don't
> think? And if someone does need that at the moment, they can just feed the
> SPIRV binary format back into the existing MLIR SPIRV dialect, right?
>
>
> PS: one more thought: SPIR-V does come with a set of builtin/intrinsic
>>> functions that expose the full capabilities of target architecture
(mostly
>>> GPU). This set of intrinsics is actually a dialect in its own. So
this is
>>> LLVM IR + SPIR-V specific intrinsics and their semantics that fully
define
>>> the SPIR-V dialect at LLVM IR level. I believe this idea could be
used in
>>> MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's
call it a LLVM IR
>>> SPIR-V dialect) -> SPIR-V binary (generated by a backend). So
the idea of
>>> 'SPIR-V dialect' still exists, it is just now expressed at
the LLVM IR
>>> level.
>>
>>
>>> regards,
>>> konrad
>>>
>>> > From: Renato Golin <rengolin at gmail.com>
>>> > Sent: Tuesday, March 2, 2021 11:12 AM
>>> > To: Trifunovic, Konrad <konrad.trifunovic at intel.com>
>>> > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal <
>>> michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <
>>> aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <
>>> andrey1.tretyakov at intel.com>
>>> > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V
backend
>>> >
>>> > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:
>>> llvm-dev at lists.llvm.org> wrote:
>>> > Hi all,
>>> >
>>> > We would like to propose this RFC for upstreaming a proper
SPIR-V
>>> backend to LLVM:
>>> >
>>> > Hi,
>>> >
>>> > Perhaps a parallel question: how does that integrate with
MLIR's SPIRV
>>> back-end?
>>> >
>>> > If this proposal goes through and we have a production-quality
SPIRV
>>> back-end in LLVM, do we remove MLIR's own version and lower to
LLVM, then
>>> to SPIRV? Or do we still need the MLIR version?
>>> >
>>> > In a perfect world, translating to LLVM IR then to SPIRV
shouldn't
>>> make a difference, but there could be some impedance mismatch
between
>>> MLIR->LLVM lowering that isn't compatible with SPIRV?
>>> >
>>> > But as a final goal, if SPIRV becomes an official LLVM target,
it
>>> would be better if we could iron out the impedance problems and
keep only
>>> one SPIRV backend.
>>> >
>>> > cheers,
>>> > --renato
>>> >
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

Thanks,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210303/060f7e28/attachment.html>

On Tue, Mar 2, 2021 at 3:18 PM James Y Knight <jyknight at google.com>
wrote:
> On Tue, Mar 2, 2021 at 4:40 PM Mehdi AMINI via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On Tue, Mar 2, 2021 at 3:07 AM Trifunovic, Konrad via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi,
>>>
>>> A very good question. I was actually expecting it 😊
>>>
>>> So, at the moment, it does not integrate into MLIR SPIRV backend
and we
>>> have not thought about it. I guess You are referring to having a
SPV
>>> dialect in MLIR and using a 'serialize' option to produce a
SPIR-V binary?
>>>
>>> I agree that developing two backends in parallel is a bit
redundant. If
>>> SPIR-V LLVM backend becomes a production quality it means actually
it could
>>> consume any LLVM IR (provided it does conform to some SPIR-V
restrictions).
>>> By any LLVM IR input I mean: it should be irrelevant whether it is
>>> produced by a clang, MLIR to LLVM IR lowering or just some other
front-end
>>> that produces LLVM IR.
>>
>> The biggest 'impedance mismatch' that I currently see is that
SPV MLIR
>>> dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend
targets
>>> compute. Besides instruction set, the fundamental difference is a
memory
>>> model.
>>> So if we want to unify those, we should actually make SPIR-V LLVM
>>> backend able to produce Vulkan dialect of SPIR-V as well.
>>>
>>> My answer is a bit elusive, but I totally agree with Your proposal:
we
>>> should work towards having a one solution, and, LLVM SPIR-V backend
seems
>>> like a more universal one (since it sits lower in the compiler
stack).
>>> My proposal would be to include some MLIR -> LLVM-IR translated
code in
>>> the testing so to have this final goal in mind.
>>>
>>
>> Something you're missing here, and maybe Lei clarified but I'll
>> reiterate: the SPIRV dialect in MLIR is equivalent to what your
GlobalISel
>> pass will produce. It can actually round-trip to/from the SPIRV binary
>> format. So it is sitting lower than your backend in my view.
>> I can't figure out a situation where it would make sense to go from
MLIR
>> SPIRV dialect to LLVM to use this new backend, but I may miss something
>> here...
>>
>> It would be really great to find a common path here before duplicating
a
>> lot of the same thing in the lllvm-project monorepo, for example being
able
>> to target the MLIR dialect from GlobalISel, or alternatively converting
the
>> MIR to it right after would be an interesting thing to explore.
>> I haven't seen it, but there was a talk last Sunday on this topic:
>> https://llvm.org/devmtg/2021-02-28/#vm1
>>
>
> This sort of problem seems like just one of those unfortunate consequences
> of MLIR being effectively an "LLVM IR 2.0 -- Generic Edition",
but not yet
> actually layered underneath LLVM where it really wants to be.
>
I don't understand what you mean here with "layered underneath
LLVM"? Can
you elaborate on this?


> I think it doesn't really make sense to tie *this* project to those
> long-term goals of layering MLIR under LLVM-IR, given the extremely long
> timescale that is likely to occur in. The "proper" solution
probably won't
> be possible any time soon.
>
I'm not sure if we're talking about the same thing here: there is
nothing
that I suggest that would operate at the level of LLVM IR. And nothing that
requires a "long timescale", it seems quite easily in scope to me
here.

>
> So, in the meantime, we could implement a special-case hack just for
> SPIRV, to enable lowering it to MLIR-SPIRV dialect. But, what's the
> purpose? It wouldn't really help move towards the longer term goal, I
don't
> think? And if someone does need that at the moment, they can just feed the
> SPIRV binary format back into the existing MLIR SPIRV dialect, right?
>
Do we want to maintain, in the LLVM monorepo, *two* different
implementations of a SPIRV IR and associated serialization (and potential
deserialization)? All the tools associated to manipulate it? I assume the
backend may even want to implement optimization passes, are we gonna
duplicate these as well?
(note that this isn't at the LLVM IR level, but post-instruction selection,
so very ad-hoc to the backend anyway).0

-- 
Mehdi

>
>
> PS: one more thought: SPIR-V does come with a set of builtin/intrinsic
>>> functions that expose the full capabilities of target architecture
(mostly
>>> GPU). This set of intrinsics is actually a dialect in its own. So
this is
>>> LLVM IR + SPIR-V specific intrinsics and their semantics that fully
define
>>> the SPIR-V dialect at LLVM IR level. I believe this idea could be
used in
>>> MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's
call it a LLVM IR
>>> SPIR-V dialect) -> SPIR-V binary (generated by a backend). So
the idea of
>>> 'SPIR-V dialect' still exists, it is just now expressed at
the LLVM IR
>>> level.
>>
>>
>>> regards,
>>> konrad
>>>
>>> > From: Renato Golin <rengolin at gmail.com>
>>> > Sent: Tuesday, March 2, 2021 11:12 AM
>>> > To: Trifunovic, Konrad <konrad.trifunovic at intel.com>
>>> > Cc: llvm-dev at lists.llvm.org; Paszkowski, Michal <
>>> michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <
>>> aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <
>>> andrey1.tretyakov at intel.com>
>>> > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V
backend
>>> >
>>> > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:
>>> llvm-dev at lists.llvm.org> wrote:
>>> > Hi all,
>>> >
>>> > We would like to propose this RFC for upstreaming a proper
SPIR-V
>>> backend to LLVM:
>>> >
>>> > Hi,
>>> >
>>> > Perhaps a parallel question: how does that integrate with
MLIR's SPIRV
>>> back-end?
>>> >
>>> > If this proposal goes through and we have a production-quality
SPIRV
>>> back-end in LLVM, do we remove MLIR's own version and lower to
LLVM, then
>>> to SPIRV? Or do we still need the MLIR version?
>>> >
>>> > In a perfect world, translating to LLVM IR then to SPIRV
shouldn't
>>> make a difference, but there could be some impedance mismatch
between
>>> MLIR->LLVM lowering that isn't compatible with SPIRV?
>>> >
>>> > But as a final goal, if SPIRV becomes an official LLVM target,
it
>>> would be better if we could iron out the impedance problems and
keep only
>>> one SPIRV backend.
>>> >
>>> > cheers,
>>> > --renato
>>> >
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210302/3270264d/attachment-0001.html>