llvm dev - Mar 2021 - [RFC] Upstreaming a proper SPIR-V backend

> As I understand it, SPIR-V is actually a mix of multiple things. It is
first and foremost 1) a binary format for encoding GPU executables that cross
the toolchain and hardware driver boundaries. Then it's 2) an intermediate
level language for expressing such GPU executables. It is also 3) a flexible and
extensible spec with all sorts of capability and extension mechanisms in order
to support the needs of multiple APIs and hardware features. It's unclear to
me what a production-quality SPIR-V LLVM backend would entail; but to actually
support various use cases SPIR-V can support (OpenCL, OpenGL, Vulkan;
shader/kernel; various levels of extensions; etc.), it looks to me that we need
a story for all the above points, where the IR aspect (2) is actually just
facet.
Agreed. Indeed, 'production quality SPIR-V backend' is vaguely defined
here and we proposed one discussion point on this. For this proposal needs, we
should focus on one subset of SPIR-V and one use-case (OpenCL). By production
quality I mean that we can correctly produce the code for that subset of SPIR-V.
I totally agree that having the 'Full SPIR-V' coverage is something very
broad and probably not achievable at all - but we are not aiming at that. I do
take the perspective of classical 'CPU backend' here: we do have to
generate the ISA code for the input LLVM-IR code. Now, besides instructions, our
backend needs to deduce proper capabilities and extensions, based on what subset
of instructions is selected. As You pointed out later, plain LLVM-IR is not
capable of describing the full SPIR-V. Some of
decorations/extensions/capabilities might be deduced by the backend, while some
need to be declared using various LLVM-IR concepts, such as metadata,
attributes, intrinsics - and that needs a clear definition.

 
>  My understanding over LLVM is it's mostly focusing on 2): we have a
very coherent single IR threading through the majority layers of the compiler
stack and the IR focuses very much as a means for compiler transformations
(i.e., no instruction versioning etc.). There isn't much native modelling
for most points for 1) and 3) (which makes sense as LLVM IR is a compiler IR).
So to make it work, one would need to shorehore through existing LLVM mechanisms
(e.g., using intrinsics for various GPU related builtins, using metadata for
SPIR-V decorations?, etc.), unless we want to evolve LLVM infrastructure to have
native support for the missing SPIR-V mechanisms, which I think might be too
much to take on.
Also agree. I do believe though, that LLVM-IR is still worth the effort, and we
can take an incremental approach into adopting some GPU concepts into LLVM-IR.
(e.g. 'convergent' attribute has been added mainly for GPU kind of
targets). The first step is defining the metadata and intrinsics that are target
specific for SPIR-V, but most of them could be generalized as GPU concepts and
even introduced into core LLVM-IR spec. I agree this is great effort - and not
really the main focus point of this proposal - yet, we should give LLVM-IR
it's own right into the world of GPUs.
> This is just general mechanisms, not mentioning the different semantics
between different SPIR-V consumers (e.g., shader vs. kernel and what that means
over memory/execution model, etc.) that needs to be sorted out too.. Just
supporting a certain use case of what SPIR-V supports is certainly simpler
though as we can bake in assumptions and avoid some infrastructure needs for the
full generality.
I would focus on just a subset and clearly define what input LLVM-IR GPU dialect
would look like for that subset.
> 
> That's why I think using MLIR as the infrastructure to build general
support for SPIR-V is more preferable as we control everything there and can
feel free to model all SPIR-V concepts in the most native way. For example we
can feel free to define all SPIR-V ops natively, including all ops introduced by
SPIR-V extensions and extended instruction sets. We can support
versions/extensions/capabilities natively and integrate it with the target
environment to automatically filter out CodeGen patterns generating ops not
available on the target, etc. To me, MLIR's open dialect/op/type/etc. system
is a perfect fit for the open SPIR-V spec with many capabilities/extensions/etc.
For example we can even make the SPIR-V dialect itself open to allow out-of-tree
extensions and development and such.
Right. For the 'General SPIR-V' support, MLIR is the right abstraction
level to use. And I would keep it that way. For the 'specific'/legacy
uses, backend is the way to fill that gap.
> With that said, I understand that software development has many reality
concerns (like existing codebase, familiarity with different components, etc.)
and we have many different use cases, which may mean that different paths make
sense. So please don't take this as a negative feedback in general. It's
just that to me it's unclear how we can unify here right now. Even when the
time arrives for unification, I'd believe going through MLIR is better to
have general SPIR-V support. :)
A very good discussion! I seem to be overly optimistic at the first place at
unifying those two approaches. Now I believe that we actually should have two
paths, for the reasons You have just explained and for the reasons of supporting
'legacy' paths/compilers that rely on a classical, years old approach:
Front-End -> LLVM-IR (opt) -> backend (llc). For that legacy path, a plain
old 'backend' approach is still (in my view) the way to go. On the other
hand, when MLIR evolves and gets wider adoption, it will be the way to go. From
the semantic point of view, MLIR is much better suited for representing
structured and extensible nature of SPIR-V. But for MLIR approach to be adopted,
new languages/front-ends need to be aware of that structure, so to take most of
the advantage of it. If Clang C/C++ start to use MLIR as its native generation
format - that would be a big case for MLIR approach, but until that happens, we
need to have some intermediate solution.

The most important use-case for the backend is the systems/languages that have
been targeting x86, ARM or NVPTX, AMDGPU. You want to replace that particular
backend with some other GPU backend (e.g. Intel GPU :) ). So the solution is to
use SPIR-V as the backend target, and then consume SPIR-V with a
proprietary/open-source GPU finalizer that eventually produces GPU assembly. (We
did not want to come up with yet another GPU intermediate language like PTX,
HSAIL etc. and want to use Khronos standard SPIR-V for that purpose). In this
use case, the stress is on points 1) and 2) and less on point 3) as You stated
earlier.

So my proposal is to keep two paths. They are complementary to each other. I
know of the maintenance cost concerns - but while there are use-cases for both,
it is still worth it. When the worlds stops using LLVM-IR, it will die silently,
so will die the SPIR-V backend - but that is a natural software lifecycle ;)

As it comes to unifying, there seems little implementation could be unified
(unless we make GlobalISel produce MLIR SPV dialect😊). Nevertheless, we might
collaborate on a conceptual level, especially on defining the subset of SPIR-V
that we want to support, what use cases (OpenCL, Vulkan compute, OGL?) are
relevant for LLVM community and to have a common vision there.

BTW: Intel is also interested in MLIR path and there is a group actively
contributing in that direction too.
...
>> My answer is a bit elusive, but I totally agree with Your proposal: we
should work towards having a one solution, and, LLVM SPIR-V backend seems like a
more universal one (since it sits lower in the compiler stack).
> I actually believe the opposite, because of the reasons I listed at the
very beginning. To me SPIR-V also stays at a higher level than LLVM. (But again,
depending on what subset we are talking about.) 
Here by 'lower level in the compiler stack' I did not mean the
higher/lower semantics level, but the place in the compiler pipeline (front-end
-> optimizer -> back-end), where I assumed MLIR is at front-end level,
optimizer is LLVM-IR, and back-end comes last.
I agree that semantically, SPIR-V is higher level than LLVM-IR, especially when
it comes to other meta-data that LLVM-IR does not support natively
(extensions/extended instruction sets/capabilities/execution model etc.)
>> My proposal would be to include some MLIR -> LLVM-IR translated code
in the testing so to have this final goal in mind.
>> PS: one more thought: SPIR-V does come with a set of builtin/intrinsic
functions that expose the full capabilities of target architecture (mostly GPU).
This set of intrinsics is actually a dialect in its own. So this is LLVM IR +
SPIR-V specific intrinsics and their semantics that fully define the SPIR-V
dialect at LLVM IR level. I believe this idea could be used in MLIR path: MLIR
-> LLVM-IR with SPIR-V intrinsics (let's call it a LLVM IR SPIR-V
dialect) -> SPIR-V binary (generated by a backend). So the idea of
'SPIR-V dialect' still exists, it is just now expressed at the LLVM IR
level.
 
> Not sure this is the prefered way, given that we can define SPIR-V ops
easily in MLIR in its own dialect with native support for various aspects. 
Agree. Having in mind that we should actually keep both paths, I believe this
path of going MLIR -> LLVM-IR -> SPIR-V then does not make sense as it
might lose some information.

regards,
konrad
> From: Renato Golin <mailto:rengolin at gmail.com> 
> Sent: Tuesday, March 2, 2021 11:12 AM
> To: Trifunovic, Konrad <mailto:konrad.trifunovic at intel.com>
> Cc: mailto:llvm-dev at lists.llvm.org; Paszkowski, Michal
<mailto:michal.paszkowski at intel.com>; Bezzubikov, Aleksandr
<mailto:aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1
<mailto:andrey1.tretyakov at intel.com>
> Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
> 
> On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:mailto:llvm-dev at lists.llvm.org> wrote:
> Hi all,
> 
> We would like to propose this RFC for upstreaming a proper SPIR-V backend
to LLVM:
> 
> Hi,
> 
> Perhaps a parallel question: how does that integrate with MLIR's SPIRV
back-end?
> 
> If this proposal goes through and we have a production-quality SPIRV
back-end in LLVM, do we remove MLIR's own version and lower to LLVM, then to
SPIRV? Or do we still need the MLIR version?
> 
> In a perfect world, translating to LLVM IR then to SPIRV shouldn't make
a difference, but there could be some impedance mismatch between MLIR->LLVM
lowering that isn't compatible with SPIRV?
> 
> But as a final goal, if SPIRV becomes an official LLVM target, it would be
better if we could iron out the impedance problems and keep only one SPIRV
backend.
> 
> cheers,
> --renato
>
_______________________________________________
LLVM Developers mailing list
mailto:llvm-dev at lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Wed, 3 Mar 2021 at 10:11, Trifunovic, Konrad <konrad.trifunovic at
intel.com>
wrote:
> > With that said, I understand that software development has many
reality
> concerns (like existing codebase, familiarity with different components,
> etc.) and we have many different use cases, which may mean that different
> paths make sense. So please don't take this as a negative feedback in
> general. It's just that to me it's unclear how we can unify here
right now.
> Even when the time arrives for unification, I'd believe going through
MLIR
> is better to have general SPIR-V support. :)
>
> A very good discussion! I seem to be overly optimistic at the first place
> at unifying those two approaches. Now I believe that we actually should
> have two paths, for the reasons You have just explained and for the reasons
> of supporting 'legacy' paths/compilers that rely on a classical,
years old
> approach: Front-End -> LLVM-IR (opt) -> backend (llc). For that
legacy
> path, a plain old 'backend' approach is still (in my view) the way
to go.
> On the other hand, when MLIR evolves and gets wider adoption, it will be
> the way to go. From the semantic point of view, MLIR is much better suited
> for representing structured and extensible nature of SPIR-V. But for MLIR
> approach to be adopted, new languages/front-ends need to be aware of that
> structure, so to take most of the advantage of it. If Clang C/C++ start to
> use MLIR as its native generation format - that would be a big case for
> MLIR approach, but until that happens, we need to have some intermediate
> solution.

I think there are two points here:

1. How many SPIRV end-points we have

This is mostly about software engineering concerns of duplication,
maintenance, etc. But it's also about IR support, with MLIR having an upper
hand here because of the existing implementation and its inherent
flexibility with dialects.

It's perfectly fine to have two back-ends for a while, but since we moved
MLIR to the monorepo, we need to treat it as part of the LLVM family, not a
side project.

LLVM IR has some "flexibility" through intrinsics, which we could use
to
translate MLIR concepts that can't be represented in LLVM IR for the
purpose of lowering only. Optimisations on these intrinsics would bring the
usual problems.

2. Where do the optimisations happen in code lowering to SPIRV

I think Ronan's points are a good basis for keeping that in MLIR, at least
for the current code. Now, if that precludes optimising in LLVM IR, than
this could be a conflict with this proposal.

If the code passes through MLIR or not will be a decision of the toolchain,
that will pick the best path for each workload. This allows us to have
concurrent approaches in tree, but also makes it hard to test and creates
corner cases that are hard to test.

So, while I appreciate this is a large proposal, that will likely take a
year or more to get into shape, I think the ultimate goal (after the
current proposal) should be that we end up with one back-end.

I'm a big fan of MLIR, and I think we should keep developing the SPIRV
dialect and possibly this could be the entry point of all SPIRV toolchains.

While Clang will take a long time (if ever) to generate MLIR for C/C++, it
could very well generate MLIR for non-C++ (OpenCL, OpenMP, SYCL, etc) which
is then optimised, compiled into LLVM IR and linked to the main module (or
not, for multi-targets) after high-level optimisations.

This would answer both questions above and create a pipeline that is
consistent, easier to test and with lower overall maintenance costs.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210303/30a4f79e/attachment.html>

On Wed, Mar 3, 2021, 2:11 AM Trifunovic, Konrad via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> > As I understand it, SPIR-V is actually a mix of multiple things. It is
> first and foremost 1) a binary format for encoding GPU executables that
> cross the toolchain and hardware driver boundaries. Then it's 2) an
> intermediate level language for expressing such GPU executables. It is also
> 3) a flexible and extensible spec with all sorts of capability and
> extension mechanisms in order to support the needs of multiple APIs and
> hardware features. It's unclear to me what a production-quality SPIR-V
LLVM
> backend would entail; but to actually support various use cases SPIR-V can
> support (OpenCL, OpenGL, Vulkan; shader/kernel; various levels of
> extensions; etc.), it looks to me that we need a story for all the above
> points, where the IR aspect (2) is actually just facet.
>
> Agreed. Indeed, 'production quality SPIR-V backend' is vaguely
defined
> here and we proposed one discussion point on this. For this proposal needs,
> we should focus on one subset of SPIR-V and one use-case (OpenCL). By
> production quality I mean that we can correctly produce the code for that
> subset of SPIR-V. I totally agree that having the 'Full SPIR-V'
coverage is
> something very broad and probably not achievable at all - but we are not
> aiming at that. I do take the perspective of classical 'CPU
backend' here:
> we do have to generate the ISA code for the input LLVM-IR code. Now,
> besides instructions, our backend needs to deduce proper capabilities and
> extensions, based on what subset of instructions is selected. As You
> pointed out later, plain LLVM-IR is not capable of describing the full
> SPIR-V. Some of decorations/extensions/capabilities might be deduced by the
> backend, while some need to be declared using various LLVM-IR concepts,
> such as metadata, attributes, intrinsics - and that needs a clear
> definition.
>
>
> >  My understanding over LLVM is it's mostly focusing on 2): we have
a
> very coherent single IR threading through the majority layers of the
> compiler stack and the IR focuses very much as a means for compiler
> transformations (i.e., no instruction versioning etc.). There isn't
much
> native modelling for most points for 1) and 3) (which makes sense as LLVM
> IR is a compiler IR). So to make it work, one would need to shorehore
> through existing LLVM mechanisms (e.g., using intrinsics for various GPU
> related builtins, using metadata for SPIR-V decorations?, etc.), unless we
> want to evolve LLVM infrastructure to have native support for the missing
> SPIR-V mechanisms, which I think might be too much to take on.
>
> Also agree. I do believe though, that LLVM-IR is still worth the effort,
> and we can take an incremental approach into adopting some GPU concepts
> into LLVM-IR. (e.g. 'convergent' attribute has been added mainly
for GPU
> kind of targets). The first step is defining the metadata and intrinsics
> that are target specific for SPIR-V, but most of them could be generalized
> as GPU concepts and even introduced into core LLVM-IR spec. I agree this is
> great effort - and not really the main focus point of this proposal - yet,
> we should give LLVM-IR it's own right into the world of GPUs.
>
> > This is just general mechanisms, not mentioning the different
semantics
> between different SPIR-V consumers (e.g., shader vs. kernel and what that
> means over memory/execution model, etc.) that needs to be sorted out too..
> Just supporting a certain use case of what SPIR-V supports is certainly
> simpler though as we can bake in assumptions and avoid some infrastructure
> needs for the full generality.
>
> I would focus on just a subset and clearly define what input LLVM-IR GPU
> dialect would look like for that subset.
>
> >
> > That's why I think using MLIR as the infrastructure to build
general
> support for SPIR-V is more preferable as we control everything there and
> can feel free to model all SPIR-V concepts in the most native way. For
> example we can feel free to define all SPIR-V ops natively, including all
> ops introduced by SPIR-V extensions and extended instruction sets. We can
> support versions/extensions/capabilities natively and integrate it with the
> target environment to automatically filter out CodeGen patterns generating
> ops not available on the target, etc. To me, MLIR's open
> dialect/op/type/etc. system is a perfect fit for the open SPIR-V spec with
> many capabilities/extensions/etc. For example we can even make the SPIR-V
> dialect itself open to allow out-of-tree extensions and development and
> such.
>
> Right. For the 'General SPIR-V' support, MLIR is the right
abstraction
> level to use. And I would keep it that way. For the
'specific'/legacy uses,
> backend is the way to fill that gap.
>
> > With that said, I understand that software development has many
reality
> concerns (like existing codebase, familiarity with different components,
> etc.) and we have many different use cases, which may mean that different
> paths make sense. So please don't take this as a negative feedback in
> general. It's just that to me it's unclear how we can unify here
right now.
> Even when the time arrives for unification, I'd believe going through
MLIR
> is better to have general SPIR-V support. :)
>
> A very good discussion! I seem to be overly optimistic at the first place
> at unifying those two approaches. Now I believe that we actually should
> have two paths, for the reasons You have just explained and for the reasons
> of supporting 'legacy' paths/compilers that rely on a classical,
years old
> approach: Front-End -> LLVM-IR (opt) -> backend (llc). For that
legacy
> path, a plain old 'backend' approach is still (in my view) the way
to go.
> On the other hand, when MLIR evolves and gets wider adoption, it will be
> the way to go. From the semantic point of view, MLIR is much better suited
> for representing structured and extensible nature of SPIR-V. But for MLIR
> approach to be adopted, new languages/front-ends need to be aware of that
> structure, so to take most of the advantage of it. If Clang C/C++ start to
> use MLIR as its native generation format - that would be a big case for
> MLIR approach, but until that happens, we need to have some intermediate
> solution.
>
> The most important use-case for the backend is the systems/languages that
> have been targeting x86, ARM or NVPTX, AMDGPU. You want to replace that
> particular backend with some other GPU backend (e.g. Intel GPU :) ). So the
> solution is to use SPIR-V as the backend target, and then consume SPIR-V
> with a proprietary/open-source GPU finalizer that eventually produces GPU
> assembly. (We did not want to come up with yet another GPU intermediate
> language like PTX, HSAIL etc. and want to use Khronos standard SPIR-V for
> that purpose). In this use case, the stress is on points 1) and 2) and less
> on point 3) as You stated earlier.
>
> So my proposal is to keep two paths. They are complementary to each other.
> I know of the maintenance cost concerns - but while there are use-cases for
> both, it is still worth it. When the worlds stops using LLVM-IR, it will
> die silently, so will die the SPIR-V backend - but that is a natural
> software lifecycle ;)
>
I agree with this viewpoint. Having a backend like this (I hesitate to just
call it a "Spir-v backend" because it is a more specific thing than
that)
would bring a level of utility and parity between GPU targets that seems
like it has a lot of value. There may be a future convergence between the
LLVM-IR and MLIR approaches, but I think we need to see and do more to get
there. So long as there is enough community support to build and maintain
this backend, it seems like a good thing that we want in repo, and having
it developed there will help us see more options for unification over time.

I'm also personally supportive of "sweating it out" on this
RFC/subsequent
discussions and exploring technical options that we may have missed for
better unification from an earlier point because these two parts of the
community have been fairly isolated. Those are good conversations that we
should have and continue. Unless if an obvious simplification emerges as
part of that, by default though, I would support moving forward with this
proposal once the details are converged.

It'd be quite a good outcome, imo, to exit these discussions with a sketch
of a plan for how these things could converge in the future, but I think we
will find that future to be a ways out and require more practical first
steps.

> As it comes to unifying, there seems little implementation could be
> unified (unless we make GlobalISel produce MLIR SPV dialect😊).
> Nevertheless, we might collaborate on a conceptual level, especially on
> defining the subset of SPIR-V that we want to support, what use cases
> (OpenCL, Vulkan compute, OGL?) are relevant for LLVM community and to have
> a common vision there.
>
> BTW: Intel is also interested in MLIR path and there is a group actively
> contributing in that direction too.
> ...
>
> >> My answer is a bit elusive, but I totally agree with Your
proposal: we
> should work towards having a one solution, and, LLVM SPIR-V backend seems
> like a more universal one (since it sits lower in the compiler stack).
>
> > I actually believe the opposite, because of the reasons I listed at
the
> very beginning. To me SPIR-V also stays at a higher level than LLVM. (But
> again, depending on what subset we are talking about.)
>
> Here by 'lower level in the compiler stack' I did not mean the
> higher/lower semantics level, but the place in the compiler pipeline
> (front-end -> optimizer -> back-end), where I assumed MLIR is at
front-end
> level, optimizer is LLVM-IR, and back-end comes last.
> I agree that semantically, SPIR-V is higher level than LLVM-IR, especially
> when it comes to other meta-data that LLVM-IR does not support natively
> (extensions/extended instruction sets/capabilities/execution model etc.)
>
> >> My proposal would be to include some MLIR -> LLVM-IR translated
code in
> the testing so to have this final goal in mind.
>
> >> PS: one more thought: SPIR-V does come with a set of
builtin/intrinsic
> functions that expose the full capabilities of target architecture (mostly
> GPU). This set of intrinsics is actually a dialect in its own. So this is
> LLVM IR + SPIR-V specific intrinsics and their semantics that fully define
> the SPIR-V dialect at LLVM IR level. I believe this idea could be used in
> MLIR path: MLIR -> LLVM-IR with SPIR-V intrinsics (let's call it a
LLVM IR
> SPIR-V dialect) -> SPIR-V binary (generated by a backend). So the idea
of
> 'SPIR-V dialect' still exists, it is just now expressed at the LLVM
IR
> level.
>
> > Not sure this is the prefered way, given that we can define SPIR-V ops
> easily in MLIR in its own dialect with native support for various aspects.
>
> Agree. Having in mind that we should actually keep both paths, I believe
> this path of going MLIR -> LLVM-IR -> SPIR-V then does not make sense
as it
> might lose some information.
>
> regards,
> konrad
>
> > From: Renato Golin <mailto:rengolin at gmail.com>
> > Sent: Tuesday, March 2, 2021 11:12 AM
> > To: Trifunovic, Konrad <mailto:konrad.trifunovic at intel.com>
> > Cc: mailto:llvm-dev at lists.llvm.org; Paszkowski, Michal <mailto:
> michal.paszkowski at intel.com>; Bezzubikov, Aleksandr <mailto:
> aleksandr.bezzubikov at intel.com>; Tretyakov, Andrey1 <mailto:
> andrey1.tretyakov at intel.com>
> > Subject: Re: [llvm-dev] [RFC] Upstreaming a proper SPIR-V backend
> >
> > On Tue, 2 Mar 2021 at 09:36, Trifunovic, Konrad via llvm-dev
<mailto:
> mailto:llvm-dev at lists.llvm.org> wrote:
> > Hi all,
> >
> > We would like to propose this RFC for upstreaming a proper SPIR-V
> backend to LLVM:
> >
> > Hi,
> >
> > Perhaps a parallel question: how does that integrate with MLIR's
SPIRV
> back-end?
> >
> > If this proposal goes through and we have a production-quality SPIRV
> back-end in LLVM, do we remove MLIR's own version and lower to LLVM,
then
> to SPIRV? Or do we still need the MLIR version?
> >
> > In a perfect world, translating to LLVM IR then to SPIRV shouldn't
make
> a difference, but there could be some impedance mismatch between
MLIR->LLVM
> lowering that isn't compatible with SPIRV?
> >
> > But as a final goal, if SPIRV becomes an official LLVM target, it
would
> be better if we could iron out the impedance problems and keep only one
> SPIRV backend.
> >
> > cheers,
> > --renato
> >
> _______________________________________________
> LLVM Developers mailing list
> mailto:llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210303/8f2f48a1/attachment.html>