llvm dev - Aug 2021 - [cfe-dev] [RFC][HIPSPV] Emitting HIP device code as SPIR-V

Hi Henry,
> The HIPSPIRV experimental branch is available at [4]. Note that it is not
yet in a state we intend to propose for upstreaming, but shaping up the patches
is a work in progress. Before proceeding to shape up and submit the patches, we
would like to get feedback for the plans we have for upstreaming. In the
following sections, we open up the above points further and sketch our plans for
changes to LLVM (mostly to the Clang tool) to achieve the goal.
Unfortunately I get 404 when I try access
https://github.com/parmance/llvm-project/tree/hip2spirv-v5. Your llvm-project
fork seems to be private.

Anyway, I see a lot of items listed in the email overlap with the SYCL
implementation, which also targets SPIR-V execution environments. Even if we
can't re-use the code as-is the approach for compiling GPU code to SPIR-V
target can be common.
> * Implement a new SPIRVTargetInfo and fill it with necessary
>   information. For HIPCL/-LZ we are planning to adjust the address
>   space mapping in a way which is discussed later in the ‘address
>   space mapping’ section.
I'm not sure if new TargetInfo class is needed for that or it can achieved
by adding HIP specific mapping in `adjust` method overload similar to how
it's done for SYCL mode
(https://github.com/llvm/llvm-project/blob/main/clang/lib/Basic/Targets/SPIR.h#L138).
> * Introduce ’hipspv’ as an OS or environment type in Triple. The
>  primary and the current use of the type is to select device offload
>  toolchain for HIPCL/-LZ.
Can't we use HIPCL toolchain if `spir` or `spirv64` target is set? We have a
new environment type for SYCL, but now I’m trying to remove it as there are no
cases where it can't be avoided. More details are in this issue [1] and pull
request if you are interested [2].
> Since the SPIR-V BE might not land in LLVM soon, we will set up the
compilation flow
>  to produce the SPIR-V binary by using the LLVM-SPIR-V translator [7]
>  which is used in our experimental branch.
+1. SYCL uses the same compilation flow. We have a patch adding declarations of
SPIR-V built-ins compatible with LLVM-SPIRV-Translator, which follows the
approach taken OpenCL built-ins and AFAIK is used by SPIR-V backend as well. I
can upload it to Phabricator for review if needed.
> HIP code expansion
Downstream SYCL implementation [3] supports all the features listed in this
section except DSM allocations + quite a few other feature specific to SYCL and
we are also doing post-link processing before SPIR-V translation. It would be
very interesting to see how we can merge our implementations.

Thanks,
Alexey

[1] https://github.com/intel/llvm/issues/3534
[2] https://github.com/intel/llvm/pull/3929
[3] https://github.com/intel/llvm/tree/sycl

Hi Bader,

On Tue, 10 Aug 2021 at 19:08, Bader, Alexey <alexey.bader at intel.com>
wrote:
>
> Hi Henry,
>
> > The HIPSPIRV experimental branch is available at [4]. Note that it is
not yet in a state we intend to propose for upstreaming, but shaping up the
patches is a work in progress. Before proceeding to shape up and submit the
patches, we would like to get feedback for the plans we have for upstreaming. In
the following sections, we open up the above points further and sketch our plans
for changes to LLVM (mostly to the Clang tool) to achieve the goal.
>
> Unfortunately I get 404 when I try access
https://github.com/parmance/llvm-project/tree/hip2spirv-v5. Your llvm-project
fork seems to be private.
>
Ah, sorry - the repo is now public. Also, we will soon start to
sending upstream-reshaped patches to the phabricator for review.
> Anyway, I see a lot of items listed in the email overlap with the SYCL
implementation, which also targets SPIR-V execution environments. Even if we
can't re-use the code as-is the approach for compiling GPU code to SPIR-V
target can be common.
>
> > * Implement a new SPIRVTargetInfo and fill it with necessary
> >   information. For HIPCL/-LZ we are planning to adjust the address
> >   space mapping in a way which is discussed later in the ‘address
> >   space mapping’ section.
>
> I'm not sure if new TargetInfo class is needed for that or it can
achieved by adding HIP specific mapping in `adjust` method overload similar to
how it's done for SYCL mode
(https://github.com/llvm/llvm-project/blob/main/clang/lib/Basic/Targets/SPIR.h#L138).
>
Yeah, It seems there is no need to have separate TargetInfo for SPIR-V
because there is not going to be much difference between SPIR and
SPIR-V target info. We also alter the mapping by overloading the
adjust method.
> > * Introduce ’hipspv’ as an OS or environment type in Triple. The
> >  primary and the current use of the type is to select device offload
> >  toolchain for HIPCL/-LZ.
>
> Can't we use HIPCL toolchain if `spir` or `spirv64` target is set? We
have a new environment type for SYCL, but now I’m trying to remove it as there
are no cases where it can't be avoided. More details are in this issue [1]
and pull request if you are interested [2].
We can drop the ‘hipspv’ component for starters. The initial thought
was we need it to select the HIPSPVToolChain but there is another
approach in sight already that does not need the component.

However, we might need the component in the future: I’m thinking of
the upstreaming of HIP expander passes which may need to be run only
if we are targeting the HIPCL/-LZ environment. But perhaps we can
leave another kind of mark in the bitcode for the passes.
>
> > Since the SPIR-V BE might not land in LLVM soon, we will set up the
compilation flow
> >  to produce the SPIR-V binary by using the LLVM-SPIR-V translator [7]
> >  which is used in our experimental branch.
>
> +1. SYCL uses the same compilation flow. We have a patch adding
declarations of SPIR-V built-ins compatible with LLVM-SPIRV-Translator, which
follows the approach taken OpenCL built-ins and AFAIK is used by SPIR-V backend
as well. I can upload it to Phabricator for review if needed.
>
> > HIP code expansion
>
> Downstream SYCL implementation [3] supports all the features listed in this
section except DSM allocations + quite a few other feature specific to SYCL and
we are also doing post-link processing before SPIR-V translation. It would be
very interesting to see how we can merge our implementations.
>
> Thanks,
> Alexey
>
> [1] https://github.com/intel/llvm/issues/3534
> [2] https://github.com/intel/llvm/pull/3929
> [3] https://github.com/intel/llvm/tree/sycl


-- 
BR,
Henry Linjamäki