llvm dev - Jun 2019 - RFC: Interface user provided vector functions with the vectorizer.

For example, Type 2 case, scalar-foo used call by value while vector-foo used
call by ref. The question Johannes is asking is whether we can decipher that
after the fact, only by looking at the two function signatures, or need some
more info (what kind, what's minimal)? I think we need to list up cases of
interest, and for each vector ABI of interest, we need to work on the
requirements and determine whether deciphering after the fact is feasible.

I think we can make further progress on trivial cases (where FE doesn't
"change" type) while we continue working out the details on
non-trivial cases.

Thanks,
Hideki

From: Doerfert, Johannes [mailto:jdoerfert at anl.gov]
Sent: Monday, June 24, 2019 9:21 AM
To: Francesco Petrogalli <Francesco.Petrogalli at arm.com>; Tian, Xinmin
<xinmin.tian at intel.com>
Cc: Saito, Hideki <hideki.saito at intel.com>; Simon Moll <moll at
cs.uni-saarland.de>; LLVM Development List <llvm-dev at
lists.llvm.org>; Clang Dev <cfe-dev at lists.llvm.org>; Renato Golin
<rengolin at gmail.com>; Finkel, Hal J. <hfinkel at anl.gov>; Andrea
Bocci <andrea.bocci at cern.ch>; Elovikov, Andrei <andrei.elovikov at
intel.com>; Alexey Bataev <a.bataev at hotmail.com>; nd <nd at
arm.com>; Roman Lebedev <lebedev.ri at gmail.com>; Philip Reames
<listmail at philipreames.com>; Shawn Landden <slandden at
gmail.com>
Subject: Re: RFC: Interface user provided vector functions with the vectorizer.

I mean, the FE will create only one of the 3 vector versions matching the one we
want for a given vector length, wouldn't it? The question now is: can we
with the scalar and one vector version correctly vectorize the call. If the
answer is no, what is the minimal amount of information, in addition to the two
version, we would need?
Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Francesco Petrogalli <Francesco.Petrogalli at
arm.com<mailto:Francesco.Petrogalli at arm.com>>
Sent: Monday, June 24, 2019 6:06:12 PM
To: Tian, Xinmin
Cc: Doerfert, Johannes; Saito, Hideki; Simon Moll; LLVM Development List; Clang
Dev; Renato Golin; Finkel, Hal J.; Andrea Bocci; Elovikov, Andrei; Alexey
Bataev; nd; Roman Lebedev; Philip Reames; Shawn Landden
Subject: Re: RFC: Interface user provided vector functions with the vectorizer.


> On Jun 24, 2019, at 10:53 AM, Tian, Xinmin <xinmin.tian at
intel.com<mailto:xinmin.tian at intel.com>> wrote:
>
> To me, it is also an issue related to SIMD signature matching when the
vectorizer kicks in. Losing info from FE to BE is not good in general.
>
Yes, we cannot loose such information. In particular, the three examples I
reported are all generating i64 in the scalar function signature:

// Type 1
typedef _Complex int S;

// Type 2
typedef struct x{
int a;
int b;
} S;

// Type 3
typedef uint64_t S;

S foo(S a, S b) {
return ...;
}


On AArch64, the correspondent vector function signature in the three cases would
be (for 2-lane unmasked vectorization):

// Type 1:

<4 x int> vectorized_foo(<4 x int>, <4 x int>)

// Type 2:

%a = type struct {I 32, i32}

<2 x %a* > vectorized_foo(<2 x %a*> , <2 x %a*>)

// Type 3:

<2 x i64> vectorized_foo(<2 x i64>, <2 x i64)

To make sure that the vectorizer knows how to map the scalar function parameters
to the vector ones, we have to make sure that the original signature 
information is stored somewhere.

I will work on this, and provide examples.

Suggestions are welcome.

Thank you

Francesco
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190624/737c6110/attachment.html>

> On Jun 24, 2019, at 12:20 PM, Saito, Hideki <hideki.saito at
intel.com> wrote:
> 
>  
> For example, Type 2 case, scalar-foo used call by value while vector-foo
used call by ref.

Yes, the call-by-ref is an important feature that can be used with linear
modifiers.
> The question Johannes is asking is whether we can decipher that after the
fact, only by looking at the two function signatures, or need some more info
(what kind, what’s minimal)?
In the draft implementation we have been working on, we came up with a ParamKind
enum that holds the following information:

enum class ParamKind 
{
    Vector,
    OMP_Linear,
    OMP_LinearRef,
    OMP_LinearVal,
    OMP_LinearUVal,
    OMP_LinearPos,
    OMP_LinearValPos,
    OMP_LinearRefPos,
    OMP_LinearUValPos,
    OMP_Uniform
};

The enum is used to classify the `ParameterType`, a class that is attached to
each parameter and describes things like uniformity, linearity (with and without
modifiers). The list of parameter types is then stored in the
VectorFunctionShape:

struct VectorFunctionShape {
            unsigned VF; // Vectorization factor 
            bool IsMasked;
            bool IsScalable;
            ISAKind ISA;
            std::vector<ParamType> Parameters;
};

Here OpenMP is used to classify the parameter types (OMP_*), but nothing
prevents the ParamKind and the VectorFunctionShape to be extended to be able to
handle other vector paradigms.

I think that we have to handle all 9 different linear and uniform cases in the
ParamKind separately, because there is no way to get such information from the
vector function shape.
> I think we need to list up cases of interest, and for each vector ABI of
interest, we need to work on the requirements and determine whether deciphering
after the fact is feasible.
>  

I will provide the cases for AArch64.

> I think we can make further progress on trivial cases (where FE doesn’t
“change” type) while we continue working out the details on non-trivial cases.
>  

I think we can agree to proceed these way. The SVFS and the VectorShapeInfo are
(I believe) designed with extendibility as a requirements. If we need more
metadata to represent some specific cases, we will extend the SVFS internal to
handle such metadata.

I will update the RFC by moving the vector function signature generation in the
FE.

Thank you everybody for their input, and for your patience. This is proving
harder than expected! :)

Francesco
> Thanks,
> Hideki
>  
> From: Doerfert, Johannes [mailto:jdoerfert at anl.gov] 
> Sent: Monday, June 24, 2019 9:21 AM
> To: Francesco Petrogalli <Francesco.Petrogalli at arm.com>; Tian,
Xinmin <xinmin.tian at intel.com>
> Cc: Saito, Hideki <hideki.saito at intel.com>; Simon Moll <moll at
cs.uni-saarland.de>; LLVM Development List <llvm-dev at
lists.llvm.org>; Clang Dev <cfe-dev at lists.llvm.org>; Renato Golin
<rengolin at gmail.com>; Finkel, Hal J. <hfinkel at anl.gov>; Andrea
Bocci <andrea.bocci at cern.ch>; Elovikov, Andrei <andrei.elovikov at
intel.com>; Alexey Bataev <a.bataev at hotmail.com>; nd <nd at
arm.com>; Roman Lebedev <lebedev.ri at gmail.com>; Philip Reames
<listmail at philipreames.com>; Shawn Landden <slandden at
gmail.com>
> Subject: Re: RFC: Interface user provided vector functions with the
vectorizer.
>  
> I mean, the FE will create only one of the 3 vector versions matching the
one we want for a given vector length, wouldn't it? The question now is: can
we with the scalar and one vector version correctly vectorize the call. If the
answer is no, what is the minimal amount of information, in addition to the two
version, we would need?
> 
> Get Outlook for Android
>  
> From: Francesco Petrogalli <Francesco.Petrogalli at arm.com>
> Sent: Monday, June 24, 2019 6:06:12 PM
> To: Tian, Xinmin
> Cc: Doerfert, Johannes; Saito, Hideki; Simon Moll; LLVM Development List;
Clang Dev; Renato Golin; Finkel, Hal J.; Andrea Bocci; Elovikov, Andrei; Alexey
Bataev; nd; Roman Lebedev; Philip Reames; Shawn Landden
> Subject: Re: RFC: Interface user provided vector functions with the
vectorizer.
>  
> 
> 
> > On Jun 24, 2019, at 10:53 AM, Tian, Xinmin <xinmin.tian at
intel.com> wrote:
> > 
> > To me, it is also an issue related to SIMD signature matching when the
vectorizer kicks in. Losing info from FE to BE is not good in general.
> >  
> 
> Yes, we cannot loose such information. In particular, the three examples I
reported are all generating i64 in the scalar function signature:
> 
> // Type 1
> typedef _Complex int S;
> 
> // Type 2 
> typedef struct x{
> int a;
> int b;
> } S;
> 
> // Type 3
> typedef uint64_t S;
> 
> S foo(S a, S b) {
> return ...;
> }
> 
> 
> On AArch64, the correspondent vector function signature in the three cases
would be (for 2-lane unmasked vectorization):
> 
> // Type 1:
> 
> <4 x int> vectorized_foo(<4 x int>, <4 x int>)
> 
> // Type 2:
> 
> %a = type struct {I 32, i32}
> 
> <2 x %a* > vectorized_foo(<2 x %a*> , <2 x %a*>)
> 
> // Type 3:
> 
> <2 x i64> vectorized_foo(<2 x i64>, <2 x i64)
> 
> To make sure that the vectorizer knows how to map the scalar function
parameters to the vector ones, we have to make sure that the original signature 
information is stored somewhere.
> 
> I will work on this, and provide examples.
> 
> Suggestions are welcome.
> 
> Thank you
> 
> Francesco

>Thank you everybody for their input, and for your patience. This is proving
harder than expected! :)
Thank you for doing the hard part of the work.

Hideki

-----Original Message-----
From: Francesco Petrogalli [mailto:Francesco.Petrogalli at arm.com] 
Sent: Monday, June 24, 2019 11:26 AM
To: Saito, Hideki <hideki.saito at intel.com>
Cc: Doerfert, Johannes <jdoerfert at anl.gov>; Tian, Xinmin
<xinmin.tian at intel.com>; Simon Moll <moll at cs.uni-saarland.de>;
LLVM Development List <llvm-dev at lists.llvm.org>; Clang Dev <cfe-dev
at lists.llvm.org>; Renato Golin <rengolin at gmail.com>; Finkel, Hal
J. <hfinkel at anl.gov>; Andrea Bocci <andrea.bocci at cern.ch>;
Elovikov, Andrei <andrei.elovikov at intel.com>; Alexey Bataev
<a.bataev at hotmail.com>; nd <nd at arm.com>; Roman Lebedev
<lebedev.ri at gmail.com>; Philip Reames <listmail at
philipreames.com>; Shawn Landden <slandden at gmail.com>
Subject: Re: RFC: Interface user provided vector functions with the vectorizer.


> On Jun 24, 2019, at 12:20 PM, Saito, Hideki <hideki.saito at
intel.com> wrote:
> 
>  
> For example, Type 2 case, scalar-foo used call by value while vector-foo
used call by ref.

Yes, the call-by-ref is an important feature that can be used with linear
modifiers.
> The question Johannes is asking is whether we can decipher that after the
fact, only by looking at the two function signatures, or need some more info
(what kind, what’s minimal)?
In the draft implementation we have been working on, we came up with a ParamKind
enum that holds the following information:

enum class ParamKind
{
    Vector,
    OMP_Linear,
    OMP_LinearRef,
    OMP_LinearVal,
    OMP_LinearUVal,
    OMP_LinearPos,
    OMP_LinearValPos,
    OMP_LinearRefPos,
    OMP_LinearUValPos,
    OMP_Uniform
};

The enum is used to classify the `ParameterType`, a class that is attached to
each parameter and describes things like uniformity, linearity (with and without
modifiers). The list of parameter types is then stored in the
VectorFunctionShape:

struct VectorFunctionShape {
            unsigned VF; // Vectorization factor 
            bool IsMasked;
            bool IsScalable;
            ISAKind ISA;
            std::vector<ParamType> Parameters; };

Here OpenMP is used to classify the parameter types (OMP_*), but nothing
prevents the ParamKind and the VectorFunctionShape to be extended to be able to
handle other vector paradigms.

I think that we have to handle all 9 different linear and uniform cases in the
ParamKind separately, because there is no way to get such information from the
vector function shape.
> I think we need to list up cases of interest, and for each vector ABI of
interest, we need to work on the requirements and determine whether deciphering
after the fact is feasible.
>  

I will provide the cases for AArch64.

> I think we can make further progress on trivial cases (where FE doesn’t
“change” type) while we continue working out the details on non-trivial cases.
>  

I think we can agree to proceed these way. The SVFS and the VectorShapeInfo are
(I believe) designed with extendibility as a requirements. If we need more
metadata to represent some specific cases, we will extend the SVFS internal to
handle such metadata.

I will update the RFC by moving the vector function signature generation in the
FE.

Thank you everybody for their input, and for your patience. This is proving
harder than expected! :)

Francesco
> Thanks,
> Hideki
>  
> From: Doerfert, Johannes [mailto:jdoerfert at anl.gov]
> Sent: Monday, June 24, 2019 9:21 AM
> To: Francesco Petrogalli <Francesco.Petrogalli at arm.com>; Tian,
Xinmin
> <xinmin.tian at intel.com>
> Cc: Saito, Hideki <hideki.saito at intel.com>; Simon Moll 
> <moll at cs.uni-saarland.de>; LLVM Development List 
> <llvm-dev at lists.llvm.org>; Clang Dev <cfe-dev at
lists.llvm.org>; Renato
> Golin <rengolin at gmail.com>; Finkel, Hal J. <hfinkel at
anl.gov>; Andrea
> Bocci <andrea.bocci at cern.ch>; Elovikov, Andrei 
> <andrei.elovikov at intel.com>; Alexey Bataev <a.bataev at
hotmail.com>; nd
> <nd at arm.com>; Roman Lebedev <lebedev.ri at gmail.com>;
Philip Reames
> <listmail at philipreames.com>; Shawn Landden <slandden at
gmail.com>
> Subject: Re: RFC: Interface user provided vector functions with the
vectorizer.
>  
> I mean, the FE will create only one of the 3 vector versions matching the
one we want for a given vector length, wouldn't it? The question now is: can
we with the scalar and one vector version correctly vectorize the call. If the
answer is no, what is the minimal amount of information, in addition to the two
version, we would need?
> 
> Get Outlook for Android
>  
> From: Francesco Petrogalli <Francesco.Petrogalli at arm.com>
> Sent: Monday, June 24, 2019 6:06:12 PM
> To: Tian, Xinmin
> Cc: Doerfert, Johannes; Saito, Hideki; Simon Moll; LLVM Development 
> List; Clang Dev; Renato Golin; Finkel, Hal J.; Andrea Bocci; Elovikov, 
> Andrei; Alexey Bataev; nd; Roman Lebedev; Philip Reames; Shawn Landden
> Subject: Re: RFC: Interface user provided vector functions with the
vectorizer.
>  
> 
> 
> > On Jun 24, 2019, at 10:53 AM, Tian, Xinmin <xinmin.tian at
intel.com> wrote:
> > 
> > To me, it is also an issue related to SIMD signature matching when the
vectorizer kicks in. Losing info from FE to BE is not good in general.
> >  
> 
> Yes, we cannot loose such information. In particular, the three examples I
reported are all generating i64 in the scalar function signature:
> 
> // Type 1
> typedef _Complex int S;
> 
> // Type 2
> typedef struct x{
> int a;
> int b;
> } S;
> 
> // Type 3
> typedef uint64_t S;
> 
> S foo(S a, S b) {
> return ...;
> }
> 
> 
> On AArch64, the correspondent vector function signature in the three cases
would be (for 2-lane unmasked vectorization):
> 
> // Type 1:
> 
> <4 x int> vectorized_foo(<4 x int>, <4 x int>)
> 
> // Type 2:
> 
> %a = type struct {I 32, i32}
> 
> <2 x %a* > vectorized_foo(<2 x %a*> , <2 x %a*>)
> 
> // Type 3:
> 
> <2 x i64> vectorized_foo(<2 x i64>, <2 x i64)
> 
> To make sure that the vectorizer knows how to map the scalar function
parameters to the vector ones, we have to make sure that the original signature 
information is stored somewhere.
> 
> I will work on this, and provide examples.
> 
> Suggestions are welcome.
> 
> Thank you
> 
> Francesco