llvm dev - Aug 2018 - [RFC][SVE] Supporting SIMD instruction sets with variable vector lengths

On 08/01/2018 02:00 PM, Graham Hunter wrote:
> Hi Hal,
>
>> On 30 Jul 2018, at 20:10, Hal Finkel <hfinkel at anl.gov> wrote:
>>
>>
>> On 07/30/2018 05:34 AM, Chandler Carruth wrote:
>>> I strongly suspect that there remains widespread concern with the
direction of this, I know I have them.
>>>
>>> I don't think that many of the people who have that concern
have had time to come back to this RFC and make progress on it, likely because
of other commitments or simply the amount of churn around SVE related patches
and such. That is at least why I haven't had time to return to this RFC and
try to write more detailed feedback.
>>>
>>> Certainly, I would want to see pretty clear and considered support
for this change to the IR type system from Hal, Chris, Eric and/or other long
time maintainers of core LLVM IR components before it moves forward, and I
don't see that in this thread.
>> At a high level, I'm happy with this approach. I think it will be
important for LLVM to support runtime-determined vector lengths - I see the
customizability and power-efficiency constraints that motivate these designs
continuing to increase in importance. I'm still undecided on whether this
makes vector code nicer even for fixed-vector-length architectures, but some of
the design decisions that it forces, such as having explicit intrinsics for
reductions and other horizontal operations, seem like the right direction
regardless.
> Thanks, that's good to hear.
>
>> 1.
>>> This is a proposal for how to deal with querying the size of
scalable types for
>>>> analysis of IR. While it has not been implemented in full,
>> Is this still true? The details here need to all work out, obviously,
and we should make sure that any issues are identified.
> Yes. I had hoped to get some more comments on the basic approach before
progressing with the implementation, but if it makes more sense to have the
implementation available to discuss then I'll start creating patches.
At least on this point, I think that we'll want to have the
implementation to help make sure there aren't important details we're
overlooking.
>
>> 2. I know that there has been some discussion around support for
changing the vector length during program execution (e.g., to account for some
(proposed?) RISC-V feature), perhaps even during the execution of a single
function. I'm very concerned about this idea because it is not at all clear
to me how to limit information transfer contaminated with the vector size from
propagating between different regions. As a result, I'm concerned about
trying to add this on later, and so if this is part of the plan, I think that we
need to think through the details up front because it could have a major impact
on the design.
> I think Robin's email yesterday covered it fairly nicely; this RFC
proposes that the hardware length of vectors will be consistent throughout an
entire function, so we don't need to limit information inside a function,
just between them. For SVE, h/w vector length will likely be consistent across
the whole program as well (assuming the programmer doesn't make a prctl call
to the kernel to change it) so we could drop that limit too, but I thought it
best to come up with a unified approach that would work for both architectures.
The 'inherits_vscale' attribute would allow us to continue optimizing
across functions for SVE where desired.
I think that this will likely work, although I think we want to invert
the sense of the attribute. vscale should be inherited by default, and
some attribute can say that this isn't so. That same attribute, I
imagine, will also forbid scalable vector function arguments and return
values on those functions. If we don't have inherited vscale as the
default, we place an implicit contract on any IR transformation hat
performs outlining that it needs to scan for certain kinds of vector
operations and add the special attribute, or just always add this
special attribute, and that just becomes another special case, which
will only actually manifest on certain platforms, that it's best to avoid.
>
> Modelling the dynamic vector length for RVV is something for Robin (or
others) to tackle later, but can be though of (at a high level) as an implicit
predicate on all operations.
My point is that, while there may be some sense in which the details can
be worked out later, we need to have a good-enough understanding of how
this will work now in order to make sure that we're not making design
decisions now that make handling the dynamic vscale in a reasonable way
later more difficult.

Thanks again,
Hal
>
> -Graham
>
>> Thanks again,
>> Hal
>>
>> -- 
>> Hal Finkel
>> Lead, Compiler Technology and Programming Languages
>> Leadership Computing Facility
>> Argonne National Laboratory
>>
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

On 1 August 2018 at 21:43, Hal Finkel <hfinkel at anl.gov>
wrote:
>
> On 08/01/2018 02:00 PM, Graham Hunter wrote:
>> Hi Hal,
>>
>>> On 30 Jul 2018, at 20:10, Hal Finkel <hfinkel at anl.gov>
wrote:
>>>
>>>
>>> On 07/30/2018 05:34 AM, Chandler Carruth wrote:
>>>> I strongly suspect that there remains widespread concern with
the direction of this, I know I have them.
>>>>
>>>> I don't think that many of the people who have that concern
have had time to come back to this RFC and make progress on it, likely because
of other commitments or simply the amount of churn around SVE related patches
and such. That is at least why I haven't had time to return to this RFC and
try to write more detailed feedback.
>>>>
>>>> Certainly, I would want to see pretty clear and considered
support for this change to the IR type system from Hal, Chris, Eric and/or other
long time maintainers of core LLVM IR components before it moves forward, and I
don't see that in this thread.
>>> At a high level, I'm happy with this approach. I think it will
be important for LLVM to support runtime-determined vector lengths - I see the
customizability and power-efficiency constraints that motivate these designs
continuing to increase in importance. I'm still undecided on whether this
makes vector code nicer even for fixed-vector-length architectures, but some of
the design decisions that it forces, such as having explicit intrinsics for
reductions and other horizontal operations, seem like the right direction
regardless.
>> Thanks, that's good to hear.
>>
>>> 1.
>>>> This is a proposal for how to deal with querying the size of
scalable types for
>>>>> analysis of IR. While it has not been implemented in full,
>>> Is this still true? The details here need to all work out,
obviously, and we should make sure that any issues are identified.
>> Yes. I had hoped to get some more comments on the basic approach before
progressing with the implementation, but if it makes more sense to have the
implementation available to discuss then I'll start creating patches.
>
> At least on this point, I think that we'll want to have the
> implementation to help make sure there aren't important details
we're
> overlooking.
+1
>>
>>> 2. I know that there has been some discussion around support for
changing the vector length during program execution (e.g., to account for some
(proposed?) RISC-V feature), perhaps even during the execution of a single
function. I'm very concerned about this idea because it is not at all clear
to me how to limit information transfer contaminated with the vector size from
propagating between different regions. As a result, I'm concerned about
trying to add this on later, and so if this is part of the plan, I think that we
need to think through the details up front because it could have a major impact
on the design.
>> I think Robin's email yesterday covered it fairly nicely; this RFC
proposes that the hardware length of vectors will be consistent throughout an
entire function, so we don't need to limit information inside a function,
just between them. For SVE, h/w vector length will likely be consistent across
the whole program as well (assuming the programmer doesn't make a prctl call
to the kernel to change it) so we could drop that limit too, but I thought it
best to come up with a unified approach that would work for both architectures.
The 'inherits_vscale' attribute would allow us to continue optimizing
across functions for SVE where desired.
>
> I think that this will likely work, although I think we want to invert
> the sense of the attribute. vscale should be inherited by default, and
> some attribute can say that this isn't so. That same attribute, I
> imagine, will also forbid scalable vector function arguments and return
> values on those functions. If we don't have inherited vscale as the
> default, we place an implicit contract on any IR transformation hat
> performs outlining that it needs to scan for certain kinds of vector
> operations and add the special attribute, or just always add this
> special attribute, and that just becomes another special case, which
> will only actually manifest on certain platforms, that it's best to
avoid.
It's a real relief to hear that you think this "will likely work".

Inverting the attribute seems good to me. I probably proposed not
inheriting by default because that's the default on RISC-V, but your
rationale is convincing.
>>
>> Modelling the dynamic vector length for RVV is something for Robin (or
others) to tackle later, but can be though of (at a high level) as an implicit
predicate on all operations.
>
> My point is that, while there may be some sense in which the details can
> be worked out later, we need to have a good-enough understanding of how
> this will work now in order to make sure that we're not making design
> decisions now that make handling the dynamic vscale in a reasonable way
> later more difficult.
Sorry if I'm a broken record, but I believe Graham was referring to
the _active vector length_ or VL here, which has nothing to do with
vscale, dynamic or not. I described earlier why I think the former
doesn't interact with the contents of this RFC in any interesting way.
If you think otherwise, could you elaborate on why you think that?


Cheers,
Robin
> Thanks again,
> Hal
>
>>
>> -Graham
>>
>>> Thanks again,
>>> Hal
>>>
>>> --
>>> Hal Finkel
>>> Lead, Compiler Technology and Programming Languages
>>> Leadership Computing Facility
>>> Argonne National Laboratory
>>>
>
> --
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory
>

On 08/01/2018 03:09 PM, Robin Kruppe wrote:
> ...
>> I think that this will likely work, although I think we want to invert
>> the sense of the attribute. vscale should be inherited by default, and
>> some attribute can say that this isn't so. That same attribute, I
>> imagine, will also forbid scalable vector function arguments and return
>> values on those functions. If we don't have inherited vscale as the
>> default, we place an implicit contract on any IR transformation hat
>> performs outlining that it needs to scan for certain kinds of vector
>> operations and add the special attribute, or just always add this
>> special attribute, and that just becomes another special case, which
>> will only actually manifest on certain platforms, that it's best to
avoid.
> It's a real relief to hear that you think this "will likely
work".
>
> Inverting the attribute seems good to me. I probably proposed not
> inheriting by default because that's the default on RISC-V, but your
> rationale is convincing.
>
>>> Modelling the dynamic vector length for RVV is something for Robin
(or others) to tackle later, but can be though of (at a high level) as an
implicit predicate on all operations.
>> My point is that, while there may be some sense in which the details
can
>> be worked out later, we need to have a good-enough understanding of how
>> this will work now in order to make sure that we're not making
design
>> decisions now that make handling the dynamic vscale in a reasonable way
>> later more difficult.
> Sorry if I'm a broken record, but I believe Graham was referring to
> the _active vector length_ or VL here, which has nothing to do with
> vscale, dynamic or not. I described earlier why I think the former
> doesn't interact with the contents of this RFC in any interesting way.
> If you think otherwise, could you elaborate on why you think that?
Was it decided that this issue is equivalent to, or a subset of, 
per-lane predication on load, stores, and similar? Or is it different?

Thanks again,
Hal
>
>
> Cheers,
> Robin
>
>> Thanks again,
>> Hal
>>
>>> -Graham
>>>
>>>> Thanks again,
>>>> Hal
>>>>
>>>> --
>>>> Hal Finkel
>>>> Lead, Compiler Technology and Programming Languages
>>>> Leadership Computing Facility
>>>> Argonne National Laboratory
>>>>
>> --
>> Hal Finkel
>> Lead, Compiler Technology and Programming Languages
>> Leadership Computing Facility
>> Argonne National Laboratory
>>
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory