[llvm-dev] [RFC] Goals for VPlan-based cost modelling

Florian Hahn via llvm-dev
Fri, 30 Oct 2020 11:38:23 -0000

Hi,

> On Oct 30, 2020, at 10:08, Anna Sophia Welker via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>
> Hi all,
>
> I am looking into the benefits of a VPlan-based cost model, and into how
such a cost model should be implemented to make the most out of these benefits.
>
> Over the last year I have been working with LLVM, mainly focused on the ARM
backend, in the course of a one-year internship at Arm Ltd. My main project from
December 2019 to August 2020 was to introduce gather/scatters for MVE
auto-vectorization. One of the recurring challenges during this work was to get
things right in the cost model.
> For example, gathers can extend the data they load, while scatters can
truncate their input, meaning that an extend following the load, or a truncate
preceding the store, is for free if it meets certain type conditions. As the
current cost model is not designed for context-based analysis, this was a pain
to model.
>


One goal is a more modular approach to cost-modeling and transformations. The
current cost-model is quite monolithic and keeps some context information in
various tables, which is hard to reason about and also hard to change and
extend. One idea with VPlan is that cost decisions could be materialized in the
VPlans directly by transformations working in concert to find the best overall
strategy.

I think the example with gathers that extend the loaded value illustrates that
quite well. In the legacy cost model, you could maintain a set of values that
are free to extend and add the loaded values to it. When visiting a zext, return
cost of zero for those.

With VPlan, instead you could add a new VPInstruction opcode ZextLoad and have a
transformation that replaces all `zext (load)` instructions with the new
ZExtLoad one. The cost-model needs to be taught about this special instruction
and how much it costs. Then you could apply this transformation to an existing
plan and check if the overall cost improved.

Of course there’s also the issue of how to generalize the TTI APIs to allow for
computing the cost of instructions with more context.


> I have done some research and found out that one of the proposed benefits
of VPlan is that a new cost model making use of it would be able to better
support context-dependent decisions like this.
> However, there does not exist much specification about what such a cost
model should look like.
>
> Also, I have read through the respective code to understand how loop
vectorization is currently done and how far the introduction of VPlan has
progressed and have realised that the only recipe that actually handles more
than one instruction from the input IR is the one for interleaved groups. When
the VPlan is generated on the VPlan-native path, every IR instruction is
considered and transformed into a recipe separately, ignoring its context (to
give a code location, I am looking at
VPlanTransforms::VPInstructionsToVPRecipes).
> And maybe there are architectures that for some cases do not have the same
vector instructions, so a pattern that works great for one could be useless for
others. So I am wondering: Is there any plan to have target-dependent flavours
of recipes, or how will those things be handled?

I think we probably want to keep things as generic as possible, try to model
generic concepts and use TTI to decide whether to use it or not (e.g. see how
masked loads/stores are handled).

>
> Right now it makes sense that nothing like this has been implemented yet,
as there is no cost model that could guide the transformation. But if recipes
are a general thing, should the cost model be the component actually providing
the target-specific pattern for a recipe, together with its cost?
>

I am not sure what patterns specifically you are thinking about, but I think the
cost model should just evaluate the cost of a given plan and not provide
anything beyond that. Of course, this still can mean that there might be certain
recipes/instructions that are not available/profitable on some targets and we
decide to never generate them, based on the cost-information.

> I am considering choosing a topic related to VPlan, possibly cost
modelling, for my Master thesis, with the goal to present a solution and
implement a prototype.
>

I am hoping to make some progress on this in the next months (hopefully the work
on modeling the def-use chains between recipes in VPlan will be wrapped up soon)
and I expect there to be a few moving parts. Not sure what that means for a
master thesis in this area.


Cheers,
Florian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201030/2590d8c3/attachment.html>