llvm dev - Dec 2020 - RFC: [GlobalISel] Representing fp types in LLT

For a while now there have been discussions here and there about
representing floating point types explicitly in LLT. Having this type
information explicitly makes RegBankSelect easier to implement, avoids
inefficiencies in disambiguating which operation is needed to lower
various operations, and is necessary for correctness in light of variant
floating point types like bfloat.

I believe there is general consensus that we need this, so this RFC is
about how we should go about modeling these types.

At the dev meeting, we discussed two approaches IIRC:

1. Explicitly encode the number of bits in the exponent and mantissa for
   flexibility
2. Encode only the types we need (ie, model the same fp types as
   llvm::Type does)

The idea with (1) was that this way we could easily handle IEEE-754
binary formats and variants like bfloat16 and tf32 without needing to
burden ISel with knowledge of specifics. The major drawback of this is
that it would struggle to represent types that don't fit the general
mold, like ppc-fp128 (which is two doubles) or the IEEE 754-2008 decimal
formats.

I think (2) is the safer way forward, and I propose that we can
implement it using only 2 extra bits in each of the scalar and vector
variants of LLT. This approach largely hinges on the fact that there are
very few interesting (that is, actively used) variants of floating point
for a given scalar size.

The two bits would be used like so:
```
enum FPInfo { NotFP = 0x0, FP = 0x1, Reserved = 0x2, VariantFP = 0x3 };
```

This uses one bit to say a scalar is floating point, which, when set
will imply that it's the "usual" format for the size, and a second
bit
to say it's a variant for that size. There's space to handle two
different variants for a size in the `Reserved` case if we need it
later.

With this, we can model every floating point type that llvm IR does
today (half, bfloat16, single, double, x86-fp80, fp128, ppc-fp128), and
is easily extensible for things like fp256, tensorfloat, or IEEE-754
decimal floats should llvm need to support those explicitly in the
future.

Backends will need to be updated to handle these extra types. Notably,
the legalizer will need to be made more precise for operations like fadd
and isel will need to be aware of when any scalar will do or if the
distinction is important.

I've attached a patch for demonstration that adds fpscalar, bfloat, and
ppcf128 and implements printing, parsing, and conversion from llvm
IR. If folks are happy with this direction I'll start working on the
backend updates to get this fully working.

WDYT?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: llt-float.patch
Type: text/x-patch
Size: 17006 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201208/e118ec56/attachment.bin>

On Tue, 8 Dec 2020 at 19:04, Justin Bogner via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> For a while now there have been discussions here and there about
> representing floating point types explicitly in LLT. Having this type
> information explicitly makes RegBankSelect easier to implement, avoids
> inefficiencies in disambiguating which operation is needed to lower
> various operations, and is necessary for correctness in light of variant
> floating point types like bfloat.
>
> I believe there is general consensus that we need this, so this RFC is
> about how we should go about modeling these types.
>
> At the dev meeting, we discussed two approaches IIRC:
>
> 1. Explicitly encode the number of bits in the exponent and mantissa for
>    flexibility
> 2. Encode only the types we need (ie, model the same fp types as
>    llvm::Type does)
>
> The idea with (1) was that this way we could easily handle IEEE-754
> binary formats and variants like bfloat16 and tf32 without needing to
> burden ISel with knowledge of specifics. The major drawback of this is
> that it would struggle to represent types that don't fit the general
> mold, like ppc-fp128 (which is two doubles) or the IEEE 754-2008 decimal
> formats.
>
> I think (2) is the safer way forward
I strongly agree!

Jay.

+Tim

Thanks for pushing this forward Justin.
> On Dec 8, 2020, at 11:02 AM, Justin Bogner <mail at justinbogner.com>
wrote:
> 
> For a while now there have been discussions here and there about
> representing floating point types explicitly in LLT. Having this type
> information explicitly makes RegBankSelect easier to implement, avoids
> inefficiencies in disambiguating which operation is needed to lower
> various operations, and is necessary for correctness in light of variant
> floating point types like bfloat.
> 
> I believe there is general consensus that we need this, so this RFC is
> about how we should go about modeling these types.
> 
> At the dev meeting, we discussed two approaches IIRC:
> 
> 1. Explicitly encode the number of bits in the exponent and mantissa for
>   flexibility
> 2. Encode only the types we need (ie, model the same fp types as
>   llvm::Type does)
> 
> The idea with (1) was that this way we could easily handle IEEE-754
> binary formats and variants like bfloat16 and tf32 without needing to
> burden ISel with knowledge of specifics. The major drawback of this is
> that it would struggle to represent types that don't fit the general
> mold, like ppc-fp128 (which is two doubles) or the IEEE 754-2008 decimal
> formats.
> 
> I think (2) is the safer way forward, and I propose that we can
> implement it using only 2 extra bits in each of the scalar and vector
> variants of LLT. This approach largely hinges on the fact that there are
> very few interesting (that is, actively used) variants of floating point
> for a given scalar size.
This argument seems reasonable to me.
> 
> The two bits would be used like so:
> ```
> enum FPInfo { NotFP = 0x0, FP = 0x1, Reserved = 0x2, VariantFP = 0x3 };
> ```
> 
> This uses one bit to say a scalar is floating point, which, when set
> will imply that it's the "usual" format for the size, and a
second bit
> to say it's a variant for that size. There's space to handle two
> different variants for a size in the `Reserved` case if we need it
> later.
> 
> With this, we can model every floating point type that llvm IR does
> today (half, bfloat16, single, double, x86-fp80, fp128, ppc-fp128), and
> is easily extensible for things like fp256, tensorfloat, or IEEE-754
> decimal floats should llvm need to support those explicitly in the
> future.
> 
> Backends will need to be updated to handle these extra types. Notably,
> the legalizer will need to be made more precise for operations like fadd
> and isel will need to be aware of when any scalar will do or if the
> distinction is important.
I’d like to elaborate further here on what the impacts would on the GISel
pipeline as a whole.

E.g. RegBankSelect. On targets like AArch64 with GPR and FPR banks, we currently
use RBS to piece together int/fp information using the instruction’s surrounding
context. With these types, this would be much simpler. Having said that, does
this effectively render the entire notion of RegBankSelect
redundant?
> 
> I've attached a patch for demonstration that adds fpscalar, bfloat, and
> ppcf128 and implements printing, parsing, and conversion from llvm
> IR. If folks are happy with this direction I'll start working on the
> backend updates to get this fully working.
> 
> WDYT?
> 
> <llt-float.patch>

> On Dec 8, 2020, at 14:02, Justin Bogner <mail at justinbogner.com>
wrote:
> 
> For a while now there have been discussions here and there about
> representing floating point types explicitly in LLT. Having this type
> information explicitly makes RegBankSelect easier to implement, avoids
> inefficiencies in disambiguating which operation is needed to lower
> various operations, and is necessary for correctness in light of variant
> floating point types like bfloat.
> 
> I believe there is general consensus that we need this, so this RFC is
> about how we should go about modeling these types.
> 
> At the dev meeting, we discussed two approaches IIRC:
> 
> 1. Explicitly encode the number of bits in the exponent and mantissa for
>   flexibility
> 2. Encode only the types we need (ie, model the same fp types as
>   llvm::Type does)
> 
Yes, I think encoding the FP mode would be overkill. I do think we shouldn’t go
too far into treating these as FP types mirroring the IR though. I would still
like to be able to directly operate on these with integer operations without
intermediate bitcasts. Only lowerings which really care about the FP semantics
would need to treat these differently.

I do think threading these through all the existing legalizer code could be
painful, since quite a few places create out of context scalar types whereas now
they would need to take care to pass through the FP-bits.

-Matt