llvm dev - Apr 2016 - Integer -> Floating point -> Integer cast optimizations

I brought this up in IRC and was told to consult someone who knows more about
floating point numbers; I propose an optimization as follows.

Let's say we have an int x, and we cast it to a float and back. Floats have
8 exponent bits and 23 mantissa bits.

If x matches the condition `countTrailingZeros(abs(x)) > (log2(abs(x)) -
23)`, then we can remove the float casts.

So, if we can establish that abs(x) is <= 2**23, we can remove the casts.
LLVM does not currently perform that optimization on this C code:

int floatcast(int x) {
    if (abs(x) <= 16777216) { // abs(x) is definitely <= 2**23 and fits
into our mantissa cleanly
        float flt = (float)x;
        return (int)flt;
    }
    return x;
}

Things get more interesting when you bring in higher integers and leading zeros.
Floating point can't exactly represent integers that don't fit neatly
into the mantissa; they have to round to a multiple of some power of 2. For
example, integers between 2**23 and 2**24 round to a multiple of 2**1 - meaning
that the result has *at least* 1 trailing zero. Integers between 2**24 and 2**25
round to a multiple of 2**2 - with the result having at least 2 trailing zeros.
Et cetera. If we can prove that the input to these casts fits in between one of
those ranges *and* has at least the correct number of leading zeros, we can
eliminate the casts. LLVM does not currently perform this optimization on this C
code:

int floatcast(int x) {
    if (16777217 <= abs(x) && abs(x) <= 33554432) { // abs(x) is
definitely between 2**23 and 2**24
        float flt = (float)(x / abs(x) * (abs(x) & (UINT32_MAX ^ 2))); //
what's being casted to float definitely has at least one trailing zero in
its absolute value
        return (int)flt;
    }
    return x;
}


 - CL

Yes, this is all correct.  What’s the actual question?

– Steve
> On Apr 14, 2016, at 11:13 AM, Carlos Liam via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> I brought this up in IRC and was told to consult someone who knows more
about floating point numbers; I propose an optimization as follows.
> 
> Let's say we have an int x, and we cast it to a float and back. Floats
have 8 exponent bits and 23 mantissa bits.
> 
> If x matches the condition `countTrailingZeros(abs(x)) > (log2(abs(x)) -
23)`, then we can remove the float casts.
> 
> So, if we can establish that abs(x) is <= 2**23, we can remove the
casts. LLVM does not currently perform that optimization on this C code:
> 
> int floatcast(int x) {
>    if (abs(x) <= 16777216) { // abs(x) is definitely <= 2**23 and
fits into our mantissa cleanly
>        float flt = (float)x;
>        return (int)flt;
>    }
>    return x;
> }
> 
> Things get more interesting when you bring in higher integers and leading
zeros. Floating point can't exactly represent integers that don't fit
neatly into the mantissa; they have to round to a multiple of some power of 2.
For example, integers between 2**23 and 2**24 round to a multiple of 2**1 -
meaning that the result has *at least* 1 trailing zero. Integers between 2**24
and 2**25 round to a multiple of 2**2 - with the result having at least 2
trailing zeros. Et cetera. If we can prove that the input to these casts fits in
between one of those ranges *and* has at least the correct number of leading
zeros, we can eliminate the casts. LLVM does not currently perform this
optimization on this C code:
> 
> int floatcast(int x) {
>    if (16777217 <= abs(x) && abs(x) <= 33554432) { // abs(x)
is definitely between 2**23 and 2**24
>        float flt = (float)(x / abs(x) * (abs(x) & (UINT32_MAX ^ 2)));
// what's being casted to float definitely has at least one trailing zero in
its absolute value
>        return (int)flt;
>    }
>    return x;
> }
> 
> - CL