Displaying 1 result from an estimated 1 matches for "0x19999ap".
2016 Mar 02
2
Incorrect return values for APFloat::convertFromString?
I noticed some odd behavior with APFloat's convertFromString method.
1. If I pass the hex representation of the closest value to 0.1 (0x19999Ap-24), everything is fine and opOk is returned. However, if I pass the same value as a decimal string (0.10000002384185791015625), opInexact is set.
2. On the lower end of the scale, the smallest denormal 0x1p-149 returns opOk, but the slightly larger value 0x1.8p-149 has opUnderflow set and (...