Displaying 1 result from an estimated 1 matches for "incresases".
Did you mean:
increases
2001 Jan 31
1
Reduced Numerical Precision in 1.2.x?
...es subtracting
relatively large numbers from each other.
For the case I am looking at (see code below), the abcnon parameters bhat
and cq should exactly equal 0.0. Version 1.1.1 did calculate these
parameters to be close to 0.0, but for versions 1.2.x the errors increase
as the size of the problem incresases:
|----------------------+---------------+---------------|
|Case |Version 1.1.1 |Version 1.2.x |
| | | |
|----------------------+---------------+---------------|
|test2.abc(10,20,10) |bhat = 5.6e-10 |bhat = 5.5e-9 |
|...