Displaying 1 result from an estimated 1 matches for "17digit".
Did you mean:
10digit
2018 Feb 26
3
Precision in R
Hi,
Why sum() on a 10-item vector produces a different value than its
counterpart on a 2-item vector? I understand the problems related to
the arithmetic precision in storing decimal numbers in binary format,
but shouldn't the errors be equal regardless of the method used?
See my example:
> options(digits=22)
> x=rep(.1,10)
> x
[1] 0.10000000000000001 0.10000000000000001