search for: extended_precision

Displaying 1 result from an estimated 1 matches for "extended_precision".

2018 Feb 26
3
Precision in R
Hi, Why sum() on a 10-item vector produces a different value than its counterpart on a 2-item vector? I understand the problems related to the arithmetic precision in storing decimal numbers in binary format, but shouldn't the errors be equal regardless of the method used? See my example: > options(digits=22) > x=rep(.1,10) > x [1] 0.10000000000000001 0.10000000000000001