Displaying 2 results from an estimated 2 matches for "dpkreil".
2009 Jun 19
3
Floating point precision / guard digits? (PR#13771)
Full_Name: D Kreil
Version: 2.8.1 and 2.9.0
OS: Debian Linux
Submission from: (NULL) (141.244.140.179)
Group: Accuracy
I understand that most floating point numbers are approximated due to their
binary storage. On the other hand, I thought that modern math CPUs used guard
digits to protect against trivial underflows. Not true?
# integers, no problem
> 1+1+1==3
[1] TRUE
# binary floating
2009 Jun 19
3
Floating point precision / guard digits? (PR#13771)
Full_Name: D Kreil
Version: 2.8.1 and 2.9.0
OS: Debian Linux
Submission from: (NULL) (141.244.140.179)
Group: Accuracy
I understand that most floating point numbers are approximated due to their
binary storage. On the other hand, I thought that modern math CPUs used guard
digits to protect against trivial underflows. Not true?
# integers, no problem
> 1+1+1==3
[1] TRUE
# binary floating