search for: precision_arithmetic

I'm using .Call() to call C code from R under Windows (on an Intel Core 2 duo). The C code involves some very small numbers, and I think I'm losing precision using doubles. I thought I might use long doubles to see if I can get that precision back. I have a few questions: 1. Does this affect the portability to other OSs or processors? 2. I'm returning the results in a matrix. Will

Any idea why this calculates the integer the way it does? irb> ("291.15".to_f * 100.0).to_i => 29114 Thanks, Tom -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to