how come <%= 19043.0 - 19032.6 %> display 10.4000000000015 how can I avoid this and get a proper result? Thanks --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
On 9/26/06, rosoft2001 <adingod-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > > > how come <%= 19043.0 - 19032.6 %>It''s a floating point error. See here: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems> display 10.4000000000015 > > how can I avoid this and get a proper result? > > ThanksUse integer arithmetic instead. For example, use cents instead of dollars by multiplying by 100 and converting to Integer. For other numbers you will need to decide what precision you need. 4 decimal places? Multiply by 10,000. Cheers, Dave --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
On 9/25/06, rosoft2001 <adingod-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > > > > how come <%= 19043.0 - 19032.6 %> > > display 10.4000000000015 > > how can I avoid this and get a proper result? >Look up bigdecimal http://www.ruby-doc.org/stdlib/ John W Higgins wishdev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
On 9/26/06, John Higgins <wishdev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On 9/25/06, rosoft2001 <adingod-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > > > > how come <%= 19043.0 - 19032.6 %> > > > > display 10.4000000000015 > > > > how can I avoid this and get a proper result? > > > > Look up bigdecimal > > http://www.ruby-doc.org/stdlib/ >You''ll also need edge rails if you plan on storing accurate values as decimal/numeric in your db. Isak> John W Higgins > wishdev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org > > > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---