Ok, I am sure this is a simplistic problem that I am just missing right now (maybe eyes are crossed??). However, I am trying to do simply math here. I have these variables initialized as follows: @amount_due = @amount_paid = @total_due = 0.0 I get @total_due and it is set to 1739.6 from the database. I then get @amount_paid and it is set to 1739.6 from the database. I then calculate the @amount_due = @total_due - @amount_paid. This value is 2.27373675443232e-013. ???? Sorry if this is on the list somewhere and I missed it. Thanks in advance for any help. ~damon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060412/bffde54c/attachment.html
Chris Schumann
2006-Apr-12 20:26 UTC
[Rails] Problems doing addition/subtraction with floats
> Date: Wed, 12 Apr 2006 14:52:54 -0500 > From: "Damon Hill" <mdamonhill@gmail.com>> Ok, I am sure this is a simplistic problem that I am just missing right now > (maybe eyes are crossed??). > However, I am trying to do simply math here. > > I have these variables initialized as follows: > @amount_due = @amount_paid = @total_due = 0.0 > > I get @total_due and it is set to 1739.6 from the database. > I then get @amount_paid and it is set to 1739.6 from the database. > I then calculate the @amount_due = @total_due - @amount_paid. > > This value is 2.27373675443232e-013.This isn''t a Rails issue: it''s a Ruby issue, and happens because 1739.6 cannot be stored exactly as a float or double (nor can 0.1). Subtracting numbers that are close to each other can often introduce this kind of error. Changing the way numbers are stored would be a huge amount of work. The easiest way to fix it is to round the result to whatever you''re working with. (for two decimal places) @amount_due = int(0.5 + 100.0 * (@total_due - @amount_paid)) / 100.0 or just print the result to two places... but that won''t help if you''re comparing to zero. Chris
Chris-- I figured that it was a Ruby issue and I apologize for posting on this list. I appreciate the reply. Thanks, ~damon On 4/12/06, Chris Schumann <chris@idlelion.net> wrote:> > > Date: Wed, 12 Apr 2006 14:52:54 -0500 > > From: "Damon Hill" <mdamonhill@gmail.com> > > > Ok, I am sure this is a simplistic problem that I am just missing right > now > > (maybe eyes are crossed??). > > However, I am trying to do simply math here. > > > > I have these variables initialized as follows: > > @amount_due = @amount_paid = @total_due = 0.0 > > > > I get @total_due and it is set to 1739.6 from the database. > > I then get @amount_paid and it is set to 1739.6 from the database. > > I then calculate the @amount_due = @total_due - @amount_paid. > > > > This value is 2.27373675443232e-013. > > This isn''t a Rails issue: it''s a Ruby issue, and happens because 1739.6 > cannot be stored exactly as a float or double (nor can 0.1). Subtracting > numbers that are close to each other can often introduce this kind of > error. Changing the way numbers are stored would be a huge amount of work. > > The easiest way to fix it is to round the result to whatever you''re > working with. > > (for two decimal places) > @amount_due = int(0.5 + 100.0 * (@total_due - @amount_paid)) / 100.0 > > or just print the result to two places... but that won''t help if you''re > comparing to zero. > > Chris > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060413/ab7513e0/attachment.html
Reasonably Related Threads
- UserEngine - rake bootstrap aborted => undefined method `synchronize_with_controllers'' for Permission:Class
- Best practice thoughts: Model helpers, mocks
- WEBrick just decided to take a hiatus on me
- ajax doesn''t show at the right place
- Re:(RESOLVED, KIND OF) auto complete and after update element