similar to: ruby floating point calculations

Displaying 20 results from an estimated 10000 matches similar to: "ruby floating point calculations"

2010 Dec 23
36
Weird issue with converting floats to integer
Any idea why this calculates the integer the way it does? irb> ("291.15".to_f * 100.0).to_i => 29114 Thanks, Tom -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to
2006 Mar 31
6
BigDecimal + Float => Float?
I was just playing around in irb and noticed that while BigDecimal + Integer => BigDecimal, BigDecimal + Float => Float. Is there any reason for this? It seems inconsistent and means you have to check the class of any number you add to a BigDecimal, which looks very un-Ruby-like to me. Ashley
2006 Feb 09
11
Ruby on AIX?
We''re looking at big IBM hardware running AIX as a potential deployment platform for the big credit card processing Rails and Ruby stuff that my team is currently building. I was wondering if anyone else is running on AIX or knows if Ruby is well-supported on this platform. I googled the subject, but http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/3daaf60b012c1921 is
2006 Jul 30
34
ruby suitable for financial apps?
hi there, I am about to embark on a mission critical financial web application, and was wondering weather ruby on rails is the best fit for this type of project. More specifically I am worried about floating point errors. Would it be more appropriate to use a more statically typed framework (.NET)? thankyou. -- Posted via http://www.ruby-forum.com/.
2006 Jul 22
4
Reading floating points into a field stored as Integers
Hi, I''m trying to follow the "AWDR" example of storing certain types of numbers (in my case, percentages not money) as integers in the database. I understand from the Depot application how to format the output to have 2 places (or n places for that matter) decimal. Now, I"m wondering about the input. It''s more natural for people to enter numbers as
2006 Apr 09
16
Mapping to BigDecimals instead of Floats
Hi everyone, I''m new to Rails and Ruby, and have been working my way through the Pickaxe and Agile Rails books for the last week or so. With a background in payroll apps I found that the default ActiveRecord mapping of decimal columns to float variables really bothered me! Financial calcs need high levels of accuracy and floats just don''t give you that. I know there are a bunch
2008 Jul 12
3
calculations on nested resources
Hi, I can''t seem to find a solution to the folowing problem. I have 3 resources that are all bound to eachother like this: class Item < ActiveRecord::Base belongs_to :item_category // has a date and amount attr end class ItemCategory < ActiveRecord::Base has_many :items has_many :item_tops // groups the items in categories end class ItemTop < ActiveRecord::Base
2006 Jul 09
6
Float -> BigDecimal
So, I''m personally cool with this change (patch from #5454, applied in [4596]), and I''ll make it work for Oracle tomorrow. But just wondering if folks considered the performance impact of the change? From a simple test it seems that BigDecimal math is about twice as slow as using Floats.
2008 Nov 07
15
any tricks re using " eql(5.5)", but where 5.5 is a decimal not float?
-- Sent from my mobile device
2009 Feb 03
4
Ruby EE: Problem with BigDecimal
I have a strange problem with my setup of Passenger and Ruby Enterprise Edition. Sometimes the loading of BigDecimals from the database and session fails (silently) and the variable containing the BigDecimal is just empty. I cannot pinpoint where/when/how it happens, and it IS only periodic. I am using the newest versions of both Passenger and REE - I have even tried to run with latest commit
2009 Jun 19
3
Floating point precision / guard digits? (PR#13771)
Full_Name: D Kreil Version: 2.8.1 and 2.9.0 OS: Debian Linux Submission from: (NULL) (141.244.140.179) Group: Accuracy I understand that most floating point numbers are approximated due to their binary storage. On the other hand, I thought that modern math CPUs used guard digits to protect against trivial underflows. Not true? # integers, no problem > 1+1+1==3 [1] TRUE # binary floating
2009 Jun 19
3
Floating point precision / guard digits? (PR#13771)
Full_Name: D Kreil Version: 2.8.1 and 2.9.0 OS: Debian Linux Submission from: (NULL) (141.244.140.179) Group: Accuracy I understand that most floating point numbers are approximated due to their binary storage. On the other hand, I thought that modern math CPUs used guard digits to protect against trivial underflows. Not true? # integers, no problem > 1+1+1==3 [1] TRUE # binary floating
2006 Jul 06
3
URI.escape() broken or misdocumented in Ruby 1.8.4
URI.escape() is supposed to be able to take a second parameter listing unsafe characters in the URI. This may be a regexp or string. If a string, it''s supposed to represent a character set listing all unsafe characters. An example given in the core documentation at: http://www.ruby-doc.org/stdlib/libdoc/uri/rdoc/classes/URI/Escape.html#M008992 ...is: p URI.escape("@?@!",
2010 Jul 08
3
Testing equality
Hello How can I test equality with two objects when they include some attribute that is BigDecimal? if I make something like this: it "should ...whatever" do obj = Factory.create(:my_object) ... MyObject.first.should == obj end. FAILS This fails because the object expected is different from the object gotten, and the only difference are the BigDecimal attributes, that are
2008 Jan 19
1
Decimals are being converted to Bigdecimals!!!!!
Hi all, I''m working on a financial app, and have set up some columns in my tables as decimals with precision => 8 and scale => 2. My migration looks like: change_column :lineitems, :price, :decimal, :precision => 8, :scale => 2 change_column :payments, :amount, :decimal, :precision => 8, :scale => 2 I''ve run the migration and restarted the server. I
2009 Sep 13
2
How can I get "predict.lm" results with manual calculations ? (a floating point problem)
Hello dear r-help group I am turning for you for help with FAQ number 7.31: "Why doesn't R think these numbers are equal?" http://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-doesn_0027t-R-think-these-numbers-are-equal_003f *My story* is this: I wish to run many lm predictions and need to have them run fast. Using predict.lm is relatively slow, so I tried having it run faster by
2008 Jun 11
1
Resourse route in namespace with prefix error
Ok so I have this route map.namespace :trades do |trades| trades.resources :buys, :path_prefix => ''/trades/:contract_id'' end I want the contract_id variable to filter a list of buys then I try to open this route http://localhost:3000/trades/1/buys I get a strange error: ActionController::RoutingError in Trades/buys#index Showing trades/buys/index.html.erb where line
2007 Nov 13
4
question about activerecord test_numeric_fields in base_test.rb
There''s a test for activerecord called test_numeric_fields in base_test.rb. here''s the test def test_numeric_fields m = NumericData.new( :bank_balance => 1586.43, :big_bank_balance => BigDecimal("1000234000567.95"), :world_population => 6000000000, :my_house_population => 3 ) assert m.save m1 =
2007 Aug 30
3
Rails - depot application
I''m doing the depot example in the Agle Web development book. I''m getting this error: "undefined method `product'' for #<BigDecimal:331ce70,''0.21E1'',8(8)> " "Extracted source (around line #9): 8: for item in @items 9: product = item.product 10: -%> 11: <tr> 12: <td><%= item.product%></td>"
2006 Jun 12
6
Storing "money" in databases
Hiall, I''m wondering what''s common practice when dealing with money values stored in databases. I like the idea of storing all the values as integers, i.e. all the values in eurocents not euros. Then I need to multiply all values the users enter by 100 before storing them in the database. Now the question is, what''s the most efficient and dryest way to do this?