Displaying 20 results from an estimated 10000 matches similar to: "ruby floating point calculations"
2010 Dec 23
36
Weird issue with converting floats to integer
Any idea why this calculates the integer the way it does?
irb> ("291.15".to_f * 100.0).to_i
=> 29114
Thanks,
Tom
--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
2006 Mar 31
6
BigDecimal + Float => Float?
I was just playing around in irb and noticed that while BigDecimal + Integer
=> BigDecimal, BigDecimal + Float => Float. Is there any reason for this?
It seems inconsistent and means you have to check the class of any number you
add to a BigDecimal, which looks very un-Ruby-like to me.
Ashley
2006 Feb 09
11
Ruby on AIX?
We''re looking at big IBM hardware running AIX as a potential
deployment platform for the big credit card processing Rails and Ruby
stuff that my team is currently building. I was wondering if anyone
else is running on AIX or knows if Ruby is well-supported on this
platform.
I googled the subject, but
http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/3daaf60b012c1921
is
2006 Jul 30
34
ruby suitable for financial apps?
hi there,
I am about to embark on a mission critical financial web application,
and was wondering weather ruby on rails is the best fit for this type
of project.
More specifically I am worried about floating point errors. Would it be
more appropriate to use a more statically typed framework (.NET)?
thankyou.
--
Posted via http://www.ruby-forum.com/.
2006 Jul 22
4
Reading floating points into a field stored as Integers
Hi, I''m trying to follow the "AWDR" example of storing certain types of
numbers (in my case, percentages not money) as integers in the
database. I understand from the Depot application how to format the
output to have 2 places (or n places for that matter) decimal.
Now, I"m wondering about the input. It''s more natural for people to
enter numbers as
2006 Apr 09
16
Mapping to BigDecimals instead of Floats
Hi everyone,
I''m new to Rails and Ruby, and have been working my way through the Pickaxe
and Agile Rails books for the last week or so. With a background in payroll
apps I found that the default ActiveRecord mapping of decimal columns to
float variables really bothered me! Financial calcs need high levels of
accuracy and floats just don''t give you that.
I know there are a bunch
2008 Jul 12
3
calculations on nested resources
Hi,
I can''t seem to find a solution to the folowing problem.
I have 3 resources that are all bound to eachother like this:
class Item < ActiveRecord::Base
belongs_to :item_category
// has a date and amount attr
end
class ItemCategory < ActiveRecord::Base
has_many :items
has_many :item_tops
// groups the items in categories
end
class ItemTop < ActiveRecord::Base
2006 Jul 09
6
Float -> BigDecimal
So, I''m personally cool with this change (patch from #5454, applied in
[4596]), and I''ll make it work for Oracle tomorrow.
But just wondering if folks considered the performance impact of the
change? From a simple test it seems that BigDecimal math is about twice
as slow as using Floats.
2008 Nov 07
15
any tricks re using " eql(5.5)", but where 5.5 is a decimal not float?
--
Sent from my mobile device
2009 Feb 03
4
Ruby EE: Problem with BigDecimal
I have a strange problem with my setup of Passenger and Ruby Enterprise
Edition.
Sometimes the loading of BigDecimals from the database and session fails
(silently) and the variable containing the BigDecimal is just empty.
I cannot pinpoint where/when/how it happens, and it IS only periodic.
I am using the newest versions of both Passenger and REE - I have even
tried to run with latest commit
2009 Jun 19
3
Floating point precision / guard digits? (PR#13771)
Full_Name: D Kreil
Version: 2.8.1 and 2.9.0
OS: Debian Linux
Submission from: (NULL) (141.244.140.179)
Group: Accuracy
I understand that most floating point numbers are approximated due to their
binary storage. On the other hand, I thought that modern math CPUs used guard
digits to protect against trivial underflows. Not true?
# integers, no problem
> 1+1+1==3
[1] TRUE
# binary floating
2009 Jun 19
3
Floating point precision / guard digits? (PR#13771)
Full_Name: D Kreil
Version: 2.8.1 and 2.9.0
OS: Debian Linux
Submission from: (NULL) (141.244.140.179)
Group: Accuracy
I understand that most floating point numbers are approximated due to their
binary storage. On the other hand, I thought that modern math CPUs used guard
digits to protect against trivial underflows. Not true?
# integers, no problem
> 1+1+1==3
[1] TRUE
# binary floating
2006 Jul 06
3
URI.escape() broken or misdocumented in Ruby 1.8.4
URI.escape() is supposed to be able to take a second parameter listing
unsafe characters in the URI. This may be a regexp or string. If a
string, it''s supposed to represent a character set listing all unsafe
characters. An example given in the core documentation at:
http://www.ruby-doc.org/stdlib/libdoc/uri/rdoc/classes/URI/Escape.html#M008992
...is:
p URI.escape("@?@!",
2010 Jul 08
3
Testing equality
Hello
How can I test equality with two objects when they include some
attribute that is BigDecimal?
if I make something like this:
it "should ...whatever" do
obj = Factory.create(:my_object)
...
MyObject.first.should == obj
end.
FAILS
This fails because the object expected is different from the object
gotten, and the only difference are the BigDecimal attributes, that are
2008 Jan 19
1
Decimals are being converted to Bigdecimals!!!!!
Hi all,
I''m working on a financial app, and have set up some columns in my
tables as decimals with precision => 8 and scale => 2. My migration
looks like:
change_column :lineitems, :price, :decimal, :precision =>
8, :scale => 2
change_column :payments, :amount, :decimal, :precision =>
8, :scale => 2
I''ve run the migration and restarted the server. I
2009 Sep 13
2
How can I get "predict.lm" results with manual calculations ? (a floating point problem)
Hello dear r-help group
I am turning for you for help with FAQ number 7.31: "Why doesn't R think
these numbers are equal?"
http://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-doesn_0027t-R-think-these-numbers-are-equal_003f
*My story* is this:
I wish to run many lm predictions and need to have them run fast.
Using predict.lm is relatively slow, so I tried having it run faster by
2008 Jun 11
1
Resourse route in namespace with prefix error
Ok so I have this route
map.namespace :trades do |trades|
trades.resources :buys, :path_prefix => ''/trades/:contract_id''
end
I want the contract_id variable to filter a list of buys
then I try to open this route http://localhost:3000/trades/1/buys
I get a strange error:
ActionController::RoutingError in Trades/buys#index
Showing trades/buys/index.html.erb where line
2007 Nov 13
4
question about activerecord test_numeric_fields in base_test.rb
There''s a test for activerecord called test_numeric_fields in
base_test.rb. here''s the test
def test_numeric_fields
m = NumericData.new(
:bank_balance => 1586.43,
:big_bank_balance => BigDecimal("1000234000567.95"),
:world_population => 6000000000,
:my_house_population => 3
)
assert m.save
m1 =
2007 Aug 30
3
Rails - depot application
I''m doing the depot example in the Agle Web development book. I''m
getting this error:
"undefined method `product'' for #<BigDecimal:331ce70,''0.21E1'',8(8)> "
"Extracted source (around line #9):
8: for item in @items
9: product = item.product
10: -%>
11: <tr>
12: <td><%= item.product%></td>"
2006 Jun 12
6
Storing "money" in databases
Hiall,
I''m wondering what''s common practice when dealing with money values
stored in databases. I like the idea of storing all the values as
integers, i.e. all the values in eurocents not euros. Then I need to
multiply all values the users enter by 100 before storing them in the
database. Now the question is, what''s the most efficient and dryest
way to do this?