similar to: Rounding error for high-precision numeric values (PR#13010)

Displaying 20 results from an estimated 1000 matches similar to: "Rounding error for high-precision numeric values (PR#13010)"

2008 Apr 22
0
numeric (decimal) WITH precision and scale at generate scaffold time?
Hello, I was wondering if anyone knew if it is (or ever will be ) possible at all to specify things like precision and scale at generate scaffold time, and if so how. Things like price:numeric(8,2) (obviously) just break the script. Tommy --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails:
2016 Aug 23
0
summary.default rounding on numeric seems inconsistent with other R behaviors
>>>>> Dirk Eddelbuettel <edd at debian.org> >>>>> on Fri, 19 Aug 2016 11:40:05 -0500 writes: > It is the old story of defined behaviour and expected outcomes. Hard to > change now. yes... not impossible though... see below > So I would suggest you do something like this in your ~/.Rprofile: R> smry <- function(...)
2016 Aug 19
0
summary.default rounding on numeric seems inconsistent with other R behaviors
John, I had raised the matter ten years ago, and I was told that the topic was already very^3 old https://stat.ethz.ch/pipermail/r-devel/2006-September/042684.html there is some discussion on its origin and also a declaration of intents to change the default behaviour, which, unfortunately, remained a declaration. I agree that R could do better here, let's hope in less than ten years
2005 Oct 19
2
Automatic rounding of values after factors , converted to numeric, are multipled by a real number
I am wondering if someone would have any suggestion about my issue? I have the following code: wgts<-aggregate(subset(lendata,select=c(Length)),list(lendata$Cruise,len data$Station,lendata$Region,lendata$Total),mean) wgts<-wgts[order(wgts$Group.3,wgts$Group.1,wgts$Group.1),] names(wgts)<-c("Cruise","Station","Region","Total","MLen")
2016 Aug 19
2
summary.default rounding on numeric seems inconsistent with other R behaviors
It is the old story of defined behaviour and expected outcomes. Hard to change now. So I would suggest you do something like this in your ~/.Rprofile: R> smry <- function(...) summary(..., digits=6) R> smry(155555L) Min. 1st Qu. Median Mean 3rd Qu. Max. 155555 155555 155555 155555 155555 155555 R> Maybe call it Summary() instead. Dirk --
2016 Aug 24
2
summary.default rounding on numeric seems inconsistent with other R behaviors
>>>>> Martin Maechler <maechler at stat.math.ethz.ch> >>>>> on Tue, 23 Aug 2016 14:33:58 +0200 writes: >>>>> Dirk Eddelbuettel <edd at debian.org> >>>>> on Fri, 19 Aug 2016 11:40:05 -0500 writes: >> It is the old story of defined behaviour and expected outcomes. Hard to >> change now. >
2016 Aug 19
3
summary.default rounding on numeric seems inconsistent with other R behaviors
I was wondering if it would make sense to change the default behavior of the following: summary(15555L) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 15560 15560 15560 15560 15560 15560 summary.default on numeric values rounds values (not just presentation) to getOption("digits")-3L (or four) digits by default, making those values surprising and less suitable for
2013 May 02
3
[Bug 2097] New: if gensalt's log_rounds parameter is set to 31 it does 0 (ZERO) rounds!
https://bugzilla.mindrot.org/show_bug.cgi?id=2097 Bug ID: 2097 Summary: if gensalt's log_rounds parameter is set to 31 it does 0 (ZERO) rounds! Classification: Unclassified Product: jBCrypt Version: unspecified Hardware: All OS: All Status: NEW Severity: security
2003 Mar 15
3
round() seems inconsistent when rounding 5s
It may be my lack of unerstanding, but round() seems to me to give inconsistent results when rounding 5s as in the following examples? > round(1.45, 1) [1] 1.4 # OK > round(2.45, 1) [1] 2.5 # shouldn't this be 2.4? > round(1.05, 1) [1] 1.1 # 1.0 ? and signif(): > signif(2.445, 3) [1] 2.44 # OK > signif(3.445, 3) [1]
2008 Jan 02
2
rounding format
Hi, The code round(0.0002234,4) will return 2e-04, how can I return 0.0002 instead? I want to print 0.0002 instead of 2e-04 for formating purpose. thanks ____________________________________________________________________________________ Looking for last minute shopping deals?
2003 Jun 04
2
Rounding problem R vs Excel.
David A. Paul wrote: > I don't have the reference, but a biologist friend of mine once > showed me a refereed journal article that purported to demonstrate > numerical errors made by MSExcel. This would have been Excel97 or > Excel2000... In any case, the journal's scope was biological in > nature and the article was of interest since Excel is heavily used in > that
2008 Aug 29
0
memisc package and rounding
Greetings, Is there a way to control the number of digits after the decimal from linear regression output using the memisc package? I have tried the following code, but it does not work: fm <- lm(y ~ X) mtable(fm, digits=9) The default seems to be 3 digits after the decimal. Also, is there a way in general to control the number of digits after the decimal? Thank you! Geoff
2008 Sep 19
0
Rounding Errors using GeoKit
Hi all, I''m trying to use the excellent GeoKit plugin to geocode addresses entered by users. I use GeoKit to get latitude and longitude values, then plot these using Google Maps. The problem I have is that the maps produced tend to be slightly off where they should be. Nearby addresses seem to be grouped to one point, which suggests that the problem is in rounding the lat and lng values
2012 Apr 27
2
get plot axis rounding method
Hello, Does anyone know how to get the rounding method used for the axis tick numbers/values in plot()? I'm using mtext() to plot the values used to plot vertical and horizontal lines (using abline()) and I'd like these vertical and horizontal line values to be rounded like the axis tick values are rounded. In other words, I want numbers plotted with mtext() to be rounded in the same
2017 Aug 22
0
Rounding of problem with sum command in R
On 2017-08-22 9:26 AM, niharika singhal wrote: > Hello I have a vector > v=c(0.08844446,0.1744455,0.1379778,0.1209769,0.1573065,0.1134463,0.2074027) > when i do > sum(v) > or > 0.08844446+0.1744455+0.1379778+0.1209769+0.1573065+0.1134463+0.2074027 > i am getting output as 1 No:? That's only the display: > sum(v)-1 [1] 1.6e-07 ????? hope this helps.? Spencer
2008 Mar 11
1
[LLVMdev] Rounding Mode for fptrunc Instruction
Hi, What is the rounding mode for the fptrunc instruction? Round to zero? Round to nearest? Or undefined? Regards, Jon
2008 Jun 24
1
[LLVMdev] rounding mode
Hello LLVMers, A question. In our system we often go from our-system -> LLVM generated functions -> callback functions in our system Is it the case that we can rely on the rounding mode being the same on both sides of the LLVM generated functions or can generated functions change the rounding mode and not restore it prior to calling another function which may or may not be one
2010 Jun 28
0
[LLVMdev] Proposal for adding rounding variant of fpto?i instructions
Hi, I would appreciate comments/feedback on this proposal to extend the fpto?i instructions to support rounding. This is my first proposed significant extension to LLVM so I'd like to know if I'm completely off base here. First of all, I noticed the convert* intrinsics corresponding to the CONVERT_RNDSAT InstructionDAG. I propose not to build my work on top of these as they seem to be
2010 Jul 12
1
[LLVMdev] Proposal for adding rounding variant of fpto?i instructions
Hi Peter, > I would appreciate comments/feedback on this proposal to extend the > fpto?i instructions to support rounding. I think this might be useful. Actually, LLVM specifies that fpto?i convert their operand into the nearest (rounding towards zero) signed integer value. Do you have other rounding modes in mind? Currently I use custom metadata to represent conversions with other rounding
2010 Jul 13
0
[LLVMdev] Proposal for adding rounding variant of fpto?i instructions
On Jul 12, 2010, at 9:28 AM, Anton Lokhmotov wrote: > Hi Peter, > >> I would appreciate comments/feedback on this proposal to extend the >> fpto?i instructions to support rounding. > I think this might be useful. Actually, LLVM specifies that fpto?i convert > their operand into the nearest (rounding towards zero) signed integer value. > Do you have other rounding modes