Displaying 20 results from an estimated 1000 matches similar to: "Rounding error for high-precision numeric values (PR#13010)"
2008 Apr 22
0
numeric (decimal) WITH precision and scale at generate scaffold time?
Hello,
I was wondering if anyone knew if it is (or ever will be ) possible at
all to specify things like precision and scale at generate scaffold
time, and if so how.
Things like price:numeric(8,2) (obviously) just break the script.
Tommy
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Ruby on Rails:
2016 Aug 23
0
summary.default rounding on numeric seems inconsistent with other R behaviors
>>>>> Dirk Eddelbuettel <edd at debian.org>
>>>>> on Fri, 19 Aug 2016 11:40:05 -0500 writes:
> It is the old story of defined behaviour and expected outcomes. Hard to
> change now.
yes... not impossible though... see below
> So I would suggest you do something like this in your ~/.Rprofile:
R> smry <- function(...)
2016 Aug 19
0
summary.default rounding on numeric seems inconsistent with other R behaviors
John,
I had raised the matter ten years ago, and I was told that the topic was
already very^3 old
https://stat.ethz.ch/pipermail/r-devel/2006-September/042684.html
there is some discussion on its origin and also a declaration of intents to
change the default behaviour, which, unfortunately, remained a declaration.
I agree that R could do better here, let's hope in less than ten years
2005 Oct 19
2
Automatic rounding of values after factors , converted to numeric, are multipled by a real number
I am wondering if someone would have any suggestion about my issue?
I have the following code:
wgts<-aggregate(subset(lendata,select=c(Length)),list(lendata$Cruise,len
data$Station,lendata$Region,lendata$Total),mean)
wgts<-wgts[order(wgts$Group.3,wgts$Group.1,wgts$Group.1),]
names(wgts)<-c("Cruise","Station","Region","Total","MLen")
2016 Aug 19
2
summary.default rounding on numeric seems inconsistent with other R behaviors
It is the old story of defined behaviour and expected outcomes. Hard to
change now.
So I would suggest you do something like this in your ~/.Rprofile:
R> smry <- function(...) summary(..., digits=6)
R> smry(155555L)
Min. 1st Qu. Median Mean 3rd Qu. Max.
155555 155555 155555 155555 155555 155555
R>
Maybe call it Summary() instead.
Dirk
--
2016 Aug 24
2
summary.default rounding on numeric seems inconsistent with other R behaviors
>>>>> Martin Maechler <maechler at stat.math.ethz.ch>
>>>>> on Tue, 23 Aug 2016 14:33:58 +0200 writes:
>>>>> Dirk Eddelbuettel <edd at debian.org>
>>>>> on Fri, 19 Aug 2016 11:40:05 -0500 writes:
>> It is the old story of defined behaviour and expected outcomes. Hard to
>> change now.
>
2016 Aug 19
3
summary.default rounding on numeric seems inconsistent with other R behaviors
I was wondering if it would make sense to change the default behavior of the following:
summary(15555L)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 15560 15560 15560 15560 15560 15560
summary.default on numeric values rounds values (not just presentation) to getOption("digits")-3L (or four) digits by default, making those values surprising and less suitable for
2013 May 02
3
[Bug 2097] New: if gensalt's log_rounds parameter is set to 31 it does 0 (ZERO) rounds!
https://bugzilla.mindrot.org/show_bug.cgi?id=2097
Bug ID: 2097
Summary: if gensalt's log_rounds parameter is set to 31 it does
0 (ZERO) rounds!
Classification: Unclassified
Product: jBCrypt
Version: unspecified
Hardware: All
OS: All
Status: NEW
Severity: security
2003 Mar 15
3
round() seems inconsistent when rounding 5s
It may be my lack of unerstanding, but round() seems to me to give
inconsistent results when rounding 5s as in the following examples?
> round(1.45, 1)
[1] 1.4 # OK
> round(2.45, 1)
[1] 2.5 # shouldn't this be 2.4?
> round(1.05, 1)
[1] 1.1 # 1.0 ?
and signif():
> signif(2.445, 3)
[1] 2.44 # OK
> signif(3.445, 3)
[1]
2008 Jan 02
2
rounding format
Hi,
The code round(0.0002234,4) will return 2e-04, how can
I return 0.0002 instead? I want to print 0.0002
instead of 2e-04 for formating purpose.
thanks
____________________________________________________________________________________
Looking for last minute shopping deals?
2003 Jun 04
2
Rounding problem R vs Excel.
David A. Paul wrote:
> I don't have the reference, but a biologist friend of mine once
> showed me a refereed journal article that purported to demonstrate
> numerical errors made by MSExcel. This would have been Excel97 or
> Excel2000... In any case, the journal's scope was biological in
> nature and the article was of interest since Excel is heavily used in
> that
2008 Aug 29
0
memisc package and rounding
Greetings,
Is there a way to control the number of digits after the decimal from linear regression output using the memisc package?
I have tried the following code, but it does not work:
fm <- lm(y ~ X)
mtable(fm, digits=9)
The default seems to be 3 digits after the decimal.
Also, is there a way in general to control the number of digits after the decimal?
Thank you!
Geoff
2008 Sep 19
0
Rounding Errors using GeoKit
Hi all,
I''m trying to use the excellent GeoKit plugin to geocode addresses
entered by users. I use GeoKit to get latitude and longitude values,
then plot these using Google Maps.
The problem I have is that the maps produced tend to be slightly off
where they should be. Nearby addresses seem to be grouped to one
point, which suggests that the problem is in rounding the lat and lng
values
2012 Apr 27
2
get plot axis rounding method
Hello,
Does anyone know how to get the rounding method used for the axis tick
numbers/values in plot()?
I'm using mtext() to plot the values used to plot vertical and horizontal
lines (using abline()) and I'd like these vertical and horizontal line
values to be rounded like the axis tick values are rounded.
In other words, I want numbers plotted with mtext() to be rounded in the
same
2017 Aug 22
0
Rounding of problem with sum command in R
On 2017-08-22 9:26 AM, niharika singhal wrote:
> Hello I have a vector
> v=c(0.08844446,0.1744455,0.1379778,0.1209769,0.1573065,0.1134463,0.2074027)
> when i do
> sum(v)
> or
> 0.08844446+0.1744455+0.1379778+0.1209769+0.1573065+0.1134463+0.2074027
> i am getting output as 1
No:? That's only the display:
> sum(v)-1
[1] 1.6e-07
????? hope this helps.? Spencer
2008 Mar 11
1
[LLVMdev] Rounding Mode for fptrunc Instruction
Hi,
What is the rounding mode for the fptrunc instruction? Round to zero?
Round to nearest? Or undefined?
Regards,
Jon
2008 Jun 24
1
[LLVMdev] rounding mode
Hello LLVMers,
A question. In our system we often go from
our-system -> LLVM generated functions -> callback functions in our
system
Is it the case that we can rely on the rounding mode being the same on
both sides of the LLVM generated functions or can generated functions
change the rounding mode and not restore it prior to calling another
function which may or may not be one
2010 Jun 28
0
[LLVMdev] Proposal for adding rounding variant of fpto?i instructions
Hi,
I would appreciate comments/feedback on this proposal to extend the
fpto?i instructions to support rounding. This is my first proposed
significant extension to LLVM so I'd like to know if I'm completely
off base here.
First of all, I noticed the convert* intrinsics corresponding to the
CONVERT_RNDSAT InstructionDAG. I propose not to build my work on
top of these as they seem to be
2010 Jul 12
1
[LLVMdev] Proposal for adding rounding variant of fpto?i instructions
Hi Peter,
> I would appreciate comments/feedback on this proposal to extend the
> fpto?i instructions to support rounding.
I think this might be useful. Actually, LLVM specifies that fpto?i convert
their operand into the nearest (rounding towards zero) signed integer value.
Do you have other rounding modes in mind? Currently I use custom metadata to
represent conversions with other rounding
2010 Jul 13
0
[LLVMdev] Proposal for adding rounding variant of fpto?i instructions
On Jul 12, 2010, at 9:28 AM, Anton Lokhmotov wrote:
> Hi Peter,
>
>> I would appreciate comments/feedback on this proposal to extend the
>> fpto?i instructions to support rounding.
> I think this might be useful. Actually, LLVM specifies that fpto?i convert
> their operand into the nearest (rounding towards zero) signed integer value.
> Do you have other rounding modes