[R] quantile() returns a value outside the data range

Jenny Bryan
Tue, 21 Aug 2007 21:11:21 -0000

Hello,

I am getting an unexpected result from quantile().  Specifically, the
return value falls outside the range of the data, which I wouldn't
have thought possible for a weighted average of 2 order statistics.
Is this an unintended accuracy issue or am I being too casual in my
comparison (is there some analogue of 'all.equal' for
"<=")?

Small example:

 > foo <- h2[,"17"][h2$plate==222] # some data I have, details
not
interesting
 > foo <- sort(foo)[1:3]            # can demo with the 3 smallest
values
 > yo <- data.frame(matrix(foo,nrow=length(foo),ncol=10))
 > fooQtile <- rep(0,9)
 > for(i in 1:9) {               # compute 0.01 quantile, for all
'types'
+   fooQtile[i] <- quantile(foo,probs=0.01,type=i)
+   yo[,i+1] <- foo <= fooQtile[i]
+ }
 > names(yo) <-
c("myData",paste("qType",1:9,sep=""))
 > yo
     myData qType1 qType2 qType3 qType4 qType5 qType6 qType7 qType8
qType9
1 6.402611   TRUE   TRUE   TRUE   TRUE   TRUE   TRUE  FALSE  FALSE
TRUE
2 6.402611   TRUE   TRUE   TRUE   TRUE   TRUE   TRUE  FALSE  FALSE
TRUE
3 6.420587  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
FALSE
 > fooQtile
[1] 6.40261 6.40261 6.40261 6.40261 6.40261 6.40261 6.40261 6.40261
6.40261
 > table(fooQtile)
fooQtile
6.40261053520674 6.40261053520674
                2                7

I expected the returned quantile to be either equal to or greater than
the minimum observed value and that is the case for types 1-6 and
9. But for types 7 (the default) and 8, the returned quantile is less
than the minimum observed value.  The difference between the type
(1-6,9) and type (7,8) return values and between the returned quantile
and the minimum is obviously very very small.

Is there any choice for 'type' that is guaranteed to return values
inside the observed range?

Thanks, Jenny

Dr. Jennifer Bryan
Assistant Professor
Department of Statistics and
  the Michael Smith Laboratories
University of British Columbia
--------------------------------------------
333-6356 Agricultural Road
Vancouver, BC Canada
V6T 1Z2