(I'm reposting this message because the original has not appeared after
about 2 days. Sorry if it shows up twice.)
Hello.
First, thanks to those who responded to my recent inquiry about using
contour() over arbitrary (x,y) by mentioning the interp() function in
the akima package. That worked nicely. Now for a new question:
I would like to use a pair of prediction intervals to graphically bound
the noise in some y(x) measurements. Here's an artificial example
showing a function y(x)=x + noise, where the noise diminishes as x
increases from 0 to 1.
x=seq(0,1,0.01)
y=x+runif(length(x),-1,1)*((1-x)/5)
fit=lm(y ~ 1 + x)
pred=predict(fit, interval="prediction")
matplot(x,pred,type="l",ylab="y")
points(x,y)
I would have expected the lower and upper prediction intervals to
converge as x increases (and the noise decreases), but they seem to
remain virtually equidistant. Can anyone explain (a) the behavior that I
see, and (b) how to obtain curves that do bound the noise?
Thanks,
Ronnen.
--
Ronnen Levinson, Ph.D. \/ RML27 at cornell.edu
scientist || http://ronnen.com
Lawrence Berkeley National Lab /\ fax 425.955.1992
=====================================I took a speed reading course and read
'War and Peace' in twenty minutes. It involves Russia.
-- Woody Allen
Ronnen Levinson wrote:> (I'm reposting this message because the original has not appeared after > about 2 days. Sorry if it shows up twice.) > > > Hello. > > First, thanks to those who responded to my recent inquiry about using > contour() over arbitrary (x,y) by mentioning the interp() function in > the akima package. That worked nicely. Now for a new question: > > I would like to use a pair of prediction intervals to graphically bound > the noise in some y(x) measurements. Here's an artificial example > showing a function y(x)=x + noise, where the noise diminishes as x > increases from 0 to 1. > > x=seq(0,1,0.01) > y=x+runif(length(x),-1,1)*((1-x)/5) > fit=lm(y ~ 1 + x) > pred=predict(fit, interval="prediction") > matplot(x,pred,type="l",ylab="y") > points(x,y) > > I would have expected the lower and upper prediction intervals to > converge as x increases (and the noise decreases), but they seem to > remain virtually equidistant. Can anyone explain (a) the behavior that I > see, and (b) how to obtain curves that do bound the noise? > > Thanks, > > Ronnen. >Well, since you use an "ordinary" lm(), for your residuals Var(e)=const. is assumed (among other assumptions), hence a global variance is also considered and estimated for prediction, of course: I(x_0) := x'_0 +- t_{n-h-1; 1-\alpha/2} \sqrt{s^2 (x'_0 (X'X)^{-1} x_0 + 1)} ^^^ You have to choose a completely different model. Uwe Ligges
The usual assumption in regression is that noise is iid, that is has the same
distribution except for a location shift at all values of x. Prediction
intervals
adopting this premise are based on an average of the noise over the whole sample
and thus produce the result you see in your figure.
Your model is what I would call a linear location scale model, that is both
location and scale of the noise are linear in x...one way to generate prediction
bands
for this sort of model is to fit conditional quantile functions. Here is an
example:
x=seq(0,1,0.01)
y=x+runif(length(x),-1,1)*((1-x)/5)
require(quantreg)
abline(rq(y~x,tau=.9)$coef[,1])
abline(rq(y~x,tau=.1)$coef[,1])
the resulting band contains 80 percent of the sample observations. You can
get some further details from ?rq and the references their.
url: www.econ.uiuc.edu/~roger/my.html Roger Koenker
email rkoenker at uiuc.edu Department of Economics
vox: 217-333-4558 University of Illinois
fax: 217-244-6678 Champaign, IL 61820
On Sat, 16 Aug 2003, Ronnen Levinson wrote:
> (I'm reposting this message because the original has not appeared after
> about 2 days. Sorry if it shows up twice.)
>
>
> Hello.
>
> First, thanks to those who responded to my recent inquiry about using
> contour() over arbitrary (x,y) by mentioning the interp() function in
> the akima package. That worked nicely. Now for a new question:
>
> I would like to use a pair of prediction intervals to graphically bound
> the noise in some y(x) measurements. Here's an artificial example
> showing a function y(x)=x + noise, where the noise diminishes as x
> increases from 0 to 1.
>
> x=seq(0,1,0.01)
> y=x+runif(length(x),-1,1)*((1-x)/5)
> fit=lm(y ~ 1 + x)
> pred=predict(fit, interval="prediction")
> matplot(x,pred,type="l",ylab="y")
> points(x,y)
>
> I would have expected the lower and upper prediction intervals to
> converge as x increases (and the noise decreases), but they seem to
> remain virtually equidistant. Can anyone explain (a) the behavior that I
> see, and (b) how to obtain curves that do bound the noise?
>
> Thanks,
>
> Ronnen.
>
> --
> Ronnen Levinson, Ph.D. \/ RML27 at cornell.edu
> scientist || http://ronnen.com
> Lawrence Berkeley National Lab /\ fax 425.955.1992
>
> =====================================> I took a speed reading course and
read 'War and Peace' in twenty minutes. It involves Russia.
> -- Woody Allen
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>