To all I have some simple questions pertaining to weights used in regression. If the variability of the dependent variable (y) is a function of the magnitude of predictor variable (x), can the use of weights give an appropriate answer to the regression parameters and the std errors? Assume that y at x=1 and 6 has a standard deviation of 0.1 and at x=11 it is 0.4 Then according to a web page on weighted regression for a calibration curve at http://member.nifty.ne.jp/mniwa/rev006.htm, I should use 1/(std^2) for each weight. i.e. for x=1 and 6, w = 100 and x=11, w = 6.25 In R the run is:>y<-c(1,6,11) >x<-c(6.7,6.7,6.6) >w<-c(100,100,6.25) >reg <-lm(x~y, weight=w) > summary(reg)Call: lm(formula = x ~ y, weights = w) Residuals: 1 2 3 -0.04762 0.09524 -0.19048 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 6.707619 0.025431 263.762 0.00241 ** y -0.002857 0.005471 -0.522 0.69361 --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 0.2182 on 1 degrees of freedom Multiple R-Squared: 0.2143, Adjusted R-squared: -0.5714 F-statistic: 0.2727 on 1 and 1 DF, p-value: 0.6936 Am I using the weight method correctly? And if so does the Estimated Std. Error for the Intercept and slope make sense? On another note. How does one do a regression with the origin fixed at 0? Merry Christmas REX [[alternative HTML version deleted]]
Rex - Yes, you have supplied an appropriate 'weight' argument given the problem description in the paragraph which begins 'Assume that ...'. Your example would be much easier to read if the variable names 'x' and 'y' in the R code matched their usage in the paragraph description, rather than transposing. But the usage within the R code is consistent, although counter- intuitive. Your example tries to predict the values of an almost constant vector c(6.7,6.7,6.6) from a highly varying one, c(1,6,11). No surprise that the intercept with the vertical axis is a bit larger than 6.7 and the slope is completely non-significant. - tom blackwell - u michigan medical school - ann arbor On Thu, 18 Dec 2003, rex_bryan at urscorp.com wrote:> To all > > I have some simple questions pertaining to weights used in regression. > If the variability of the dependent variable (y) is a function of the magnitude of predictor > variable (x), can the use of weights give an appropriate answer to the regression parameters > and the std errors? > > Assume that y at x=1 and 6 has a standard deviation of 0.1 and at x=11 it is 0.4 > Then according to a web page on weighted regression for a calibration curve at > http://member.nifty.ne.jp/mniwa/rev006.htm, I should use 1/(std^2) for each weight. > > i.e. for x=1 and 6, w = 100 and x=11, w = 6.25 > > In R the run is: > > >y<-c(1,6,11) > >x<-c(6.7,6.7,6.6) > >w<-c(100,100,6.25) > >reg <-lm(x~y, weight=w) > > summary(reg) > > Call: > lm(formula = x ~ y, weights = w) > > Residuals: > 1 2 3 > -0.04762 0.09524 -0.19048 > > Coefficients: > Estimate Std. Error t value Pr(>|t|) > (Intercept) 6.707619 0.025431 263.762 0.00241 ** > y -0.002857 0.005471 -0.522 0.69361 > --- > Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 > > Residual standard error: 0.2182 on 1 degrees of freedom > Multiple R-Squared: 0.2143, Adjusted R-squared: -0.5714 > F-statistic: 0.2727 on 1 and 1 DF, p-value: 0.6936 > > Am I using the weight method correctly? > And if so does the Estimated Std. Error for the Intercept and slope make sense? > > On another note. How does one do a regression with the origin fixed at 0? > > Merry Christmas > > REX > > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help >
Paul Thanks for your quick response. Yes I have x and y backward. Thomas Blackwell pointed that out to me. Oops. And yes I will be considering more than 3 points for any regression. This little data set was first presented by a math wizard in a MathCAD forum explaining the mathematics behind weighted regression. When I tried Statistica on the data set I found that their definition of weights was a replication counter NOT in the spirit of 1/var. It put me into a blue funk ... I hate it when my state-of-the-art tools turn out to be a sham. I then turned to R and all appears to be mathemagically working well. As I walk the dogs I'm thinking (amittedly confusedly) on how prediction and confidence intervals could be generated around a weighted regression line. My thoughts range from purloining concepts from kriging, or bootstapping ... in each case I come to the conclusion that this is more complex topic than I thought.... Oh well...it keeps me interested in talking to smart people like you. Have a Merry Christmas REX ----- Original Message ----- From: "Paul E. Johnson" <pauljohn at ku.edu> To: "rex_bryan at urscorp.com" <Rex_Bryan at urscorp.com> Sent: Friday, December 19, 2003 2:43 PM Subject: Re: [R] weighted regression> > I keep new user tips at http://www.ku.edu/~pauljohn/R/Rtips.html > > I have some observations on your example: > > 1. You have y and x backward in your lm code > > 2. You get rid of the intercept by lm(y~x-1) but it is not generally > advisable. it will bias the slope parameter if your assumption is wrong. > > 3. You are right that Weighted Least Squares can be accomplished by > weights in lm. Please read the description of weight in ?lm > > weights: an optional vector of weights to be used in the fitting > process. If specified, weighted least squares is used with > weights 'weights' (that is, minimizing 'sum(w*e^2)'); > otherwise ordinary least squares is used. > > If you understand WLS, then you will see how to enter the variable w. > > 4. I hope you aren't really doing regression with 3 observations. > > pj > > > rex_bryan at urscorp.com wrote: > > >To all > > > >I have some simple questions pertaining to weights used in regression. > >If the variability of the dependent variable (y) is a function of themagnitude of predictor> >variable (x), can the use of weights give an appropriate answer to theregression parameters> >and the std errors? > > > >Assume that y at x=1 and 6 has a standard deviation of 0.1 and at x=11 itis 0.4> >Then according to a web page on weighted regression for a calibrationcurve at> >http://member.nifty.ne.jp/mniwa/rev006.htm, I should use 1/(std^2) foreach weight.> > > >i.e. for x=1 and 6, w = 100 and x=11, w = 6.25 > > > >In R the run is: > > > > > > > >>y<-c(1,6,11) > >>x<-c(6.7,6.7,6.6) > >>w<-c(100,100,6.25) > >>reg <-lm(x~y, weight=w) > >>summary(reg) > >> > >> > > > >Call: > >lm(formula = x ~ y, weights = w) > > > >Residuals: > > 1 2 3 > >-0.04762 0.09524 -0.19048 > > > >Coefficients: > > Estimate Std. Error t value Pr(>|t|) > >(Intercept) 6.707619 0.025431 263.762 0.00241 ** > >y -0.002857 0.005471 -0.522 0.69361 > >--- > >Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 > > > >Residual standard error: 0.2182 on 1 degrees of freedom > >Multiple R-Squared: 0.2143, Adjusted R-squared: -0.5714 > >F-statistic: 0.2727 on 1 and 1 DF, p-value: 0.6936 > > > >Am I using the weight method correctly? > >And if so does the Estimated Std. Error for the Intercept and slope makesense?> > > >On another note. How does one do a regression with the origin fixed at0?> > > >Merry Christmas > > > >REX > > > > > > > > > > > > [[alternative HTML version deleted]] > > > >______________________________________________ > >R-help at stat.math.ethz.ch mailing list > >https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > > > > > > -- > Paul E. Johnson email: pauljohn at ukans.edu > Dept. of Political Science http://lark.cc.ukans.edu/~pauljohn > University of Kansas Office: (785) 864-9086 > Lawrence, Kansas 66045 FAX: (785) 864-5700 >