Greetings, I would like to fit a multiple linear regression model in which the residuals are expected to follow a multivariate normal distribution, using weighted least squares. I know that the data in question have biases that would result in correlated residuals, and I have a means for quantifying those biases as a covariance matrix. I cannot, unfortunately, correct the data for these biases. It seems that this should be a straightforward task, but so much of the literature is concerned with the probability model in which the residuals are uncorrelated that I can't find a good reference. So in order of importance, please, can someone point me to a definitive reference for least squares with correlated residuals, and is there a standard R package to handle this case? Many thanks in advance, Anthony
The function gls() in the nlme library will handle correlated observations assuming that you have an easily specified pattern to the variances and covariances (e.g. an AR(1) process). If you just have an arbitrary variance matrix for the y-vector do not despair. You just have to do a few matrix algebra computations, with which I am sure you are quite expert... The functions you will probably need are simple ones like chol(), crossprod(), "%*%" and solve() so nothing too difficult there. You might want to start with model.matrix(), though, and that's slightly more challenging for most people. Bill Venables. -----Original Message----- From: Andy Jacobson [mailto:junkmail at mu.met.psu.edu] Sent: Friday, June 13, 2003 1:40 PM To: r-help at stat.math.ethz.ch Subject: [R] covariate data errors Greetings, I would like to fit a multiple linear regression model in which the residuals are expected to follow a multivariate normal distribution, using weighted least squares. I know that the data in question have biases that would result in correlated residuals, and I have a means for quantifying those biases as a covariance matrix. I cannot, unfortunately, correct the data for these biases. It seems that this should be a straightforward task, but so much of the literature is concerned with the probability model in which the residuals are uncorrelated that I can't find a good reference. So in order of importance, please, can someone point me to a definitive reference for least squares with correlated residuals, and is there a standard R package to handle this case? Many thanks in advance, Anthony ______________________________________________ R-help at stat.math.ethz.ch mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
Dear Anthony, The gls (generalized least squares) function in the nlme package should do what you want. (I assume that your analysis leads you to expect an error-covariance matrix of a specific form with some free parameters to estimate.) Generalized least squares estimation is a common topic in regression texts. You'll find a brief appendix on the subject from my R and S-PLUS Companion to Applied Regression, in the context of time-series regression, at <http://www.socsci.mcmaster.ca/jfox/Books/Companion/appendix-timeseries-regression.pdf>. I hope that this helps, John At 11:40 PM 6/12/2003 -0400, Andy Jacobson wrote:>Greetings, > > I would like to fit a multiple linear regression model in >which the residuals are expected to follow a multivariate normal >distribution, using weighted least squares. I know that the data in >question have biases that would result in correlated residuals, and I >have a means for quantifying those biases as a covariance matrix. I >cannot, unfortunately, correct the data for these biases. > > It seems that this should be a straightforward task, but so >much of the literature is concerned with the probability model in >which the residuals are uncorrelated that I can't find a good >reference. So in order of importance, please, can someone point me to >a definitive reference for least squares with correlated residuals, >and is there a standard R package to handle this case? > > Many thanks in advance, > > Anthony----------------------------------------------------- John Fox Department of Sociology McMaster University Hamilton, Ontario, Canada L8S 4M4 email: jfox at mcmaster.ca phone: 905-525-9140x23604 web: www.socsci.mcmaster.ca/jfox
Hello, I'm not qualified to judge about definitiveness, but you may find the discussion of "Time Series Regression and Generalized Least Squares" in Chapter 14 of "Applied Regression Analysis, Linear Models, and Related Models" by John Fox (1997, Sage Publications) helpful. He also has an accompanying package "car" on the CRAN R site. Duncan ***************************************** Dr. Duncan Mackay School of Biological Sciences Flinders University GPO Box 2100 Adelaide S.A. 5001 AUSTRALIA Ph (08) 8201 2627 FAX (08) 8201 3015 http://www.scieng.flinders.edu.au/biology/people/mackay_d/index.html -----Original Message----- From: r-help-bounces at stat.math.ethz.ch [mailto:r-help-bounces at stat.math.ethz.ch]On Behalf Of Andy Jacobson Sent: Friday, 13 June 2003 1:10 PM To: r-help at stat.math.ethz.ch Subject: [R] covariate data errors Greetings, I would like to fit a multiple linear regression model in which the residuals are expected to follow a multivariate normal distribution, using weighted least squares. I know that the data in question have biases that would result in correlated residuals, and I have a means for quantifying those biases as a covariance matrix. I cannot, unfortunately, correct the data for these biases. It seems that this should be a straightforward task, but so much of the literature is concerned with the probability model in which the residuals are uncorrelated that I can't find a good reference. So in order of importance, please, can someone point me to a definitive reference for least squares with correlated residuals, and is there a standard R package to handle this case? Many thanks in advance, Anthony ______________________________________________ R-help at stat.math.ethz.ch mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
Do you mean correlations in the *errors*? The residuals are always correlated. What does this have to do with your subject line -- it is errors in the dependent variable I think you mean? If you have correlated errors, you should be using generalized least squares not least squares or weighted least squares. (That is covered in all good books on regression: I don't know your level, but Seber's has a comprehensive account.) There are several R functions to fit GLS, including gls(nlme) and lm.gls(MASS). On Thu, 12 Jun 2003, Andy Jacobson wrote:> Greetings, > > I would like to fit a multiple linear regression model in > which the residuals are expected to follow a multivariate normal > distribution, using weighted least squares. I know that the data in > question have biases that would result in correlated residuals, and I > have a means for quantifying those biases as a covariance matrix. I > cannot, unfortunately, correct the data for these biases. > > It seems that this should be a straightforward task, but so > much of the literature is concerned with the probability model in > which the residuals are uncorrelated that I can't find a good > reference. So in order of importance, please, can someone point me to > a definitive reference for least squares with correlated residuals, > and is there a standard R package to handle this case? > > Many thanks in advance, > > Anthony > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
I have found "Mixed Effects Models in S and Splus" by Drs. Pinheiro and Bates to be enormously helpful. I highly recommend the book - it contains excellent examples. Best, david paul -----Original Message----- From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk] Sent: Friday, June 13, 2003 2:54 AM To: Andy Jacobson Cc: r-help at stat.math.ethz.ch Subject: Re: [R] covariate data errors Do you mean correlations in the *errors*? The residuals are always correlated. What does this have to do with your subject line -- it is errors in the dependent variable I think you mean? If you have correlated errors, you should be using generalized least squares not least squares or weighted least squares. (That is covered in all good books on regression: I don't know your level, but Seber's has a comprehensive account.) There are several R functions to fit GLS, including gls(nlme) and lm.gls(MASS). On Thu, 12 Jun 2003, Andy Jacobson wrote:> Greetings, > > I would like to fit a multiple linear regression model in which the > residuals are expected to follow a multivariate normal distribution, > using weighted least squares. I know that the data in question have > biases that would result in correlated residuals, and I have a means > for quantifying those biases as a covariance matrix. I cannot, > unfortunately, correct the data for these biases. > > It seems that this should be a straightforward task, but so much of > the literature is concerned with the probability model in which the > residuals are uncorrelated that I can't find a good reference. So in > order of importance, please, can someone point me to a definitive > reference for least squares with correlated residuals, and is there a > standard R package to handle this case? > > Many thanks in advance, > > Anthony > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595 ______________________________________________ R-help at stat.math.ethz.ch mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
Seemingly Similar Threads
- Constructing Matrices
- How can I test if a not independently and not identically distributed time series residuals' are uncorrelated ?
- Testing for normality of residuals in a regression model
- understanding the output from gls
- Advice on picking a regression method