R help - May 2017 - adding predictor to linear model without changing existing coefficients

Dear list members,

I want to add a predictor to a linear model without changing the
coefficients of the existing model. How is that done with R?

So if I have a response y and predictors x1, x2, x3 I want to make a model lm1
like

lm1 = lm(y~x1+x2)

After this model is computed I want to add x3 like

lm2 = lm(y~x1+x2+x3)

However, unlike it is done by the notation above or by update or add1
(as far as I understand) I don't want a new model with all predictors
estimated anew but I want a model lm2 where the coefficients for x1 and
x2 stay exactly as in lm1 and the coefficent for x3 is estimated
additionally. The reasons for this are theoretical. I guess what I want
is similar to calculating a new regression on the residuals of lm1.

lm2 = lm(residuals(lm1)~x3)

however, I would prefer to to that in the common framework of the lm
command in order to calculate statistics, perform anova on the models
and so on.

thanks for your help!
Urs

You could use an offset term. An example:

n <- 100
x1 <- rnorm(n)
x2 <- rnorm(n)
x3 <- rnorm(n)
y <- 0 + .2 * x1 - .5 * x2 + .3 * x3 + rnorm(n)
res1 <- lm(y ~ x1 + x2)
summary(res1)
res2 <- lm(y ~ 1 + offset(coef(res1)[2] * x1 + coef(res1)[3] * x2))
summary(res2) ### identical intercept as in res1
res3 <- lm(y ~ offset(coef(res1)[2] * x1 + coef(res1)[3] * x2) + x3)
summary(res3)
 
You may need to consider whether you want to fix up the dfs, since the
coefficients in the offset term are obviously not counted. Also, you did not say
whether you want to reestimate the intercept; in res3 the intercept is
reestimated.

Best,
Wolfgang

-- 
Wolfgang Viechtbauer, Ph.D., Statistician | Department of Psychiatry and    
Neuropsychology | Maastricht University | P.O. Box 616 (VIJV1) | 6200 MD    
Maastricht, The Netherlands | +31 (43) 388-4170 | http://www.wvbauer.com    
>-----Original Message-----
>From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of Urs
>Kleinholdermann
>Sent: Wednesday, May 17, 2017 09:12
>To: r-help at r-project.org
>Subject: [R] adding predictor to linear model without changing existing
>coefficients
>
>Dear list members,
>
>I want to add a predictor to a linear model without changing the
>coefficients of the existing model. How is that done with R?
>
>So if I have a response y and predictors x1, x2, x3 I want to make a model
>lm1 like
>
>lm1 = lm(y~x1+x2)
>
>After this model is computed I want to add x3 like
>
>lm2 = lm(y~x1+x2+x3)
>
>However, unlike it is done by the notation above or by update or add1
>(as far as I understand) I don't want a new model with all predictors
>estimated anew but I want a model lm2 where the coefficients for x1 and
>x2 stay exactly as in lm1 and the coefficent for x3 is estimated
>additionally. The reasons for this are theoretical. I guess what I want
>is similar to calculating a new regression on the residuals of lm1.
>
>lm2 = lm(residuals(lm1)~x3)
>
>however, I would prefer to to that in the common framework of the lm
>command in order to calculate statistics, perform anova on the models
>and so on.
>
>thanks for your help!
>Urs
>
>______________________________________________
>R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide http://www.R-project.org/posting-
>guide.html
>and provide commented, minimal, self-contained, reproducible code.

Hello, Urs,

you may have seen Wolfgang Viechtbauer's answer already which offers
an R-technical solution, but this may leave the mathematical grounds
of linear models. See inline below for my concern and a hint.

Am 17.05.2017 um 09:12 schrieb Urs Kleinholdermann:
> Dear list members,
>
> I want to add a predictor to a linear model without changing the
> coefficients of the existing model. How is that done with R?
>
> So if I have a response y and predictors x1, x2, x3 I want to make a model
lm1 like
>
> lm1 = lm(y~x1+x2)
>
> After this model is computed I want to add x3 like
>
> lm2 = lm(y~x1+x2+x3)
>
> However, unlike it is done by the notation above or by update or add1
> (as far as I understand) I don't want a new model with all predictors
> estimated anew but I want a model lm2 where the coefficients for x1 and
> x2 stay exactly as in lm1 and the coefficent for x3 is estimated
> additionally. The reasons for this are theoretical.
And the reasons why this is usually impossible (for a valid linear
model as a projection of the response vector onto a linear subspace
spanned by the columns of the design matrix) are also theoretical:
It is not an R problem, but a mathematical fact that unless the vector
of values of a new model term is orthogonal to the vectors of all model
terms already included in the model (i.e., to all columns of its design
matrix) the estimated coefficients of the "old" model are correlated
with the estimated coefficient of the "new" one, and hence the already
existing ones change. So, if you manage to obtain orthogonality you can
achieve what you desire. (You may want to consult with a (theoretical)
book on linear models ... or a local statistician.)

Hth  --  Gerrit

> I guess what I want is similar to calculating a new regression on
> the residuals of lm1.
>
> lm2 = lm(residuals(lm1)~x3)
>
> however, I would prefer to to that in the common framework of the lm
> command in order to calculate statistics, perform anova on the models
> and so on.
>
> thanks for your help!
> Urs
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

You should consult a linear models text, but, assuming I have
correctly understood your post, the procedure is this (the translation
to R code is trivial, and I leave it to you):

Let y be the response variable, P1 be the first set of predictors and
z be your new predictor to be added.

1. regress y on P1; let r1 be the residuals from this model, call it M1.

2. regress z on P1 and let r2 be the residuals from this model

3. regress r1 on r2, Call this model M2

Than the predictions obtained from the model y ~ P1 + z are the same
(within numerical error) as those obtained by adding the predictions
from M1 to the predictions of M2. As Gerrit mentioned, the
coefficients from the full fit model will be different than those
obtained from the seprate model fitting procedure.

Cheers,
Bert





Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Wed, May 17, 2017 at 12:12 AM, Urs Kleinholdermann
<urs at kleinholdermann.de> wrote:
> Dear list members,
>
> I want to add a predictor to a linear model without changing the
> coefficients of the existing model. How is that done with R?
>
> So if I have a response y and predictors x1, x2, x3 I want to make a model
lm1 like
>
> lm1 = lm(y~x1+x2)
>
> After this model is computed I want to add x3 like
>
> lm2 = lm(y~x1+x2+x3)
>
> However, unlike it is done by the notation above or by update or add1
> (as far as I understand) I don't want a new model with all predictors
> estimated anew but I want a model lm2 where the coefficients for x1 and
> x2 stay exactly as in lm1 and the coefficent for x3 is estimated
> additionally. The reasons for this are theoretical. I guess what I want
> is similar to calculating a new regression on the residuals of lm1.
>
> lm2 = lm(residuals(lm1)~x3)
>
> however, I would prefer to to that in the common framework of the lm
> command in order to calculate statistics, perform anova on the models
> and so on.
>
> thanks for your help!
> Urs
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.