李俊杰
2007-May-21 01:53 UTC
[R] How to compare linear models with intercept and those without intercept using minimizing adjs R^2 strategy
Dear R-list, I apologize for my many emails but I think I know how to desctribe my problem differently and more clearly. My question is how to compare linear models with intercept and those without intercept using maximizing adjusted R^2 strategy. Now I do it like the following:> library(leaps) > n=20 > x=matrix(rnorm(n*3),ncol=3) > b=c(1,2,0) > intercept=1 > y=x%*%b+rnorm(n,0,1)+intercept > > var.selection=leaps(cbind(rep(1,n),x),y,int=F,method="adjr2") > ##### Choose the model with maximum adjr2 > var.selection$which[var.selection$adjr2==max(var.selection$adjr2),]1 2 3 4 TRUE TRUE TRUE FALSE Actually, I use the definition of R-square in which the model is without a intercept term. Is what I am doing is correct? Thanks for any suggestion or correction. -- Junjie Li, klijunjie@gmail.com Undergranduate in DEP of Tsinghua University, [[alternative HTML version deleted]]
Lucke, Joseph F
2007-May-21 14:53 UTC
[R] How to compare linear models with intercept and those withoutintercept using minimizing adjs R^2 strategy
One issue is whether you want your estimators to be based on central moments (covariances) or on non-central moments. Removing the intercept changes the statistics from central to non-central moments. The adjusted R2, by which I think you mean Fisher's adjusted R2, is based on central moments (ratio of unbiased estimators of variances---central moments). So if you remove the intercept, you must re-derive the adjusted R2 for non-central moments --- you can't just plug in the number of independent variables as zero. -----Original Message----- From: r-help-bounces at stat.math.ethz.ch [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of ??? Sent: Sunday, May 20, 2007 8:53 PM To: r-help at stat.math.ethz.ch Subject: [R] How to compare linear models with intercept and those withoutintercept using minimizing adjs R^2 strategy Dear R-list, I apologize for my many emails but I think I know how to desctribe my problem differently and more clearly. My question is how to compare linear models with intercept and those without intercept using maximizing adjusted R^2 strategy. Now I do it like the following:> library(leaps) > n=20 > x=matrix(rnorm(n*3),ncol=3) > b=c(1,2,0) > intercept=1 > y=x%*%b+rnorm(n,0,1)+intercept > > var.selection=leaps(cbind(rep(1,n),x),y,int=F,method="adjr2") > ##### Choose the model with maximum adjr2 > var.selection$which[var.selection$adjr2==max(var.selection$adjr2),]1 2 3 4 TRUE TRUE TRUE FALSE Actually, I use the definition of R-square in which the model is without a intercept term. Is what I am doing is correct? Thanks for any suggestion or correction. -- Junjie Li, klijunjie at gmail.com Undergranduate in DEP of Tsinghua University, [[alternative HTML version deleted]] ______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
李俊杰
2007-May-21 15:42 UTC
[R] How to compare linear models with intercept and those withoutintercept using minimizing adjs R^2 strategy
I have a question about what you've wrote in your pdf file. Why must we view my problem in the viewpoint of hypothesis testing? Is testing the original philosophy of maximizing Fisher's A-statistic to choose a optimum model? Thanks. 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke@uth.tmc.edu>:> > I taken the conversation offline and used a pdf file to better display > equations. > > ------------------------------ > *From:* Àî¿¡½Ü [mailto:klijunjie@gmail.com] > *Sent:* Monday, May 21, 2007 10:14 AM > *To:* Lucke, Joseph F > *Cc:* r-help@stat.math.ethz.ch > *Subject:* Re: [R] How to compare linear models with intercept and those > withoutintercept using minimizing adjs R^2 strategy > > > > > 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke@uth.tmc.edu>: > > > > One issue is whether you want your estimators to be based on central > > moments (covariances) or on non-central moments. Removing the intercept > > > > changes the statistics from central to non-central moments. The > > adjusted R2, by which I think you mean Fisher's adjusted R2, is based on > > central moments (ratio of unbiased estimators of variances---central > > moments). So if you remove the intercept, you must re-derive the > > adjusted R2 for non-central moments --- you can't just plug in the > > number of independent variables as zero. > > > I have consulted A.J. Miller's Subset Selection in Regression(1990), and I > found what I was talking about adjusted R^2 was exactly as you > said--Fisher's A-statisitc. The formula of adjusted R^2 without the > intercept in that book was also the same as what summary(lm)$adj.r.squared > does in R. I guess what you want me to derive is the formula in that book. > > Though I know the formula of adjusted R2 for non-central moments, I still > want to know whether I am in the right way to compare *linear models with > intercept and those without intercept using maximizing adjs R^2 strategy. > * > ** > Actually, I consider the left column consisted of all 1 in predictor > matrix Z as the intercept term. Then I apply maximizing adjs R^2 strategy > to decide which variables to select. Z is the term in the model: Y=Zb+e. > > Thanks for your suggestion, and I am looking forward for your reply. > > > > -----Original Message----- > > From: r-help-bounces@stat.math.ethz.ch > > [mailto:r-help-bounces@stat.math.ethz.ch] On Behalf Of ??? > > Sent: Sunday, May 20, 2007 8:53 PM > > To: r-help@stat.math.ethz.ch > > Subject: [R] How to compare linear models with intercept and those > > withoutintercept using minimizing adjs R^2 strategy > > > > Dear R-list, > > > > I apologize for my many emails but I think I know how to desctribe my > > problem differently and more clearly. > > > > My question is how to compare linear models with intercept and those > > without intercept using maximizing adjusted R^2 strategy. > > > > Now I do it like the following: > > > > > library(leaps) > > > n=20 > > > x=matrix(rnorm(n*3),ncol=3) > > > b=c(1,2,0) > > > intercept=1 > > > y=x%*%b+rnorm(n,0,1)+intercept > > > > > > var.selection=leaps(cbind(rep(1,n),x),y,int=F,method="adjr2") > > > ##### Choose the model with maximum adjr2 > > > var.selection$which[var.selection$adjr2==max(var.selection$adjr2),] > > 1 2 3 4 > > TRUE TRUE TRUE FALSE > > > > > > Actually, I use the definition of R-square in which the model is without > > > > a intercept term. > > > > Is what I am doing is correct? > > > > Thanks for any suggestion or correction. > > -- > > Junjie Li, klijunjie@gmail.com > > Undergranduate in DEP of Tsinghua University, > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > R-help@stat.math.ethz.ch mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > > http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html> > > and provide commented, minimal, self-contained, reproducible code. > > > > > > -- > Junjie Li, klijunjie@gmail.com > Undergranduate in DEP of Tsinghua University, > >-- Junjie Li, klijunjie@gmail.com Undergranduate in DEP of Tsinghua University, [[alternative HTML version deleted]]
李俊杰
2007-May-21 16:34 UTC
[R] How to compare linear models with intercept and those withoutintercept using minimizing adjs R^2 strategy
So when I am using the adjusted R2 and as a penalized optimality criterion, and I have to compare models with intercept and those without intercept to decide the final model selected, does my crierion in my first email make sense? Because we know that in leaps(leaps), if we want to select a model by the adjusted R2 criterion, we have to decide whether the intercept should be added in advance. But with my adjusted R2 criterion, we don't have to decide that in advance. Thank you so much for your patient clarification. 2007/5/22, Lucke, Joseph F <Joseph.F.Lucke@uth.tmc.edu>:> > You don't have to embed model selection as hypothesis testing. You are > using the adjusted R2 and as a penalized optimality criterion. > > ------------------------------ > *From:* Àî¿¡½Ü [mailto:klijunjie@gmail.com] > *Sent:* Monday, May 21, 2007 10:43 AM > *To:* Lucke, Joseph F > *Cc:* r-help@stat.math.ethz.ch > *Subject:* Re: [R] How to compare linear models with intercept and those > withoutintercept using minimizing adjs R^2 strategy > > > I have a question about what you've wrote in your pdf file. Why must we > view my problem in the viewpoint of hypothesis testing? Is testing the > original philosophy of maximizing Fisher's A-statistic to choose a optimum > model? > > Thanks. > > > 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke@uth.tmc.edu>: > > > > I taken the conversation offline and used a pdf file to better display > > equations. > > > > ------------------------------ > > *From:* Àî¿¡½Ü [mailto:klijunjie@gmail.com] > > *Sent: *Monday, May 21, 2007 10:14 AM > > *To:* Lucke, Joseph F > > *Cc:* r-help@stat.math.ethz.ch > > *Subject:* Re: [R] How to compare linear models with intercept and those > > withoutintercept using minimizing adjs R^2 strategy > > > > > > > > > > 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke@uth.tmc.edu>: > > > > > > One issue is whether you want your estimators to be based on central > > > moments (covariances) or on non-central moments. Removing the > > > intercept > > > changes the statistics from central to non-central moments. The > > > adjusted R2, by which I think you mean Fisher's adjusted R2, is based > > > on > > > central moments (ratio of unbiased estimators of variances---central > > > moments). So if you remove the intercept, you must re-derive the > > > adjusted R2 for non-central moments --- you can't just plug in the > > > number of independent variables as zero. > > > > > > I have consulted A.J. Miller's Subset Selection in Regression(1990), and > > I found what I was talking about adjusted R^2 was exactly as you > > said--Fisher's A-statisitc. The formula of adjusted R^2 without the > > intercept in that book was also the same as what summary(lm)$adj.r.squared > > does in R. I guess what you want me to derive is the formula in that book. > > > > Though I know the formula of adjusted R2 for non-central moments, I > > still want to know whether I am in the right way to compare *linear > > models with intercept and those without intercept using maximizing adjs R^2 > > strategy. * > > ** > > Actually, I consider the left column consisted of all 1 in predictor > > matrix Z as the intercept term. Then I apply maximizing adjs R^2 > > strategy to decide which variables to select. Z is the term in the model: > > Y=Zb+e. > > > > Thanks for your suggestion, and I am looking forward for your reply. > > > > > > > > -----Original Message----- > > > From: r-help-bounces@stat.math.ethz.ch > > > [mailto:r-help-bounces@stat.math.ethz.ch] On Behalf Of ??? > > > Sent: Sunday, May 20, 2007 8:53 PM > > > To: r-help@stat.math.ethz.ch > > > Subject: [R] How to compare linear models with intercept and those > > > withoutintercept using minimizing adjs R^2 strategy > > > > > > Dear R-list, > > > > > > I apologize for my many emails but I think I know how to desctribe my > > > problem differently and more clearly. > > > > > > My question is how to compare linear models with intercept and those > > > without intercept using maximizing adjusted R^2 strategy. > > > > > > Now I do it like the following: > > > > > > > library(leaps) > > > > n=20 > > > > x=matrix(rnorm(n*3),ncol=3) > > > > b=c(1,2,0) > > > > intercept=1 > > > > y=x%*%b+rnorm(n,0,1)+intercept > > > > > > > > var.selection=leaps(cbind(rep(1,n),x),y,int=F,method="adjr2") > > > > ##### Choose the model with maximum adjr2 > > > > var.selection$which[var.selection$adjr2==max(var.selection$adjr2),] > > > 1 2 3 4 > > > TRUE TRUE TRUE FALSE > > > > > > > > > Actually, I use the definition of R-square in which the model is > > > without > > > a intercept term. > > > > > > Is what I am doing is correct? > > > > > > Thanks for any suggestion or correction. > > > -- > > > Junjie Li, klijunjie@gmail.com > > > Undergranduate in DEP of Tsinghua University, > > > > > > [[alternative HTML version deleted]] > > > > > > ______________________________________________ > > > R-help@stat.math.ethz.ch mailing list > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > PLEASE do read the posting guide > > > http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html> > > > and provide commented, minimal, self-contained, reproducible code. > > > > > > > > > > > -- > > Junjie Li, klijunjie@gmail.com > > Undergranduate in DEP of Tsinghua University, > > > > > > > -- > Junjie Li, klijunjie@gmail.com > Undergranduate in DEP of Tsinghua University, >-- Junjie Li, klijunjie@gmail.com Undergranduate in DEP of Tsinghua University, [[alternative HTML version deleted]]
李俊杰
2007-May-24 06:29 UTC
[R] How to compare linear models with intercept and those withoutintercept using minimizing adjs R^2 strategy
2007/5/24, Lucke, Joseph F <Joseph.F.Lucke at uth.tmc.edu>:> > > > ------------------------------ > *From:* ?????? [mailto:klijunjie at gmail.com] > *Sent:* Monday, May 21, 2007 8:12 PM > *To:* Lucke, Joseph F > *Subject:* Re: [R] How to compare linear models with intercept and those > withoutintercept using minimizing adjs R^2 strategy > > > > > 2007/5/22, Lucke, Joseph F <Joseph.F.Lucke at uth.tmc.edu>: > > > > Bottom line: > > You use the adjusted R2 with the intercept in your leaps(). If the case > > arises that the the intercept-only model (no other predictors) is the result > > of you leaps(), then you may test for whether the intercept itself is zero. > > > > > > You cannot compare models with predictors for intercept versus no > > intercept as this violates marginality. > > > > I've wrote my question in pdf file. Sorry for my slow understanding. > > > > ------------------------------ > > *From:* ?????? [mailto:klijunjie at gmail.com] > > *Sent: *Monday, May 21, 2007 11:34 AM > > *To:* Lucke, Joseph F > > *Cc:* r-help at stat.math.ethz.ch > > *Subject:* Re: [R] How to compare linear models with intercept and those > > withoutintercept using minimizing adjs R^2 strategy > > > > > > So when I am using the adjusted R2 and as a penalized optimality > > criterion, and I have to compare models with intercept and those without > > intercept to decide the final model selected, does my crierion in my > > first email make sense? > > > > Because we know that in leaps(leaps), if we want to select a model by > > the adjusted R2 criterion, we have to decide whether the intercept should be > > added in advance. But with my adjusted R2 criterion, we don't have to decide > > that in advance. > > > > Thank you so much for your patient clarification. > > > > > > > > 2007/5/22, Lucke, Joseph F <Joseph.F.Lucke at uth.tmc.edu>: > > > > > > You don't have to embed model selection as hypothesis testing. You > > > are using the adjusted R2 and as a penalized optimality criterion. > > > > > > ------------------------------ > > > *From:* ?????? [mailto:klijunjie at gmail.com] > > > *Sent: *Monday, May 21, 2007 10:43 AM > > > *To:* Lucke, Joseph F > > > *Cc:* r-help at stat.math.ethz.ch > > > *Subject:* Re: [R] How to compare linear models with intercept and > > > those withoutintercept using minimizing adjs R^2 strategy > > > > > > > > > I have a question about what you've wrote in your pdf file. Why must > > > we view my problem in the viewpoint of hypothesis testing? Is testing the > > > original philosophy of maximizing Fisher's A-statistic to choose a optimum > > > model? > > > > > > Thanks. > > > > > > > > > 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke at uth.tmc.edu>: > > > > > > > > I taken the conversation offline and used a pdf file to better > > > > display equations. > > > > > > > > ------------------------------ > > > > *From:* ?????? [mailto:klijunjie at gmail.com] > > > > *Sent: *Monday, May 21, 2007 10:14 AM > > > > *To:* Lucke, Joseph F > > > > *Cc:* r-help at stat.math.ethz.ch > > > > *Subject:* Re: [R] How to compare linear models with intercept and > > > > those withoutintercept using minimizing adjs R^2 strategy > > > > > > > > > > > > > > > > > > > > 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke at uth.tmc.edu>: > > > > > > > > > > One issue is whether you want your estimators to be based on > > > > > central > > > > > moments (covariances) or on non-central moments. Removing the > > > > > intercept > > > > > changes the statistics from central to non-central moments. The > > > > > adjusted R2, by which I think you mean Fisher's adjusted R2, is > > > > > based on > > > > > central moments (ratio of unbiased estimators of > > > > > variances---central > > > > > moments). So if you remove the intercept, you must re-derive the > > > > > adjusted R2 for non-central moments --- you can't just plug in the > > > > > number of independent variables as zero. > > > > > > > > > > > > I have consulted A.J. Miller's Subset Selection in Regression(1990), > > > > and I found what I was talking about adjusted R^2 was exactly as you > > > > said--Fisher's A-statisitc. The formula of adjusted R^2 without the > > > > intercept in that book was also the same as what summary(lm)$adj.r.squared > > > > does in R. I guess what you want me to derive is the formula in that book. > > > > > > > > Though I know the formula of adjusted R2 for non-central moments, I > > > > still want to know whether I am in the right way to compare *linear > > > > models with intercept and those without intercept using maximizing adjs R^2 > > > > strategy. * > > > > ** > > > > Actually, I consider the left column consisted of all 1 in > > > > predictor matrix Z as the intercept term. Then I apply maximizing > > > > adjs R^2 strategy to decide which variables to select. Z is the term in the > > > > model: Y=Zb+e. > > > > > > > > Thanks for your suggestion, and I am looking forward for your reply. > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: r-help-bounces at stat.math.ethz.ch > > > > > [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of ??? > > > > > Sent: Sunday, May 20, 2007 8:53 PM > > > > > To: r-help at stat.math.ethz.ch > > > > > Subject: [R] How to compare linear models with intercept and those > > > > > > > > > > withoutintercept using minimizing adjs R^2 strategy > > > > > > > > > > Dear R-list, > > > > > > > > > > I apologize for my many emails but I think I know how to desctribe > > > > > my > > > > > problem differently and more clearly. > > > > > > > > > > My question is how to compare linear models with intercept and > > > > > those > > > > > without intercept using maximizing adjusted R^2 strategy. > > > > > > > > > > Now I do it like the following: > > > > > > > > > > > library(leaps) > > > > > > n=20 > > > > > > x=matrix(rnorm(n*3),ncol=3) > > > > > > b=c(1,2,0) > > > > > > intercept=1 > > > > > > y=x%*%b+rnorm(n,0,1)+intercept > > > > > > > > > > > > var.selection=leaps(cbind(rep(1,n),x),y,int=F,method="adjr2") > > > > > > ##### Choose the model with maximum adjr2 > > > > > > var.selection$which[var.selection$adjr2==max(var.selection$adjr2 > > > > > ),] > > > > > 1 2 3 4 > > > > > TRUE TRUE TRUE FALSE > > > > > > > > > > > > > > > Actually, I use the definition of R-square in which the model is > > > > > without > > > > > a intercept term. > > > > > > > > > > Is what I am doing is correct? > > > > > > > > > > Thanks for any suggestion or correction. > > > > > -- > > > > > Junjie Li, klijunjie at gmail.com > > > > > Undergranduate in DEP of Tsinghua University, > > > > > > > > > > [[alternative HTML version deleted]] > > > > > > > > > > ______________________________________________ > > > > > R-help at stat.math.ethz.ch mailing list > > > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > > > PLEASE do read the posting guide > > > > > http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html> > > > > > and provide commented, minimal, self-contained, reproducible code. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Junjie Li, klijunjie at gmail.com > > > > Undergranduate in DEP of Tsinghua University, > > > > > > > > > > > > > > > > > -- > > > Junjie Li, klijunjie at gmail.com > > > Undergranduate in DEP of Tsinghua University, > > > > > > > > > > > -- > > Junjie Li, klijunjie at gmail.com > > Undergranduate in DEP of Tsinghua University, > > > > > > -- > Junjie Li, klijunjie at gmail.com > Undergranduate in DEP of Tsinghua University, > >-- Junjie Li, klijunjie at gmail.com Undergranduate in DEP of Tsinghua University, -------------- next part -------------- A non-text attachment was scrubbed... Name: ZeroIntercept_query2.pdf Type: application/pdf Size: 7713 bytes Desc: not available Url : https://stat.ethz.ch/pipermail/r-help/attachments/20070524/be500e69/attachment.pdf