R-packages leaps and subselect implement various methods of selecting best or good subsets of predictor variables for linear regression models, but they do not seem to be applicable to logistic regression models. Does anyone know of software for finding good subsets of predictor variables for linear regression models? Thanks. -Ben p.s., The leaps package references "Subset Selection in Regression" by Alan Miller. On page 2 of the 2nd edition of that text it states the following: "All of the models which will be considered in this monograph will be linear; that is they will be linear in the regression coefficients.Though most of the ideas and problems carry over to the fitting of nonlinear models and generalized linear models (particularly the fitting of logistic relationships), the complexity is greatly increased."
Wittner, Ben wrote:> R-packages leaps and subselect implement various methods of selecting best or > good subsets of predictor variables for linear regression models, but they do > not seem to be applicable to logistic regression models. > > Does anyone know of software for finding good subsets of predictor variables for > linear regression models? > > Thanks. > > -BenWhy are these procedures still being used? The performance is known to be bad in almost every sense (see r-help archives). Frank Harrell> > p.s., The leaps package references "Subset Selection in Regression" by Alan > Miller. On page 2 of the > 2nd edition of that text it states the following: > > "All of the models which will be considered in this monograph will be linear; > that is they > will be linear in the regression coefficients.Though most of the ideas and > problems carry > over to the fitting of nonlinear models and generalized linear models > (particularly the fitting > of logistic relationships), the complexity is greatly increased."-- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University
> -----Original Message----- > From: r-help-bounces at stat.math.ethz.ch > [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Wittner, Ben > Sent: 02 March 2005 11:33 > To: r-help at lists.R-project.org > Subject: [R] subset selection for logistic regression > > R-packages leaps and subselect implement various methods of > selecting best or good subsets of predictor variables for > linear regression models, but they do not seem to be > applicable to logistic regression models. > > Does anyone know of software for finding good subsets of > predictor variables for linear regression models? > > Thanks. > > -Ben > > p.s., The leaps package references "Subset Selection in > Regression" by Alan Miller. On page 2 of the 2nd edition of > that text it states the following: > > "All of the models which will be considered in this > monograph will be linear; that is they > will be linear in the regression coefficients.Though most > of the ideas and problems carry > over to the fitting of nonlinear models and generalized > linear models (particularly the fitting > of logistic relationships), the complexity is greatly increased." > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html >The LASSO method and the Least Angle Regression method are two such that have both been implemented (efficiently IMHO - only one least squares for all levels of shrinkage IIRC) in the lars package for R of Hastie and Efron. There is a paper by Madigan and Ridgeway that discusses the use of the Least Angle Regresson approach in the context of logistic regression - available for download from Madigan's space at Ruttgers: www.stat.rutgers.edu/~madigan/PAPERS/lars3.pdf HTH Mike