R help - Mar 2005 - subset selection for logistic regression

R-packages leaps and subselect implement various methods of selecting best or
good subsets of predictor variables for linear regression models, but they do
not seem to be applicable to logistic regression models.
 
Does anyone know of software for finding good subsets of predictor variables for
linear regression models?
 
Thanks.
 
-Ben
 
p.s., The leaps package references "Subset Selection in Regression" by
Alan
Miller. On page 2 of the
2nd edition of that text it states the following:
 
  "All of the models which will be considered in this monograph will be
linear;
that is they
   will be linear in the regression coefficients.Though most of the ideas and
problems carry
   over to the fitting of nonlinear models and generalized linear models
(particularly the fitting
   of logistic relationships), the complexity is greatly increased."

Wittner, Ben wrote:
> R-packages leaps and subselect implement various methods of selecting best
or
> good subsets of predictor variables for linear regression models, but they
do
> not seem to be applicable to logistic regression models.
>  
> Does anyone know of software for finding good subsets of predictor
variables for
> linear regression models?
>  
> Thanks.
>  
> -Ben
Why are these procedures still being used?  The performance is known to 
be bad in almost every sense (see r-help archives).

Frank Harrell
>  
> p.s., The leaps package references "Subset Selection in
Regression" by Alan
> Miller. On page 2 of the
> 2nd edition of that text it states the following:
>  
>   "All of the models which will be considered in this monograph will
be linear;
> that is they
>    will be linear in the regression coefficients.Though most of the ideas
and
> problems carry
>    over to the fitting of nonlinear models and generalized linear models
> (particularly the fitting
>    of logistic relationships), the complexity is greatly increased."

-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University

> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch 
> [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Wittner, Ben
> Sent: 02 March 2005 11:33
> To: r-help at lists.R-project.org
> Subject: [R] subset selection for logistic regression
> 
> R-packages leaps and subselect implement various methods of 
> selecting best or good subsets of predictor variables for 
> linear regression models, but they do not seem to be 
> applicable to logistic regression models.
>  
> Does anyone know of software for finding good subsets of 
> predictor variables for linear regression models?
>  
> Thanks.
>  
> -Ben
>  
> p.s., The leaps package references "Subset Selection in 
> Regression" by Alan Miller. On page 2 of the 2nd edition of 
> that text it states the following:
>  
>   "All of the models which will be considered in this 
> monograph will be linear; that is they
>    will be linear in the regression coefficients.Though most 
> of the ideas and problems carry
>    over to the fitting of nonlinear models and generalized 
> linear models (particularly the fitting
>    of logistic relationships), the complexity is greatly increased."
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
> 
The LASSO method and the Least Angle Regression method are two such that
have both been implemented (efficiently IMHO - only one least squares for
all levels of shrinkage IIRC) in the lars package for R of Hastie and Efron.
There is a paper by Madigan and Ridgeway that discusses the use of the Least
Angle Regresson approach in the context of logistic regression - available
for download from Madigan's space at Ruttgers: 
www.stat.rutgers.edu/~madigan/PAPERS/lars3.pdf 

HTH

Mike