R help - Dec 2012 - Ridge Regression variable selection

Unlike L1 (lasso) regression or elastic net (mixture of L1 and L2), L2 norm
regression (ridge regression) does not select variables.  Selection of
variables would not work properly, and it's unclear why you would want to
omit "apparently" weak variables anyway.
Frank

maths123 wrote
> I have a .txt file containing a dataset with 500 samples. There are 10
> variables.
> 
> I am trying to perform variable selection using the ridge regression
> method but I am very confused. 
> 
> I have input the following:
> diabetes10<-read.table("diabetes10.txt", header=TRUE)
> diabetes10
> library(MASS)
> select(lm.ridge(y=diabetes10 ~ age+sex+bmi+map+tc , diabetes10,
>                lambda = seq(0,0.1,0.0001)))
> 
> First of all, i am confused about the lamda values,
> Second of all, my output is:
> 
> modified HKB estimator is -1.334073e-29 
> modified L-W estimator is -5.610557e-28 
> smallest value of GCV  at 1e-04 
> 
> 
> I have no idea what that is telling me and where I am supposed to work out
> which variables have been selected.
> 
> Someone help me out please!




-----
Frank Harrell
Department of Biostatistics, Vanderbilt University
--
View this message in context:
http://r.789695.n4.nabble.com/Ridge-Regression-variable-selection-tp4653984p4654051.html
Sent from the R help mailing list archive at Nabble.com.

Frank Harrell <f.harrell <at> vanderbilt.edu> writes:
> 
> Unlike L1 (lasso) regression or elastic net (mixture of L1 and L2), L2 norm
> regression (ridge regression) does not select variables.  Selection of
> variables would not work properly, and it's unclear why you would want
to
> omit "apparently" weak variables anyway.
> Frank
> 
  ... and this was cross-posted from StackOverflow, where I said more
or less the same thing about ridge regression (I didn't get into the
"don't do variable selection" issue yet, I was waiting ...)

http://stackoverflow.com/questions/14046569/ridge-regression-in-r

  For the other questions (what are the lambda values?  What does
the output mean?) I would suggest getting a copy of _Modern
Applied Statistics in S_ [the book that the package, MASS, was
written to accompany] and reading the relevant chapter.
> maths123 wrote
> > I have a .txt file containing a dataset with 500 samples. There are 10
> > variables.
> > 
> > I am trying to perform variable selection using the ridge regression
> > method but I am very confused. 
> > 
> > I have input the following:
> > diabetes10<-read.table("diabetes10.txt", header=TRUE)
> > diabetes10
> > library(MASS)
> > select(lm.ridge(y=diabetes10 ~ age+sex+bmi+map+tc , diabetes10,
> >                lambda = seq(0,0.1,0.0001)))
> > 
> > First of all, i am confused about the lamda values,
> > Second of all, my output is:
> > 
> > modified HKB estimator is -1.334073e-29 
> > modified L-W estimator is -5.610557e-28 
> > smallest value of GCV  at 1e-04 
> > 
> > 
> > I have no idea what that is telling me and where I am supposed to work
out
> > which variables have been selected.
> >