R help - Jun 2013 - Coefficients paths - comparison of ridge, lasso and elastic net regression

I would like to compare models selected with ridge, lasso and elastic net.
Fig. below shows coefficients paths using all 3 methods: ridge (Fig A,
alpha=0), lasso (Fig B; alpha=1) and elastic net (Fig C; alpha=0.5). The
optimal solution depends on the selected value of lambda, which is chosen
based on cross validation.

<http://r.789695.n4.nabble.com/file/n4668722/regularization.jpg> 

When looking at these plots, I would expect the elastic net (Fig C) to
exhibit a grouping effect. However it is not clear in the presented case.
The coefficients path for lasso and elastic net are very similar. What could
be the reason for this ? Is it just a coding mistake ? I used the following
code in R:


library(glmnet)
X<- as.matrix(mydata[,2:22])
Y<- mydata[,23]
par(mfrow=c(1,3))
ans1<-cv.glmnet(X, Y, alpha=0) # ridge
plot(ans1$glmnet.fit, "lambda", label=FALSE)
    text (6, 0.4, "A", cex=1.8, font=1)
    ans2<-cv.glmnet(X, Y, alpha=1) # lasso
    plot(ans2$glmnet.fit, "lambda", label=FALSE)
text (-0.8, 0.48, "B", cex=1.8, font=1)
ans3<-cv.glmnet(X, Y, alpha=0.5) # elastic net 
plot(ans3$glmnet.fit, "lambda", label=FALSE)
text (0, 0.62, "C", cex=1.8, font=1)


The code used to plot elastic net coefficients paths is exactly the same as
for ridge and lasso. The only difference is in the value of alpha. Alpha
parameter for elastic net regression was selected based on the lowest MSE
(mean squared error) for corresponding lambda values.

Thank you for your help !



--
View this message in context:
http://r.789695.n4.nabble.com/Coefficients-paths-comparison-of-ridge-lasso-and-elastic-net-regression-tp4668722.html
Sent from the R help mailing list archive at Nabble.com.

Hi, beginner
Do you have reproducible data?  I think your question is more related to
statistical learning theory than R.  You may want to watch Prof.Hastie's
webinar.  http://www.stanford.edu/~hastie/lectures.htm


On Wed, Jun 5, 2013 at 10:22 AM, beginner <paxkn@nottingham.ac.uk> wrote:
> I would like to compare models selected with ridge, lasso and elastic net.
> Fig. below shows coefficients paths using all 3 methods: ridge (Fig A,
> alpha=0), lasso (Fig B; alpha=1) and elastic net (Fig C; alpha=0.5). The
> optimal solution depends on the selected value of lambda, which is chosen
> based on cross validation.
>
> <http://r.789695.n4.nabble.com/file/n4668722/regularization.jpg>
>
> When looking at these plots, I would expect the elastic net (Fig C) to
> exhibit a grouping effect. However it is not clear in the presented case.
> The coefficients path for lasso and elastic net are very similar. What
> could
> be the reason for this ? Is it just a coding mistake ? I used the following
> code in R:
>
>
> library(glmnet)
> X<- as.matrix(mydata[,2:22])
> Y<- mydata[,23]
> par(mfrow=c(1,3))
> ans1<-cv.glmnet(X, Y, alpha=0) # ridge
> plot(ans1$glmnet.fit, "lambda", label=FALSE)
>     text (6, 0.4, "A", cex=1.8, font=1)
>     ans2<-cv.glmnet(X, Y, alpha=1) # lasso
>     plot(ans2$glmnet.fit, "lambda", label=FALSE)
> text (-0.8, 0.48, "B", cex=1.8, font=1)
> ans3<-cv.glmnet(X, Y, alpha=0.5) # elastic net
> plot(ans3$glmnet.fit, "lambda", label=FALSE)
> text (0, 0.62, "C", cex=1.8, font=1)
>
>
> The code used to plot elastic net coefficients paths is exactly the same as
> for ridge and lasso. The only difference is in the value of alpha. Alpha
> parameter for elastic net regression was selected based on the lowest MSE
> (mean squared error) for corresponding lambda values.
>
> Thank you for your help !
>
>
>
> --
> View this message in context:
>
http://r.789695.n4.nabble.com/Coefficients-paths-comparison-of-ridge-lasso-and-elastic-net-regression-tp4668722.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
	[[alternative HTML version deleted]]