I'm doing some linear modeling and am new to the ridge/lasso/elasticnet procedures. In my case I have N>>p (p=15 based on variables used in past literature and some physical reasoning) so my understanding is that I should be interested in ridge regression to avoid the issue of multicollinearity of predictors. Lasso is useful when p>>N. In the past I have performed step-wise regression with stepAIC in both directions to choose my variables and then used VIF to determine if any of these variables are correlated. My understanding is that ridge regression is a more robust approach for this workflow. Reading the glmnet_beta vignette, it describes the alpha parameter where alpha=1 is a lasso regression and alpha=0 is a ridge regression. Farther down the authors suggest a 10 fold validation to determine an alpha value and based on the plots shown, say that alpha=1 does the best here. However, all the models look like they approach the same MSE and alpha=0 is the lowest curve for all lambda (but maybe this second point doesn't matter?). With my data I get a very similar looking set of curves so I'm trying to decide if I should stick with alpha=1 instead of alpha=0. Is there a way to extract MSE for a lambda, e.g. lambda.1se? Any advice or clarification is appreciated. Thanks. Dominik [[alternative HTML version deleted]]
> Is there a way to extract MSE for a lambda, e.g. lambda.1se?nevermind this specific question. it's now obvious. However my overall question stands. On Fri, Sep 16, 2016 at 10:10 AM, Dominik Schneider < dominik.schneider at colorado.edu> wrote:> I'm doing some linear modeling and am new to the ridge/lasso/elasticnet > procedures. In my case I have N>>p (p=15 based on variables used in past > literature and some physical reasoning) so my understanding is that I > should be interested in ridge regression to avoid the issue of > multicollinearity of predictors. Lasso is useful when p>>N. > > In the past I have performed step-wise regression with stepAIC in both > directions to choose my variables and then used VIF to determine if any of > these variables are correlated. My understanding is that ridge regression > is a more robust approach for this workflow. > > Reading the glmnet_beta vignette, it describes the alpha parameter where > alpha=1 is a lasso regression and alpha=0 is a ridge regression. Farther > down the authors suggest a 10 fold validation to determine an alpha value > and based on the plots shown, say that alpha=1 does the best here. However, > all the models look like they approach the same MSE and alpha=0 is the > lowest curve for all lambda (but maybe this second point doesn't matter?). > With my data I get a very similar looking set of curves so I'm trying to > decide if I should stick with alpha=1 instead of alpha=0. Is there a way to > extract MSE for a lambda, e.g. lambda.1se? > > Any advice or clarification is appreciated. Thanks. > Dominik > >[[alternative HTML version deleted]]
You seem to be mainly asking for help with statistical methodology, which is generally off topic for this list, which is about help with R programming. I suggest you study the references given in the vignette/package and/or post to a statistical list like stats.stackexchange.com instead. Cheers, Bert Bert Gunter "The trouble with having an open mind is that people keep coming along and sticking things into it." -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) On Fri, Sep 16, 2016 at 9:33 AM, Dominik Schneider <dominik.schneider at colorado.edu> wrote:>> Is there a way to extract MSE for a lambda, e.g. lambda.1se? > nevermind this specific question. it's now obvious. However my overall > question stands. > > On Fri, Sep 16, 2016 at 10:10 AM, Dominik Schneider < > dominik.schneider at colorado.edu> wrote: > >> I'm doing some linear modeling and am new to the ridge/lasso/elasticnet >> procedures. In my case I have N>>p (p=15 based on variables used in past >> literature and some physical reasoning) so my understanding is that I >> should be interested in ridge regression to avoid the issue of >> multicollinearity of predictors. Lasso is useful when p>>N. >> >> In the past I have performed step-wise regression with stepAIC in both >> directions to choose my variables and then used VIF to determine if any of >> these variables are correlated. My understanding is that ridge regression >> is a more robust approach for this workflow. >> >> Reading the glmnet_beta vignette, it describes the alpha parameter where >> alpha=1 is a lasso regression and alpha=0 is a ridge regression. Farther >> down the authors suggest a 10 fold validation to determine an alpha value >> and based on the plots shown, say that alpha=1 does the best here. However, >> all the models look like they approach the same MSE and alpha=0 is the >> lowest curve for all lambda (but maybe this second point doesn't matter?). >> With my data I get a very similar looking set of curves so I'm trying to >> decide if I should stick with alpha=1 instead of alpha=0. Is there a way to >> extract MSE for a lambda, e.g. lambda.1se? >> >> Any advice or clarification is appreciated. Thanks. >> Dominik >> >> > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.