sikander hayat
2007-Apr-17 14:38 UTC
[R] value of complexity parameter in ridge regression
Hi, What is the optimum range to look for a value of lambda while doing ridge regression. Can/ should lambda be greater than 1 ? I have conflicting (or what appears conflicting to me) sources that use lambda >= 0, without any upper limit, but that makes the search space infinite.. right ?? So, perhaps my question is: is there an upper limit to lambda. Does the value of lambda convey something about my data ? Thanks a lot, Sikander --------------------------------- [[alternative HTML version deleted]]
> What is the optimum range to look for a value of lambda while doing ridge > regression. Can/ should lambda be greater than 1 ?-- I think it's data dependent, but lambda can certainly be greater than one. For many ridge regression problems you can choose lambda `objectively' by generalized cross validation (GCV). package `mgcv' provides a routine `magic' for doing this (although it doesn't use the most efficient method if you only have one lambda/ridge penalty). Of course this won't be appropriate if the ridge penalty is only being used to stabilize the fit and you want the minimum lambda that e.g. makes X'X+ \lambda I +ve definite.> I have conflicting (or what appears conflicting to me) sources that use > lambda >= 0, without any upper limit, but that makes the search space > infinite.. right ?? > > So, perhaps my question is: is there an upper limit to lambda.I don't think so.> Does the value of lambda convey something about my data ?Depends on the details of the model. For some models ridge penalties can be viewed as inverses of random effect covariance matrices, in which case lambda is related to the random effect variance. (see e.g. section 6.2.6. of 2006 book referenced in ?gam from mgcv package). best, Simon> > Thanks a lot, > Sikander > > > --------------------------------- > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html and provide commented, minimal, > self-contained, reproducible code.--> Simon Wood, Mathematical Sciences, University of Bath, Bath, BA2 7AY UK > +44 1225 386603 www.maths.bath.ac.uk/~sw283