R help - Oct 2010 - Ridge regression and mixed models

Dear R users,
  An equivalence between linear mixed model formulation and penalized regression
models (including the ridge regression and penalized regression splines) has 
proven to be very useful in many aspects. Examples include the use of the lme() 
function in the library(nlme) to fit smooth models including the estimation of a
smoothing parameter using REML. My question concerns the use of the linear mixed
model software to fit a ridge regression with the number of columns in the 
design matrix X (p) exceeding the number of observations (n). Has anybody in the
R community implemented the LME-like approach with estimation of the variance 
components using REML to find the coefficient estimates (BLUEs) and predictors 
(BLUPs) in the ridge regression problem in the "p > n"  setting?

Sample code below summarizes my problem:
####################################################
version$version.string
# [1] "R version 2.11.1 (2010-05-31)"

library(nlme)

# DATA generation:
dim <- 200
n <- 50
XX <- matrix(rnorm(dim*n, 0, 0.1), ncol=dim, nrow=n)
beta <- matrix(c(rep(1, 40), rep(2,20), rep(0,140)), ncol=1)
Y <- XX %*% beta + rnorm(n)

# MODEL fit:
dummyId <- factor(rep(1,n))
Z.block <- list(dummyId=pdIdent(~-1+XX))
data.fr <- data.frame(Y,XX)
fit <- lme(Y~1,
        data=data.fr, 
        random=Z.block)

# ERROR:
Warning message:
In lme.formula(Y ~ 1, data = data.fr, random = Z.block) :
  Fewer observations than random effects in all level 1 groups
#############################################################

Thank you in advance,
Jarek  Harezlak


      
	[[alternative HTML version deleted]]

Curious - what would be the purpose of this regression?

On Mon, Oct 4, 2010 at 4:39 PM, harezlak at post.harvard.edu
<jarek67h at yahoo.com> wrote:
> Dear R users,
> ?An equivalence between linear mixed model formulation and penalized
regression
> models (including the ridge regression and penalized regression splines)
has
> proven to be very useful in many aspects. Examples include the use of the
lme()
> function in the library(nlme) to fit smooth models including the estimation
of a
> smoothing parameter using REML. My question concerns the use of the linear
mixed
> model software to fit a ridge regression with the number of columns in the
> design matrix X (p) exceeding the number of observations (n). Has anybody
in the
> R community implemented the LME-like approach with estimation of the
variance
> components using REML to find the coefficient estimates (BLUEs) and
predictors
> (BLUPs) in the ridge regression problem in the "p > n"
?setting?
>
> Sample code below summarizes my problem:
> ####################################################
> version$version.string
> # [1] "R version 2.11.1 (2010-05-31)"
>
> library(nlme)
>
> # DATA generation:
> dim <- 200
> n <- 50
> XX <- matrix(rnorm(dim*n, 0, 0.1), ncol=dim, nrow=n)
> beta <- matrix(c(rep(1, 40), rep(2,20), rep(0,140)), ncol=1)
> Y <- XX %*% beta + rnorm(n)
>
> # MODEL fit:
> dummyId <- factor(rep(1,n))
> Z.block <- list(dummyId=pdIdent(~-1+XX))
> data.fr <- data.frame(Y,XX)
> fit <- lme(Y~1,
> ? ? ? ?data=data.fr,
> ? ? ? ?random=Z.block)
>
> # ERROR:
> Warning message:
> In lme.formula(Y ~ 1, data = data.fr, random = Z.block) :
> ?Fewer observations than random effects in all level 1 groups
> #############################################################
>
> Thank you in advance,
> Jarek ?Harezlak
>
>
>
> ? ? ? ?[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Dimitri Liakhovitski
Ninah Consulting
www.ninah.com