[R] Ridge regression and mixed models

harezlak@post.harvard.edu
Mon, 04 Oct 2010 20:39:02 -0000

Dear R users,
  An equivalence between linear mixed model formulation and penalized regression
models (including the ridge regression and penalized regression splines) has
proven to be very useful in many aspects. Examples include the use of the lme()
function in the library(nlme) to fit smooth models including the estimation of a
smoothing parameter using REML. My question concerns the use of the linear mixed
model software to fit a ridge regression with the number of columns in the
design matrix X (p) exceeding the number of observations (n). Has anybody in the
R community implemented the LME-like approach with estimation of the variance
components using REML to find the coefficient estimates (BLUEs) and predictors
(BLUPs) in the ridge regression problem in the "p > n"  setting?

Sample code below summarizes my problem:
####################################################
version$version.string
# [1] "R version 2.11.1 (2010-05-31)"

library(nlme)

# DATA generation:
dim <- 200
n <- 50
XX <- matrix(rnorm(dim*n, 0, 0.1), ncol=dim, nrow=n)
beta <- matrix(c(rep(1, 40), rep(2,20), rep(0,140)), ncol=1)
Y <- XX %*% beta + rnorm(n)

# MODEL fit:
dummyId <- factor(rep(1,n))
Z.block <- list(dummyId=pdIdent(~-1+XX))
data.fr <- data.frame(Y,XX)
fit <- lme(Y~1,
        data=data.fr,
        random=Z.block)

# ERROR:
Warning message:
In lme.formula(Y ~ 1, data = data.fr, random = Z.block) :
  Fewer observations than random effects in all level 1 groups
#############################################################

Thank you in advance,
Jarek  Harezlak



	[[alternative HTML version deleted]]