R help - May 2009 - Linear Regression with Constraints

Hi!
I am a bit new to R.
I am looking for the right function to use for a multiple regression problem
of the form:

y = c1 + x1 + (c2 * x2) - (c3 * x3)

Where c1, c2, and c3 are the desired regression coefficients that are
subject to the following constraints:

0.0 < c2 < 1.0, and
0.0 < c3 < 1.0

y, x1, x2, and x3 are observed data.  
I have a total of 6 rows of data in a data set.

Is "optim" in the stats package the right function to use?
Also, I can't quite figure out how to specify the constraints.
Thank you!

-Stu

Here is a demonstration of how to solve your problem :


n <- 30  # You might need more than 6 data points to get good estimates for
3 parameters

x1 <- rnorm(n)

x2 <- runif(n)

x3 <- rbinom(n, size=1, prob=0.4)

A <- cbind(x1, x2, x3) # 30 x 3 matrix of independent variables

b <- c(-1, 0.5, 0.2) # Note: the last component is out of bounds!

y <- A %*% b + rnorm(n, sd=0.1)

qr.solve(A, y)  # unconstrained LS solution

# Implementing the bounds (there is probably a better way to do this)
#
nc <- ncol(A)

c1 <- matrix(0, nc, nc)

diag(c1) <- 1

c2 <- matrix(0, nc, nc)

diag(c2) <- -1

cmat <- rbind(c1, c2)

Cmat <- cmat[c(2,5,3,6), ]  # Constraint matrix G

b0 <- c(0, -1, -1, 0)

 require(limSolve)

ans <- lsei(A = A, B = y, G = Cmat, H = b0)

ans 

While ans$X gives you the point estimates, it is a bit tricky to get
standard errors.  
 

Hope this helps,
Ravi.

----------------------------------------------------------------------------
-------

Ravi Varadhan, Ph.D.

Assistant Professor, The Center on Aging and Health

Division of Geriatric Medicine and Gerontology 

Johns Hopkins University

Ph: (410) 502-2619

Fax: (410) 614-9625

Email: rvaradhan at jhmi.edu

Webpage:
http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
tml

 

----------------------------------------------------------------------------
--------


-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On
Behalf Of Stu @ AGS
Sent: Tuesday, May 26, 2009 2:12 PM
To: r-help at r-project.org
Subject: [R] Linear Regression with Constraints

Hi!
I am a bit new to R.
I am looking for the right function to use for a multiple regression problem
of the form:

y = c1 + x1 + (c2 * x2) - (c3 * x3)

Where c1, c2, and c3 are the desired regression coefficients that are
subject to the following constraints:

0.0 < c2 < 1.0, and
0.0 < c3 < 1.0

y, x1, x2, and x3 are observed data.  
I have a total of 6 rows of data in a data set.

Is "optim" in the stats package the right function to use?
Also, I can't quite figure out how to specify the constraints.
Thank you!

-Stu

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Le mardi 26 mai 2009 ? 14:11 -0400, Stu @ AGS a ?crit :
> Hi!
> I am a bit new to R.
> I am looking for the right function to use for a multiple regression
problem
> of the form:
> 
> y = c1 + x1 + (c2 * x2) - (c3 * x3)
> 
> Where c1, c2, and c3 are the desired regression coefficients that are
> subject to the following constraints:
> 
> 0.0 < c2 < 1.0, and
> 0.0 < c3 < 1.0
Sounds rather like an in-the-closet Bayesian problem (with a very
strange prior...). Did you consider to submit it to WinBUGS (or JAGS) ?

If you still want a direct optimization, you could have started :

RSiteSearch("optimization constraint")

Which would have quickly led you to ask :

? constrOptim
> y, x1, x2, and x3 are observed data.  
> I have a total of 6 rows of data in a data set.
??? I that's real-life data, I wonder what kind of situation forces you
to estimate 3+1 parameters (c1, c2, c3 and the residual, which is not
really a parameter) with 6 data points ? Your problem can be written as
a system of 6 linear equations with 3 unknowns (c1, c2, c3), leaving you
room to search in (a small piece of) R^3 (the residual is another way to
express your objective function, not an independent parameter).

Of course, if it's homework, get lost !

					Emmanuel Charpentier
> Is "optim" in the stats package the right function to use?
> Also, I can't quite figure out how to specify the constraints.
> Thank you!
> 
> -Stu
>