R help - Jun 2008 - optim, constrOptim: setting some parameters equal to each other

Hello, and apologies for the upcoming naive questions. I am a biologist who is
trying to teach himself the appropriate areas of math and stats. I welcome
pointers to suggested background reading just as much as I do direct answers to
my question.

Let's say I have a function F() that takes variables (a,b,c,a1,b1,c1) and
returns x, and I want to find the values of these variables that result in a
minimum value of x. That's a straightforward application of optim().
However, for the same function I also need to obtain values that return the
minimum value of x subject to the following constraints: a=a1, b=b1, c=c1, a=a1
&& b=b1, a=a1 && c=c1, ... and so on, for any combination of
these constraints including a=a1, b=b1, c=c1. The brute force way to do this
with optim() would be to write into F() an immense switch statement anticipating
every possible combination of constrained variables. Obviously this is
inefficient and unmaintainable. Therefore, my question is:

Is the purpose of constrOptim() this exact type of problem? If so, how does one
express the constraint I described above in terms of the ui, ci, and theta
parameters? Are there any introductory texts I should have read for this to be
obvious to me from constrOptim's documentation?

If constrOptim() is not the answer to this problem, can anybody suggest any more
elegant alterntives to the switch-statement-from-hell approach?

Thank you very, very much in advance. I thought I understood R reasonably well
until I started banging my head against this problem!

If 'F' is twice differentiable, this could be done fairly easily 
by writing 'F' as a function to be maximized with a1 = a + da, etc., and
using the 'activePars' argument in maxNR{maxLik} to specify the 
constraints you want. 

      More specifically, consider the following: 

      Fmax <- function(x){
           -F(x[1], x[2], x[3], x[1]+x[4], x[2]+x[5], x[3]+x[6])
      }

      You may know that if your 'F' is the negative of a 
log(likelihood), then you can test statistical hypothesis about whether 
a=a1, etc., using the fact that under commonly met regularity 
conditions, 2*log(likelihood ratio) is approximately distributed as 
chi-square.  Moreover, this approximation is often quite good -- and can 
be evaluated with a simple Monte Carlo, permuting the order of your 
response variable if you have one. 

      Hope this helps. 
      Spencer Graves    

Alex F. Bokov wrote:
> Hello, and apologies for the upcoming naive questions. I am a biologist who
is trying to teach himself the appropriate areas of math and stats. I welcome
pointers to suggested background reading just as much as I do direct answers to
my question.
>
> Let's say I have a function F() that takes variables (a,b,c,a1,b1,c1)
and returns x, and I want to find the values of these variables that result in a
minimum value of x. That's a straightforward application of optim().
However, for the same function I also need to obtain values that return the
minimum value of x subject to the following constraints: a=a1, b=b1, c=c1, a=a1
&& b=b1, a=a1 && c=c1, ... and so on, for any combination of
these constraints including a=a1, b=b1, c=c1. The brute force way to do this
with optim() would be to write into F() an immense switch statement anticipating
every possible combination of constrained variables. Obviously this is
inefficient and unmaintainable. Therefore, my question is:
>
> Is the purpose of constrOptim() this exact type of problem? If so, how does
one express the constraint I described above in terms of the ui, ci, and theta
parameters? Are there any introductory texts I should have read for this to be
obvious to me from constrOptim's documentation?
>
> If constrOptim() is not the answer to this problem, can anybody suggest any
more elegant alterntives to the switch-statement-from-hell approach?
>
> Thank you very, very much in advance. I thought I understood R reasonably
well until I started banging my head against this problem!
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

Here is an example w/optim where you have an objective function
F(x) where you have (possibly a few) constraints of form x_i=x_j, and you
can specify the constraints flexibly, which is what I _think_ you want.
An example from you would have been nice.

## objective function that depends on all parameters, here a, b, c
obfun <- function(a,b,c,dd)
    sum((dd - (exp(a * 1:100) + exp(b * 1:50) + exp(c * 1:25) ))^2)

## fn for optim
fr <- function(x, eqspec, dd, obfun) {
  ## assign variables for parameter values given in x
  for(i in 1:length(x))
    assign(names(x)[i], x[[i]])
  ## use eqspec to assign parameter values determined w/equ. constr.
  for(i in 1:length(eqspec))
    assign( names(eqspec)[i], x[[ eqspec[[i]] ]])
  ## now have all parameter values, call objective function
  obfun(a,b,c,d, dd)
}

dd <- exp(-.05 * 1:100) + exp(-.05 * 1:50) + exp(.005 * 1:25)

## let par contain all parameters that are not eq. constrained and
## all parameters on right side of constraints of form a=b

## let eqspec give all dependent parameters on left side of
## the constraints of form a=b

## e.g.
## for constraint a=b
xx <- optim(par=list(b=-4, c=-.1), fn=fr, eqspec=list(a="b"),
            dd=dd, obfun=obfun)

## for constraint b=a
x1 <- optim(par=list(a=-4, c=-.1), fn=fr, eqspec=list(b="a"),
            dd=dd, obfun=obfun)

On Sun, 8 Jun 2008, Alex F. Bokov wrote:
> Hello, and apologies for the upcoming naive questions. I am a biologist
> who is trying to teach himself the appropriate areas of math and stats.
> I welcome pointers to suggested background reading just as much as I do
> direct answers to my question.
>
> Let's say I have a function F() that takes variables (a,b,c,a1,b1,c1)
> and returns x, and I want to find the values of these variables that
> result in a minimum value of x. That's a straightforward application of
> optim(). However, for the same function I also need to obtain values
> that return the minimum value of x subject to the following constraints:
> a=a1, b=b1, c=c1, a=a1 && b=b1, a=a1 && c=c1, ... and so
on, for any
> combination of these constraints including a=a1, b=b1, c=c1. The brute
> force way to do this with optim() would be to write into F() an immense
> switch statement anticipating every possible combination of constrained
> variables. Obviously this is inefficient and unmaintainable. Therefore,
> my question is:
>
> Is the purpose of constrOptim() this exact type of problem? If so, how
> does one express the constraint I described above in terms of the ui,
> ci, and theta parameters? Are there any introductory texts I should have
> read for this to be obvious to me from constrOptim's documentation?
>
> If constrOptim() is not the answer to this problem, can anybody suggest
> any more elegant alterntives to the switch-statement-from-hell approach?
>
> Thank you very, very much in advance. I thought I understood R
> reasonably well until I started banging my head against this problem!
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>