Greetings, R-Wizards:
I'm trying to find an extremum subject to a nonlinear constraint. (Yes, I
have perused the archives but have found nothing positive.) The details of
the problem are these:
In a paper published some years ago in Technometrics, ("Confidence bands
for
cumulative distribution functions of continuous random variables"
Technometrics, 25, 77-86. 1983), Cheng and Iles describe an ingenious method
for placing confidence bounds on an entire cdf by defining the likelihood
ratio confidence "ellipse" for the model parameters, and then
traversing the
periphery and finding the most extreme values for x, at a given F(x), such
that the distribution parameters reside on that confidence contour. (Well,
their method is more sophisticated than that, but that's essentially how it
works.) I implemented the thing in a spreadsheet 15 years ago, and would
like to do the same in R. But EXCEL's solver can find an extremum subject
to a constraint, and I haven't figured out how to get nlm() to do that.
I would be grateful for any algorithmic suggestions.
Many Thanks.
Charles Annis, P.E.
Charles.Annis at StatisticalEngineering.com
phone: 561-352-9699
eFAX: 503-217-5849
http://www.StatisticalEngineering.com
>>>>> "FCharles" == Charles Annis, P E <Charles.Annis at statisticalengineering.com> >>>>> on Mon, 16 Jun 2003 13:19:50 -0400 writes:FCharles> Greetings, R-Wizards: I'm trying to find an FCharles> extremum subject to a nonlinear constraint. (Yes, FCharles> I have perused the archives but have found nothing FCharles> positive.) The details of the problem are these: FCharles> In a paper published some years ago in FCharles> Technometrics, ("Confidence bands for cumulative FCharles> distribution functions of continuous random FCharles> variables" Technometrics, 25, 77-86. 1983), Cheng FCharles> and Iles describe an ingenious method for placing FCharles> confidence bounds on an entire cdf by defining the FCharles> likelihood ratio confidence "ellipse" for the FCharles> model parameters, and then traversing the FCharles> periphery and finding the most extreme values for FCharles> x, at a given F(x), such that the distribution FCharles> parameters reside on that confidence contour. FCharles> (Well, their method is more sophisticated than FCharles> that, but that's essentially how it works.) I FCharles> implemented the thing in a spreadsheet 15 years FCharles> ago, and would like to do the same in R. But FCharles> EXCEL's solver can find an extremum subject to a FCharles> constraint, and I haven't figured out how to get FCharles> nlm() to do that. FCharles> I would be grateful for any algorithmic FCharles> suggestions. I'm astonished that your perusing of the archives and the on-line help system didn't find optim(). It has many methods, one of which working with box constraints. I'm urging you to read help(optim) {and please read to the end even if it's long}. If you have non-box constraints, there are other tricks (eg. continuous border penalizing) but it really depends on your application. Martin Maechler <maechler at stat.math.ethz.ch> http://stat.ethz.ch/~maechler/ Seminar fuer Statistik, ETH-Zentrum LEO C16 Leonhardstr. 27 ETH (Federal Inst. Technology) 8092 Zurich SWITZERLAND phone: x-41-1-632-3408 fax: ...-1228 <><
It seems that you have a nonlinear constraint (rather than box
constraints). Provided your constraint is reasonably well behaved
(differentiable) you could still use optim on the Lagrangian, and then
run an "outer" optimisation to find the langrangian multiplier and
slack
variable (that is, assuming you have an INequality constraint. Set z to
zero below if you just have an equality constraint). For example if your
original problem was:
Max_{x} f(x)
such that g(x) >= 0
You could instead try:
Max_{x;y;z} f(x) - y * (g(x) - z)
such that z >= 0
i.e. define a function L(x,l,z) = f(x) - y * (g(x) - z)
Then feed L into optim and maximise it w.r.t: x (your original
variables), y (the Lagrange multiplier) and z (a slack variable).
Don't hold me to this as my optimisation theory is a little rusty...
-----Original Message-----
From: Martin Maechler [mailto:maechler at stat.math.ethz.ch]
Sent: 17 June 2003 08:42
To: Charles.Annis at statisticalengineering.com
Cc: r-help at stat.math.ethz.ch
Subject: Re: [R] Constrained optimization
>>>>> "FCharles" == Charles Annis, P E
<Charles.Annis at statisticalengineering.com>>>>>> on Mon, 16 Jun 2003 13:19:50 -0400 writes:
FCharles> Greetings, R-Wizards: I'm trying to find an
FCharles> extremum subject to a nonlinear constraint. (Yes,
FCharles> I have perused the archives but have found nothing
FCharles> positive.) The details of the problem are these:
FCharles> In a paper published some years ago in
FCharles> Technometrics, ("Confidence bands for cumulative
FCharles> distribution functions of continuous random
FCharles> variables" Technometrics, 25, 77-86. 1983), Cheng
FCharles> and Iles describe an ingenious method for placing
FCharles> confidence bounds on an entire cdf by defining the
FCharles> likelihood ratio confidence "ellipse" for the
FCharles> model parameters, and then traversing the
FCharles> periphery and finding the most extreme values for
FCharles> x, at a given F(x), such that the distribution
FCharles> parameters reside on that confidence contour.
FCharles> (Well, their method is more sophisticated than
FCharles> that, but that's essentially how it works.) I
FCharles> implemented the thing in a spreadsheet 15 years
FCharles> ago, and would like to do the same in R. But
FCharles> EXCEL's solver can find an extremum subject to a
FCharles> constraint, and I haven't figured out how to get
FCharles> nlm() to do that.
FCharles> I would be grateful for any algorithmic
FCharles> suggestions.
I'm astonished that your perusing of the archives and the on-line help
system didn't find
optim().
It has many methods, one of which working with box constraints. I'm
urging you to read help(optim) {and please read to the end even if it's
long}.
If you have non-box constraints, there are other tricks
(eg. continuous border penalizing) but it really depends on
your application.
Martin Maechler <maechler at stat.math.ethz.ch>
http://stat.ethz.ch/~maechler/
Seminar fuer Statistik, ETH-Zentrum LEO C16 Leonhardstr. 27
ETH (Federal Inst. Technology) 8092 Zurich SWITZERLAND
phone: x-41-1-632-3408 fax: ...-1228 <><
______________________________________________
R-help at stat.math.ethz.ch mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
Greetings, R-Wizards: Earlier this week I requested help with trying to find an extremum subject to a nonlinear constraint. Many thanks to Martin Maechler, Spencer Graves, and Jonathan Baron, who all suggested optim() rather than nlm(), and to Robert Gentleman who suggested using a half-interval search, and special thanks to Patrick Burns and David Khabie-Zeitoune who suggested ways to create an appropriate objective function to use with optim(). While working on the solution I took a closer look at the likelihood profiles and reparameterized to avoid precipitous behavior in the neighborhood of zero. Thus the solution for me was three-fold: 1) Re-parameterize to provide a less-asymmetric likelihood surface. 2) Use box constraints on the parameters to keep the solution-finding under control and thus prevent excursions beyond the domain of parameter feasibility. 3) Define an objective function that had the right objective! ~:-) Thanks to all for your help. Charles Annis, P.E. Charles.Annis at StatisticalEngineering.com phone: 561-352-9699 eFAX: 503-217-5849 http://www.StatisticalEngineering.com