Hello, I've just a little problem using the function optim. Here is the function I want to optimize : test_function(x){(exp(-0.06751 + 0.25473*((x[1]-350)/150) + 0.04455*((x[2]-40)/20) + 0.09399*((x[3]-400)/100) - 0.17238*((x[4]-250)/50)- 0.45984*((x[5]-550)/150)-0.39508*((x[1]-350)/150)* ((x[1]-350)/150) - 0.05116*((x[2]-40)/20)* ((x[2]-40)/20) - 0.27735*((x[3]-400)/100)*((x[3]-400)/100) - 0.12902*((x[4]-250)/50)* ((x[4]-250)/50)-1.07812*((x[5]-550)/150)* ((x[5]-550)/150) - 0.14650*((x[1]-350)/150)*((x[2]-40)/20) - 0.39647*((x[1]-350)/150)* ((x[3]-400)/100) + 0.09800*((x[1]-350)/150)* ((x[4]-250)/50) + 1.01607*((x[1]-350)/150)*((x[5]-550)/150) -0.04202*((x[2]-40)/20)* ((x[3]-400)/100) + 0.05057*((x[2]-40)/20)* ((x[4]-250)/50) + 0.17933*((x[2]-40)/20)*((x[5]-550)/150) + 0.10944*((x[3]-400)/100)* ((x[4]-250)/50) + 0.62105*((x[3]-400)/100)* ((x[5]-550)/150) - 0.22426*((x[4]-250)/50)*((x[5]-550)/150))/219741978624 )} Then I run the following command, and I get a pretty good result (I know approximatively the theoritical maximum) optim(c(200,60,500,300,700),test,control=list(fnscale=-1))$par [1] 398.17520 31.34863 372.17182 217.30874 533.49458 Fortunately, all the optimal values fall in their range of possible value. But to be rigourous, I should perform a box-constrained optimization. Then, I do so with the following command (in fixing the lower and upper args and not): optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method="L-BFGS-B",lower=c(200,20,300,200,400),upper=c(500,60,500,300,700))$par [1] 200 60 500 300 700 optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method="L-BFGS-B")$par [1] 200.00000 60.00007 500.00000 300.00000 700.00000 Here the result I get is wrong, and in fact it is more or less the initial guess I 've set. Can someone explain me what is going wrong with the method "L-BFGS-B" Thanks in advance Isabelle Zabalza-Mezghani -- Isabelle Zabalza-Mezghani Tel : 01 47 52 61 99 Institut Fran?ais du P?trole E-mail : isabelle.zabalza-mezghani at ifp.fr 1-4 Av. Bois Preau - Bat Lauriers 92852 Rueil Malmaison Cedex, France -------------- next part -------------- An HTML attachment was scrubbed... URL: https://stat.ethz.ch/pipermail/r-help/attachments/20011108/89539955/attachment.html
A few general suggestions: 1. you should always check the $convergence of your result -- in this case it tells you that the optimization failed, and tells you something about why (although admittedly not much you could interpret without looking at the source code). 2. control(fnscale=-1,trace=4) gives you some details of the progress of the optimization. 3. I would probably never start a box-constrained algorithm on the boundaries of the allowable region -- that's just making things hard for yourself ... 4. nonlinear optimizers are notoriously finicky. I found that, even starting from the same place, changing fnscale from -1 to -1e-12 (so that the values given by the function are around 1.0) allowed me to get to the right answer. Whenever possible, too, providing analytical derivatives for the function will make the algorithms much faster and more stable. If you're going to do a lot of this it's worthwhile -- particularly as it looks pretty easy for your functions, if this one is at all typical. Ben Bolker On Thu, 8 Nov 2001, Isabelle ZABALZA wrote:> Hello, > > I've just a little problem using the function optim. > Here is the function I want to optimize : > > test_function(x){(exp(-0.06751 + 0.25473*((x[1]-350)/150) + > 0.04455*((x[2]-40)/20) + 0.09399*((x[3]-400)/100) - > 0.17238*((x[4]-250)/50)- > 0.45984*((x[5]-550)/150)-0.39508*((x[1]-350)/150)* ((x[1]-350)/150) - > 0.05116*((x[2]-40)/20)* ((x[2]-40)/20) - > 0.27735*((x[3]-400)/100)*((x[3]-400)/100) - 0.12902*((x[4]-250)/50)* > ((x[4]-250)/50)-1.07812*((x[5]-550)/150)* ((x[5]-550)/150) - > 0.14650*((x[1]-350)/150)*((x[2]-40)/20) - 0.39647*((x[1]-350)/150)* > ((x[3]-400)/100) + 0.09800*((x[1]-350)/150)* ((x[4]-250)/50) + > 1.01607*((x[1]-350)/150)*((x[5]-550)/150) -0.04202*((x[2]-40)/20)* > ((x[3]-400)/100) + 0.05057*((x[2]-40)/20)* ((x[4]-250)/50) + > 0.17933*((x[2]-40)/20)*((x[5]-550)/150) + 0.10944*((x[3]-400)/100)* > ((x[4]-250)/50) + 0.62105*((x[3]-400)/100)* ((x[5]-550)/150) - > 0.22426*((x[4]-250)/50)*((x[5]-550)/150))/219741978624 )} > > Then I run the following command, and I get a pretty good result (I know > approximatively the theoritical maximum) > > optim(c(200,60,500,300,700),test,control=list(fnscale=-1))$par > [1] 398.17520 31.34863 372.17182 217.30874 533.49458 > > Fortunately, all the optimal values fall in their range of possible > value. But to be rigourous, I should perform a box-constrained > optimization. > Then, I do so with the following command (in fixing the lower and upper > args and not): > > optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method="L-BFGS-B",lower=c(200,20,300,200,400),upper=c(500,60,500,300,700))$par > > [1] 200 60 500 300 700 > > optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method="L-BFGS-B")$par > > [1] 200.00000 60.00007 500.00000 300.00000 700.00000 > > Here the result I get is wrong, and in fact it is more or less the > initial guess I 've set. > > Can someone explain me what is going wrong with the method "L-BFGS-B" > > Thanks in advance > > Isabelle Zabalza-Mezghani > > > > -- > Isabelle Zabalza-Mezghani Tel : 01 47 52 61 99 > Institut Fran?ais du P?trole E-mail : isabelle.zabalza-mezghani at ifp.fr > 1-4 Av. Bois Preau - Bat Lauriers > 92852 Rueil Malmaison Cedex, France > > >-- 318 Carr Hall bolker at zoo.ufl.edu Zoology Department, University of Florida http://www.zoo.ufl.edu/bolker Box 118525 (ph) 352-392-5697 Gainesville, FL 32611-8525 (fax) 352-392-3704 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
The problem is with scaling of the objective function. The values of test near the starting point are of the order of 1e-13 or so. I used the fnscale value of -1e-15 (the scaling is done by the inverse of fnscale) and I got essentially the same results from Nelder-Mead, BFGS, and L-BFGS-B. Cheers, Andy __________________________________ Andy Jaworski Engineering Systems Technology Center 3M Center, 518-1-01 St. Paul, MN 55144-1000 ----- E-mail: apjaworski at mmm.com Tel: (651) 733-6092 Fax: (651) 736-3122 |--------+----------------------------------> | | Isabelle ZABALZA | | | <isabelle.zabalza-mezgha| | | ni at ifp.fr> | | | | | | 11/08/2001 10:04 AM | | | | |--------+----------------------------------> >----------------------------------------------------------------------------------------------------------| | | | To: mailingr <r-help at stat.math.ethz.ch> | | cc: (bcc: Andrzej P. Jaworski/US-Corporate/3M/US) | | Subject: [R] Problem with optim (method L-BFGS-B) | >----------------------------------------------------------------------------------------------------------| Hello, I've just a little problem using the function optim. Here is the function I want to optimize : test_function(x){(exp(-0.06751 + 0.25473*((x[1]-350)/150) + 0.04455*((x[2]-40)/20) + 0.09399*((x[3]-400)/100) - 0.17238*((x[4]-250)/50)- 0.45984*((x[5]-550)/150)-0.39508*((x[1]-350)/150)* ((x[1]-350)/150) - 0.05116*((x[2]-40)/20)* ((x[2]-40)/20) - 0.27735*((x[3]-400)/100)*((x[3]-400)/100) - 0.12902*((x[4]-250)/50)* ((x[4]-250)/50)-1.07812*((x[5]-550)/150)* ((x[5]-550)/150) - 0.14650*((x[1]-350)/150)*((x[2]-40)/20) - 0.39647*((x[1]-350)/150)* ((x[3]-400)/100) + 0.09800*((x[1]-350)/150)* ((x[4]-250)/50) + 1.01607*((x[1]-350)/150)*((x[5]-550)/150) -0.04202*((x[2]-40)/20)* ((x[3]-400)/100) + 0.05057*((x[2]-40)/20)* ((x[4]-250)/50) + 0.17933*((x[2]-40)/20)*((x[5]-550)/150) + 0.10944*((x[3]-400)/100)* ((x[4]-250)/50) + 0.62105*((x[3]-400)/100)* ((x[5]-550)/150) - 0.22426*((x[4]-250)/50)*((x[5]-550)/150))/219741978624 )} Then I run the following command, and I get a pretty good result (I know approximatively the theoritical maximum) optim(c(200,60,500,300,700),test,control=list(fnscale=-1))$par [1] 398.17520 31.34863 372.17182 217.30874 533.49458 Fortunately, all the optimal values fall in their range of possible value. But to be rigourous, I should perform a box-constrained optimization. Then, I do so with the following command (in fixing the lower and upper args and not): optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method ="L-BFGS-B",lower=c(200,20,300,200,400),upper=c(500,60,500,300,700))$par [1] 200 60 500 300 700 optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method ="L-BFGS-B")$par [1] 200.00000 60.00007 500.00000 300.00000 700.00000 Here the result I get is wrong, and in fact it is more or less the initial guess I 've set. Can someone explain me what is going wrong with the method "L-BFGS-B" Thanks in advance Isabelle Zabalza-Mezghani -- Isabelle Zabalza-Mezghani Tel : 01 47 52 61 99 Institut Fran?ais du P?trole E-mail : isabelle.zabalza-mezghani at ifp.fr 1-4 Av. Bois Preau - Bat Lauriers 92852 Rueil Malmaison Cedex, France << File att1.htm not included with reply >> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
On Thu, 8 Nov 2001, Isabelle ZABALZA wrote:> Hello, > > I've just a little problem using the function optim. > Here is the function I want to optimize : > > test_function(x){(exp(-0.06751 + 0.25473*((x[1]-350)/150) + > 0.04455*((x[2]-40)/20) + 0.09399*((x[3]-400)/100) - > 0.17238*((x[4]-250)/50)- > 0.45984*((x[5]-550)/150)-0.39508*((x[1]-350)/150)* ((x[1]-350)/150) - > 0.05116*((x[2]-40)/20)* ((x[2]-40)/20) - > 0.27735*((x[3]-400)/100)*((x[3]-400)/100) - 0.12902*((x[4]-250)/50)* > ((x[4]-250)/50)-1.07812*((x[5]-550)/150)* ((x[5]-550)/150) - > 0.14650*((x[1]-350)/150)*((x[2]-40)/20) - 0.39647*((x[1]-350)/150)* > ((x[3]-400)/100) + 0.09800*((x[1]-350)/150)* ((x[4]-250)/50) + > 1.01607*((x[1]-350)/150)*((x[5]-550)/150) -0.04202*((x[2]-40)/20)* > ((x[3]-400)/100) + 0.05057*((x[2]-40)/20)* ((x[4]-250)/50) + > 0.17933*((x[2]-40)/20)*((x[5]-550)/150) + 0.10944*((x[3]-400)/100)* > ((x[4]-250)/50) + 0.62105*((x[3]-400)/100)* ((x[5]-550)/150) - > 0.22426*((x[4]-250)/50)*((x[5]-550)/150))/219741978624 )} > > Then I run the following command, and I get a pretty good result (I know > approximatively the theoritical maximum) > > optim(c(200,60,500,300,700),test,control=list(fnscale=-1))$par > [1] 398.17520 31.34863 372.17182 217.30874 533.49458 > > Fortunately, all the optimal values fall in their range of possible > value. But to be rigourous, I should perform a box-constrained > optimization. > Then, I do so with the following command (in fixing the lower and upper > args and not): > > optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method="L-BFGS-B",lower=c(200,20,300,200,400),upper=c(500,60,500,300,700))$par > > [1] 200 60 500 300 700 > > optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method="L-BFGS-B")$par > > [1] 200.00000 60.00007 500.00000 300.00000 700.00000 > > Here the result I get is wrong, and in fact it is more or less the > initial guess I 've set. > > Can someone explain me what is going wrong with the method "L-BFGS-B"Not looking at all the output. There's a typo in your code, but I get>optim(c(200,60,500,300,700),test,control=list(fnscale=-1),method="L-BFGS-B",lower=c(200,20,300,200,400),upper=c(500,60,500,300,700)) $par [1] 200 60 500 300 700 $value [1] 3.301069e-13 $counts function gradient 1 1 $convergence [1] 52 $message [1] "ERROR: ABNORMAL_TERMINATION_IN_LNSRCH" In short, the line search is failing. MORAL: always check that optim says it has converged. You need to scale your problem. -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._