R help - Dec 2024 - Non linear optimization with nloptr package fail to produce true optimal result

Hi,

I have below non-linear constraint optimization problem

#Original artificial data

library(nloptr)

set.seed(1)
A <- 1.34
B <- 0.5673
C <- 6.356
D <- -1.234
x <- seq(0.5, 20, length.out = 500)
y <- A + B * x + C * x^2 + D * log(x) + runif(500, 0, 3)

#Objective function

X <- cbind(1, x, x^2, log(x))
f <- function(theta) {
sum(abs(X %*% theta - y))
}

#Constraint

eps <- 1e-4

hin <- function(theta) {
  abs(sum(X %*% theta) - sum(y)) - 1e-3 + eps
}

Hx <- function(theta) {
  X[100, , drop = FALSE] %*% theta - (120 - eps)
}

#Optimization with nloptr

Sol = nloptr(rep(0, 4), f, eval_g_ineq = hin, eval_g_eq = Hx, opts
list("algorithm" = "NLOPT_LN_COBYLA", "xtol_rel" =
1.0e-8))$solution
# -0.2186159 -0.5032066  6.4458823 -0.4125948

However this does not appear to be optimal value. For example, if I
use below set,
0.222, 6.999, 6.17, -19.371, value of my objective function is lower
that that using nloptr

I just wonder in the package nloptr is good for non-linear optimization?

COBYLA stands for Contrained Optimization by Linear Approximation.

You seem to have some squares in your functions. Maybe BOBYQA would
be a better choice, though it only does bounds, so you'd have to introduce
a penalty, but then more of the optimx solvers would be available. With
only 4 parameters, possibly one of the Nelder-Mead variants (anms?) would
be suitable at least for tryout.

Optimizers are like other tools. Some are chainsaws, others are scalpels.
Don't do neurosurgery with a chainsaw unless you want a mess.

Have you checked that the objective and contraint are computed correctly?
 > 50% of "your software doesn't work" in optimization are due
to such errors.

John Nash


On 2024-12-13 12:52, Daniel Lobo wrote:
> Hi,
> 
> I have below non-linear constraint optimization problem
> 
> #Original artificial data
> 
> library(nloptr)
> 
> set.seed(1)
> A <- 1.34
> B <- 0.5673
> C <- 6.356
> D <- -1.234
> x <- seq(0.5, 20, length.out = 500)
> y <- A + B * x + C * x^2 + D * log(x) + runif(500, 0, 3)
> 
> #Objective function
> 
> X <- cbind(1, x, x^2, log(x))
> f <- function(theta) {
> sum(abs(X %*% theta - y))
> }
> 
> #Constraint
> 
> eps <- 1e-4
> 
> hin <- function(theta) {
>    abs(sum(X %*% theta) - sum(y)) - 1e-3 + eps
> }
> 
> Hx <- function(theta) {
>    X[100, , drop = FALSE] %*% theta - (120 - eps)
> }
> 
> #Optimization with nloptr
> 
> Sol = nloptr(rep(0, 4), f, eval_g_ineq = hin, eval_g_eq = Hx, opts >
list("algorithm" = "NLOPT_LN_COBYLA", "xtol_rel" =
1.0e-8))$solution
> # -0.2186159 -0.5032066  6.4458823 -0.4125948
> 
> However this does not appear to be optimal value. For example, if I
> use below set,
> 0.222, 6.999, 6.17, -19.371, value of my objective function is lower
> that that using nloptr
> 
> I just wonder in the package nloptr is good for non-linear optimization?
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
https://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

A small correction, the below combination

2.02, 6.764, 6.186, -20.095

Gives better result.

On Fri, 13 Dec 2024 at 23:22, Daniel Lobo <danielobo9976 at gmail.com>
wrote:
>
> Hi,
>
> I have below non-linear constraint optimization problem
>
> #Original artificial data
>
> library(nloptr)
>
> set.seed(1)
> A <- 1.34
> B <- 0.5673
> C <- 6.356
> D <- -1.234
> x <- seq(0.5, 20, length.out = 500)
> y <- A + B * x + C * x^2 + D * log(x) + runif(500, 0, 3)
>
> #Objective function
>
> X <- cbind(1, x, x^2, log(x))
> f <- function(theta) {
> sum(abs(X %*% theta - y))
> }
>
> #Constraint
>
> eps <- 1e-4
>
> hin <- function(theta) {
>   abs(sum(X %*% theta) - sum(y)) - 1e-3 + eps
> }
>
> Hx <- function(theta) {
>   X[100, , drop = FALSE] %*% theta - (120 - eps)
> }
>
> #Optimization with nloptr
>
> Sol = nloptr(rep(0, 4), f, eval_g_ineq = hin, eval_g_eq = Hx, opts >
list("algorithm" = "NLOPT_LN_COBYLA", "xtol_rel" =
1.0e-8))$solution
> # -0.2186159 -0.5032066  6.4458823 -0.4125948
>
> However this does not appear to be optimal value. For example, if I
> use below set,
> 0.222, 6.999, 6.17, -19.371, value of my objective function is lower
> that that using nloptr
>
> I just wonder in the package nloptr is good for non-linear optimization?

You posted a version of this question on StackOverflow, and were given 
advice there that you ignored.

nloptr() clearly indicates that it is quitting without reaching an 
optimum, but you are hiding that message.  Don't do that.

Duncan Murdoch

On 2024-12-13 12:52 p.m., Daniel Lobo wrote:
> library(nloptr)
> 
> set.seed(1)
> A <- 1.34
> B <- 0.5673
> C <- 6.356
> D <- -1.234
> x <- seq(0.5, 20, length.out = 500)
> y <- A + B * x + C * x^2 + D * log(x) + runif(500, 0, 3)
> 
> #Objective function
> 
> X <- cbind(1, x, x^2, log(x))
> f <- function(theta) {
> sum(abs(X %*% theta - y))
> }
> 
> #Constraint
> 
> eps <- 1e-4
> 
> hin <- function(theta) {
>    abs(sum(X %*% theta) - sum(y)) - 1e-3 + eps
> }
> 
> Hx <- function(theta) {
>    X[100, , drop = FALSE] %*% theta - (120 - eps)
> }
> 
> #Optimization with nloptr
> 
> Sol = nloptr(rep(0, 4), f, eval_g_ineq = hin, eval_g_eq = Hx, opts >
list("algorithm" = "NLOPT_LN_COBYLA", "xtol_rel" =
1.0e-8))$solution
> # -0.2186159 -0.5032066  6.4458823 -0.4125948