R devel - Aug 2023 - feature request: optim() iteration of functions that return multiple values

Dear all,

I have used optim a lot in contexts where it would useful to be able to iterate
function myfun that, in addition to the primary objective to be minimized
('minimize.me'), could return other values such as alternative metrics
of the minimization, informative intermediate values from the calculations, etc.

myfun  <- function()
{
...
return(list(minimize.me = minimize.me, R2 = R2, pval = pval, etc.))
}

During the iteration, optim could utilize just the first value from the myfun
return list; all the other values calculated and returned by myfun could be
ignored by optim.
After convergence, the other return values of myfun could be finally extracted
and appended into the optim return value (which is a list) as additional entry
e.g.: $aux <- list(R2, pval, etc.), (without 'minimize.me' as it is
already returned as $value).

The usual ways for accessing optim return values, e.g., $par, $value, etc. are
not affected.  Computational cost may not be prohibitive either.  Is this
feasible to consider?


	[[alternative HTML version deleted]]

On Thu, 03 Aug 2023, Sami Tuomivaara writes:
> Dear all,
>
> I have used optim a lot in contexts where it would
> useful to be able to iterate function myfun that, in
> addition to the primary objective to be minimized
> ('minimize.me'), could return other values such as
> alternative metrics of the minimization, informative
> intermediate values from the calculations, etc.
>
> myfun  <- function()
> {
> ...
> return(list(minimize.me = minimize.me, R2 = R2, pval = pval, etc.))
> }
>
> During the iteration, optim could utilize just the first value from the
myfun return list; all the other values calculated and returned by myfun could
be ignored by optim.
> After convergence, the other return values of myfun
> could be finally extracted and appended into the optim
> return value (which is a list) as additional entry
> e.g.: $aux <- list(R2, pval, etc.), (without
> 'minimize.me' as it is already returned as $value).
>
> The usual ways for accessing optim return values, e.g.,
> $par, $value, etc. are not affected.  Computational
> cost may not be prohibitive either.  Is this feasible
> to consider?
>
If you only wish to store additional information, you could do
so with an environment, without changing optim.  For instance,
like so (using the first example from ?optim):

    data <- new.env()
    data$i <- 0
    data$fun.value <- numeric(1000)
    
    fr <- function(x, data) {   ## Rosenbrock Banana function
        x1 <- x[1]
        x2 <- x[2]
        ans <- 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
        data$i <- data$i + 1
        data$fun.value[data$i] <- ans
        ans
    }
    optim(c(-1.2,1), fr, data = data)
    ## $par
    ## [1] 1.000260 1.000506
    ## 
    ## $value
    ## [1] 8.825241e-08
    ## 
    ## $counts
    ## function gradient 
    ##      195       NA 
    ## 
    ## ....

    data$i
    ## 195
    
    plot(data$fun.value[1:data$i])




-- 
Enrico Schumann
Lucerne, Switzerland
http://enricoschumann.net

Enrico gave you a workaround that stores the extra values in an environment.

Another possible workaround is an optional argument to myfun() that asks 
it to return more information, e.g.

     fr <- function(x, data, extraInfo = FALSE) {   ## Rosenbrock Banana 
function
         x1 <- x[1]
         x2 <- x[2]
         ans <- 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
         if (extraInfo) {
           list(ans=ans, extras = ...)
         else
           ans
     }

Then after optim() finishes, call fr() again with parameters as returned 
by optim, and extraInfo = TRUE.

Duncan Murdoch

On 03/08/2023 4:21 p.m., Sami Tuomivaara wrote:
> Dear all,
> 
> I have used optim a lot in contexts where it would useful to be able to
iterate function myfun that, in addition to the primary objective to be
minimized ('minimize.me'), could return other values such as alternative
metrics of the minimization, informative intermediate values from the
calculations, etc.
> 
> myfun  <- function()
> {
> ...
> return(list(minimize.me = minimize.me, R2 = R2, pval = pval, etc.))
> }
> 
> During the iteration, optim could utilize just the first value from the
myfun return list; all the other values calculated and returned by myfun could
be ignored by optim.
> After convergence, the other return values of myfun could be finally
extracted and appended into the optim return value (which is a list) as
additional entry e.g.: $aux <- list(R2, pval, etc.), (without
'minimize.me' as it is already returned as $value).
> 
> The usual ways for accessing optim return values, e.g., $par, $value, etc.
are not affected.  Computational cost may not be prohibitive either.  Is this
feasible to consider?
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

For a solution that does not require any change to the original function 
being optimized, the following one-liner could be used, which converts 
existing functions to functions that return only the first element:

returnFirst <- function(fun) function(...) do.call(fun,list(...))[[1]]

Example:

fr <- function(x) {   ## Rosenbrock Banana function
   x1 <- x[1]
   x2 <- x[2]
   ans <- 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
   list(ans=ans, extra1 = 1:10, extra2 = letters)
}

fr2 <- returnFirst(fr)
tmp <- optim(c(-1.2,1), fr2)
fr(tmp$par)


Am 03.08.23 um 22:21 schrieb Sami Tuomivaara:
> Dear all,
> 
> I have used optim a lot in contexts where it would useful to be able to
iterate function myfun that, in addition to the primary objective to be
minimized ('minimize.me'), could return other values such as alternative
metrics of the minimization, informative intermediate values from the
calculations, etc.
> 
> myfun  <- function()
> {
> ...
> return(list(minimize.me = minimize.me, R2 = R2, pval = pval, etc.))
> }
> 
> During the iteration, optim could utilize just the first value from the
myfun return list; all the other values calculated and returned by myfun could
be ignored by optim.
> After convergence, the other return values of myfun could be finally
extracted and appended into the optim return value (which is a list) as
additional entry e.g.: $aux <- list(R2, pval, etc.), (without
'minimize.me' as it is already returned as $value).
> 
> The usual ways for accessing optim return values, e.g., $par, $value, etc.
are not affected.  Computational cost may not be prohibitive either.  Is this
feasible to consider?
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel