R help - Mar 2020 - Relatively Simple Maximization Using Optim Doesnt Optimize

> L= 1006.536
> L= 1006.537
> L= 1006.535
> It appears to have chosen step size 0.001, not 0.00001.  It should be
> getting adequate accuracy in both 1st and 2nd derivatives.
> Those little ripples you see in the plot are not relevant.
I'm impressed.
But you're still wrong.

Try this:
---------
#not good R code!
v = numeric ()

production3 <- function(L){
    #store in vector
    v <<- c (v, L)

   budget=100000
   Lcost=12
   Kcost=15
   K=(budget-L*Lcost)/Kcost
   machines=0.05*L^(2/3)*K^(1/3)
   return(machines)
}

optim.sol <- optim (1001, production3 ,method="CG", control =
list(fnscale=-1) )

n = length (v)
print (n)

plot (1:n ,v, type="l")
---------

After 401 iterations (on my computer), the algorithm hasn't converged.
And I note it's converging extremely slowly, so I don't see any
argument for increasing the number of iterations.

And try this:
(The first 30 steps).
---------
plot (1:30 ,v [1:30], type="l")
---------

Little ripples aren't going anywhere...

On 12/03/2020 8:52 p.m., Abby Spurdle wrote:
>> L= 1006.536
>> L= 1006.537
>> L= 1006.535
>> It appears to have chosen step size 0.001, not 0.00001.  It should be
>> getting adequate accuracy in both 1st and 2nd derivatives.
>> Those little ripples you see in the plot are not relevant.
> 
> I'm impressed.
> But you're still wrong.
> 
> Try this:
> ---------
> #not good R code!
> v = numeric ()
> 
> production3 <- function(L){
>      #store in vector
>      v <<- c (v, L)
> 
>     budget=100000
>     Lcost=12
>     Kcost=15
>     K=(budget-L*Lcost)/Kcost
>     machines=0.05*L^(2/3)*K^(1/3)
>     return(machines)
> }
> 
> optim.sol <- optim (1001, production3 ,method="CG", control =
list(fnscale=-1) )
> 
> n = length (v)
> print (n)
> 
> plot (1:n ,v, type="l")
> ---------
> 
> After 401 iterations (on my computer), the algorithm hasn't converged.
> And I note it's converging extremely slowly, so I don't see any
> argument for increasing the number of iterations.
> 
> And try this:
> (The first 30 steps).
> ---------
> plot (1:30 ,v [1:30], type="l")
> ---------
> 
> Little ripples aren't going anywhere...
> 
That's the bug.

It is correctly signalling that it hasn't converged (look at 
optim.sol$convergence, which "indicates that the iteration limit maxit 
had been reached".)  But CG should be taking bigger steps.  On a 1D 
quadratic objective function with no errors in the derivatives, it 
should take one step to convergence.  Here we're not quadratic (though 
it's pretty close), and we don't have exact derivatives (your ripples), 
so the fact that it is sticking to one step size is a sign that it is 
not working.   If those ripples are big enough to matter (and I'm not 
convinced of that), it should take highly variable steps.

The fact that it doesn't give a warning() when it knows it has failed to 
converge is also a pretty serious design flaw.

Duncan Murdoch

Once again, CG and its successors aren't envisaged for 1D problems. Do you
really want to perform brain surgery with a chain saw?

Note that

production4 <- function(L) { - production3(L) }

sjn2 <- optimize(production3, c(900, 1100))
sjn2

gives

$minimum
[1] 900.0001

$objective
[1] 84.44156

Whether that is a good optimum I haven't checked.

JN


On 2020-03-13 6:47 a.m., Duncan Murdoch wrote:
> On 12/03/2020 8:52 p.m., Abby Spurdle wrote:
>>> L= 1006.536
>>> L= 1006.537
>>> L= 1006.535
>>> It appears to have chosen step size 0.001, not 0.00001.? It should
be
>>> getting adequate accuracy in both 1st and 2nd derivatives.
>>> Those little ripples you see in the plot are not relevant.
>>
>> I'm impressed.
>> But you're still wrong.
>>
>> Try this:
>> ---------
>> #not good R code!
>> v = numeric ()
>>
>> production3 <- function(L){
>> ???? #store in vector
>> ???? v <<- c (v, L)
>>
>> ??? budget=100000
>> ??? Lcost=12
>> ??? Kcost=15
>> ??? K=(budget-L*Lcost)/Kcost
>> ??? machines=0.05*L^(2/3)*K^(1/3)
>> ??? return(machines)
>> }
>>
>> optim.sol <- optim (1001, production3 ,method="CG",
control = list(fnscale=-1) )
>>
>> n = length (v)
>> print (n)
>>
>> plot (1:n ,v, type="l")
>> ---------
>>
>> After 401 iterations (on my computer), the algorithm hasn't
converged.
>> And I note it's converging extremely slowly, so I don't see any
>> argument for increasing the number of iterations.
>>
>> And try this:
>> (The first 30 steps).
>> ---------
>> plot (1:30 ,v [1:30], type="l")
>> ---------
>>
>> Little ripples aren't going anywhere...
>>
> 
> That's the bug.
> 
> It is correctly signalling that it hasn't converged (look at
optim.sol$convergence, which "indicates that the iteration
> limit maxit had been reached".)? But CG should be taking bigger
steps.? On a 1D quadratic objective function with no
> errors in the derivatives, it should take one step to convergence.? Here
we're not quadratic (though it's pretty close),
> and we don't have exact derivatives (your ripples), so the fact that it
is sticking to one step size is a sign that it
> is not working.?? If those ripples are big enough to matter (and I'm
not convinced of that), it should take highly
> variable steps.
> 
> The fact that it doesn't give a warning() when it knows it has failed
to converge is also a pretty serious design flaw.
> 
> Duncan Murdoch
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

> It is correctly signalling that it hasn't converged (look at
> optim.sol$convergence, which "indicates that the iteration limit maxit
> had been reached".)  But CG should be taking bigger steps.  On a 1D
> quadratic objective function with no errors in the derivatives, it
> should take one step to convergence.  Here we're not quadratic (though
> it's pretty close), and we don't have exact derivatives (your
ripples),
> so the fact that it is sticking to one step size is a sign that it is
> not working.   If those ripples are big enough to matter (and I'm not
> convinced of that), it should take highly variable steps.
Hi Duncan,

I need to apologize.
The problem has nothing to do with little ripples.
(My bad...)

I tried approximating the function with a cubic Hermite spline.
(Essentially smoothing the function).

However, the optim function still returns the wrong result.
Which surprised me...

Then I tried changing the max number of iterations, and found
something quite interesting:
---------
production.wr <- function(L){
  cat (L, "\n")
  budget=100000
  Lcost=12
  Kcost=15
  K=(budget-L*Lcost)/Kcost
  machines=0.05*L^(2/3)*K^(1/3)
  return(machines)
}

S1=optim(1001,production.wr,method="CG",control=list(fnscale=-1,
maxit=1))
S1=optim(1001,production.wr,method="CG",control=list(fnscale=-1,
maxit=2))
S1=optim(1001,production.wr,method="CG",control=list(fnscale=-1,
maxit=3))
S1=optim(1001,production.wr,method="CG",control=list(fnscale=-1,
maxit=4))
---------

The first iteration calls the function (3 + 2) times.
Subsequent iterations call the function (2 + 2) times.
In the subset-of-3, the step size is exactly 0.001.
And in subsequent leading (but not trailing) subsets-of-2, the step
size is exactly 0.002.

I was wondering (hypothetically) if the first iteration is
approximating the second derivative, and subsequent iterations are
not...???