R devel - Mar 2010 - Improved Nelder-Mead algorithm - a potential replacement for optim's Nelder-Mead

Hi,

I have written an R translation of C.T. Kelley's Matlab version of the
Nelder-Mead algorithm.  This algorithm is discussed in detail in his book
"Iterative methods for optimization" (SIAM 1999, Chapter 8).  I have
tested this relatively extensively on a number of smooth and non-smooth
problems.  It performs well, in general, and it almost always outperforms
optim's implementation of Nelder-Mead.  I have obtained written permissions
from both SIAM (publishers of Kelley's text) and from C.T. Kelley himself to
make this publicly available in R.

I have been planning on releasing this function either as a separate package or
as one of the algorithms in a package containing derivative-free methods. 
However, it was recently suggested to me by Soren Hosgaard that a better option
would be to incorporate this as part of `optim' itself, either replacing
existing Nelder-Mead or as another `method' in optim, so that it will be
more widely used.   Would anyone from the R-core be interested in incorporting
this into optim? It should be mentioned that my code is entirely written in R. 
Therefore, speed gains could be achieved if translated into C (I am not
proficient in C).

Best,
Ravi.

____________________________________________________________________

Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology
School of Medicine
Johns Hopkins University

Ph. (410) 502-2619
email: rvaradhan at jhmi.edu
-------------- next part --------------
An embedded message was scrubbed...
From: =?iso-8859-1?Q?S=F8ren_H=F8jsgaard?= <Soren.Hojsgaard at agrsci.dk>
Subject: SV: [R] Improved Nelder-Mead algorithm and Matlab's fminsearch.
Was: Hi
Date: Fri, 05 Mar 2010 20:47:11 +0100
Size: 5724
URL:
<https://stat.ethz.ch/pipermail/r-devel/attachments/20100305/9404b759/attachment.mht>

On Fri, Mar 5, 2010 at 8:26 PM, Ravi Varadhan <rvaradhan at jhmi.edu>
wrote:
> Hi,
>
> I have written an R translation of C.T. Kelley's Matlab version of the
Nelder-Mead algorithm. ?This algorithm is discussed in detail in his book
"Iterative methods for optimization" (SIAM 1999, Chapter 8). ?I have
tested this relatively extensively on a number of smooth and non-smooth
problems. ?It performs well, in general, and it almost always outperforms
optim's implementation of Nelder-Mead. ?I have obtained written permissions
from both SIAM (publishers of Kelley's text) and from C.T. Kelley himself to
make this publicly available in R.
 By 'in R' do you mean 'written in R' or 'in the R package
as you get
from CRAN'?

 I think the terms and conditions of that permission would need to be
studied if the code can be redistributed and modified, who holds the
copyright, can it be stuck in with R under an open license and so
on...
>  ?Therefore, speed gains could be achieved if translated into C (I am not
proficient in C).
 Not necessarily - if most of the time is spent in the objective
function then even an instantaneous implementation isn't going to
speed things up enough to be worthwhile. More R users know R than C so
the R implementation is always going to be more useful for R users to
study, tweak, and possibly improve!

 Sounds good though!

Barry

-- 
blog: http://geospaced.blogspot.com/
web: http://www.maths.lancs.ac.uk/~rowlings
web: http://www.rowlingson.com/
twitter: http://twitter.com/geospacedman
pics: http://www.flickr.com/photos/spacedman

Hi,

i have a question concerning the Nelder-Mead algorithm in R. As far as i can
see, the shrink operation is not included in the optim() function. Does
anyone know an implementation of the Nelder-Mead algorithm including this
operation in R? Could maybe someone send me one? I would try to write the
code on my own, but due to my doctor thesis i am a little under time
pressure and would appreciate any help. Thank you in advance!

Greetings
Ralf

--
View this message in context:
http://r.789695.n4.nabble.com/Improved-Nelder-Mead-algorithm-a-potential-replacement-for-optim-s-Nelder-Mead-tp1580101p3678732.html
Sent from the R devel mailing list archive at Nabble.com.