R help - Jan 2001 - Levenberg-Marquardt algorithm

Hi All,

Is the Levenberg-Marquardt algorithm available in R. This method combines the 
steepest descent algorithm and Newton's method.

Thanks in Advance,
Dermot MacSweeney.


**************************************************************

Dermot MacSweeney			NMRC, 
Email: dsweeney at nmrc.ucc.ie		Lee Maltings, 
Tel: +353 21 904178			Prospect Row, 
Fax: +353 21 270271			Cork, 
WWW: http://nmrc.ucc.ie			Ireland

**************************************************************


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

On Wed, 10 Jan 2001, Dermot MacSweeney wrote:
> Hi All,
>
> Is the Levenberg-Marquardt algorithm available in R. This method combines
the
> steepest descent algorithm and Newton's method.
My books have it as a modified Gauss-Newton method for non-linear least
squares problems, and although its some years since I read the papers,
I am pretty sure that is what the author's tackled.  Another way to look
at it is that it uses ridge regression on the local linearizations
to find the step.

1) R does not specialised software for non-linear least squares: nls is
Gauss-Newton inside, I believe.  (I could not find that documented, except
for the S version.)  But NLS is a specialized problem and not that common
in my experience.

2) Levenberg-Marquardt is hardly competitive these days. It probably
was not by the time of Marquardt(1963) in statistical problems, which are
normally in optimization parlance `large residual problems'.  That is, the
best fit is not nearly exact.

For such problems general optimization algorithms (e.g. those in optim)
are often at least as good as specialized NLS methods.  So I did not
think it worth implementing the current crop of specialized NLS methods.

The Nocedal and Wright (1999) reference in ?optim is an excellent
overview: I had page 267 open as I wrote this.

`Performance of the Gauss-Newton and Levenberg-Marquardt algorithms is
usual poor in the large-residual case.'

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

On Wed, 10 Jan 2001, Prof Brian Ripley wrote:
> 1) R does not specialised software for non-linear least squares: nls
> is Gauss-Newton inside, I believe. (I could not find that
> documented, except for the S version.) But NLS is a specialized
> problem and not that common in my experience.
Maybe in statistics, but not quite in econometrics. Most econometrics
textbooks discuss non-linear least squares in quite details. See, for
example, my favorite textbook

Davidson and MacKinnon (1993) Estimation and Inference in
Econometrics, Oxford University Press.
> For such problems general optimization algorithms (e.g. those in
> optim) are often at least as good as specialized NLS methods. So I
> did not think it worth implementing the current crop of specialized
> NLS methods.
There is an issue of what is meant by "good." Most NLS models
estimated in econometrics are small dimension "dense" models and for
these problems there is evidence that specialized algorithms produce
"better" results in terms of accuracy. There is a recent article^(1)
that compares the numerical accuracy of NLS estimates in canned
statistical packages using the NLS problems posted at NIST's
statistical reference datasets page at

http://www.nist.gov/itl/div898/strd/nls/nls_main.shtml

It is my experience that, at least for the dense NIST problems, the
specialized algorithms (for example, TOMS algorithm 573, NL2SOL
written by Dennis, Gay, and Welsch in 1981) return more accurate
estimates than general minimization routines that do not exploit the
least squares structure. I do not recall whether R was tested in the
article sited below, but it would be great if someone could volunteer
to test the accuracy of the R optimizer for the NIST problems.

(1) McCullough, BD. "Assessing the reliability of statistical
software"
Part  I: AMERICAN STATISTICIAN, 1998 NOV, V52 N4:358-366.
Part II: AMERICAN STATISTICIAN, 1999 MAY, V53 N2:149-159.

h.
--------------------------------------------------------------------
Time series regression studies give no sign of converging toward the
truth. (Phillip Cagan)

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._