R devel - Dec 2012 - Found explanation for R-2.15.2 slowdown in one case; caution for any users of La_chol

2 days ago, I posted my long message about the observed slowdown in a
package between R-2.15.0 and R-2.15.2.

Uwe Ligges urged me to make a self-contained R example. That was the
encouragement I needed. I tracked the problem down to a failing use of
a LAPACK routine.

R's LAPACK C interface changed one variable in one function. But it
turned out to be an important change.  In case others have code that
is behaving in unexpected says, I'd urge package writers to
double-check their usage of the Cholesky inverse. Here are details:

In R 2.15.0, src/main/lapack.c, we have the prototype:

SEXP La_chol (SEXP A)

BUT in R 2.15.2, the prototype changed:

SEXP La_chol (SEXP A, SEXP pivot)

In the problem case I was studying, the effort to use La_chol was
wrapped in a "try" catch framework, and when Cholesky failed, it fell
back to a singular value decomposition. That's much slower, of course.

Hence the program seemed slower under R-2.15.2, but it was really
failing in a way that I had not noticed.

pj

-- 
Paul E. Johnson
Professor, Political Science      Assoc. Director
1541 Lilac Lane, Room 504      Center for Research Methods
University of Kansas                 University of Kansas
http://pj.freefaculty.org               http://quant.ku.edu

On 14.12.2012 23:31, Paul Johnson wrote:
> 2 days ago, I posted my long message about the observed slowdown in a
> package between R-2.15.0 and R-2.15.2.
>
> Uwe Ligges urged me to make a self-contained R example. That was the
> encouragement I needed. I tracked the problem down to a failing use of
> a LAPACK routine.
>
> R's LAPACK C interface changed one variable in one function. But it
> turned out to be an important change.  In case others have code that
> is behaving in unexpected says, I'd urge package writers to
> double-check their usage of the Cholesky inverse. Here are details:
>
> In R 2.15.0, src/main/lapack.c, we have the prototype:
>
> SEXP La_chol (SEXP A)
>
> BUT in R 2.15.2, the prototype changed:
>
> SEXP La_chol (SEXP A, SEXP pivot)
>
> In the problem case I was studying, the effort to use La_chol was
> wrapped in a "try" catch framework, and when Cholesky failed, it
fell
> back to a singular value decomposition. That's much slower, of course.
>
> Hence the program seemed slower under R-2.15.2, but it was really
> failing in a way that I had not noticed.
>
> pj
>

That is the reason why R CMD check gives a WARNING in the checks of your 
package for quite some time now:


checking foreign function calls ... WARNING

Foreign function calls with 'PACKAGE' argument in a base package:
.Call("La_chol", ..., PACKAGE = "base")
.Call("La_chol2inv", ..., PACKAGE = "base")
Packages should not make .C/.Call/.Fortran calls to base packages. They
are not part of the API, for use only by R itself and subject to change
without notice.


So time to fix that. See also:
http://cran.r-project.org/web/checks/check_results_Amelia.html


Best,
Uwe Ligges