Paul Johnson
2012-Dec-11 16:54 UTC
[R-sig-Debian] Debian packaging and openblas related crash when profiling in R
Hello R-sig-debian and (hopefully) Dirk:
On Debian wheezy, I have the R packaging that CRAN (you) provide. I
run into a little trouble while trying to fiddle with alternative
BLAS.
I know you and I went around on this last year and I think perhaps
I've found something wrong in the framework, or I've just done
something wrong.
I installed the packages openblas-base and openblas-dev, and they do
the "update alternatives" thing so that their shared library is at the
right spot. I can't tell if R runs any faster, but it does run, which
is encouraging.
Now here's the problem I see. Using Rprof causes R to crash, but only
when openblas stuff is in the picture. I wondered what you think?
Here's the code to reproduce this:
datM <-
read.csv("http://pj.freefaculty.org/scraps/profile/missingDat.csv")
library(Amelia)
set.seed(12345)
Rprof("Amelia-1.out")
impA <- amelia(datM, m = 5, idvars="group")
Rprof(NULL)
summaryRprof("Amelia-1.out")
Here's the crashed session
> datM <-
read.csv("http://pj.freefaculty.org/scraps/profile/missingDat.csv")
> library(Amelia)
Loading required package: foreign
##
## Amelia II: Multiple Imputation
## (Version 1.6.3, built: 2012-06-21)
## Copyright (C) 2005-2012 James Honaker, Gary King and Matthew Blackwell
## Refer to http://gking.harvard.edu/amelia/ for more information
##> set.seed(12345)
> Rprof("Amelia-1.out")
> impA <- amelia(datM, m = 5, idvars="group")
-- Imputation 1 --
1
*** caught segfault ***
address 0x11000088, cause 'memory not mapped'
Warning: stack imbalance in 'length', 121 then 122
Warning: stack imbalance in '==', 120 then 121
Warning: stack imbalance in 'if', 118 then 119
Warning: stack imbalance in '<-', 120 then 124
Warning: stack imbalance in '<-', 97 then 94
Warning: stack imbalance in 'min', 117 then 116
Warning: stack imbalance in 'min', 117 then 118
Traceback:
1: am.inv(a = g11)
2: doTryCatch(return(expr), name, parentenv, handler)
3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
4: tryCatchList(expr, classes, parentenv, handlers)
5: tryCatch(expr, error = function(e) { call <- conditionCall(e)
if (!is.null(call)) { if (identical(call[[1L]],
quote(doTryCatch))) call <- sys.call(-4L) dcall <-
deparse(call)[1L] prefix <- paste("Error in", dcall, ":
")
LONG <- 75L msg <- conditionMessage(e) sm <-
strsplit(msg, "\n")[[1L]] w <- 14L + nchar(dcall, type =
"w") +
nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L
+
nchar(dcall, type = "b") + nchar(sm[1L], type =
"b")
if (w > LONG) prefix <- paste0(prefix, "\n ")
}
else prefix <- "Error : " msg <- paste0(prefix,
conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if
(!silent && identical(getOption("show.error.messages"),
TRUE))
{ cat(msg, file = stderr())
.Internal(printDeferredWarnings()) } invisible(structure(msg,
class = "try-error", condition = e))})
6: try(am.inv(a = g11), silent = TRUE)
7: amsweep(thetareal, c(FALSE, o[ss, ]))
8: emfred(x, thetaold, indx$o, indx$m, indx$ivector, indx$icap,
indx$AMr1, indx$AMr2, AM1stln = AM1stln, returntype = "theta",
priors = priors, empri = empri, collect = collect)
9: emarch(x.stacked$x, p2s = p2s, thetaold = NULL, tolerance tolerance,
startvals = startvals, x.stacked$priors, empri = empri,
frontend = frontend, collect = collect, autopri = prepped$autopri,
emburn = emburn)
10: amelia.default(datM, m = 5, idvars = "group")
11: amelia(datM, m = 5, idvars = "group")
Warning: stack imbalance in '<-', 108 then 110
Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 3
Process R exited abnormally with code 70 at Tue Dec 11 10:36:51 2012
I double-checked /etc/alternatives and as you can see it does now
point to openbas
pauljohn at pjlap-124:alternatives$ ls -la libblas.*
lrwxrwxrwx 1 root root 36 Dec 11 10:34 libblas.a ->
/usr/lib/openblas-base/libopenblas.a
lrwxrwxrwx 1 root root 37 Dec 11 10:34 libblas.so ->
/usr/lib/openblas-base/libopenblas.so
lrwxrwxrwx 1 root root 39 Dec 11 10:34 libblas.so.3 ->
/usr/lib/openblas-base/libopenblas.so.0
lrwxrwxrwx 1 root root 39 Dec 11 10:34 libblas.so.3gf ->
/usr/lib/openblas-base/libopenblas.so.0
--
Paul E. Johnson
Professor, Political Science Assoc. Director
1541 Lilac Lane, Room 504 Center for Research Methods
University of Kansas University of Kansas
http://pj.freefaculty.org http://quant.ku.edu
Dirk Eddelbuettel
2012-Dec-11 18:33 UTC
[R-sig-Debian] Debian packaging and openblas related crash when profiling in R
On 11 December 2012 at 10:54, Paul Johnson wrote:
| Hello R-sig-debian and (hopefully) Dirk:
|
| On Debian wheezy, I have the R packaging that CRAN (you) provide. I
Not so -- I provide the distro package. Johannes provides the Debian
"backport" build via CRAN, Michael does likewise for Ubuntu.
| run into a little trouble while trying to fiddle with alternative
| BLAS.
|
| I know you and I went around on this last year and I think perhaps
| I've found something wrong in the framework, or I've just done
| something wrong.
|
| I installed the packages openblas-base and openblas-dev, and they do
| the "update alternatives" thing so that their shared library is at
the
| right spot. I can't tell if R runs any faster, but it does run, which
| is encouraging.
|
| Now here's the problem I see. Using Rprof causes R to crash, but only
| when openblas stuff is in the picture. I wondered what you think?
|
| Here's the code to reproduce this:
May I humbly suggest we file this under user-error?
R is famously single-threaded, OpenBLAS is not and I *strongly* suspect that
by profiling you are getting yourself into callbacks that R is simply not
built for. If you want to profile, constrain yourself to a single-threaded
blas such as reference blas or Atlas.
Dirk
| datM <-
read.csv("http://pj.freefaculty.org/scraps/profile/missingDat.csv")
| library(Amelia)
| set.seed(12345)
| Rprof("Amelia-1.out")
| impA <- amelia(datM, m = 5, idvars="group")
| Rprof(NULL)
| summaryRprof("Amelia-1.out")
|
|
| Here's the crashed session
|
| > datM <-
read.csv("http://pj.freefaculty.org/scraps/profile/missingDat.csv")
| > library(Amelia)
| Loading required package: foreign
| ##
| ## Amelia II: Multiple Imputation
| ## (Version 1.6.3, built: 2012-06-21)
| ## Copyright (C) 2005-2012 James Honaker, Gary King and Matthew Blackwell
| ## Refer to http://gking.harvard.edu/amelia/ for more information
| ##
| > set.seed(12345)
| > Rprof("Amelia-1.out")
| > impA <- amelia(datM, m = 5, idvars="group")
| -- Imputation 1 --
|
| 1
| *** caught segfault ***
| address 0x11000088, cause 'memory not mapped'
| Warning: stack imbalance in 'length', 121 then 122
| Warning: stack imbalance in '==', 120 then 121
| Warning: stack imbalance in 'if', 118 then 119
| Warning: stack imbalance in '<-', 120 then 124
| Warning: stack imbalance in '<-', 97 then 94
| Warning: stack imbalance in 'min', 117 then 116
| Warning: stack imbalance in 'min', 117 then 118
|
| Traceback:
| 1: am.inv(a = g11)
| 2: doTryCatch(return(expr), name, parentenv, handler)
| 3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
| 4: tryCatchList(expr, classes, parentenv, handlers)
| 5: tryCatch(expr, error = function(e) { call <- conditionCall(e)
| if (!is.null(call)) { if (identical(call[[1L]],
| quote(doTryCatch))) call <- sys.call(-4L) dcall <-
| deparse(call)[1L] prefix <- paste("Error in", dcall,
": ")
| LONG <- 75L msg <- conditionMessage(e) sm <-
| strsplit(msg, "\n")[[1L]] w <- 14L + nchar(dcall, type =
"w") +
| nchar(sm[1L], type = "w") if (is.na(w)) w <-
14L +
| nchar(dcall, type = "b") + nchar(sm[1L], type =
"b")
| if (w > LONG) prefix <- paste0(prefix, "\n
") }
| else prefix <- "Error : " msg <- paste0(prefix,
| conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L]))
if
| (!silent && identical(getOption("show.error.messages"),
TRUE))
| { cat(msg, file = stderr())
| .Internal(printDeferredWarnings()) } invisible(structure(msg,
| class = "try-error", condition = e))})
| 6: try(am.inv(a = g11), silent = TRUE)
| 7: amsweep(thetareal, c(FALSE, o[ss, ]))
| 8: emfred(x, thetaold, indx$o, indx$m, indx$ivector, indx$icap,
| indx$AMr1, indx$AMr2, AM1stln = AM1stln, returntype = "theta",
| priors = priors, empri = empri, collect = collect)
| 9: emarch(x.stacked$x, p2s = p2s, thetaold = NULL, tolerance | tolerance,
startvals = startvals, x.stacked$priors, empri = empri,
| frontend = frontend, collect = collect, autopri = prepped$autopri,
| emburn = emburn)
| 10: amelia.default(datM, m = 5, idvars = "group")
| 11: amelia(datM, m = 5, idvars = "group")
| Warning: stack imbalance in '<-', 108 then 110
|
| Possible actions:
| 1: abort (with core dump, if enabled)
| 2: normal R exit
| 3: exit R without saving workspace
| 4: exit R saving workspace
| Selection: 3
|
| Process R exited abnormally with code 70 at Tue Dec 11 10:36:51 2012
|
| I double-checked /etc/alternatives and as you can see it does now
| point to openbas
|
| pauljohn at pjlap-124:alternatives$ ls -la libblas.*
| lrwxrwxrwx 1 root root 36 Dec 11 10:34 libblas.a ->
| /usr/lib/openblas-base/libopenblas.a
| lrwxrwxrwx 1 root root 37 Dec 11 10:34 libblas.so ->
| /usr/lib/openblas-base/libopenblas.so
| lrwxrwxrwx 1 root root 39 Dec 11 10:34 libblas.so.3 ->
| /usr/lib/openblas-base/libopenblas.so.0
| lrwxrwxrwx 1 root root 39 Dec 11 10:34 libblas.so.3gf ->
| /usr/lib/openblas-base/libopenblas.so.0
|
| --
| Paul E. Johnson
| Professor, Political Science Assoc. Director
| 1541 Lilac Lane, Room 504 Center for Research Methods
| University of Kansas University of Kansas
| http://pj.freefaculty.org http://quant.ku.edu
|
| _______________________________________________
| R-SIG-Debian mailing list
| R-SIG-Debian at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-debian
--
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com