Dear R-users, we are using library(snow) for computation on a linux cluster with RMPI. We have a problem with clusterEvalQ: after launching clusterEvalQ it seems loading the required library on each node but if we type a function belonging to the loaded package R doesn't find it.> library(snow)# making cluster with 3 nodes> cl <- makeCluster(3, type = "MPI")Loading required package: Rmpi 3 slaves are spawned successfully. 0 failed. #loading library abind on each node> clusterEvalQ(cl,library(abind))[[1]] [1] "abind" "EMsspir" "mvtnorm" "MASS" "snow" "Rmpi" [7] "methods" "stats" "graphics" "grDevices" "utils" "datasets" [13] "base" [[2]] [1] "abind" "EMsspir" "mvtnorm" "MASS" "snow" "Rmpi" [7] "methods" "stats" "graphics" "grDevices" "utils" "datasets" [13] "base" [[3]] [1] "abind" "EMsspir" "mvtnorm" "MASS" "snow" "Rmpi" [7] "methods" "stats" "graphics" "grDevices" "utils" "datasets" [13] "base" #searching for function "abind" belonging to library "abind"> abindError: object "abind" not found Is it possible to know which libraries each node has loaded? Thank you very much best regards Michela and Marco
Michela Cameletti wrote:> Dear R-users, > we are using library(snow) for computation on a linux cluster with RMPI. > We have a problem with clusterEvalQ: after launching clusterEvalQ it seems > loading the required library on each node but if we type a function > belonging to the loaded package R doesn't find it. > >> library(snow) > > # making cluster with 3 nodes >> cl <- makeCluster(3, type = "MPI") > Loading required package: Rmpi > 3 slaves are spawned successfully. 0 failed. > > #loading library abind on each node >> clusterEvalQ(cl,library(abind)) > [[1]] > [1] "abind" "EMsspir" "mvtnorm" "MASS" "snow" "Rmpi" > [7] "methods" "stats" "graphics" "grDevices" "utils" "datasets" > [13] "base" > > [[2]] > [1] "abind" "EMsspir" "mvtnorm" "MASS" "snow" "Rmpi" > [7] "methods" "stats" "graphics" "grDevices" "utils" "datasets" > [13] "base" > > [[3]] > [1] "abind" "EMsspir" "mvtnorm" "MASS" "snow" "Rmpi" > [7] "methods" "stats" "graphics" "grDevices" "utils" "datasets" > [13] "base" > > #searching for function "abind" belonging to library "abind" >> abind > Error: object "abind" not found > > Is it possible to know which libraries each node has loaded?clusterEvalQ(cl,library()) You have loaded the ***package*** on the nodes, but not in your session directly! Uwe Ligges> Thank you very much > best regards > Michela and Marco > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
On 23 November 2006 at 16:08, Michela Cameletti wrote:
| Dear R-users,
| we are using library(snow) for computation on a linux cluster with RMPI.
| We have a problem with clusterEvalQ: after launching clusterEvalQ it seems
| loading the required library on each node but if we type a function
| belonging to the loaded package R doesn't find it.
|
| > library(snow)
|
| # making cluster with 3 nodes
| > cl <- makeCluster(3, type = "MPI")
| Loading required package: Rmpi
| 3 slaves are spawned successfully. 0 failed.
|
| #loading library abind on each node
| > clusterEvalQ(cl,library(abind))
| [[1]]
| [1] "abind" "EMsspir" "mvtnorm"
"MASS" "snow" "Rmpi"
| [7] "methods" "stats" "graphics"
"grDevices" "utils" "datasets"
| [13] "base"
|
| [[2]]
| [1] "abind" "EMsspir" "mvtnorm"
"MASS" "snow" "Rmpi"
| [7] "methods" "stats" "graphics"
"grDevices" "utils" "datasets"
| [13] "base"
|
| [[3]]
| [1] "abind" "EMsspir" "mvtnorm"
"MASS" "snow" "Rmpi"
| [7] "methods" "stats" "graphics"
"grDevices" "utils" "datasets"
| [13] "base"
|
| #searching for function "abind" belonging to library
"abind"
| > abind
| Error: object "abind" not found
|
| Is it possible to know which libraries each node has loaded?
Two quick hunches:
i) do you have the same directory layout on all nodes, i.e. is it say
/usr/local/lib/R/site-packages/abind everywhere ? I seem to recall that
this matters, but it may only have been relevant for binaries and the path.
ii) did you try manually on each machine if abind can in fact be loaded, eg
$ echo "stopifnot(require(abind,quiet=TRUE)); cat('Ok\n')"
| R --slave
I tend to just keep one installation of /usr/local/lib/R/site-packages and
share that via NFS to the nodes in my MPI/PVM/snow cluster as this helps to
keep tabs on i) and ii).
Hope this helps, Dirk
--
Hell, there are no rules here - we're trying to accomplish something.
-- Thomas A. Edison