search for: clusterapply

...parLapply(cl = clu, X = 1:2, fun = fun, c = 1) shouldn't give an error. This could be easily avoided by using all the argument names in the custerApply call of parLapply which means changing, parLapply <- function(cl = NULL, X, fun, ...) { cl <- defaultCluster(cl) do.call(c, clusterApply(cl, x = splitList(X, length(cl)), fun = lapply, fun, ...), quote = TRUE) } to parLapply <- function (cl = NULL, X, fun, ...) { cl <- defaultCluster(cl) do.call(c, clusterApply(cl = cl, x = splitList(X, length(cl)), fun = lapply, fun, ...), quote = TRUE) }...

Hi! I recognized that the argument matching of clusterApply (and therefore parLapply) goes wrong when one of the arguments of the function is called "c". In this case, the argument "c" is used as cluster and the functions give the following error message "Error in checkCluster(cl) : not a valid cluster". Of course, "c&quo...

...give an error. >> >> This could be easily avoided by using all the argument >> names in the custerApply call of parLapply which means changing, >> >> parLapply <- function(cl = NULL, X, fun, ...) { >> cl <- defaultCluster(cl) >> do.call(c, clusterApply(cl, x = splitList(X, length(cl)), >> fun = lapply, fun, ...), quote = TRUE) >> } >> >> to >> >> parLapply <- function (cl = NULL, X, fun, ...) { >> cl <- defaultCluster(cl) >> do.call(c, clusterApply(cl = cl, x = splitLis...

...fun, c = 1) > shouldn't give an error. > > This could be easily avoided by using all the argument > names in the custerApply call of parLapply which means changing, > > parLapply <- function(cl = NULL, X, fun, ...) { > cl <- defaultCluster(cl) > do.call(c, clusterApply(cl, x = splitList(X, length(cl)), > fun = lapply, fun, ...), quote = TRUE) > } > > to > > parLapply <- function (cl = NULL, X, fun, ...) { > cl <- defaultCluster(cl) > do.call(c, clusterApply(cl = cl, x = splitList(X, length(cl)), >...

This is nothing specific to parallel::clusterApply() per se. It is the default behavior of R where it allows for partial argument names. I don't think there's much that can be done here except always using fully named arguments to the "apply" function itself as you show. You can "alert" yourself when there's a mist...

...rkers, specifically, forking inside socket connections. When mclapply is called inside a SOCK, PSOCK or FORK worker I get an error in unserialize(). cl <- makeCluster(1, "SOCK") fun = function(i) { library(parallel) mclapply(1:2, sqrt) } Failure occurs after multiple calls to clusterApply: > clusterApply(cl, 1, fun) [[1]] [[1]][[1]] [1] 1 [[1]][[2]] [1] 1.414214 > clusterApply(cl, 1, fun) [[1]] [[1]][[1]] [1] 1 [[1]][[2]] [1] 1.414214 > clusterApply(cl, 1, fun) Error in unserialize(node$con) : error reading from connection This example is from Martin and may be a...

...(a) Found: It a parallel version of ?evalq?, "is" missing. (b) Found: 'parLapplyLB', 'parSapplyLB' are load-balancing versions, intended for use when applying ?FUN? to 'parLapplyLB' has no 'FUN' arg (more on this below). (c) Found: 'clusterApply' calls 'fun' on the first node with arguments 'seq[[1]]' and 'clusterApply' has no 'seq' arg. There are a few other occurences of mistake (c) in the \details and \value section, for clusterApply() and clusterApplyLB(). Omissions --------- o X: A vec...

...de a SOCK, PSOCK or FORK worker I get an >> error in unserialize(). >> >> cl <- makeCluster(1, "SOCK") >> >> fun = function(i) { >> library(parallel) >> mclapply(1:2, sqrt) >> } >> >> Failure occurs after multiple calls to clusterApply: >> >>> clusterApply(cl, 1, fun) >> [[1]] >> [[1]][[1]] >> [1] 1 >> >> [[1]][[2]] >> [1] 1.414214 >> >>> clusterApply(cl, 1, fun) >> [[1]] >> [[1]][[1]] >> [1] 1 >> >> [[1]][[2]] >> [1] 1.414214 &g...

...graphs induced by the permutations of node labels (to preserve the graph topology). Since I ll have many permutations to perform, I was thinking of using the snow package and in particular "parSapply" to divide the work between my 4 CPUs. But I get the following error message : Error in clusterApply(cl, splitList(x, length(cl)), lapply, fun, ...) : recursive default argument reference What am i doing wrong ? Some details about my platform and R version : ----------------------------------------------- $platform [1] "x86_64-redhat-linux-gnu" $version.string [1] "R vers...

...mon > When mclapply is called inside a SOCK, PSOCK or FORK worker I get an > error in unserialize(). > > cl <- makeCluster(1, "SOCK") > > fun = function(i) { > library(parallel) > mclapply(1:2, sqrt) > } > > Failure occurs after multiple calls to clusterApply: > > > clusterApply(cl, 1, fun) > [[1]] > [[1]][[1]] > [1] 1 > > [[1]][[2]] > [1] 1.414214 > > > clusterApply(cl, 1, fun) > [[1]] > [[1]][[1]] > [1] 1 > > [[1]][[2]] > [1] 1.414214 > > > clusterApply(cl, 1, fun) > Error in unser...

...ter only checks the variable cl. And the variable is still available after stopping the cluster! ( a simple solution would be deleting the cluster variable cl in the function stopCluster) > library(snow) > cl <- makeCluster(5) 5 slaves are spawned successfully. 0 failed. > clusterApply(cl, 1:2, get("+"), 3) [[1]] [1] 4 [[2]] [1] 5 > stopCluster(c1) [1] 1 > clusterApply(cl, 1:2, get("+"), 3) Fehler in mpi.probe(source, tag, comm, status) : MPI_Error_string: invalid communicator > sessionInfo() R version 2.6.0 (2007-10-03) x86_64-unknown-linux-...

...ng was several times larger than that taken by parallel processing. I'm very curious how this happened. Thank you very much. > library(snow) > > cc <- makePVMcluster(2) > > temp <- list(matrix(rnorm(1000000),1000),matrix(rnorm(1000000),1000)) > > system.time(tt <- clusterApply(cc,temp,"solve")) user system elapsed 0.584 0.144 4.355 > system.time(ttt <- sapply(temp,"solve")) user system elapsed 4.777 0.100 4.901 I'm using Ubuntu 8.10. And here's my CPU info: processor : 0 vendor_id : GenuineIntel cpu fami...

...>>> error in unserialize(). >>> >>> cl <- makeCluster(1, "SOCK") >>> >>> fun = function(i) { >>> library(parallel) >>> mclapply(1:2, sqrt) >>> } >>> >>> Failure occurs after multiple calls to clusterApply: >>> >>>> clusterApply(cl, 1, fun) >>> [[1]] >>> [[1]][[1]] >>> [1] 1 >>> >>> [[1]][[2]] >>> [1] 1.414214 >>> >>>> clusterApply(cl, 1, fun) >>> [[1]] >>> [[1]][[1]] >>> [1]...

...[1]: https://stackoverflow.com/questions/38230831/why-does-parlapplylb-not-actually-balance-load [2]: https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16792 ## The Call Chain First, we traced the relevant R function calls through the code, beginning with `parLapplyLB`: 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then **clusterApplyLB** 2. **splitList:** clusterApply.R:157 3. **clusterApplyLB:** clusterApply.R:87, calls **dynamicClusterApply** 4. **dynamicClusterApply:** clusterApply.R:39 ## splitList We used both our whiteboard and an R session to manually *run* a few exam...

Hi, here's an obscure question someone can hopefully help with. I have some R code that uses stuff from parallel (now a part of the R core in 2.15 I believe), especially clusterApply. However, this seems to cause problems in R.app, and I've seen advice to not use these multicore functions, e.g. doMC, in R.app. So, I want to make this optional. How can have a program check whether or not it is running in R.app or regular R? Thanks so much for your help!! Nick PS: I s...

...mbers. I copied the following from the command space for a simple example: cl<-makeCluster(cl, replicate,1,runif(2)) clusterCall(cl, replicate, 2, runif(2)) [[1]] 0.6533959 0.6533959 0.1071051 0.1071051 [[2]] 0.6533959 0.6533959 0.1071051 0.1071051 This is not alleviated by using clusterApply to set a random seed for each processor and seems to be related to the use of the replicate function within clusterCall. I have rearranged the function so that replicate is used to call the clusterCall function (ie. replicate(2, clusterCall(cl, runif,2),simplify=F) ) and resolved the random nu...

...ylb-not-actually-balance-load >> [2]: https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16792 >> >> >> ## The Call Chain >> >> First, we traced the relevant R function calls through the code, beginning with `parLapplyLB`: >> >> 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then **clusterApplyLB** >> 2. **splitList:** clusterApply.R:157 >> 3. **clusterApplyLB:** clusterApply.R:87, calls **dynamicClusterApply** >> 4. **dynamicClusterApply:** clusterApply.R:39 >> >> >> ## splitList >> >> We u...

...s/38230831/why-does-parlapplylb-not-actually-balance-load > [2]: https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16792 > > > ## The Call Chain > > First, we traced the relevant R function calls through the code, beginning with `parLapplyLB`: > > 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then **clusterApplyLB** > 2. **splitList:** clusterApply.R:157 > 3. **clusterApplyLB:** clusterApply.R:87, calls **dynamicClusterApply** > 4. **dynamicClusterApply:** clusterApply.R:39 > > > ## splitList > > We used both our whiteboard and a...

...you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP terminology, which has already been an inspiration for the present code (parLapply already implements static scheduling via internal function staticClusterApply, yet with a fixed chunk size; parLapplyLB already implements dynamic scheduling via internal function dynamicClusterApply, but with a fixed chunk size set to an unlucky value so that it behaves like static scheduling). The default chunk size for parallelLapplyLB will be set so that there is som...

...cts variables to export as in the amazing `future` package), the lack of copy-on-write functionality would cause scripts everywhere to break. A simple example illustrating these two points: `x <- 5e8; mclapply(1:24, sum, x, 8)` Using fork, `mclapply` takes 5 seconds. Using "psock", `clusterApply` does not complete. Travers On Fri, Apr 12, 2019 at 2:32 AM I?aki Ucar <iucar at fedoraproject.org> wrote: > > On Thu, 11 Apr 2019 at 22:07, Henrik Bengtsson > <henrik.bengtsson at gmail.com> wrote: > > > > ISSUE: > > Using *forks* for parallel processing in...