search for: parlapply

Dear R-user, I am trying to use the function 'parLapply' from the 'snow' package which is supposed to work the same wys as 'lapply' but for a parallelized cluster of computers. The function I am trying to call in parallel is 'dudi.pca' (from the 'ade4' package) which performs principal component analyses. When I c...

Hi R-developers In the package Parallel, the function parLapply(cl, x, f) seems to allow transmission of only one parameter (x) to the function f. Hence in order to compute f(x, y) parallelly, I had to define f(x, y) as f(x) and tried to access y within the function, whereas y was defined outside of f(x). Script: library(parallel) f <- function(x) { z &...

Thank you for your answer! I agree with you except for the 3 (Error) example and I realize now I should have started with that in the explanation. >From my point of view parLapply(cl = clu, X = 1:2, fun = fun, c = 1) shouldn't give an error. This could be easily avoided by using all the argument names in the custerApply call of parLapply which means changing, parLapply <- function(cl = NULL, X, fun, ...) { cl <- defaultCluster(cl) do.call(c, clusterAppl...

invoking parLapply without a cluster fails to find a previously registered cluster > library(parallel) > setDefaultCluster(makePSOCKcluster(2)) > parLapply(X=1:2, fun=function(...) {}) Error in cut.default(i, breaks) : invalid number of intervals This is because in parLapply length(cl) is determined be...

Dear R users, I have some problems with the parLapply function from the "parallel" package: I use parLapply on a pretty big R object without changing the object within the called function. If I execute parLapply alone, everything works fine. It seems that the object resides only once in the memory. But if I use the same call within another...

Hello everybody, I've got a bit of a problem with parLapply that's left me scratching my head today. I've tried this in R 2.11 and the 23 bit Revolution R Enterprise and gotten the same result, OS in question is Windows XP, the package involved is the snow package. I've got a list of 20 rain/no rain (1/0) situations for these two stations i an...

The runtime of parallel::parLapply depends on variables unrelated to the parLapply call. However, this is not clearly documented. Therefore I would like to suggest expanding the relevant documentation to explain this behaviour. Consider this example: parallel_demo <- function(random_values_count) { some_data <- runif(rando...

I am slightly confused by the way the standard output is redirected in a R snow cluster environment. I am using parLapply from the snow package to execute a function on my MPI/LAM cluster. How can I redirect standard output (produced using "cat") from this function back to the terminal where I invoked it? I intend to transmit some status information in advance to the final result of the function. I investiga...

Hi! I recognized that the argument matching of clusterApply (and therefore parLapply) goes wrong when one of the arguments of the function is called "c". In this case, the argument "c" is used as cluster and the functions give the following error message "Error in checkCluster(cl) : not a valid cluster". Of course, "c" is for many reasons an...

...Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: >> Thank you for your answer! >> I agree with you except for the 3 (Error) example and >> I realize now I should have started with that in the explanation. >> >> From my point of view >> parLapply(cl = clu, X = 1:2, fun = fun, c = 1) >> shouldn't give an error. >> >> This could be easily avoided by using all the argument >> names in the custerApply call of parLapply which means changing, >> >> parLapply <- function(cl = NULL, X, fun, ...) { >&gt...

In principle, I'd like to be able to do something like this: sge.parLapply(seq(10), function(x) parLapply(seq(x), function(x) x^2)) In practice, however, I have to resort to acrobatics like this: sge.options(sge.remove.files=FALSE) sge.options(sge.qsub.options='-cwd -V') sge.parLapply(seq(10), function(x) { sge.options(sge...

Hi, Here are the issues I found: Typos ----- (a) Found: It a parallel version of ?evalq?, "is" missing. (b) Found: 'parLapplyLB', 'parSapplyLB' are load-balancing versions, intended for use when applying ?FUN? to 'parLapplyLB' has no 'FUN' arg (more on this below). (c) Found: 'clusterApply' calls 'fun' on the first node with arguments 'seq[[1]]' and &...

...for 2 nodes 2 slaves are spawned successfully. 0 failed. Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: no applicable method for 'lapply' applied to an object of class "list" ". Where this is the problem line of code "yusum <- parLapply(cl, yu, sum)" The function is CallSnow <- function (Nnodes = 2, Nsamples = Nnodes) { require(Rmpi) require(snow) cat("Using snow package, asking for ", Nnodes, "nodes \n") cl <- makeCluster(Nnodes, type="MPI") on.exit(stopCluster(cl)) #print(do.ca...

...") library("snow") a <- as.data.frame(replicate(1000, rnorm(20))) b <- as.data.frame(replicate(1000, rnorm(20))) c <- as.data.frame(replicate(1000, rnorm(20))) d <- as.data.frame(replicate(1000, rnorm(20))) abcd <- list(a, b, c, d) cl <- makeCluster(4) results <- parLapply(cl,    X = abcd,    FUN = function(i) {      clhs(x = i, size = round(nrow(i) / 5), iter = 2000, simple = FALSE)    }, ) stopCluster(cl) ****************************************** Before running the last line, R is throwing an error: "Error in length(x) : 'x' is missing". Any ide...

On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: > Thank you for your answer! > I agree with you except for the 3 (Error) example and > I realize now I should have started with that in the explanation. > > From my point of view > parLapply(cl = clu, X = 1:2, fun = fun, c = 1) > shouldn't give an error. > > This could be easily avoided by using all the argument > names in the custerApply call of parLapply which means changing, > > parLapply <- function(cl = NULL, X, fun, ...) { > cl <- defaultCluste...

...R seems to clone all environments (that are normally passed by reference) that are returned from a child process. In particular, consider the following example: library(parallel) env1 <- new.env() envs2 <- lapply(1:4, function(x) env1) cl<-makeCluster(2, type="FORK") envs3 <- parLapply(cl, 1:4, function(x) env1) envs4 <- parLapply(cl, 1:4, function(x) capture.output(str(env1))) stopCluster(cl) First I make an environment (env1). Then using the non-parallel lapply I get a list (envs2) where all entries contain a pointer to env1. Now when using the parallel parLapply the entrie...

R Version : 2.14.1 x64 Running on Windows 7 Connecting to a database on a remote Microsoft SQL Server 2012 The short form of my problem is the following. I have an unordered vectors of names, say: names<-c("A", "B", "A", "C","C") each of which have an id in a table in my db. I need to convert the names to their corresponding ids. I

...silent = mc.silent)) res <- mccollect(jobs) On 10/24/2012 05:07 PM, Cook, Malcolm wrote: > On 10/24/12 12:44 AM, "Michael Lawrence" <lawrence.michael at gene.com> wrote: > >> I agree that it would fruitful to have parLapply in BiocGenerics. It looks >> to be a flexible abstraction and its presence in the parallel package >> makes >> it ubiquitous. If it hasn't been done already, mclapply (and mcmapply) >> would be good candidates, as well. The fork-based parallelism is >> substantivel...

This is all normal, a fork cluster works with processes, that do not share memory. When you create a fork cluster, you create a new process, that has the same memory layout as the parent. But from this moment its memory is independent of the parent process. When parLapply is done, the results are serialized and copied back to the parent process. The serialized environment is independent of the original environment in the parent process, when parLapply unserializes the results it creates new objects. Environments have reference semantics, but not across processes....

...iocGenerics package] Hi, I am currently running a lengthy simulation study (no details necessary) on a large multi-core system. The simulated data sets are stored in a long list and they are unevenly sized (hence, the computation times vary greatly between data sets), so I naively thought that parLapplyLB() would be the right function to parallelize my computations. Presently, only two thirds of my computations are finished, however, only 16 of my 72 workers seem to be active, whereas the remaining 56 seem to run idle. I tried to find out in more detail what parLapplyLB() actually does and I...