search for: clusterappli

Thank you for your answer! I agree with you except for the 3 (Error) example and I realize now I should have started with that in the explanation. >From my point of view parLapply(cl = clu, X = 1:2, fun = fun, c = 1) shouldn't give an error. This could be easily avoided by using all the argument names in the custerApply call of parLapply which means changing, parLapply <-

Hi! I recognized that the argument matching of clusterApply (and therefore parLapply) goes wrong when one of the arguments of the function is called "c". In this case, the argument "c" is used as cluster and the functions give the following error message "Error in checkCluster(cl) : not a valid cluster". Of course, "c" is for many reasons an unfortunate

On 03/15/2018 05:25 PM, Henrik Bengtsson wrote: > On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: >> Thank you for your answer! >> I agree with you except for the 3 (Error) example and >> I realize now I should have started with that in the explanation. >> >> From my point of view >> parLapply(cl = clu, X = 1:2, fun = fun, c =

On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: > Thank you for your answer! > I agree with you except for the 3 (Error) example and > I realize now I should have started with that in the explanation. > > From my point of view > parLapply(cl = clu, X = 1:2, fun = fun, c = 1) > shouldn't give an error. > > This could be easily avoided by

This is nothing specific to parallel::clusterApply() per se. It is the default behavior of R where it allows for partial argument names. I don't think there's much that can be done here except always using fully named arguments to the "apply" function itself as you show. You can "alert" yourself when there's a mistake by using: options(warnPartialMatchArgs =

Hi Simon, I'm having trouble with nested parallel workers, specifically, forking inside socket connections. When mclapply is called inside a SOCK, PSOCK or FORK worker I get an error in unserialize(). cl <- makeCluster(1, "SOCK") fun = function(i) { library(parallel) mclapply(1:2, sqrt) } Failure occurs after multiple calls to clusterApply: > clusterApply(cl, 1,

Hi, Here are the issues I found: Typos ----- (a) Found: It a parallel version of ?evalq?, "is" missing. (b) Found: 'parLapplyLB', 'parSapplyLB' are load-balancing versions, intended for use when applying ?FUN? to 'parLapplyLB' has no 'FUN' arg (more on this below). (c) Found: 'clusterApply' calls 'fun' on the first

On 03/25/2015 07:48 PM, Simon Urbanek wrote: > On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: > >> Hi Simon, >> >> I'm having trouble with nested parallel workers, specifically, forking inside socket connections. >> > > You simply can't by definition - when you fork *all* the workers share the same connection

Hi, I want to compute a distribution of the intersection of a graph and 'randomized' graphs induced by the permutations of node labels (to preserve the graph topology). Since I ll have many permutations to perform, I was thinking of using the snow package and in particular "parSapply" to divide the work between my 4 CPUs. But I get the following error message : Error in

On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: > Hi Simon, > > I'm having trouble with nested parallel workers, specifically, forking inside socket connections. > You simply can't by definition - when you fork *all* the workers share the same connection inherited from the parent, so you cannot use any I/O operations that you

Hello, is there any function in the package snow to check for a really running cluster? The function checkCluster only checks the variable cl. And the variable is still available after stopping the cluster! ( a simple solution would be deleting the cluster variable cl in the function stopCluster) > library(snow) > cl <- makeCluster(5) 5 slaves are spawned successfully. 0

Hey my R buddies, I installed the "snow" and "rpvm" package on my Lenovo Thinkpad T400 today. The experiment below gave me a surprise. The time consumed by serial processing was several times larger than that taken by parallel processing. I'm very curious how this happened. Thank you very much. > library(snow) > > cc <- makePVMcluster(2) > > temp <-

On Mar 30, 2015, at 4:40 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: > On 03/25/2015 07:48 PM, Simon Urbanek wrote: >> On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: >> >>> Hi Simon, >>> >>> I'm having trouble with nested parallel workers, specifically, forking inside socket connections.

Dear R-Devel List, **TL;DR:** The function **parLapplyLB** of the parallel package has [reportedly][1] (see also attached RRD output) not been doing its job, i.e. not actually balancing the load. My colleague Dirk Sarpe and I found the cause of the problem and we also have a patch to fix it (attached). A similar fix has also been provided [here][2]. [1]:

Hi, here's an obscure question someone can hopefully help with. I have some R code that uses stuff from parallel (now a part of the R core in 2.15 I believe), especially clusterApply. However, this seems to cause problems in R.app, and I've seen advice to not use these multicore functions, e.g. doMC, in R.app. So, I want to make this optional. How can have a program check whether

I am trying to run a monte carlo process using snow with a MPI cluster. I have ~thirty processors to run the algorithm on and I want to run it 5000 times and take the average of the output. A very simple way to do this is to divide 5000 by the number of processors to get a number n and tell each processor to run the algorithm n times. I realize there are more efficient ways to manage the

Hi, I'm trying to understand the rationale for your proposed amount of splitting and more precisely why that one is THE one. If I put labels on your example numbers in one of your previous post: nbrOfElements <- 97 nbrOfWorkers <- 5 With these, there are two extremes in how you can split up the processing in chunks such that all workers are utilized: (A) Each worker, called

Dear R-Devel List, I have installed R 3.4.3 with the patch applied on our cluster and ran a *real-world* job of one of our users to confirm that the patch works to my satisfaction. Here are the results. The original was a series of jobs, all essentially doing the same stuff using bootstrapped data, so for the original there is more data and I show the arithmetic mean with standard deviation. The

Dear Christian and Henrik, thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP terminology, which has already been an inspiration for the present code (parLapply already implements static scheduling via internal function staticClusterApply, yet with a fixed chunk size; parLapplyLB already

Just throwing my two cents in: I think removing/deprecating fork would be a bad idea for two reasons: 1) There are no performant alternatives 2) Removing fork would break existing workflows Even if replaced with something using the same interface (e.g., a function that automatically detects variables to export as in the amazing `future` package), the lack of copy-on-write functionality would