similar to: [parallel] fixes load balancing of parLapplyLB

Hi, I'm trying to understand the rationale for your proposed amount of splitting and more precisely why that one is THE one. If I put labels on your example numbers in one of your previous post: nbrOfElements <- 97 nbrOfWorkers <- 5 With these, there are two extremes in how you can split up the processing in chunks such that all workers are utilized: (A) Each worker, called

Dear Christian and Henrik, thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP terminology, which has already been an inspiration for the present code (parLapply already implements static scheduling via internal function staticClusterApply, yet with a fixed chunk size; parLapplyLB already

Dear Henrik, The rationale is just that it is within these extremes and that it is really simple to calculate, without making any assumptions and knowing that it won't be perfect. The extremes A and B you are mentioning are special cases based on assumptions. Case A is based on the assumption that the function has a long runtime or varying runtime, then you are likely to get the best load

Dear Tomas, Thanks for your commitment to fix this issue and also to add the chunk size as an argument. If you want our input, let us know ;) Best Regards On 02/26/2018 04:01 PM, Tomas Kalibera wrote: > Dear Christian and Henrik, > > thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP

Looking at parLapplyLB, one sees that it takes in X and then passes splitList(X, length(cl)) to clusterApplyLB, which then calls dynamicClusterApply. Thus while dynamicClusterApply does handle tasks in a load-balancing fashion, sending out individual tasks as previous tasks complete, parLapplyLB preempts that by splitting up the tasks in advance into as many groups of tasks as there are cluster

Dear R-Devel List, I have installed R 3.4.3 with the patch applied on our cluster and ran a *real-world* job of one of our users to confirm that the patch works to my satisfaction. Here are the results. The original was a series of jobs, all essentially doing the same stuff using bootstrapped data, so for the original there is more data and I show the arithmetic mean with standard deviation. The

[cross-posted on R-devel and Bioc-devel, since the functions from the parallel package discussed here are mirrored in the BiocGenerics package] Hi, I am currently running a lengthy simulation study (no details necessary) on a large multi-core system. The simulated data sets are stored in a long list and they are unevenly sized (hence, the computation times vary greatly between data sets), so

Greetings, I am having an interesting problem and I wonder if anyone else has seen this behavior. I am running R 2.11.1 with SNOW 0.3-3 on a Dell cluster running CentOS 5.5. I create my cluster using: cluster<- makeCluster(nodes,type="SOCK",port=10191) # nodes is a vector of compute nodes I then wrap a loop around clusterApplyLB to evaluate my function multiple times, with

Hi folks, Suppose I create the character vector charvec by > charvec<-c("a1.b1","a2.b2") > charvec [1] "a1.b1" "a2.b2" and then I use strsplit on charvec as follows: > splitlist<-strsplit(charvec,split=".",fixed=TRUE) > splitlist [[1]] [1] "a1" "b1" [[2]] [1] "a2" "b2" I was wondering

Thank you for your answer! I agree with you except for the 3 (Error) example and I realize now I should have started with that in the explanation. >From my point of view parLapply(cl = clu, X = 1:2, fun = fun, c = 1) shouldn't give an error. This could be easily avoided by using all the argument names in the custerApply call of parLapply which means changing, parLapply <-

Hi, Here are the issues I found: Typos ----- (a) Found: It a parallel version of ?evalq?, "is" missing. (b) Found: 'parLapplyLB', 'parSapplyLB' are load-balancing versions, intended for use when applying ?FUN? to 'parLapplyLB' has no 'FUN' arg (more on this below). (c) Found: 'clusterApply' calls 'fun' on the first

Hi, I'm turning my wheels on this and keep coming around to the same wrong solution - please have a look and give a hand ... The premise is: a DF like so > loremIpsum <- "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque leo ipsum, ultricies scelerisque volutpat non, volutpat et nulla. Curabitur consequat ullamcorper tellus id imperdiet. Duis semper malesuada

On 03/15/2018 05:25 PM, Henrik Bengtsson wrote: > On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: >> Thank you for your answer! >> I agree with you except for the 3 (Error) example and >> I realize now I should have started with that in the explanation. >> >> From my point of view >> parLapply(cl = clu, X = 1:2, fun = fun, c =

Hello, I am new to R. I often collect data at multiple sites and need to create separate graphs (such as scatterplots or histograms) of specific variables for each site. I have tried to do this by splitting the data frame and then using lapply, but it seems that the graphing commands cannot be called as functions. Here is a sample of my data, called "seeddist2": site

Hi, I want to compute a distribution of the intersection of a graph and 'randomized' graphs induced by the permutations of node labels (to preserve the graph topology). Since I ll have many permutations to perform, I was thinking of using the snow package and in particular "parSapply" to divide the work between my 4 CPUs. But I get the following error message : Error in

On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: > Thank you for your answer! > I agree with you except for the 3 (Error) example and > I realize now I should have started with that in the explanation. > > From my point of view > parLapply(cl = clu, X = 1:2, fun = fun, c = 1) > shouldn't give an error. > > This could be easily avoided by

Hi, I am currently building an R package and I am facing a peculiar problem where some of the functions does not work within the package. However, if I source the script the function works. For example, in a method for parallelization of analysis on each chromosome simultaneously I am receiving error at the following position of the code: # this profile the information chromosome wise and

Hi! I recognized that the argument matching of clusterApply (and therefore parLapply) goes wrong when one of the arguments of the function is called "c". In this case, the argument "c" is used as cluster and the functions give the following error message "Error in checkCluster(cl) : not a valid cluster". Of course, "c" is for many reasons an unfortunate

Hi, I wish to NEATLY store, and later display one table per file (similar to capabilities of write.table()). BUT BY NEATLY I MEAN: 1. Column Headings right aligned with data values. 2. Decimal points line-up per column. 3. Data values padded trailing zeros per column (if not integers). 4. "Tricky" formats such as certain characters/dates possibly chosen by the user? bla bla bla I do

Hello everybody, I've got a bit of a problem with parLapply that's left me scratching my head today. I've tried this in R 2.11 and the 23 bit Revolution R Enterprise and gotten the same result, OS in question is Windows XP, the package involved is the snow package. I've got a list of 20 rain/no rain (1/0) situations for these two stations i and j, all the items in this list look