search for: nbrofchunks

...nbrOfElements <- 97 nbrOfWorkers <- 5 With these, there are two extremes in how you can split up the processing in chunks such that all workers are utilized: (A) Each worker, called multiple times, processes one element each time: > nbrOfElements <- 97 > nbrOfWorkers <- 5 > nbrOfChunks <- nbrOfElements > sapply(parallel:::splitList(1:nbrOfElements, nbrOfChunks), length) [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 [30] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 [59] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 [88] 1 1 1 1 1 1 1...

...two extremes in how you can split up the >> processing in chunks such that all workers are utilized: >> >> (A) Each worker, called multiple times, processes one element each >> time: >> >>> nbrOfElements <- 97 >>> nbrOfWorkers <- 5 >>> nbrOfChunks <- nbrOfElements >>> sapply(parallel:::splitList(1:nbrOfElements, nbrOfChunks), length) >> [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 >> [30] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 >> [59] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1...

...;- 5 > >With these, there are two extremes in how you can split up the >processing in chunks such that all workers are utilized: > >(A) Each worker, called multiple times, processes one element each >time: > >> nbrOfElements <- 97 >> nbrOfWorkers <- 5 >> nbrOfChunks <- nbrOfElements >> sapply(parallel:::splitList(1:nbrOfElements, nbrOfChunks), length) > [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 >[30] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 >[59] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 &g...

...it up the >>> processing in chunks such that all workers are utilized: >>> >>> (A) Each worker, called multiple times, processes one element each >>> time: >>> >>>> nbrOfElements <- 97 >>>> nbrOfWorkers <- 5 >>>> nbrOfChunks <- nbrOfElements >>>> sapply(parallel:::splitList(1:nbrOfElements, nbrOfChunks), length) >>> [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 >>> [30] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 >>> [59] 1 1 1 1 1 1 1 1 1 1 1 1 1...

Dear R-Devel List, **TL;DR:** The function **parLapplyLB** of the parallel package has [reportedly][1] (see also attached RRD output) not been doing its job, i.e. not actually balancing the load. My colleague Dirk Sarpe and I found the cause of the problem and we also have a patch to fix it (attached). A similar fix has also been provided [here][2]. [1]: