similar to: Typos/omissions/inconsistencies in man page for clusterApply

[cross-posted on R-devel and Bioc-devel, since the functions from the parallel package discussed here are mirrored in the BiocGenerics package] Hi, I am currently running a lengthy simulation study (no details necessary) on a large multi-core system. The simulated data sets are stored in a long list and they are unevenly sized (hence, the computation times vary greatly between data sets), so

Dear Tomas, Thanks for your commitment to fix this issue and also to add the chunk size as an argument. If you want our input, let us know ;) Best Regards On 02/26/2018 04:01 PM, Tomas Kalibera wrote: > Dear Christian and Henrik, > > thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP

Dear Henrik, The rationale is just that it is within these extremes and that it is really simple to calculate, without making any assumptions and knowing that it won't be perfect. The extremes A and B you are mentioning are special cases based on assumptions. Case A is based on the assumption that the function has a long runtime or varying runtime, then you are likely to get the best load

Dear R-Devel List, I have installed R 3.4.3 with the patch applied on our cluster and ran a *real-world* job of one of our users to confirm that the patch works to my satisfaction. Here are the results. The original was a series of jobs, all essentially doing the same stuff using bootstrapped data, so for the original there is more data and I show the arithmetic mean with standard deviation. The

According to ?parCapply: parRapply and parCapply always return a vector. This appears not to be the case in the following minimal reproducible example: > library(parallel) > nslaves <- 2 > cl <- makeCluster(nslaves) > X <- matrix(2,nrow=3,ncol=4) > X <- rbind(c(1,1,0,1),X) > tv <- parCapply(cl,X,FUN=function(x){ +

Dear Christian and Henrik, thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP terminology, which has already been an inspiration for the present code (parLapply already implements static scheduling via internal function staticClusterApply, yet with a fixed chunk size; parLapplyLB already

On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: > Thank you for your answer! > I agree with you except for the 3 (Error) example and > I realize now I should have started with that in the explanation. > > From my point of view > parLapply(cl = clu, X = 1:2, fun = fun, c = 1) > shouldn't give an error. > > This could be easily avoided by

On 03/15/2018 05:25 PM, Henrik Bengtsson wrote: > On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: >> Thank you for your answer! >> I agree with you except for the 3 (Error) example and >> I realize now I should have started with that in the explanation. >> >> From my point of view >> parLapply(cl = clu, X = 1:2, fun = fun, c =

Hi, I'm trying to understand the rationale for your proposed amount of splitting and more precisely why that one is THE one. If I put labels on your example numbers in one of your previous post: nbrOfElements <- 97 nbrOfWorkers <- 5 With these, there are two extremes in how you can split up the processing in chunks such that all workers are utilized: (A) Each worker, called

Thank you for your answer! I agree with you except for the 3 (Error) example and I realize now I should have started with that in the explanation. >From my point of view parLapply(cl = clu, X = 1:2, fun = fun, c = 1) shouldn't give an error. This could be easily avoided by using all the argument names in the custerApply call of parLapply which means changing, parLapply <-

Hi! I recognized that the argument matching of clusterApply (and therefore parLapply) goes wrong when one of the arguments of the function is called "c". In this case, the argument "c" is used as cluster and the functions give the following error message "Error in checkCluster(cl) : not a valid cluster". Of course, "c" is for many reasons an unfortunate

This is nothing specific to parallel::clusterApply() per se. It is the default behavior of R where it allows for partial argument names. I don't think there's much that can be done here except always using fully named arguments to the "apply" function itself as you show. You can "alert" yourself when there's a mistake by using: options(warnPartialMatchArgs =

Hi, I want to compute a distribution of the intersection of a graph and 'randomized' graphs induced by the permutations of node labels (to preserve the graph topology). Since I ll have many permutations to perform, I was thinking of using the snow package and in particular "parSapply" to divide the work between my 4 CPUs. But I get the following error message : Error in

Dear R-Devel List, **TL;DR:** The function **parLapplyLB** of the parallel package has [reportedly][1] (see also attached RRD output) not been doing its job, i.e. not actually balancing the load. My colleague Dirk Sarpe and I found the cause of the problem and we also have a patch to fix it (attached). A similar fix has also been provided [here][2]. [1]:

The build system rolled up R-2.15.0.tar.gz (codename "Easter Beagle") at 9:00 this morning. This is the first release of the 2.15 series and contains several new features and changes; see the list below for details. You can get the source code from http://cran.r-project.org/src/base/R-2/R-2.15.0.tar.gz or wait for it to be mirrored at a CRAN site nearer to you. Binaries for various

The build system rolled up R-2.15.0.tar.gz (codename "Easter Beagle") at 9:00 this morning. This is the first release of the 2.15 series and contains several new features and changes; see the list below for details. You can get the source code from http://cran.r-project.org/src/base/R-2/R-2.15.0.tar.gz or wait for it to be mirrored at a CRAN site nearer to you. Binaries for various

Hi, This is my first message to this list. It would be great if people here could help me with the following problem (self-contained code example below): I have a list of matrices and would like to apply a summary function to the matrices in the list. The matrices represent social networks, therefore I need to apply some specialized summary functions provided by the ergm package. These

R Version : 2.14.1 x64 Running on Windows 7 Connecting to a database on a remote Microsoft SQL Server 2012 The short form of my problem is the following. I have an unordered vectors of names, say: names<-c("A", "B", "A", "C","C") each of which have an id in a table in my db. I need to convert the names to their corresponding ids. I

Hello R-devel, A function to be run inside lapply() or one of its friends is trivial to augment with side effects to show a progress bar. When the code is intended to be run on a 'parallel' cluster, it generally cannot rely on its own side effects to report progress. I've found three approaches to progress bars for parallel processes on CRAN: - Importing 'snow' (not

Hi there, I have a data frame DF with 40 millions strings and their frequency. I am searching for strings with a given pattern and I am trying to speed up this part of my code. I try many options but so far I am not satisfied. I tried: - grepl and subset are equivalent in term of processing time grepl(paste0("^",pattern),df$Strings) subset(df,