similar to: Simple example for High performance computing with R?

On a Unix machine I ran caret::rfe using the multicore package, and I saved the resulting object using save(lm2, file = "lm2.RData"). [Reproducible example below.] When I try to load("lm2.RData") on my Windows laptop, I get Error in loadNamespace(name) : there is no package called 'multicore' I completely understand the error and I would like to ignore it and

dear r experts---Is there a multicore equivalent of by(), just like mclapply() is the multicore equivalent of lapply()? if not, is there a fast way to convert a data.table into a list based on a column that lapply and mclapply can consume? advice appreciated...as always. regards, /iaw ---- Ivo Welch (ivo.welch at gmail.com)

Hi all, I have R installed on a box, which is running on a machine with 16 core and Redhat - Linux. I am handling huge (size of dataset will be 5 GB) dataset. Lets assume that my data is in the form of structured (multiple) logs. I access the data by using all.files(). Since by default basic version of R utilizes single core, the processing of my analysis code is taking too much time. I got to

Hello all, I upgraded my R workstation, and to my dismay, only one core appears to be used during intensive computation of a bioconductor function. What I have now is two dual-core Xeon 5160 CPUs and 10 GB RAM. When I fully load it, top reports about 25% user, 75% idle and 0.98 short-term load. The archives gave nothing helpful besides mention of snow. I thought of posting to HPC, but this system

Dear list, When I use VennDiagram package, I got a error as follow: venn.plot <- draw.pairwise.venn( area1 = 3186, area2 = 325, cross.area = 5880); Error in draw.pairwise.venn(area1 = 3186, area2 = 325, cross.area = 588) : Impossible: cross section area too large. Does anyone have suggestion? Thank you.

Thanks Simon, I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don?t see a reason to not make use of it. The goal for the current piece of code I?m working on is to bootstrap many

The attached code produces corrupted pdfs (test2.pdf, test4.pdf and test5.pdf). The resulting pdf depends on how many cores are available on the machine. I don't see why there should be any difference between the pdfs (exept for the timestamp). Doing many operations involving mclapply can increase the size of the resulting pdf by ten times! Thank you for checking this. require(multicore)

Sir, I want to apply mclapply() function for my analysis. So, I have to install multicore package. But I can not install the package. >install.packages("multicore") It gives that package multicore is not available. Can you help me? Regards, Suman Dhara [[alternative HTML version deleted]]

Dear all, I have an R script that uses Rcpp, and I have been trying to parallelize it using mclapply (I tried with the multicore and the parallel library) Sometimes (not always, interestingly), the CPU use for each core drops, usually so that the total over all cores reaches 100%, i.e., as fast as if using just one single core fully. I tried my code directly from within emacs, and also using a

Hello, I spent a lot of a time on a weird bug, and I just managed to narrow it down. In parallel code (here with parallel::mclappy, but I got it doMC/multicore too), if the library(tcltk) is loaded, R hangs when trying to open a DB connection. I got the same behaviour on two different computers, one dual-core, and one 2 xeon quad-core. Here's the code: library(parallel) library(RSQLite)

Thanks Henrik, That clears things up significantly. I did see the warning but failed to include it my initial email. It sounds like an RStudio issue, and it seems like that it?s quite intrinsic to how forks interact with RStudio. Given this code is eventually going to be a part of a package, should I expect it to fail mysteriously in RStudio for my users? Is the best solution here to migrate all

Dear R developers, with the inclusion of the package "parallel" in the upcoming release of R, users and package developers are likely to make increasing usage of parallelization features. In part, these features rely on forking the R process. As ?mcfork points out, fork()ing in a GUI process is typically a bad idea. In RKWard, we "only" seem to have problems with signals

On Windows there are a few 'pthreads' implementation, e.g. pthreads-w32 and winpthreads [https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Using-pthreads]. We're thinking of giving them a try for the matrixStats package, and basic tests indicates it works, but since Windows pthreads are not used by core R (or?) I've got a little bit worried that we will face overwhelming

I'm running r 2. on a mac running 10.6.4 and a dual-core macbook pro. I'm having a funny time with multicore. When I run it with 2 cores, mclapply, R borks with the following error. The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()

Dear list, Is there a elegant way to name multiple ordered plot as A, B, C and D? Just put A, B, C and D on the top left corner of each plot. Usually I do it in Illustrator, but I think in R should there is a way. Could you please provide a example? Thank you very much in advance.

Is there any reason to expect a problem ? i'm running this script on the cluster down the hall: module load R/2.11.0 R library(multicore) fxx<-function(ll) runif(1) mclapply(1:10,fxx) i get: Error in fork() : Unable to fork. less /proc/cpuinfo yields: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Xeon(R) CPU

On 03/25/2015 07:48 PM, Simon Urbanek wrote: > On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: > >> Hi Simon, >> >> I'm having trouble with nested parallel workers, specifically, forking inside socket connections. >> > > You simply can't by definition - when you fork *all* the workers share the same connection

This is a 'rather than re-invent the wheel' post. Has anyone out there re-written combn so that it can be parallelized - with multicore, snow, or otherwise? I have a job that requires large numbers of combinations, and rather than get all of the index values, then crank it through mclapply, I was wondering if there was a way to just do this natively within a function. Just curious.

Hi! I have a data.frame "data" and splitted it. data <- split(data, data[,1]) This is a quite slow procedure; and I do not want to do it again. So, any unsplit and "resplit" is no option for me. But: I have to cbind "variables" to the splitted data from another list, that contains of vectors with matching sizes, so for (i in 1:length(data)) { data[[i]]

Aaaah ... and argh - I should have better not to post R question at midnight, especially when I know it forks the process and it's not using threads. Brain meltdown. (So, we'll proceed trying to use pthreads in matrixStats also for Windows). Sorry for the noise and thanks Kasper. Henrik On Aug 15, 2015 02:52, "Kasper Daniel Hansen" <kasperdanielhansen at gmail.com> wrote: