similar to: Multiple cores are used in simple for loop

On Fri, 2016-01-15 at 15:03 +0100, Daniel Kaschek wrote: > Dear all, > > I run different R versions (3.2.1, 3.2.2 and 3.2.3) on different > platforms (Arch, Ubuntu, Debian) with a different number of available > cores (24, 4, 24). The following line produces very different behavior > on the three machines: > > for(i in 1:1e6) {n <- 100; M <- matrix(rnorm(n^2), n,

Dear Martyn, On Fr, Jan 15, 2016 at 4:01 , Martyn Plummer <plummerm at iarc.fr> wrote: > > Alternatively, you may be able to control the maximum number of > threads > by setting and exporting an appropriate environment variable depending > on what backend you are using, e.g. OPENBLAS_NUM_THREADS or > MKL_NUM_THREADS. Thanks a lot. Running export OPENBLAS_NUM_THREADS

Dear R-devel, I am running mclapply with many iterations over a function that modifies nothing and makes no copies of anything. It is taking up a lot of memory, so it seems to me like this is a bug. Should I post this to bugs.r-project.org? A minimal reproducible example can be obtained by first starting a memory monitoring program such as htop, and then executing the following code while

Hi, While I suspect that this is an issue peculiar to my machine (Debian squeeze amd64, R version 3.0.1, up-to-date packages), I'm hoping that somebody on this list may be able to give me suggestions on how to troubleshoot and fix the following: > library (snow) > cl <- makeSOCKcluster(c("localhost","localhost")) sh: 1: RunSnowWorker: not found I presume/hope

From: Daniel Kaschek <daniel.kaschek at physik.uni-freiburg.de> > ... When I evaluate this list of functions by > another lapply/sapply, I get an unexpected result: all values coincide. > However, when I uncomment the print(), it works as expected. Is this a > bug or a feature? > > conditions <- 1:4 > test <- lapply(conditions, function(mycondition){ >

Yes I am running on Rstudio 1.2.5033. I was also running this code without error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed work fine even with large data.frames. Any idea as to what interaction between Rstudio and mclapply causes this? Thanks, Shian On 28 Apr 2020, at 7:29 pm, Simon Urbanek <simon.urbanek at R-project.org<mailto:simon.urbanek at

Thanks Henrik, That clears things up significantly. I did see the warning but failed to include it my initial email. It sounds like an RStudio issue, and it seems like that it?s quite intrinsic to how forks interact with RStudio. Given this code is eventually going to be a part of a package, should I expect it to fail mysteriously in RStudio for my users? Is the best solution here to migrate all

Well it's only a leak if you don't get the memory back after it returns, right? Anyway, one (untested by me) possibility is the copying of memory pages when the garbage collector touches objects, as pointed out by Radford Neal here: http://r.789695.n4.nabble.com/Re-R-devel-Digest-Vol-149-Issue-22-td4710367.html If so, I don't think this would be easily avoidable, but there may be

Thanks Simon, I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don?t see a reason to not make use of it. The goal for the current piece of code I?m working on is to bootstrap many

Toby, > On Sep 2, 2015, at 1:12 PM, Toby Hocking <tdhock5 at gmail.com> wrote: > > Dear R-devel, > > I am running mclapply with many iterations over a function that modifies > nothing and makes no copies of anything. It is taking up a lot of memory, > so it seems to me like this is a bug. Should I post this to > bugs.r-project.org? > > A minimal reproducible

Dear R-devel, I am experiencing issues with running GAM models using mclapply, it fails to return any values if the data input becomes large. For example here the code runs fine with a df of 100 rows, but fails at 1000. library(mgcv) library(parallel) > df <- data.frame( + x = 1:100, + y = 1:100 + ) > > mclapply(1:2, function(i, df) { + fit <- gam(y ~ s(x, bs =

I've been using mclapply and have encountered situations where it gives errors or returns incorrect results. Here's a minimal example, which gives the error on R 2.15.2 on Mac and Linux: library(parallel) f <- function(x) NULL mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) # Error in sum(sapply(res, inherits, "try-error")) : # invalid 'type' (list) of argument

Dear R-devel friends: I'm back to bother you again about the conflict between mclapply and tcltk. I've been monitoring several packages that want to use mclapply to parallelize computations and need to figure out what should be done. It appears tcltk cannot be safely unloaded, so the best we can do is check for the presence of tcltk and stop if it is found before mclapply() is used. I

Hello, I have been having issues using parallel::mclapply in a memory-efficient way and would like some guidance. I am using a 40 core machine with 96 GB of RAM. I've tried to run mclapply with 20, 30, and 40 mc.cores and it has practically brought the machine to a standstill each time to the point where I do a hard reset. When running mclapply with 10 mc.cores, I can see that each process

I have this function and want to run it parallel with different sets of data. Using SNOW and clusterApplyLB. system.time(out <- mclapply(cData, plotGraph)) #each cData contains 100X6000 doubles system.time(out <- mclapply(cData2, plotGraph)) system.time(out <- mclapply(cData3, plotGraph)) system.time(out <- mclapply(cData4, plotGraph)) system.time(out <- mclapply(cData5,

> On Feb 25, 2015, at 5:35 PM, Benjamin Tyner <btyner at gmail.com> wrote: > > Actually, it depends on the number of cores: Under current semantics, yes. Each 'stream' of function calls is lazily capturing the last value of `i` on that core. Under Luke's proposed semantics (IIUC), the result would be the same (2,4,6,8) for both parallel and serial execution. This is

Dear members, I am using arfima in an mclapply construction (from the parallel package): Browse[2]> LYG <- mclapply(LYGH, FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:10], FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:2], FUN = arfima, mc.cores = detectCores()) ^C You can see that I am

Hello, I am running large simulations, which unfortunately I can't really replicate here because the code is so extensive. I rely heavily on mclapply, but I realize that I'm losing data somewhere. There are two worrisome symptoms: 1) I am getting 'NULL' as a return value for some (but not all) elements of the output when I use mclapply, but not if I use lapply > tmp2[1:3]

Dear parallel users and developers, I might have encountered a bug in the function 'mclapply' of package 'parallel'. I construct a matrix using the same input data and code with a single difference: Once I use mclapply and the other time lapply. Shockingly the result is NOT the same. To evaluate please unpack the attached archive and execute Rscript mclapply_test.R I put the

Dear Jeff, There was a problem in LYGH and lapply threw an error, but mclapply got stuck in an infinite loop. The doc for mclapply says that mclapply runs under try() with silent = TRUE. So that means mclapply should run properly, i.e output a try class object and exit. But it didn't. Can you shed some light on why this happened? THanking you, Yours sincerely, AKSHAY M