similar to: Multiple cores are used in simple for loop

On Fri, 2016-01-15 at 15:03 +0100, Daniel Kaschek wrote: > Dear all, > > I run different R versions (3.2.1, 3.2.2 and 3.2.3) on different > platforms (Arch, Ubuntu, Debian) with a different number of available > cores (24, 4, 24). The following line produces very different behavior > on the three machines: > > for(i in 1:1e6) {n <- 100; M <- matrix(rnorm(n^2), n,

Dear Martyn, On Fr, Jan 15, 2016 at 4:01 , Martyn Plummer <plummerm at iarc.fr> wrote: > > Alternatively, you may be able to control the maximum number of > threads > by setting and exporting an appropriate environment variable depending > on what backend you are using, e.g. OPENBLAS_NUM_THREADS or > MKL_NUM_THREADS. Thanks a lot. Running export OPENBLAS_NUM_THREADS

Dear R-devel, I am running mclapply with many iterations over a function that modifies nothing and makes no copies of anything. It is taking up a lot of memory, so it seems to me like this is a bug. Should I post this to bugs.r-project.org? A minimal reproducible example can be obtained by first starting a memory monitoring program such as htop, and then executing the following code while

Hi, While I suspect that this is an issue peculiar to my machine (Debian squeeze amd64, R version 3.0.1, up-to-date packages), I'm hoping that somebody on this list may be able to give me suggestions on how to troubleshoot and fix the following: > library (snow) > cl <- makeSOCKcluster(c("localhost","localhost")) sh: 1: RunSnowWorker: not found I presume/hope

From: Daniel Kaschek <daniel.kaschek at physik.uni-freiburg.de> > ... When I evaluate this list of functions by > another lapply/sapply, I get an unexpected result: all values coincide. > However, when I uncomment the print(), it works as expected. Is this a > bug or a feature? > > conditions <- 1:4 > test <- lapply(conditions, function(mycondition){ >

Yes I am running on Rstudio 1.2.5033. I was also running this code without error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed work fine even with large data.frames. Any idea as to what interaction between Rstudio and mclapply causes this? Thanks, Shian On 28 Apr 2020, at 7:29 pm, Simon Urbanek <simon.urbanek at R-project.org<mailto:simon.urbanek at

Thanks Henrik, That clears things up significantly. I did see the warning but failed to include it my initial email. It sounds like an RStudio issue, and it seems like that it?s quite intrinsic to how forks interact with RStudio. Given this code is eventually going to be a part of a package, should I expect it to fail mysteriously in RStudio for my users? Is the best solution here to migrate all

Hi I just upgraded my system to xen-3.0-testing, hoping it would solve my problems with the virtual network interfaceses (I cannot connect between two domains) and memory usage (I sometime "miss" a few hundret megabytes). The update didn''t help on the networking issues, but it did on memory usage. However, I created a new problem: using htop, the cpu is used at 50% in both

Thanks Simon, I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don?t see a reason to not make use of it. The goal for the current piece of code I?m working on is to bootstrap many

Well it's only a leak if you don't get the memory back after it returns, right? Anyway, one (untested by me) possibility is the copying of memory pages when the garbage collector touches objects, as pointed out by Radford Neal here: http://r.789695.n4.nabble.com/Re-R-devel-Digest-Vol-149-Issue-22-td4710367.html If so, I don't think this would be easily avoidable, but there may be

Hey all, I have a Windows 10 pro (64bit) long time running as a libvirt/KVM guest that I think Windows Update finally narfed. The hardware is a supermicro motherboard with dual Intel E5-2640 CPUs for a total of 40 threads and 64GB The guest is allocated 2sockets, 5cores, 2 threads and 32GB of RAM. (I've also tried 1 socket, 20 cores, 1 thread and 1s10c2t -- no help) In an SSH session to

Dear R-devel, I am experiencing issues with running GAM models using mclapply, it fails to return any values if the data input becomes large. For example here the code runs fine with a df of 100 rows, but fails at 1000. library(mgcv) library(parallel) > df <- data.frame( + x = 1:100, + y = 1:100 + ) > > mclapply(1:2, function(i, df) { + fit <- gam(y ~ s(x, bs =

Toby, > On Sep 2, 2015, at 1:12 PM, Toby Hocking <tdhock5 at gmail.com> wrote: > > Dear R-devel, > > I am running mclapply with many iterations over a function that modifies > nothing and makes no copies of anything. It is taking up a lot of memory, > so it seems to me like this is a bug. Should I post this to > bugs.r-project.org? > > A minimal reproducible

Hi list ! We have a very busy webserver hosted in a clustered environment where the document root and data is on a GFS2 partition off a fiber-attached disk array. Now on busy moments, I can see in htop, nmon that there is a fair percentage of cpu that is waiting for I/O. In nmon, I can spot that the most busy block device correspond to our gfs2 partition where many times, it shows that

Hi everybody, with the following code I generate a list of functions. Each function reflects a "condition". When I evaluate this list of functions by another lapply/sapply, I get an unexpected result: all values coincide. However, when I uncomment the print(), it works as expected. Is this a bug or a feature? conditions <- 1:4 test <- lapply(conditions, function(mycondition){

I've been using mclapply and have encountered situations where it gives errors or returns incorrect results. Here's a minimal example, which gives the error on R 2.15.2 on Mac and Linux: library(parallel) f <- function(x) NULL mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) # Error in sum(sapply(res, inherits, "try-error")) : # invalid 'type' (list) of argument

Dear R-devel friends: I'm back to bother you again about the conflict between mclapply and tcltk. I've been monitoring several packages that want to use mclapply to parallelize computations and need to figure out what should be done. It appears tcltk cannot be safely unloaded, so the best we can do is check for the presence of tcltk and stop if it is found before mclapply() is used. I

Hi I tried to restrict usage of some running VM by cpu.shares (i.e. set to 10 from original 1024) on loaded system and it seem doesn't work as I expected... all running processes has same CPU usage (by htop) :-/ Does anyone has same experience? Fedora 19, libvirt-1.0.5.9-1.fc19.x86_64 Thanks a lot Martin -- Martin Pavlásek <mpavlase@redhat.com> OpenStack QA Associate/Red Hat

Hello, I have been having issues using parallel::mclapply in a memory-efficient way and would like some guidance. I am using a 40 core machine with 96 GB of RAM. I've tried to run mclapply with 20, 30, and 40 mc.cores and it has practically brought the machine to a standstill each time to the point where I do a hard reset. When running mclapply with 10 mc.cores, I can see that each process

Hello, I am running large simulations, which unfortunately I can't really replicate here because the code is so extensive. I rely heavily on mclapply, but I realize that I'm losing data somewhere. There are two worrisome symptoms: 1) I am getting 'NULL' as a return value for some (but not all) elements of the output when I use mclapply, but not if I use lapply > tmp2[1:3]