search for: mcmappli

Hello, I have a minimal example that shows a problem I'm having with parallel processing. library(parallel) mcmapply(function(x, y) { print("Running") Sys.sleep(10) }, as.list(1:10), as.list(10:1), mc.cores = 16, SIMPLIFY = FALSE) I see "Running" printed once every ten seconds. I read the documentation for mcmapply, but I don't understand why it wouldn't

Quick question regarding multicore versions of mapply. Package 'multicore' provides a parallelized version of 'lapply', called 'mclapply'. I haven't found any parallelized versions of 'mapply', however (although one can use the lower level function 'parallel', it becomes harder to control the number of spawned processes etc). Is anyone aware of a

Hola erreros Estoy rizando el rizo con esto de las matrices sparse y he encontrado la forma de hacerlo en python, que va rapidísimo. Por tanto tengo una matriz sparse en un objeto python con esta info: <138493x26744 sparse matrix of type '<type 'numpy.int64'> Quiero hacer lo mismo con R pero me he topado con la lentitud de los bucles para rellenar la sparse (también por mi

I'd like to pick up this thread started on 2019-04-11 (https://hypatia.math.ethz.ch/pipermail/r-devel/2019-April/077632.html). Modulo all the other suggestions in this thread, would my proposal of being able to disable forked processing via an option or an environment variable make sense? I've prototyped a working patch that works like: > options(fork.allowed = FALSE) >

If I understand the thread correctly this is an RStudio issue and I would suggest that the developers consider using pthread_atfork() so RStudio can handle forking as they deem fit (bail out with an error or make RStudio work). Note that in principle the functionality requested here can be easily implemented in a package so R doesn?t need to be modified. Cheers, Simon Sent from my iPhone

On 1/10/20 7:33 AM, Henrik Bengtsson wrote: > I'd like to pick up this thread started on 2019-04-11 > (https://hypatia.math.ethz.ch/pipermail/r-devel/2019-April/077632.html). > Modulo all the other suggestions in this thread, would my proposal of > being able to disable forked processing via an option or an > environment variable make sense? I don't think R should be doing

Henrik, the example from the post works just fine in CRAN R for me - the post was about homebrew build so it's conceivably a bug in their libraries. That's exactly why I was proposing a more general solution where you can simply define a function in user-space that will issue a warning or stop on fork, it doesn't have to be part of core R, there are other packages that use fork() as

On Mon, 15 Apr 2019 at 08:44, Tomas Kalibera <tomas.kalibera at gmail.com> wrote: > > On 4/13/19 12:05 PM, I?aki Ucar wrote: > > On Sat, 13 Apr 2019 at 03:51, Kevin Ushey <kevinushey at gmail.com> wrote: > >> I think it's worth saying that mclapply() works as documented > > Mostly, yes. But it says nothing about fork's copy-on-write and memory >

> On Jan 10, 2020, at 3:10 PM, G?bor Cs?rdi <csardi.gabor at gmail.com> wrote: > > On Fri, Jan 10, 2020 at 7:23 PM Simon Urbanek > <simon.urbanek at r-project.org> wrote: >> >> Henrik, >> >> the example from the post works just fine in CRAN R for me - the post was about homebrew build so it's conceivably a bug in their libraries. > > I

The RStudio GUI was just one example. AFAIK, and please correct me if I'm wrong, another example is where multi-threaded code is used in forked processing and that's sometimes unstable. Yes another, which might be multi-thread related or not, is https://stat.ethz.ch/pipermail/r-devel/2018-September/076845.html: res <- parallel::mclapply(urls, function(url) { download.file(url,

On Fri, Jan 10, 2020 at 7:23 PM Simon Urbanek <simon.urbanek at r-project.org> wrote: > > Henrik, > > the example from the post works just fine in CRAN R for me - the post was about homebrew build so it's conceivably a bug in their libraries. I think it works now, because Apple switched to a different SSL library for libcurl. It usually crashes or fails on older macOS

Henrik, the whole point and only purpose of mc* functions is to fork. That's what the multicore package was about, so if you don't want to fork, don't use mc* functions - they don't have any other purpose. I really fail to see the point - if you use mc* functions you're very explicitly asking for forking - so your argument is like saying that print() should have an option to

On Fri, Jan 10, 2020 at 11:23 AM Simon Urbanek <simon.urbanek at r-project.org> wrote: > > Henrik, > > the example from the post works just fine in CRAN R for me - the post was about homebrew build so it's conceivably a bug in their libraries. Thanks for ruling that example out. > That's exactly why I was proposing a more general solution where you can simply define

In pvec(list(1, 2), FUN, mc.cores=2) FUN sees integer() arguments whereas pvec(list(1, 2, 3), FUN, mc.cores=2) FUN sees list() arguments; the latter seems consistent with pvec's description. This came up in a complicated Bioconductor thread about generics and parallel evaluation https://stat.ethz.ch/pipermail/bioc-devel/2012-October/003745.html One relevant point is that a

> On Jan 10, 2020, at 3:10 PM, G?bor Cs?rdi <csardi.gabor at gmail.com> wrote: > > On Fri, Jan 10, 2020 at 7:23 PM Simon Urbanek > <simon.urbanek at r-project.org> wrote: >> >> Henrik, >> >> the example from the post works just fine in CRAN R for me - the post was about homebrew build so it's conceivably a bug in their libraries. > > I

On Fri, Jan 10, 2020 at 7:23 PM Simon Urbanek <simon.urbanek at r-project.org> wrote: > > Henrik, > > the whole point and only purpose of mc* functions is to fork. That's what the multicore package was about, so if you don't want to fork, don't use mc* functions - they don't have any other purpose. But, with that same argument I'm surprised we have fake

The build system rolled up R-2.15.0.tar.gz (codename "Easter Beagle") at 9:00 this morning. This is the first release of the 2.15 series and contains several new features and changes; see the list below for details. You can get the source code from http://cran.r-project.org/src/base/R-2/R-2.15.0.tar.gz or wait for it to be mirrored at a CRAN site nearer to you. Binaries for various

The build system rolled up R-2.15.0.tar.gz (codename "Easter Beagle") at 9:00 this morning. This is the first release of the 2.15 series and contains several new features and changes; see the list below for details. You can get the source code from http://cran.r-project.org/src/base/R-2/R-2.15.0.tar.gz or wait for it to be mirrored at a CRAN site nearer to you. Binaries for various

The build system rolled up R-4.0.0.tar.gz (codename "Arbor Day") this morning. The list below details the changes in this release. You can get the source code from http://cran.r-project.org/src/base/R-4/R-4.0.0.tar.gz or wait for it to be mirrored at a CRAN site nearer to you. Binaries for various platforms will appear in due course. For the R Core Team, Peter Dalgaard These are

The build system rolled up R-4.0.0.tar.gz (codename "Arbor Day") this morning. The list below details the changes in this release. You can get the source code from http://cran.r-project.org/src/base/R-4/R-4.0.0.tar.gz or wait for it to be mirrored at a CRAN site nearer to you. Binaries for various platforms will appear in due course. For the R Core Team, Peter Dalgaard These are