similar to: parallel::mclapply does not return try-error objects with mc.preschedule=TRUE

I've been using mclapply and have encountered situations where it gives errors or returns incorrect results. Here's a minimal example, which gives the error on R 2.15.2 on Mac and Linux: library(parallel) f <- function(x) NULL mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) # Error in sum(sapply(res, inherits, "try-error")) : # invalid 'type' (list) of argument

Hello, I spent a lot of a time on a weird bug, and I just managed to narrow it down. In parallel code (here with parallel::mclappy, but I got it doMC/multicore too), if the library(tcltk) is loaded, R hangs when trying to open a DB connection. I got the same behaviour on two different computers, one dual-core, and one 2 xeon quad-core. Here's the code: library(parallel) library(RSQLite)

Dear all, I got an issue trying to parse excel files in parallel using XLConnect, the process hangs forever. Martin Studer, the maintainer of XLConnect kindly investigated the issue, identified rJava as a possible cause of the problem: This does not work (hangs): library(parallel) require(rJava) .jinit() res <- mclapply(1:2, function(i) {

Dear members, I am using pbmcapply to parellise my code. But the following code doesn't work: > LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule = FALSE) | | 0%, ETA NA^ It just hangs. But the

Hi, there seem to be some (small) bugs in the mclapply function in parallel. I discovered this in the current R release version, and I checked that it is still present in R-devel. I think it only occurs in the part of the code corresponding to argument option mc.preschedule = FALSE. Here are two examples: a) library(parallel) mclapply(list(), identity, mc.preschedule=FALSE) Error in

On Fri, 9 Jun 2023 18:01:44 +0000 akshay kulkarni <akshay_e4 at hotmail.com> wrote: > > LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule = > > FALSE) > | > | > 0%, ETA NA^ > > It just hangs. My questions from the last time still stand: 0) What is your

Hey folks, Is there any way to exit an mclapply early on error? For example, in the following mclapply loop, I have to wait for all the processes to finish before the error is returned. ``` mclapply(X = 1:12, FUN = function(x) {Sys.sleep(0.1); if(x == 4) stop()}, mc.cores = 4, mc.preschedule = F) ``` When there are many calculations in FUN, it takes a long time before the error is returned.

Hello, I have been having issues using parallel::mclapply in a memory-efficient way and would like some guidance. I am using a 40 core machine with 96 GB of RAM. I've tried to run mclapply with 20, 30, and 40 mc.cores and it has practically brought the machine to a standstill each time to the point where I do a hard reset. When running mclapply with 10 mc.cores, I can see that each process

Dear parallel users and developers, I might have encountered a bug in the function 'mclapply' of package 'parallel'. I construct a matrix using the same input data and code with a single difference: Once I use mclapply and the other time lapply. Shockingly the result is NOT the same. To evaluate please unpack the attached archive and execute Rscript mclapply_test.R I put the

dear R experts---I am looking at a fairly uninformative error in my program: Error in mclapply(1:nrow(opts), solveme) : (converted from warning) all scheduled cores encountered errors in user code the doc on ?mclapply tells me that In addition, each process is running the job inside try(..., silent=TRUE) so if error occur they will be stored as try-error objects in the list. I looked up

ISSUE: Using *forks* for parallel processing in R is not always safe. The `parallel::mclapply()` function uses forked processes to parallelize. One example where it has been confirmed that forked processing causes problems is when running R via RStudio. It is recommended to use PSOCK clusters (`parallel::makeCluster()`) rather than *forked* processes when running R from RStudio (

Just throwing my two cents in: I think removing/deprecating fork would be a bad idea for two reasons: 1) There are no performant alternatives 2) Removing fork would break existing workflows Even if replaced with something using the same interface (e.g., a function that automatically detects variables to export as in the amazing `future` package), the lack of copy-on-write functionality would

Hi Inaki, > "Performant"... in terms of what. If the cost of copying the data > predominates over the computation time, maybe you didn't need > parallelization in the first place. Performant in terms of speed. There's no copying in that example using `mclapply` and so it is significantly faster than other alternatives. It is a very simple and contrived example, but

On Sat, 13 Apr 2019 at 18:41, Simon Urbanek <simon.urbanek at r-project.org> wrote: > > Sure, but that a completely bogus argument because in that case it would fail even more spectacularly with any other method like PSOCK because you would *have to* allocate n times as much memory so unlike mclapply it is guaranteed to fail. With mclapply it is simply much more efficient as it will

On Sat, 13 Apr 2019 at 03:51, Kevin Ushey <kevinushey at gmail.com> wrote: > > I think it's worth saying that mclapply() works as documented Mostly, yes. But it says nothing about fork's copy-on-write and memory overcommitment, and that this means that it may work nicely or fail spectacularly depending on whether, e.g., you operate on a long vector. -- I?aki ?car

Quick question regarding multicore versions of mapply. Package 'multicore' provides a parallelized version of 'lapply', called 'mclapply'. I haven't found any parallelized versions of 'mapply', however (although one can use the lower level function 'parallel', it becomes harder to control the number of spawned processes etc). Is anyone aware of a

I'd like to pick up this thread started on 2019-04-11 (https://hypatia.math.ethz.ch/pipermail/r-devel/2019-April/077632.html). Modulo all the other suggestions in this thread, would my proposal of being able to disable forked processing via an option or an environment variable make sense? I've prototyped a working patch that works like: > options(fork.allowed = FALSE) >

On Mon, 15 Apr 2019 at 08:44, Tomas Kalibera <tomas.kalibera at gmail.com> wrote: > > On 4/13/19 12:05 PM, I?aki Ucar wrote: > > On Sat, 13 Apr 2019 at 03:51, Kevin Ushey <kevinushey at gmail.com> wrote: > >> I think it's worth saying that mclapply() works as documented > > Mostly, yes. But it says nothing about fork's copy-on-write and memory >

> On Jan 10, 2020, at 3:10 PM, G?bor Cs?rdi <csardi.gabor at gmail.com> wrote: > > On Fri, Jan 10, 2020 at 7:23 PM Simon Urbanek > <simon.urbanek at r-project.org> wrote: >> >> Henrik, >> >> the example from the post works just fine in CRAN R for me - the post was about homebrew build so it's conceivably a bug in their libraries. > > I

Hi all, Would it be possible to have the new 'parallel' library export a dummy function, something akin to if(Windows) mclapply <- lapply to paper over the lack of fork() support on said platform? This may not be the world's greatest idea, but it would make it easier for me to maintain my package and still offer most users good parallel support. Plus, I can't really see