similar to: exiting mclapply early on error

Hello Giovanni, I don't know if my workflow would suit you but I tend to want the opposite when I launch a parallel process. I tend to want to keep the processes alive as long as they can. If the computation time is long I would not want to lose everything. lapply..8 <- function(X,FUN,...){ max..cores <- as.numeric(system("grep ^processor /proc/cpuinfo 2>/dev/null | wc

Thanks for the response, Jeremie. I wholeheartedly agree about testing. In my case, this feature would be used purely to reduce computation time. I?m calculating an expensive (and embarrassingly parallel) likelihood function, and for some parameter combinations my objective function diverges to infinity. In those instances, I?d like to throw an error that stops subsequent computations and that I

I've been using mclapply and have encountered situations where it gives errors or returns incorrect results. Here's a minimal example, which gives the error on R 2.15.2 on Mac and Linux: library(parallel) f <- function(x) NULL mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) # Error in sum(sapply(res, inherits, "try-error")) : # invalid 'type' (list) of argument

Hello, Consider this: 1) library(parallel) res <- mclapply(1:2, stop) #Warning message: #In mclapply(1:2, stop) : # all scheduled cores encountered errors in user code is(res[[1]], 'try-error') #[1] FALSE 2) library(parallel) res <- mclapply(1:2, stop, mc.preschedule=FALSE) #Warning message: #In mclapply(1:2, stop, mc.preschedule = FALSE) : # 2 function calls resulted in an

Hi, there seem to be some (small) bugs in the mclapply function in parallel. I discovered this in the current R release version, and I checked that it is still present in R-devel. I think it only occurs in the part of the code corresponding to argument option mc.preschedule = FALSE. Here are two examples: a) library(parallel) mclapply(list(), identity, mc.preschedule=FALSE) Error in

Dear members, I am using pbmcapply to parellise my code. But the following code doesn't work: > LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule = FALSE) | | 0%, ETA NA^ It just hangs. But the

On Fri, 9 Jun 2023 18:01:44 +0000 akshay kulkarni <akshay_e4 at hotmail.com> wrote: > > LYG <- pbmclapply(LYGH,FUN = arfima,mc.cores = 2,mc.preschedule = > > FALSE) > | > | > 0%, ETA NA^ > > It just hangs. My questions from the last time still stand: 0) What is your

Quick question regarding multicore versions of mapply. Package 'multicore' provides a parallelized version of 'lapply', called 'mclapply'. I haven't found any parallelized versions of 'mapply', however (although one can use the lower level function 'parallel', it becomes harder to control the number of spawned processes etc). Is anyone aware of a

Dear R-devel friends: I'm back to bother you again about the conflict between mclapply and tcltk. I've been monitoring several packages that want to use mclapply to parallelize computations and need to figure out what should be done. It appears tcltk cannot be safely unloaded, so the best we can do is check for the presence of tcltk and stop if it is found before mclapply() is used. I

Hello, I have been having issues using parallel::mclapply in a memory-efficient way and would like some guidance. I am using a 40 core machine with 96 GB of RAM. I've tried to run mclapply with 20, 30, and 40 mc.cores and it has practically brought the machine to a standstill each time to the point where I do a hard reset. When running mclapply with 10 mc.cores, I can see that each process

Dear members, I am using arfima in an mclapply construction (from the parallel package): Browse[2]> LYG <- mclapply(LYGH, FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:10], FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:2], FUN = arfima, mc.cores = detectCores()) ^C You can see that I am

Dear parallel users and developers, I might have encountered a bug in the function 'mclapply' of package 'parallel'. I construct a matrix using the same input data and code with a single difference: Once I use mclapply and the other time lapply. Shockingly the result is NOT the same. To evaluate please unpack the attached archive and execute Rscript mclapply_test.R I put the

Dear Jeff, There was a problem in LYGH and lapply threw an error, but mclapply got stuck in an infinite loop. The doc for mclapply says that mclapply runs under try() with silent = TRUE. So that means mclapply should run properly, i.e output a try class object and exit. But it didn't. Can you shed some light on why this happened? THanking you, Yours sincerely, AKSHAY M

Hello, I am running large simulations, which unfortunately I can't really replicate here because the code is so extensive. I rely heavily on mclapply, but I realize that I'm losing data somewhere. There are two worrisome symptoms: 1) I am getting 'NULL' as a return value for some (but not all) elements of the output when I use mclapply, but not if I use lapply > tmp2[1:3]

On Wed, 17 May 2023 13:55:59 +0000 akshay kulkarni <akshay_e4 at hotmail.com> wrote: > So that means mclapply should run properly, i.e output a try class > object and exit. But it didn't. Can you shed some light on why this > happened? What's your sessionInfo()? Are you using a GUI frontend? mclapply() relies on the fork() system call, which is tricky to get right in a

Dear all, I got an issue trying to parse excel files in parallel using XLConnect, the process hangs forever. Martin Studer, the maintainer of XLConnect kindly investigated the issue, identified rJava as a possible cause of the problem: This does not work (hangs): library(parallel) require(rJava) .jinit() res <- mclapply(1:2, function(i) {

Dear R-devel, I am experiencing issues with running GAM models using mclapply, it fails to return any values if the data input becomes large. For example here the code runs fine with a df of 100 rows, but fails at 1000. library(mgcv) library(parallel) > df <- data.frame( + x = 1:100, + y = 1:100 + ) > > mclapply(1:2, function(i, df) { + fit <- gam(y ~ s(x, bs =

dear R experts---I am looking at a fairly uninformative error in my program: Error in mclapply(1:nrow(opts), solveme) : (converted from warning) all scheduled cores encountered errors in user code the doc on ?mclapply tells me that In addition, each process is running the job inside try(..., silent=TRUE) so if error occur they will be stored as try-error objects in the list. I looked up

Hi Inaki, > "Performant"... in terms of what. If the cost of copying the data > predominates over the computation time, maybe you didn't need > parallelization in the first place. Performant in terms of speed. There's no copying in that example using `mclapply` and so it is significantly faster than other alternatives. It is a very simple and contrived example, but

ISSUE: Using *forks* for parallel processing in R is not always safe. The `parallel::mclapply()` function uses forked processes to parallelize. One example where it has been confirmed that forked processing causes problems is when running R via RStudio. It is recommended to use PSOCK clusters (`parallel::makeCluster()`) rather than *forked* processes when running R from RStudio (