similar to: Random behavior of mclapply

Hi Thibault, mclapply has been designed to signal an error in two ways. User code errors are returned as special objects (of class "try-error") in the respective element of the result list. All other errors (including a process killed) are returned as NULL in the respective elements of the result list. To detect these errors reliably, one needs to implement FUN so that it never

Hi, I am still new to R and this is my first post on this mailing-list. I have two .csv (each one being a column of real numbers) coming from the same database (the first one is just longer than the second) and I read them in R the following way: returns <- read.csv("test.csv", header = FALSE) returns2 <- read.csv("test2.csv", header = FALSE) However, the two

I have an lapply function call that I want to parallelize. Below is a very simplified version of the code: url_base <- "https://cloud.r-project.org/src/contrib/" files <- c("A3_1.0.0.tar.gz", "ABC.RAP_0.9.0.tar.gz") res <- parallel::mclapply(files, function(s) download.file(paste0(url_base, s), s)) Instead of download a couple of files in parallel, I get a

I've been using mclapply and have encountered situations where it gives errors or returns incorrect results. Here's a minimal example, which gives the error on R 2.15.2 on Mac and Linux: library(parallel) f <- function(x) NULL mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) # Error in sum(sapply(res, inherits, "try-error")) : # invalid 'type' (list) of argument

Dear members, I am using arfima in an mclapply construction (from the parallel package): Browse[2]> LYG <- mclapply(LYGH, FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:10], FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:2], FUN = arfima, mc.cores = detectCores()) ^C You can see that I am

Hello, I have been having issues using parallel::mclapply in a memory-efficient way and would like some guidance. I am using a 40 core machine with 96 GB of RAM. I've tried to run mclapply with 20, 30, and 40 mc.cores and it has practically brought the machine to a standstill each time to the point where I do a hard reset. When running mclapply with 10 mc.cores, I can see that each process

Dear parallel users and developers, I might have encountered a bug in the function 'mclapply' of package 'parallel'. I construct a matrix using the same input data and code with a single difference: Once I use mclapply and the other time lapply. Shockingly the result is NOT the same. To evaluate please unpack the attached archive and execute Rscript mclapply_test.R I put the

Dear R-devel friends: I'm back to bother you again about the conflict between mclapply and tcltk. I've been monitoring several packages that want to use mclapply to parallelize computations and need to figure out what should be done. It appears tcltk cannot be safely unloaded, so the best we can do is check for the presence of tcltk and stop if it is found before mclapply() is used. I

On Sat, 13 Apr 2019 at 03:51, Kevin Ushey <kevinushey at gmail.com> wrote: > > I think it's worth saying that mclapply() works as documented Mostly, yes. But it says nothing about fork's copy-on-write and memory overcommitment, and that this means that it may work nicely or fail spectacularly depending on whether, e.g., you operate on a long vector. -- I?aki ?car

Just throwing my two cents in: I think removing/deprecating fork would be a bad idea for two reasons: 1) There are no performant alternatives 2) Removing fork would break existing workflows Even if replaced with something using the same interface (e.g., a function that automatically detects variables to export as in the amazing `future` package), the lack of copy-on-write functionality would

Hello, I am running large simulations, which unfortunately I can't really replicate here because the code is so extensive. I rely heavily on mclapply, but I realize that I'm losing data somewhere. There are two worrisome symptoms: 1) I am getting 'NULL' as a return value for some (but not all) elements of the output when I use mclapply, but not if I use lapply > tmp2[1:3]

dear r experts---Is there a multicore equivalent of by(), just like mclapply() is the multicore equivalent of lapply()? if not, is there a fast way to convert a data.table into a list based on a column that lapply and mclapply can consume? advice appreciated...as always. regards, /iaw ---- Ivo Welch (ivo.welch at gmail.com)

Dear all, I got an issue trying to parse excel files in parallel using XLConnect, the process hangs forever. Martin Studer, the maintainer of XLConnect kindly investigated the issue, identified rJava as a possible cause of the problem: This does not work (hangs): library(parallel) require(rJava) .jinit() res <- mclapply(1:2, function(i) {

On Wed, 17 May 2023 13:55:59 +0000 akshay kulkarni <akshay_e4 at hotmail.com> wrote: > So that means mclapply should run properly, i.e output a try class > object and exit. But it didn't. Can you shed some light on why this > happened? What's your sessionInfo()? Are you using a GUI frontend? mclapply() relies on the fork() system call, which is tricky to get right in a

Dear R-devel, I am experiencing issues with running GAM models using mclapply, it fails to return any values if the data input becomes large. For example here the code runs fine with a df of 100 rows, but fails at 1000. library(mgcv) library(parallel) > df <- data.frame( + x = 1:100, + y = 1:100 + ) > > mclapply(1:2, function(i, df) { + fit <- gam(y ~ s(x, bs =

Hello, Consider this: 1) library(parallel) res <- mclapply(1:2, stop) #Warning message: #In mclapply(1:2, stop) : # all scheduled cores encountered errors in user code is(res[[1]], 'try-error') #[1] FALSE 2) library(parallel) res <- mclapply(1:2, stop, mc.preschedule=FALSE) #Warning message: #In mclapply(1:2, stop, mc.preschedule = FALSE) : # 2 function calls resulted in an

Yes I am running on Rstudio 1.2.5033. I was also running this code without error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed work fine even with large data.frames. Any idea as to what interaction between Rstudio and mclapply causes this? Thanks, Shian On 28 Apr 2020, at 7:29 pm, Simon Urbanek <simon.urbanek at R-project.org<mailto:simon.urbanek at

Dear all. I am using the mclapply function to split my code to the many cores my system has. It seems that is working fine. This is the parallel version of lcapply. The only problem that I seem to have is that the printf cannot print messages. The ideal to me is to have fro my function an output of the form Shadowlist<-mclapply(1:dimz, function(i) { print(sprintf('Creating the

Dear Jeff, There was a problem in LYGH and lapply threw an error, but mclapply got stuck in an infinite loop. The doc for mclapply says that mclapply runs under try() with silent = TRUE. So that means mclapply should run properly, i.e output a try class object and exit. But it didn't. Can you shed some light on why this happened? THanking you, Yours sincerely, AKSHAY M

Dear R wizards--- I have a wrapper on mclapply() that makes it a little easier for me to do multiprocessing. (Posting this may make life easier for other googlers.) I pass a data frame, a vector that tells me what rows should be recomputed, and the function; and I get back a vector or matrix of answers. d <- data.frame( id=1:6, val=11:16 ) loc <- c(TRUE,TRUE,FALSE,TRUE,FALSE,TRUE)