similar to: mclapply hanging occasionally on macos

? Mon, 30 Dec 2024 19:16:11 -0800 Ivo Welch <ivo.welch at gmail.com> ?????: > useless.function <- function( ) { > y <- rnorm(3); x <- rnorm(3) > summary( lm( y ~ x )) ## useless > NULL > } > > run30 <- function(i) { > message("run30=", i) > useless.function() > } > > run30( 0 ) > message("many mc")

just a plain macos desktop, latest sequoia. no clusters or anything complex. when I ^c out of it, a `traceback()` on macos tells me that ``` [highest number]: selectChildren(ac[!fin], -1) [-1]: mclapply(... ``` I don't know what selectChildren is waiting for (internal? macos?), but it certainly is patient. I wonder whether others have encounted similar problems on occasion. it's not

dear r experts---Is there a multicore equivalent of by(), just like mclapply() is the multicore equivalent of lapply()? if not, is there a fast way to convert a data.table into a list based on a column that lapply and mclapply can consume? advice appreciated...as always. regards, /iaw ---- Ivo Welch (ivo.welch at gmail.com)

The following program is whittled down from a much larger program that always works on Intel, and always works on AMD's threadripper with lapply but not mclappy. With mclapply on AMD, all processes go into "suspend" mode and the program then hangs. This bug is replicable on an AMD Ryzen Threadripper 2950X 16-Core Processor (128GB RAM), running latest ubuntu 18.04. The R version

Dear R wizards--- I have a wrapper on mclapply() that makes it a little easier for me to do multiprocessing. (Posting this may make life easier for other googlers.) I pass a data frame, a vector that tells me what rows should be recomputed, and the function; and I get back a vector or matrix of answers. d <- data.frame( id=1:6, val=11:16 ) loc <- c(TRUE,TRUE,FALSE,TRUE,FALSE,TRUE)

dear R experts---I am looking at a fairly uninformative error in my program: Error in mclapply(1:nrow(opts), solveme) : (converted from warning) all scheduled cores encountered errors in user code the doc on ?mclapply tells me that In addition, each process is running the job inside try(..., silent=TRUE) so if error occur they will be stored as try-error objects in the list. I looked up

Dear R-devel, I am experiencing issues with running GAM models using mclapply, it fails to return any values if the data input becomes large. For example here the code runs fine with a df of 100 rows, but fails at 1000. library(mgcv) library(parallel) > df <- data.frame( + x = 1:100, + y = 1:100 + ) > > mclapply(1:2, function(i, df) { + fit <- gam(y ~ s(x, bs =

Yes I am running on Rstudio 1.2.5033. I was also running this code without error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed work fine even with large data.frames. Any idea as to what interaction between Rstudio and mclapply causes this? Thanks, Shian On 28 Apr 2020, at 7:29 pm, Simon Urbanek <simon.urbanek at R-project.org<mailto:simon.urbanek at

Thanks Henrik, That clears things up significantly. I did see the warning but failed to include it my initial email. It sounds like an RStudio issue, and it seems like that it?s quite intrinsic to how forks interact with RStudio. Given this code is eventually going to be a part of a package, should I expect it to fail mysteriously in RStudio for my users? Is the best solution here to migrate all

Dear R experts--- Has anyone written parallel versions of "by" (i.e., mcby) and "ave" (i.e. mcave) ? I did ask a question like this a year ago, and then the answer was no. for those who are googling the group for the answer to this question, in the meantime, the poor man's version of "by" is mclapply( split( ds, factor ), FUN ) I don't know the poor

dear R experts: ?apologies for all my speed and memory questions. ?I have a bet with my coauthors that I can make R reasonably efficient through R-appropriate programming techniques. this is not just for kicks, but for work. for benchmarking, my [3 year old] Mac Pro has 2.8GHz Xeons, 16GB of RAM, and R 2.13.1. right now, it seems that 'split()' is why I am losing my bet. ?(split is an

Thanks Simon, I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don?t see a reason to not make use of it. The goal for the current piece of code I?m working on is to bootstrap many

I have found more general questions, but I have a specific one. I have a few million (independent) short regressions that I would like to run (each reg has about 60 observations, though they can have missing observations [yikes]). So, I would like to be running as many `lm` and `coef(lm)` in parallel as possible. my hardware is Mac, with nice GPUs and integrated memory --- and so far completely

Dear R experts: could someone please point me to a page that explains how to set up more than 1 machine for library parallel (which is quickly becoming my favorite!) my dream setup would be a design where I just pass a list of hostnames:user:password to my parallel master, and then start R listener processes on each of my slaves by hand. R would start slave processes automatically on each slave

I've been using mclapply and have encountered situations where it gives errors or returns incorrect results. Here's a minimal example, which gives the error on R 2.15.2 on Mac and Linux: library(parallel) f <- function(x) NULL mclapply(1, f, mc.preschedule = FALSE, mc.cores = 1) # Error in sum(sapply(res, inherits, "try-error")) : # invalid 'type' (list) of argument

Dear members, I am using arfima in an mclapply construction (from the parallel package): Browse[2]> LYG <- mclapply(LYGH, FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:10], FUN = arfima, mc.cores = detectCores()) ^C Browse[2]> LYG <- mclapply(LYGH[1:2], FUN = arfima, mc.cores = detectCores()) ^C You can see that I am

Hello, I have been having issues using parallel::mclapply in a memory-efficient way and would like some guidance. I am using a 40 core machine with 96 GB of RAM. I've tried to run mclapply with 20, 30, and 40 mc.cores and it has practically brought the machine to a standstill each time to the point where I do a hard reset. When running mclapply with 10 mc.cores, I can see that each process

Dear parallel users and developers, I might have encountered a bug in the function 'mclapply' of package 'parallel'. I construct a matrix using the same input data and code with a single difference: Once I use mclapply and the other time lapply. Shockingly the result is NOT the same. To evaluate please unpack the attached archive and execute Rscript mclapply_test.R I put the

Dear R developers: ... 7: lapply(seq_len(cores), inner.do) 8: FUN(1:3[[3]], ...) 9: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) Selection: .....................Error in sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) : long vectors not supported yet: memory.c:3100 admittedly, my outcome will be a very big list, with 30,000 elements, each containing data frames

Dear R-devel friends: I'm back to bother you again about the conflict between mclapply and tcltk. I've been monitoring several packages that want to use mclapply to parallelize computations and need to figure out what should be done. It appears tcltk cannot be safely unloaded, so the best we can do is check for the presence of tcltk and stop if it is found before mclapply() is used. I