similar to: Listing all spawned jobs/processed after parallel::mcparallel()?

On 20/06/2015 22:21, Henrik Bengtsson wrote: > QUESTION: > Is it possible to query number of active jobs running after launching > them with parallel::mcparallel()? > > For example, if I launch 3 jobs using: > >> library(parallel) >> f <- lapply(1:3, FUN=mcparallel) > > then I can inspect them as: > >> str(f) > List of 3 > $ :List of 2 >

I stumbled upon the following: f <- parallel::mcparallel(raw(0L)) parallel::mccollect(f) # $`77083` # NULL but f <- parallel::mcparallel(raw(1L)) parallel::mccollect(f) # Error in unserialize(r) : read error traceback() # 2: unserialize(r) # 1: parallel::mccollect(f) (restarting because the above appears to corrupt the R session) f <- parallel::mcparallel(raw(2L))

Hi there, I've tried to implement an asynchronous job scheduler using parallel::mcparallel() and parallel::mccollect(..., wait=FALSE). My goal was to send processes to the background, leaving the R session open for interactive use while all jobs store their results on the file system. To keep track of the running jobs I've stored the process ids and written a little helper to not spawn

Dear All, Since upgrading to 3.6.0, I've been getting a strange error messages from the child process when using mcparallel/mccollect. Before filing a report in the Bugzilla, I want to figure out whether I had been doing something wrong all this time and R 3.6.0 has exposed it, or whether something else is going on. # Background # Ultimately, what I want to do is to be able to set a time

On Fri, Aug 31, 2018 at 2:51 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote: [...] > kill(sig=0) is specified by POSIX but indeed as you say there is a race > condition due to PID-reuse. In principle, detecting that a worker > process is still alive cannot be done correctly outside base R. I am not sure why you think so. > At user-level I would probably consider some

I'd like to importFrom(parallel, mccollect, mcparallel) but on Windows these are not exported because this if(tools:::.OStype() == "unix") { export(mccollect, mcparallel, mc.reset.stream, mcaffinity) } appears at src/library/parallel/NAMESPACE:6 of svn r61857. So should I be doing if (tools:::.OStype() == "unix") { importFrom(parallel, mccollect, mcparallel) }

On Thu, May 2, 2019 at 7:24 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote: > > On 5/1/19 12:25 AM, Gergely Dar?czi wrote: > > Dear All, > > > > I'm running into issues with calling mccollect on a list containing NULL > > using R 3.6 (this used to work in 3.5.3): > > > > jobs <- lapply( > > list(NULL, 'foobar'), >

Dear All, I'm running into issues with calling mccollect on a list containing NULL using R 3.6 (this used to work in 3.5.3): jobs <- lapply( list(NULL, 'foobar'), function(x) mcparallel(identity(x))) mccollect(jobs, wait = FALSE, timeout = 0) #> Error in names(res) <- pnames[match(s, pids)] : #> 'names' attribute [2] must be the same length as the vector

On Fri, Aug 31, 2018 at 3:35 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote: > > On 08/31/2018 03:13 PM, G?bor Cs?rdi wrote: > > On Fri, Aug 31, 2018 at 2:51 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote: > > [...] > >> kill(sig=0) is specified by POSIX but indeed as you say there is a race > >> condition due to PID-reuse. In

Dear maintainers of the parallel package, I ran into an issue with the parallel package in R-3.1.0. The following code prints the message "NULL!" quite a lot. library(parallel) for (n in 1:1000) { p <- mcparallel(sqrt(n)) res <- mccollect(p, wait=FALSE, timeout=1000) mccollect(p) if (is.null(res)) cat(n," NULL!\n") } It does not happen in

On 03/25/2015 07:48 PM, Simon Urbanek wrote: > On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: > >> Hi Simon, >> >> I'm having trouble with nested parallel workers, specifically, forking inside socket connections. >> > > You simply can't by definition - when you fork *all* the workers share the same connection

Hi, I'd like to test whether a (localhost) PSOCK cluster node is still running or not by its PID, e.g. it may have crashed / core dumped. I'm ok with getting false-positive results due to *another* process with the same PID has since started. I can the PID of each cluster nodes by querying them for their Sys.getpid(), e.g. pids <- parallel::clusterEvalQ(cl, Sys.getpid()) Is there

Hi Simon, I'm having trouble with nested parallel workers, specifically, forking inside socket connections. When mclapply is called inside a SOCK, PSOCK or FORK worker I get an error in unserialize(). cl <- makeCluster(1, "SOCK") fun = function(i) { library(parallel) mclapply(1:2, sqrt) } Failure occurs after multiple calls to clusterApply: > clusterApply(cl, 1,

Hello everyone, I'm trying to parallelize an R script I have written. To do this, I am first trying to use the multicore package, because I've had some previous success with that. The function I'm trying to parallelize is illumqc. I'd like to create a separate process for each of 8 files, contained in the vector "files". Below is my code: for(i in

I think recreating tempdir() is ok in an emergency situation, but package code should not be removing tempdir() - it may contain important information. Bill Dunlap TIBCO Software wdunlap tibco.com On Wed, Nov 8, 2017 at 4:55 PM, Henrik Bengtsson <henrik.bengtsson at gmail.com > wrote: > Related to this problem - from R-devel NEWS >

Hi, I am facing a very weird problem with parallel::mclapply. I have a script which does some data wrangling on an input dataset in parallel and then writes the results to disk. I have been using this script daily for more than one year always on an EC2 instance launched from the same AMI (no updates installed after launch) and processed thousands of different input data sets successfully. I now

Hi Andreas, the error is reported when some child process cannot send results to the master process, which originates from an error returned by write() - when write() returns -1 or 0. The logic around the writing has not changed since R 3.5.2. It should not be related to the printing in the child, only to returning the value. The problem may be originating from the execution environment,

Hi Andreas, thank you very much, good job finding it was EBADF. Now the question is why the pipe has been closed prematurely; it could be accidentally by R (a race condition in the cleanup code in fork.c) or possibly by some other code running in the same process (maybe the R program itself or some other code it runs). Maybe we can take this off the list and come back when we know the cause

On 08/31/2018 03:13 PM, G?bor Cs?rdi wrote: > On Fri, Aug 31, 2018 at 2:51 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote: > [...] >> kill(sig=0) is specified by POSIX but indeed as you say there is a race >> condition due to PID-reuse. In principle, detecting that a worker >> process is still alive cannot be done correctly outside base R. > I am not sure

Dear list, I am running into 2 problems when using the optimize function from the stats package (note: I've also tried unsuccessfully to use optim, nlm, & nlminb). The second problem is caused by my solution to the first, so I am asking if anyone has a better solution to the first question, or if there exists a solution to the second problem. I should also mention that what I am