similar to: optimize and mcparallel problem

Dear List, After running a compound symmetric model using gls, I realized that the default contrasts were not the ones that made the most sense given the biological relationships among the factor levels. When I either changed the factor levels to re-arrange the order they occur in the gls model (not shown below) OR specifically change the contrasts I get the exact same estimates for the

Hello everyone, I'm trying to parallelize an R script I have written. To do this, I am first trying to use the multicore package, because I've had some previous success with that. The function I'm trying to parallelize is illumqc. I'd like to create a separate process for each of 8 files, contained in the vector "files". Below is my code: for(i in

I'm trying to understand how to use the multicore package. In particular, I'm trying to work out what is covered where it says this in the help file for the parallel function: expr: expression to evaluate (do _not_ use any on-screen devices or GUI elements in this code) Can a function that produces an EPS file be considered an "expression" in this case? I've

I stumbled upon the following: f <- parallel::mcparallel(raw(0L)) parallel::mccollect(f) # $`77083` # NULL but f <- parallel::mcparallel(raw(1L)) parallel::mccollect(f) # Error in unserialize(r) : read error traceback() # 2: unserialize(r) # 1: parallel::mccollect(f) (restarting because the above appears to corrupt the R session) f <- parallel::mcparallel(raw(2L))

On 20/06/2015 22:21, Henrik Bengtsson wrote: > QUESTION: > Is it possible to query number of active jobs running after launching > them with parallel::mcparallel()? > > For example, if I launch 3 jobs using: > >> library(parallel) >> f <- lapply(1:3, FUN=mcparallel) > > then I can inspect them as: > >> str(f) > List of 3 > $ :List of 2 >

QUESTION: Is it possible to query number of active jobs running after launching them with parallel::mcparallel()? For example, if I launch 3 jobs using: > library(parallel) > f <- lapply(1:3, FUN=mcparallel) then I can inspect them as: > str(f) List of 3 $ :List of 2 ..$ pid: int 142225 ..$ fd : int [1:2] 8 13 ..- attr(*, "class")= chr [1:3] "parallelJob"

Hi there, I've tried to implement an asynchronous job scheduler using parallel::mcparallel() and parallel::mccollect(..., wait=FALSE). My goal was to send processes to the background, leaving the R session open for interactive use while all jobs store their results on the file system. To keep track of the running jobs I've stored the process ids and written a little helper to not spawn

Dear All, Since upgrading to 3.6.0, I've been getting a strange error messages from the child process when using mcparallel/mccollect. Before filing a report in the Bugzilla, I want to figure out whether I had been doing something wrong all this time and R 3.6.0 has exposed it, or whether something else is going on. # Background # Ultimately, what I want to do is to be able to set a time

On Tue, Apr 28, 2020 at 9:00 PM Shian Su <su.s at wehi.edu.au> wrote: > > Thanks Simon, > > I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don?t see a reason to not make

On Mar 30, 2015, at 4:40 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: > On 03/25/2015 07:48 PM, Simon Urbanek wrote: >> On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: >> >>> Hi Simon, >>> >>> I'm having trouble with nested parallel workers, specifically, forking inside socket connections.

On Fri, Jan 10, 2020 at 7:23 PM Simon Urbanek <simon.urbanek at r-project.org> wrote: > > Henrik, > > the whole point and only purpose of mc* functions is to fork. That's what the multicore package was about, so if you don't want to fork, don't use mc* functions - they don't have any other purpose. But, with that same argument I'm surprised we have fake

On 03/25/2015 07:48 PM, Simon Urbanek wrote: > On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote: > >> Hi Simon, >> >> I'm having trouble with nested parallel workers, specifically, forking inside socket connections. >> > > You simply can't by definition - when you fork *all* the workers share the same connection

Do NOT use mcparallel() in packages except as a non-default option that user can set for the reasons Henrik explained. Multicore is intended for HPC applications that need to use many cores for computing-heavy jobs, but it does not play well with RStudio and more importantly you don't know the resource available so only the user can tell you when it's safe to use. Multi-core machines are

Full_Name: Jeremy Fox Version: 1.5.0 OS: Solaris 8, 64 bit Submission from: (NULL) (171.64.232.63) I am compiling R under Solaris and the compilation stops at this point. The problem only happens when I link in readline. The R binary is actually compiled and readlne works. The main problem with the compiled binary (other than that all the supporting files were not correctly installed) is that

Henrik, the whole point and only purpose of mc* functions is to fork. That's what the multicore package was about, so if you don't want to fork, don't use mc* functions - they don't have any other purpose. I really fail to see the point - if you use mc* functions you're very explicitly asking for forking - so your argument is like saying that print() should have an option to

Thanks Simon, I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don?t see a reason to not make use of it. The goal for the current piece of code I?m working on is to bootstrap many

Hi again, One important correction of my first message: I misinterpreted the output. Actually in that R session 2 input files were processed one after the other in a loop. The first (with 88 parts went fine). The second (with 85 parts) produced the sendMaster errors and failed. If (in a new session via Rscript) I only process the second input file it will work. The other observations on R vs

Hi Tomas, Thanks for your prompt reply and your offer to help. I might need to get back to this since I am not too experienced in debugging these kinds of issues. Anyway, I gave it a try and I think I have found the immediate cause: I installed the debug symbols (r-base-core-dbg), placed https://github.com/wch/r-source/blob/tags/R-3-5-2/src/library/parallel/src/fork.c in cwd and changed the

Hi all, With the help of Tomas, I was able to track the issue down: Prior to R v3.6.0 the parallel package passes an uninitialized variable as the file descriptor argument to the close system call. In my particular R session this uninitialized variable (reproducibly) was holding the value 7, which corresponded to the file descriptor of the write end of the pipe the second child would use to

Hi all, With the help of Tomas, I was able to track the issue down: Prior to R v3.6.0 the parallel package passes an uninitialized variable as the file descriptor argument to the close system call. In my particular R session this uninitialized variable (reproducibly) was holding the value 7, which corresponded to the file descriptor of the write end of the pipe the second child would use to