Displaying 20 results from an estimated 1000 matches similar to: "DOCUMENTATION(?): parallel::mcparallel() gives various types of "Error in unserialize(r) : ..." errors if value is of type raw"
2015 Mar 25
2
nested parallel workers
Hi Simon,
I'm having trouble with nested parallel workers, specifically, forking
inside socket connections.
When mclapply is called inside a SOCK, PSOCK or FORK worker I get an
error in unserialize().
cl <- makeCluster(1, "SOCK")
fun = function(i) {
library(parallel)
mclapply(1:2, sqrt)
}
Failure occurs after multiple calls to clusterApply:
> clusterApply(cl, 1,
2015 Mar 30
2
nested parallel workers
On 03/25/2015 07:48 PM, Simon Urbanek wrote:
> On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote:
>
>> Hi Simon,
>>
>> I'm having trouble with nested parallel workers, specifically, forking inside socket connections.
>>
>
> You simply can't by definition - when you fork *all* the workers share the same connection
2016 Aug 30
1
mcparallel / mccollect
Hi there,
I've tried to implement an asynchronous job scheduler using
parallel::mcparallel() and parallel::mccollect(..., wait=FALSE). My
goal was to send processes to the background, leaving the R session
open for interactive use while all jobs store their results on the
file system. To keep track of the running jobs I've stored the process
ids and written a little helper to not spawn
2019 May 03
1
Strange error messages from parallel::mcparallel family under 3.6.0
Dear All,
Since upgrading to 3.6.0, I've been getting a strange error messages
from the child process when using mcparallel/mccollect. Before filing a report in the Bugzilla, I want to figure out whether I had been doing something wrong all this time and R 3.6.0 has exposed it, or whether something else is going on.
# Background #
Ultimately, what I want to do is to be able to set a time
2015 Jun 20
2
Listing all spawned jobs/processed after parallel::mcparallel()?
QUESTION:
Is it possible to query number of active jobs running after launching
them with parallel::mcparallel()?
For example, if I launch 3 jobs using:
> library(parallel)
> f <- lapply(1:3, FUN=mcparallel)
then I can inspect them as:
> str(f)
List of 3
$ :List of 2
..$ pid: int 142225
..$ fd : int [1:2] 8 13
..- attr(*, "class")= chr [1:3] "parallelJob"
2018 Aug 31
2
Detecting whether a process exists or not by its PID?
On Fri, Aug 31, 2018 at 2:51 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:
[...]
> kill(sig=0) is specified by POSIX but indeed as you say there is a race
> condition due to PID-reuse. In principle, detecting that a worker
> process is still alive cannot be done correctly outside base R.
I am not sure why you think so.
> At user-level I would probably consider some
2013 Feb 07
1
How to NAMESPACE OS-specific importFrom?
I'd like to importFrom(parallel, mccollect, mcparallel) but on Windows these are
not exported because this
if(tools:::.OStype() == "unix") {
export(mccollect, mcparallel, mc.reset.stream, mcaffinity)
}
appears at src/library/parallel/NAMESPACE:6 of svn r61857. So should I be doing
if (tools:::.OStype() == "unix") {
importFrom(parallel, mccollect, mcparallel)
}
2019 May 03
2
mccollect with NULL in R 3.6
On Thu, May 2, 2019 at 7:24 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:
>
> On 5/1/19 12:25 AM, Gergely Dar?czi wrote:
> > Dear All,
> >
> > I'm running into issues with calling mccollect on a list containing NULL
> > using R 3.6 (this used to work in 3.5.3):
> >
> > jobs <- lapply(
> > list(NULL, 'foobar'),
>
2019 Apr 30
2
mccollect with NULL in R 3.6
Dear All,
I'm running into issues with calling mccollect on a list containing NULL
using R 3.6 (this used to work in 3.5.3):
jobs <- lapply(
list(NULL, 'foobar'),
function(x) mcparallel(identity(x)))
mccollect(jobs, wait = FALSE, timeout = 0)
#> Error in names(res) <- pnames[match(s, pids)] :
#> 'names' attribute [2] must be the same length as the vector
2014 May 21
2
issue with parallel package
Dear maintainers of the parallel package,
I ran into an issue with the parallel package in R-3.1.0.
The following code prints the message "NULL!" quite a lot.
library(parallel)
for (n in 1:1000) {
p <- mcparallel(sqrt(n))
res <- mccollect(p, wait=FALSE, timeout=1000)
mccollect(p)
if (is.null(res)) cat(n," NULL!\n")
}
It does not happen in
2018 Aug 31
1
Detecting whether a process exists or not by its PID?
On Fri, Aug 31, 2018 at 3:35 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:
>
> On 08/31/2018 03:13 PM, G?bor Cs?rdi wrote:
> > On Fri, Aug 31, 2018 at 2:51 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:
> > [...]
> >> kill(sig=0) is specified by POSIX but indeed as you say there is a race
> >> condition due to PID-reuse. In
2015 Mar 30
0
nested parallel workers
On Mar 30, 2015, at 4:40 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote:
> On 03/25/2015 07:48 PM, Simon Urbanek wrote:
>> On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote:
>>
>>> Hi Simon,
>>>
>>> I'm having trouble with nested parallel workers, specifically, forking inside socket connections.
2012 Nov 16
0
Bug in parallel / mclapply
Hi,
there seem to be some (small) bugs in the mclapply function in parallel.
I discovered this in the current R release version, and I checked that it is
still present in R-devel.
I think it only occurs in the part of the code corresponding to argument option
mc.preschedule = FALSE.
Here are two examples:
a)
library(parallel)
mclapply(list(), identity, mc.preschedule=FALSE)
Error in
2015 Mar 26
0
nested parallel workers
On Mar 25, 2015, at 3:46 PM, Valerie Obenchain <vobencha at fredhutch.org> wrote:
> Hi Simon,
>
> I'm having trouble with nested parallel workers, specifically, forking inside socket connections.
>
You simply can't by definition - when you fork *all* the workers share the same connection inherited from the parent, so you cannot use any I/O operations that you
2015 Jun 21
0
Listing all spawned jobs/processed after parallel::mcparallel()?
On 20/06/2015 22:21, Henrik Bengtsson wrote:
> QUESTION:
> Is it possible to query number of active jobs running after launching
> them with parallel::mcparallel()?
>
> For example, if I launch 3 jobs using:
>
>> library(parallel)
>> f <- lapply(1:3, FUN=mcparallel)
>
> then I can inspect them as:
>
>> str(f)
> List of 3
> $ :List of 2
>
2018 Aug 30
3
Detecting whether a process exists or not by its PID?
Hi, I'd like to test whether a (localhost) PSOCK cluster node is still
running or not by its PID, e.g. it may have crashed / core dumped.
I'm ok with getting false-positive results due to *another* process
with the same PID has since started.
I can the PID of each cluster nodes by querying them for their
Sys.getpid(), e.g.
pids <- parallel::clusterEvalQ(cl, Sys.getpid())
Is there
2011 Dec 06
1
unserialize and eager execution
Hi,
While debugging a network server I'm developing I noticed something unusual
- call to unserialize() resulted in
an error about loading a namespace.
I was a bit taken back by this - why should unserializing an object cause a
namespace lookup?
Are there any other side-effects of unserialize() that I should be cautious
about? I've been
digging through the R_Unserialize() call, I
2018 Aug 31
0
Detecting whether a process exists or not by its PID?
On 08/31/2018 03:13 PM, G?bor Cs?rdi wrote:
> On Fri, Aug 31, 2018 at 2:51 PM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:
> [...]
>> kill(sig=0) is specified by POSIX but indeed as you say there is a race
>> condition due to PID-reuse. In principle, detecting that a worker
>> process is still alive cannot be done correctly outside base R.
> I am not sure
2019 May 20
1
Race condition on parallel package's mcexit and rmChild
Have read the latest code, but I still don't understand why mc_exit
needs to write zero on exit. If a child closes its pipe, parent will
know that on next select.
Best,
Yijiang
Tomas Kalibera <tomas.kalibera at gmail.com> ?2019?5?20??? ??10:52???
>
> This issue has already been addressed in 76462 (R-devel) and also ported
> to R-patched. In fact rmChild() is used in
2020 Oct 29
2
Something is wrong with the unserialize function
Hi all,
I am not able to export an ALTREP object when `gctorture` is on in the
worker. The package simplemmap can be used to reproduce the problem. See
the example below
```
## Create a temporary file
filePath <- tempfile()
con <- file(filePath, "wrb")
writeBin(rep(0.0,10),con)
close(con)
library(simplemmap)
library(parallel)
cl <- makeCluster(1)
x <- mmap(filePath,