On May 21, 2014, at 1:17 PM, Josef Leydold <josef.leydold at wu.ac.at>
wrote:
> Dear maintainers of the parallel package,
>
> I ran into an issue with the parallel package in R-3.1.0.
>
> The following code prints the message "NULL!" quite a lot.
>
> library(parallel)
> for (n in 1:1000) {
> p <- mcparallel(sqrt(n))
> res <- mccollect(p, wait=FALSE, timeout=1000)
> mccollect(p)
> if (is.null(res)) cat(n," NULL!\n")
> }
>
>
> It does not happen in R-3.0.3.
> (To be exact, I have checked out R-devel revision r65130
> for my current tests.)
>
> I found that this behavior is introduced in revision r65143.
>
> When I insert
> Sys.sleep(0.1)
> in the loop then this behavior disappears in R-3.1.0.
>
> There is also another difference between R-3.1.0 and R-3.0.3:
> When the line
> mccollect(p)
> is replaced by
> mccollect(p, wait=FALSE, timeout=1000)
> then in R-3.1.0 some child processes are not terminated and some
> pipes remain open. This does not happen in R-3.0.3.
>
> Remark:
> If the line
> mccollect(p)
> is omitted at all then "NULL!" is never printed but
> R aborts after 510 iterations of the loop with the message
> "Error in mcfork(detached) : unable to create a pipe".
> Then none of the child processes is terminated and
> number of possible open pipes is exhausted.
>
> Is this behavior intended or a bug?
>
As noted in the follow-up this is absolutely expected. wait=FALSE merely checks
if the result is available - if it's not at that point then NULL is returned
- that's what you see and is exactly what one would expect. Analogously, if
you don't collect the results then the child processes will be still hanging
around waiting for you to collect the results and therefore you'll run out
of descriptors. The fact that it worked in 3.0.3 is just a coincidence that it
used more time managing the children and checking their status before you got to
your attempted collect so the result was more likely to be already available.
Because 3.1.0 is more efficient (it no longer manages children manually but
rather uses signal-based cleanup), it comes faster to the collect point and the
results may not be available yet.
Cheers,
Simon
>
>> version
> _
> platform x86_64-pc-linux-gnu
> arch x86_64
> os linux-gnu
> system x86_64, linux-gnu
> status
> major 3
> minor 1.0
> year 2014
> month 04
> day 10
> svn rev 65387
> language R
> version.string R version 3.1.0 (2014-04-10)
> nickname Spring Dance
>
>
> best whishes
> Josef
>
> --
>
>
>
-----------------------------------------------------------------------------
>
> Josef Leydold
>
> Institute for Statistics and Mathematics
> WU (Vienna University of Economics and Business)
>
> Welthandelsplatz 1, Building D4, 1020 Vienna, Austria
>
> Phone +43 1 31336 4695 FAX +43 1 31336 904695 email josef.leydold at
wu.ac.at
>
>
-----------------------------------------------------------------------------
> Alles Unglueck kam daher, dass die Denkenden nicht mehr handeln konnten,
> und die Handelnden keine Zeit mehr fanden zu denken. (Marlen
Haushofer)
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>