R devel - May 2017 - R 3.4 and mclapply assertion failure - is this a bug?

That's great news that OpenMP is now enabled in MacOS CRAN binaries! Thanks
Simon.

data.table is supposed to automatically switch down to single-threaded mode
when explicitly parallelized via the fork mechanism.   It does that via
functions registered in init.c as follows :

    pthread_atfork(&when_fork, &when_fork_end, NULL)

Tests 1705 and 1706 test this is working.   Those tests are passing ok on
CRAN currently including x86_64-apple-darwin15.6.0.

I'll work with Yan off list to see what the difference is.  Either the fork
catch isn't working in that environment for some reason, the use case is
different to the tests somehow or the fork catch is working but something
somewhere is asserting its surprise at being single threaded.  In the
meantime setDTthreads(1) can be called manually before the explicit
parallelism as a workaround.

Matt

On Tue, May 2, 2017 at 1:25 PM, Yan Alperovych <y_alperovych at yahoo.fr>
wrote:
> Dear Matt,
>
> I follow the Simon?s advice on this. Since R 3.4 the mclapply is producing
> a strange error which was not present in R 3.3.3. Simon?s answer is below
> and the relevant info (the console output, the code, and the sample data)
> are all below.
>
> Thank you very much for helping,
> Yan
>
> Begin forwarded message:
>
> *From: *Simon Urbanek <simon.urbanek at r-project.org>
> *Subject: **Re: [Rd] R 3.4 and mclapply assertion failure - is this a
> bug?*
> *Date: *May 2, 2017 at 7:09:36 PM GMT+2
> *To: *Yan Alperovych <y_alperovych at yahoo.fr>
> *Cc: *"r-devel at r-project.org" <r-devel at r-project.org>
>
> Yan,
>
> this looks like an interaction of OpenMP in data.table and forking - you
> may want to report that to Matt as he can shed more light on that. He has
> been complaining that we didn't enable OpenMP before 3.4 so he probably
> knows about the issue and how to fix it.
>
> Cheers,
> Simon
>
>
> On May 2, 2017, at 7:12 AM, Yan Alperovych via R-devel <
> r-devel at r-project.org> wrote:
>
> Dear all,
>
>
> I am not sure if this is a bug, so I prefer to post it here before filing.
>
>
> After upgrading to 3.4 I encounter the following message with mclapply:
>
>
> Assertion failure at kmp_runtime.cpp(6480): __kmp_thread_pool =>
__null.OMP : Error #13: Assertion failure at kmp_runtime.cpp(6480).OMP:
> Hint: Please submit a bug report with this message, compile and run
> commands used, and machine configuration info including native compiler and
> operating system versions. Faster response will be obtained by including
> all program sources. For information on submitting this issue, please see
> http://www.intel.com/software/products/support/.
>
>
> The complete pdf of what console produces is here:
>
> https://www.dropbox.com/s/navugg28gqne14p/R%20Console.pdf?dl=0
>
>
> The batch with commands to (hopefully) reproduce it is here:
>
> https://www.dropbox.com/s/8pd0ij3kowigdlc/20170430%20-%
> 20R%20mclapply%20bug%20report.r?dl=0
>
>
> The data for the reproducible example is here:
> https://www.dropbox.com/s/2enoeapu7jgcxwd/z.Rdata?dl=0
>
>
> Note that this behavior was not present in R 3.3.3 on the same machine and
> all the batch was working without any errors.
>
>
> The sessionInfo():
> R version 3.4.0 (2017-04-21)
>
> Platform: x86_64-apple-darwin15.6.0 (64-bit)
>
> Running under: macOS Sierra 10.12.4
>
>
> Matrix products: default
>
>
> BLAS: /Library/Frameworks/R.framework/Versions/3.4/
> Resources/lib/libRblas.0.dylib
>
> LAPACK: /Library/Frameworks/R.framework/Versions/3.4/
> Resources/lib/libRlapack.dylib
>
>
> *locale:*
> *[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8*
>
> *attached base packages:*
> *[1] parallel  compiler  stats     graphics  grDevices utils     datasets
>  methods   base     *
>
> *other attached packages:*
> * [1] data.table_1.10.4      numbers_0.6-6          microbenchmark_1.4-2.1
> zoo_1.8-0              doParallel_1.0.10      iterators_1.0.8       *
> * [7] foreach_1.4.3          RSclient_0.7-3         stringi_1.1.5
>  stringr_1.2.0          lubridate_1.6.0        plyr_1.8.4            *
>
> *loaded via a namespace (and not attached):*
> * [1] Rcpp_0.12.10     lattice_0.20-35  codetools_0.2-15 grid_3.4.0
> gtable_0.2.0     magrittr_1.5     scales_0.4.1     ggplot2_2.2.1   *
> * [9] lazyeval_0.2.0   tools_3.4.0      munsell_0.4.3    colorspace_1.3-2
> tibble_1.3.0    *
>
>
>
> Thank you in advance for help/hints, Yan
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
>
	[[alternative HTML version deleted]]