Henrik Bengtsson
2015-Aug-14 21:00 UTC
[Rd] Why not pthreads on Windows in 'parallel' package?
On Windows there are a few 'pthreads' implementation, e.g. pthreads-w32 and winpthreads [https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Using-pthreads]. We're thinking of giving them a try for the matrixStats package, and basic tests indicates it works, but since Windows pthreads are not used by core R (or?) I've got a little bit worried that we will face overwhelming problems. So, why are the above Windows implementations not used in the 'parallel' package in order to add multicore support for mclapply() on Windows? Was it tried but found to be unreliable? Was it that no one had the time to do it? License issues? Are there any pointers to old R-devel threads discussing this? Thanks Henrik
Kasper Daniel Hansen
2015-Aug-15 00:52 UTC
[Rd] Why not pthreads on Windows in 'parallel' package?
mclapply uses fork which is different from pthreads. As I understand it, pthreads requires you to rewrite code, fork is a system call which takes care of completely replicating the current state of the process. Kasper On Fri, Aug 14, 2015 at 5:00 PM, Henrik Bengtsson <henrik.bengtsson at ucsf.edu> wrote:> On Windows there are a few 'pthreads' implementation, e.g. > pthreads-w32 and winpthreads > [https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Using-pthreads > ]. > We're thinking of giving them a try for the matrixStats package, and > basic tests indicates it works, but since Windows pthreads are not > used by core R (or?) I've got a little bit worried that we will face > overwhelming problems. > > So, why are the above Windows implementations not used in the > 'parallel' package in order to add multicore support for mclapply() on > Windows? Was it tried but found to be unreliable? Was it that no one > had the time to do it? License issues? Are there any pointers to old > R-devel threads discussing this? > > Thanks > > Henrik > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >[[alternative HTML version deleted]]
Henrik Bengtsson
2015-Aug-15 03:16 UTC
[Rd] Why not pthreads on Windows in 'parallel' package?
Aaaah ... and argh - I should have better not to post R question at midnight, especially when I know it forks the process and it's not using threads. Brain meltdown. (So, we'll proceed trying to use pthreads in matrixStats also for Windows). Sorry for the noise and thanks Kasper. Henrik On Aug 15, 2015 02:52, "Kasper Daniel Hansen" <kasperdanielhansen at gmail.com> wrote:> mclapply uses fork which is different from pthreads. As I understand it, > pthreads requires you to rewrite code, fork is a system call which takes > care of completely replicating the current state of the process. > > Kasper > > On Fri, Aug 14, 2015 at 5:00 PM, Henrik Bengtsson < > henrik.bengtsson at ucsf.edu> wrote: > >> On Windows there are a few 'pthreads' implementation, e.g. >> pthreads-w32 and winpthreads >> [ >> https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Using-pthreads >> ]. >> We're thinking of giving them a try for the matrixStats package, and >> basic tests indicates it works, but since Windows pthreads are not >> used by core R (or?) I've got a little bit worried that we will face >> overwhelming problems. >> >> So, why are the above Windows implementations not used in the >> 'parallel' package in order to add multicore support for mclapply() on >> Windows? Was it tried but found to be unreliable? Was it that no one >> had the time to do it? License issues? Are there any pointers to old >> R-devel threads discussing this? >> >> Thanks >> >> Henrik >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> > >[[alternative HTML version deleted]]