R help - Oct 2013 - speed of makeCluster (package parallel)

Hi all,

I am quite new in the world of parallelization and I wonder if there is a
way to increase the speed of creation of a parallel socket cluster. The
time spend to include threads increase exponentially with the number of
thread considered and I use of computer with two 8 cores CPU and thus
showing a total of 32 threads in windows 7.
Currently, I use the default parameters (type = "PSOCK"), but is there
any
fine tuning parameters that I can use to take advantage of this system ?

Thanks in advance for your help !

Arnaud

R version 3.0.1 (2013-05-16)
Platform: x86_64-w64-mingw32/x64 (64-bit)

	[[alternative HTML version deleted]]

See library(help = "parallel?)


On 28 Oct 2013, at 17:19, Arnaud Mosnier <a.mosnier at gmail.com> wrote:
> Hi all,
> 
> I am quite new in the world of parallelization and I wonder if there is a
> way to increase the speed of creation of a parallel socket cluster. The
> time spend to include threads increase exponentially with the number of
> thread considered and I use of computer with two 8 cores CPU and thus
> showing a total of 32 threads in windows 7.
> Currently, I use the default parameters (type = "PSOCK"), but is
there any
> fine tuning parameters that I can use to take advantage of this system ?
> 
> Thanks in advance for your help !
> 
> Arnaud
> 
> R version 3.0.1 (2013-05-16)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

On 28/10/2013 16:19, Arnaud Mosnier wrote:
> Hi all,
>
> I am quite new in the world of parallelization and I wonder if there is a
> way to increase the speed of creation of a parallel socket cluster. The
> time spend to include threads increase exponentially with the number of
It increases linearly in my tests (or a decent OS).  But really if 
parallel computing is worthwhile you will be doing minutes of work on 
each worker process and the startup time will not be signifcant.
> thread considered and I use of computer with two 8 cores CPU and thus
> showing a total of 32 threads in windows 7.
The first way to speed things up: use a decent OS:  forking clusters is 
much faster.
> Currently, I use the default parameters (type = "PSOCK"), but is
there any
> fine tuning parameters that I can use to take advantage of this system ?
>
> Thanks in advance for your help !
>
> Arnaud
>
> R version 3.0.1 (2013-05-16)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
>
> 	[[alternative HTML version deleted]]

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Thanks Brian, I thought that forking clusters was better ... but as you
mentioned, it is not available on windows.
Unfortunately, you do not always choose the OS used by your company !

Arnaud



Date: Mon, 28 Oct 2013 17:59:10 +0000
From: Prof Brian Ripley <ripley@stats.ox.ac.uk>
To: r-help@r-project.org
Subject: Re: [R] speed of makeCluster (package parallel)
Message-ID: <526EA5EE.9060109@stats.ox.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

On 28/10/2013 16:19, Arnaud Mosnier wrote:
> Hi all,
>
> I am quite new in the world of parallelization and I wonder if there is a
> way to increase the speed of creation of a parallel socket cluster. The
> time spend to include threads increase exponentially with the number of
It increases linearly in my tests (or a decent OS).  But really if
parallel computing is worthwhile you will be doing minutes of work on
each worker process and the startup time will not be signifcant.
> thread considered and I use of computer with two 8 cores CPU and thus
> showing a total of 32 threads in windows 7.
The first way to speed things up: use a decent OS:  forking clusters is
much faster.
> Currently, I use the default parameters (type = "PSOCK"), but is
there any
> fine tuning parameters that I can use to take advantage of this system ?
>
> Thanks in advance for your help !
>
> Arnaud
>
> R version 3.0.1 (2013-05-16)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
>
>       [[alternative HTML version deleted]]

--
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

	[[alternative HTML version deleted]]