R help - May 2002 - Summary of Suggestions for poor man's parallel processing

Thanks to Luke Tierney, Agustin Lobo, Stephen Eglen, A.J. Rossini, Simon Wood
and Timothy H. Keitt for responding to my question about poor man's parallel
processing. Much of the substantive material was cc'ed to r-help already,
but
my take is as follows.

1) Currently, there is nothing "simple" for solving my problem.

2) I thought that Timothy Keitt's description of the use of a relational
   database for keeping track of different runs was quite interesting.

3) It seems clear that the "future" of this sort of activity lies with
RPVM
   (already available on CRAN) and SNOW by Luke Tierney. I found the
   description of SNOW available at
   http://www.stat.umn.edu/~luke/R/cluster/cluster.html to be intriguing. 

4) Since my needs are so simple, I will probably just hack something together
   myself that involves splitting my big.list into 4 chunks in Perl and then
   starting 4 jobs in R from Perl. Once those jobs are complete, I will then
   start an R job that collects the results. At least that is my current
   plan. I plan to follow the progress of RPVM and SNOW closely.

Thanks to all who responded.

Dave Kane
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

A couple of years ago I wrote a PERL script for distributing tasks across as
set of machines.  It's available at
http://www.warnes.net/downloads/distribute_pl.txt and you might find it a
useful place to start.

-Greg


> -----Original Message-----
> From: David Kane <David Kane [mailto:a296180 at mica.fmr.com]
> Sent: Friday, May 10, 2002 11:34 AM
> To: r-help at stat.math.ethz.ch
> Subject: [R] Summary of Suggestions for poor man's parallel processing
> 
> 
> Thanks to Luke Tierney, Agustin Lobo, Stephen Eglen, A.J. 
> Rossini, Simon Wood
> and Timothy H. Keitt for responding to my question about poor 
> man's parallel
> processing. Much of the substantive material was cc'ed to 
> r-help already, but
> my take is as follows.
> 
> 1) Currently, there is nothing "simple" for solving my problem.
> 
> 2) I thought that Timothy Keitt's description of the use of a 
> relational
>    database for keeping track of different runs was quite interesting.
> 
> 3) It seems clear that the "future" of this sort of activity 
> lies with RPVM
>    (already available on CRAN) and SNOW by Luke Tierney. I found the
>    description of SNOW available at
>    http://www.stat.umn.edu/~luke/R/cluster/cluster.html to be 
> intriguing. 
> 
> 4) Since my needs are so simple, I will probably just hack 
> something together
>    myself that involves splitting my big.list into 4 chunks 
> in Perl and then
>    starting 4 jobs in R from Perl. Once those jobs are 
> complete, I will then
>    start an R job that collects the results. At least that is 
> my current
>    plan. I plan to follow the progress of RPVM and SNOW closely.
> 
> Thanks to all who responded.
> 
> Dave Kane
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
> -.-.-.-.-.-.-.-.-
> r-help mailing list -- Read 
> http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: 
> r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
> _._._._._._._._._
> 

LEGAL NOTICE
Unless expressly stated otherwise, this message is confidential and may be
privileged. It is intended for the addressee(s) only. Access to this E-mail by
anyone else is unauthorized. If you are not an addressee, any disclosure or
copying of the contents of this E-mail or any action taken (or not taken) in
reliance on it is unauthorized and may be unlawful. If you are not an addressee,
please inform the sender immediately.
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._