> From: "Yves Gauvreau" <cyg at sympatico.ca>
> References: <Pine.GSO.4.05.10006170711230.12526-100000 at auk.stats>
> Date: Sat, 17 Jun 2000 08:41:57 -0400
> Sender: owner-r-help at stat.math.ethz.ch
> Precedence: bulk
>
> This discussion is quite interesting. From the little I know of these
issues
> may I humbly suggest that parallelism may be the ultimate solution for the
> problem of speed and data size.
....> I would be curious to know what it would involve
> to provide R with parallelism enlisting PVM or other similar package?
>
> YG
I agree with you about the parallel approach and this is what we have
been building in the past year or two.
R, S and Omegahat have packages that allow them to
a) call methods in other applications, and
b) allow other applications to call methods on R objects
This uses CORBA to handle the remote method invocations,
but the user is insulated from this for both a) and b)
as they need only use R/S functions.
The framework allows R to call an application like Gnumeric (the GNOME
spreadsheet), or xgobi. However, it can also be used to have one R
session invoke methods in numerous others and do so as background
calls. The way we intend it to be used is by having a collection of R
"slaves" running on different machines, and a single
"master" that
dispatches individual tasks from a collection to the next idle R
server.
For example, to execute the command
lapply(elements, foo)
one would create several R "slaves" which provided
a foo method (by calling the R function foo()).
Then the lapply() is decomposed into a collection of
tasks corresponding to each value in the elements object
and dispatched.
Since CORBA is language neutral, we actuall use a Java
application/Omegahat script to create the tasks and manage the task
dispatching. One very nice aspect of this is that if a machine
fails or "slave" dies, we catch this and reschedule the task
it was currently working on.
There are definitely complexities here, but most
are data management issues.
This code has been in existence for quite some time and needs people
to bang on it. It is available from http://www.omegahat.org.
(Note: We are just working out some conflicts with CORBA and
R's header files that we recently discovered.)
Before the CORBA packages, I did do some work with PVM/MPI and S. I
think it is too low-level and requires much more effort from the R/S
user as type definitions cannot be dynamically read as they are in
CORBA.
Duncan.
--
_______________________________________________________________
Duncan Temple Lang duncan at research.bell-labs.com
Bell Labs, Lucent Technologies office: (908)582-3217
700 Mountain Avenue, Room 2C-259 fax: (908)582-3340
Murray Hill, NJ 07974-2070
http://cm.bell-labs.com/stat/duncan
"Languages shape the way we think, and determine what
we can think about."
Benjamin Whorf
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at
stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._