R devel - Jun 2010 - GUI's and R background processes

Hello,

I am new to R and have created an application using R 2.10, with a graphical UI
using TclTk 8.5, running on windows 7, quad core machine.
The intention of the application is to launch calculations and display results
on a graphical dashboard.

I've reached a roadblock, and I need to confirm that the following CANNOT be
done. I've been trying to find a mechanism for doing the following:

1. From the dashboard, start a huge calculation (i.e. call a function) that take
at least 30 seconds to run, but without tying up the other dashboard features
2. Dashboard can detect when the calculation is finished
3. Dashboard can display incremental results as the calculation runs (i.e.
status/progress)

The requirement is to kickoff 4 calculations (#1), but I don't want the user
to wait for the others to finish.
The calculations are not dependent. I just need to display results.

I've been reading that threading in R is not an option. I tried using the
multicore package, but that is still synchronous. I looked at multicore, fork(),
addTaskCallback(), and TclTk threading, and none of these seem like an option.

Is there a way to launch multiple R scripts from controller that can communicate
back and forth? I believe this means that the separate processes are able to
communicate.

I certainly appreciate any direction you can provide. I really want to find some
good news to tell the boss, though, since we went down this path before
realizing the limitations.

Thanks very much,

Anne


	[[alternative HTML version deleted]]

Anne George wrote:
> Hello,
>
> I am new to R and have created an application using R 2.10, with a
graphical UI using TclTk 8.5, running on windows 7, quad core machine.
> The intention of the application is to launch calculations and display
results on a graphical dashboard.
>
> I've reached a roadblock, and I need to confirm that the following
CANNOT be done. I've been trying to find a mechanism for doing the
following:
>
> 1. From the dashboard, start a huge calculation (i.e. call a function) that
take at least 30 seconds to run, but without tying up the other dashboard
features
> 2. Dashboard can detect when the calculation is finished
> 3. Dashboard can display incremental results as the calculation runs (i.e.
status/progress)
>
> The requirement is to kickoff 4 calculations (#1), but I don't want the
user to wait for the others to finish.
> The calculations are not dependent. I just need to display results.
>
> I've been reading that threading in R is not an option. I tried using
the multicore package, but that is still synchronous. I looked at multicore,
fork(), addTaskCallback(), and TclTk threading, and none of these seem like an
option.
>
> Is there a way to launch multiple R scripts from controller that can
communicate back and forth? I believe this means that the separate processes are
able to communicate.
>
> I certainly appreciate any direction you can provide. I really want to find
some good news to tell the boss, though, since we went down this path before
realizing the limitations.
This is possible, but not easy.  You need to work at the C level.

The key is that the long running processes need to periodically tell R 
to check for user events by calling

R_ProcessEvents();
R_CheckUserInterrupt();

at the C level.  This works in Windows in current versions, and I 
believe on all platforms in R-devel.

The code should be prepared for R to call it again during 
R_ProcessEvents, so if it is not re-entrant, it should arrange to signal 
an error.  It should be prepared for the R_CheckUserInterrupt call to 
never return if the user happened to interrupt the calculation.

You probably only want one processor to do this, and let it check on the 
other cores to see if there are any results ready.


Duncan Murdoch

> I am new to R and have created an application using R 2.10, with a
graphical UI using TclTk 8.5, running on windows 7, quad core machine.
> The intention of the application is to launch calculations and display
results on a graphical dashboard.
> 
> I've reached a roadblock, and I need to confirm that the following
CANNOT be done. I've been trying to find a mechanism for doing the
following:
> 
> 1. From the dashboard, start a huge calculation (i.e. call a function) that
take at least 30 seconds to run, but without tying up the other dashboard
features
> 2. Dashboard can detect when the calculation is finished 
> 3. Dashboard can display incremental results as the calculation runs (i.e.
status/progress)
> 
> The requirement is to kickoff 4 calculations (#1), but I don't want the
user to wait for the others to finish.
> The calculations are not dependent. I just need to display results. 
> 
> I've been reading that threading in R is not an option. I tried using
the multicore package, but that is still synchronous. I looked at multicore,
fork(), addTaskCallback(), and TclTk threading, and none of these seem like an
option.
> 
> Is there a way to launch multiple R scripts from controller that can
communicate back and forth? I believe this means that the separate processes are
able to communicate.
> 
> I certainly appreciate any direction you can provide. I really want to find
some good news to tell the boss, though, since we went down this path before
realizing the limitations.
Another option to consider is moving to a client/server architecture. Even if
the dashboard and the R engine are on the same machine now, in the future you
might want to deploy the dashboard to the web (or other client) to make it
available to a wider audience. This also might make the asynchronous
communication easier to manage, and offload computations from the client
machine.

There are several options out there, including:
- Rserve, open source project maintained by Simon Urbanek.
http://rosuda.org/Rserve/
- Biocep-R. http://biocep-distrib.r-forge.r-project.org/. Also open source.
- Spotfire Statistics Services: commercial solution from TIBCO (my company),
that providers server access to a cluster of R and S+ engines via Java and C#
APIs.
http://spotfire.tibco.com/products/statistics-services/predictive-analytics.aspx

Good luck,
Lou

--------------------------
Lou Bajuk-Yorgan
Sr. Director, Product Management
Spotfire, TIBCO Software
206-802-2328
lbajuk at tibco.com
http://spotfire.tibco.com

TIBCO Spotfire Analytics
Better than BI.? Smarter than Spreadsheets.
(click here for the Top 5 reasons why...)