R help - May 2006 - "Pipelining" programs in R

Hello...

I would like to use R for 'pipelining' data among several programs. 
I'm
wondering how I can use R to call another program, feed that program a set 
of parameters, and retrieve the output.

E.g., I have an executable that, when opened, prompts the user to enter a 
set of parameters.  The program then executes & prompts the user for the 
name of an output file.  I need to run this program on a large batch of 
parameters, such that it would clearly be desirable to automate the 
process.  Is there a straightforward way to do this?  I can't find any 
online documentation addressing this topic, but perhaps I've not been 
looking in the right place.

In pseudocode, supposing I have a large array of parameters in R:

For each set of parameters
         -Open Program.
         -Enter Parameters.
         -Cause program to execute (typically done by simply entering " \n 
" after manually entering parameters).
         -Enter name of output file.
         -Close program.

Any advice will be greatly appreciated!

Thanks,
Dan Rabosky



Dan Rabosky
Department of Ecology and Evolutionary Biology
237 Corson Hall
Cornell University
Ithaca, NY14853-2701 USA
web: http://www.birds.cornell.edu/evb/Graduates_Dan.htm

You can use system() or pipe() and friends.  This is covered in Section 
5.1 of `Writing R Extensions'.

Perhaps the simplest way is to something like

tmp <- tempfile()
zz <- file(tmp, "w")
# write the input script to zz, e.g. via cat
close(zz)
res <- system(paste("myprog <", tmp), intern = TRUE)

and then parse the output from the character vector 'res'.

If you have a Unix-like OS and understand pipes, fifos and sockets you can 
also use those: see ?pipe.  (With a higher degree of understanding you can 
use some of these on Windows NT.)

It would have been very helpful to have known your OS, a piece of 
information the posting guide asked for.  (Since you talk about
'open'ing
an executable, I surmise you are not familiar with traditional OSes which 
'run' programs.)

On Thu, 18 May 2006, Dan Rabosky wrote:
>
> Hello...
>
> I would like to use R for 'pipelining' data among several programs.
I'm
> wondering how I can use R to call another program, feed that program a set
> of parameters, and retrieve the output.
>
> E.g., I have an executable that, when opened, prompts the user to enter a
> set of parameters.  The program then executes & prompts the user for
the
> name of an output file.  I need to run this program on a large batch of
> parameters, such that it would clearly be desirable to automate the
> process.  Is there a straightforward way to do this?  I can't find any
> online documentation addressing this topic, but perhaps I've not been
> looking in the right place.
>
> In pseudocode, supposing I have a large array of parameters in R:
>
> For each set of parameters
>         -Open Program.
>         -Enter Parameters.
>         -Cause program to execute (typically done by simply entering "
\n
> " after manually entering parameters).
>         -Enter name of output file.
>         -Close program.
>
> Any advice will be greatly appreciated!
>
> Thanks,
> Dan Rabosky
>
>
>
> Dan Rabosky
> Department of Ecology and Evolutionary Biology
> 237 Corson Hall
> Cornell University
> Ithaca, NY14853-2701 USA
> web: http://www.birds.cornell.edu/evb/Graduates_Dan.htm
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
>
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

On Thu, May 18, 2006 at 11:50:24PM -0400, Dan Rabosky
wrote:
> 
> Hello...
> 
> I would like to use R for 'pipelining' data among several programs.
I'm
> wondering how I can use R to call another program, feed that program a set 
> of parameters, and retrieve the output.
> 
> E.g., I have an executable that, when opened, prompts the user to enter a 
> set of parameters.  The program then executes & prompts the user for
the
> name of an output file.  I need to run this program on a large batch of 
> parameters, such that it would clearly be desirable to automate the 
> process.  Is there a straightforward way to do this?  I can't find any 
> online documentation addressing this topic, but perhaps I've not been 
> looking in the right place.
> 
> In pseudocode, supposing I have a large array of parameters in R:
> 
> For each set of parameters
>          -Open Program.
>          -Enter Parameters.
>          -Cause program to execute (typically done by simply entering
" \n
> " after manually entering parameters).
>          -Enter name of output file.
>          -Close program.
If your program gets all the input it requires from the command line,
you might use the pipe function, as in

    f <- pipe("ls -l");
    l <- readLines(f);

However, your executable seems to expect its input via its standard input
and you want to read the output it writes to its standard output. To my
knowledge, this is not possible with R. I've written some stuff to add
such functionality to R, this is available from

    http://www2.cmp.uea.ac.uk/~jtk/software/

The filterpipe patch is against an older version of R, though.

Best regards, Jan
-- 
 +- Jan T. Kim -------------------------------------------------------+
 |             email: jtk at cmp.uea.ac.uk                               |
 |             WWW:   http://www.cmp.uea.ac.uk/people/jtk             |
 *-----=<  hierarchical systems are for files, not for humans  >=-----*