R devel - Feb 2005 - Process to both write to and read from (pipe/fork)?

Dear all,

I would like to start a process from an R program in such a way that
I can both feed input into the process and read the process's output.
It seems that in R, I can have a pipe for writing into another process's
input or a pipe for reading from another process's output, but not both.

Doing both necessitates forking, such that the child can start the
external process and feed that with some input, and the parent can
read the output from the external process. Additionally, this requires
obtaining a plain pipe, i.e. one with an input handle (for writing to)
and an output handle (for reading from) prior to forking, so the child
can connect the external process's stdout to the input handle and the
parent can read that from the output handle.

My problem is that I cannot find a way in R to set up such a pipe.

For forking, I've found the fork package, but I can't seem to get a
pipe. I know about the pipe function in the base package, but it seems
to me that that is an interface to popen(2), rather than to pipe(2)
(see attached C source). At least, I can't seem to get anything else but
a popen equivalent from it. I've looked for a pipe package, for pipe in
the fork package, and googled around, to no avail.

For illustrating what I'd like to do, I attach a C program that
replicates the pipe example from the base docs, with the difference
that the input data to be processed with sed are present in the
program rather than in a file.

I start to feel silly and stupid for being unable to figure out such a
basic thing. Thanks in advance for any help -- RTFMs very welcome.

Best regards, Jan
-- 
 +- Jan T. Kim -------------------------------------------------------+
 |    *NEW*    email: jtk at cmp.uea.ac.uk                               |
 |    *NEW*    WWW:   http://www.cmp.uea.ac.uk/people/jtk             |
 *-----=<  hierarchical systems are for files, not for humans  >=-----*
-------------- next part --------------
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>


int main(int argc, char *argv[])
{
  int fd[2];

  /* get a pipe, fd[0] is "input end", fd[1] is "output end"
*/
  pipe(fd);

  /* fork a child process to feed something into the pipe */
  if (fork())
  {
    /* this is the parent process which reads from the pipe */
    FILE *f = fdopen(fd[0], "r");
    char buf[100];

    close(fd[1]);
    while (!feof(f))
    {
      if (fgets(buf, 100, f))
	printf("got line: %s", buf);
    }
    fclose(f);
  }
  else
  {
    /* this is the child process which feeds something into the pipe */
    FILE *f = popen("sed -e 's/,$//'", "w");

    close(fd[0]);
    /* connect stdout to pipe's output end, so parent ends up reading
      the outut of sed */
    dup2(fd[1], fileno(stdout));
    fprintf(f, "450, 390, 467, 654,  30, 542, 334, 432, 421,\n");
    fprintf(f, "357, 497, 493, 550, 549, 467, 575, 578, 342,\n");
    fprintf(f, "446, 547, 534, 495, 979, 479\n");
    pclose(f);
  }
  return (0);
}

[Moved from R-help.]

One reason you cannot easily do this with basic R functions is that it is 
not portable.  E.g. pipes on Windows 98 do not work like that, and 
system() on that OS has to work hard to do something similar.

If we only had to consider standard Unices, pipe() would allow read-write 
modes.  As it is, it is easy for you to write an OS-specific extension.

BTW, please re-read the distinction between R-devel and R-help in the 
posting guide: this (and most of your other recent postings) seem to me to 
fall unambiguously within the specification for R-devel.


On Tue, 1 Feb 2005, Jan T. Kim wrote:
> Dear all,
>
> I would like to start a process from an R program in such a way that
> I can both feed input into the process and read the process's output.
> It seems that in R, I can have a pipe for writing into another
process's
> input or a pipe for reading from another process's output, but not
both.
>
> Doing both necessitates forking, such that the child can start the
> external process and feed that with some input, and the parent can
> read the output from the external process. Additionally, this requires
> obtaining a plain pipe, i.e. one with an input handle (for writing to)
> and an output handle (for reading from) prior to forking, so the child
> can connect the external process's stdout to the input handle and the
> parent can read that from the output handle.
>
> My problem is that I cannot find a way in R to set up such a pipe.
>
> For forking, I've found the fork package, but I can't seem to get a
> pipe. I know about the pipe function in the base package, but it seems
> to me that that is an interface to popen(2), rather than to pipe(2)
> (see attached C source). At least, I can't seem to get anything else
but
> a popen equivalent from it. I've looked for a pipe package, for pipe in
> the fork package, and googled around, to no avail.
>
> For illustrating what I'd like to do, I attach a C program that
> replicates the pipe example from the base docs, with the difference
> that the input data to be processed with sed are present in the
> program rather than in a file.
>
> I start to feel silly and stupid for being unable to figure out such a
> basic thing. Thanks in advance for any help -- RTFMs very welcome.
>
> Best regards, Jan
> -- 
> +- Jan T. Kim -------------------------------------------------------+
> |    *NEW*    email: jtk@cmp.uea.ac.uk                               |
> |    *NEW*    WWW:   http://www.cmp.uea.ac.uk/people/jtk             |
> *-----=<  hierarchical systems are for files, not for humans 
>=-----*
>
-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595