R devel - Sep 2008 - suggestion of new API function for embedded programming.

While doing some embedded programming and trying to figure out how to generate
a hand coded SEXP equivalent of the line
"t.test(x,conf.level=(1-p))$conf.int[2]" I had an idea for an addition
to the
embedded API.

There are a number of hidden or static parse functions (R_ParseBuffer,
R_Parse1Buffer, etc.) which take an IoBuffer* and returns a parsed tree.  If
one or more of these functions were exported to the Rembedded.h API we could
do something like the following:

  R_Expr = R_Parse1Buffer(&R_ConsoleIob, 0, &status);
  if (PARSE_OK==status) {
    ...
    value = eval(R_CurrentExpr, rho);
    ...
  }

or possibly simplifying the interface to take the CMDL string:

  R_Expr = R_Parse1Line("t.test(x,conf.level=(1-p))$conf.int[2]",
&status);

I think this would be a useful addition to the embedding interface, and
hopefully not difficult to incorporate by someone more experienced with the
internals than I currently am.  I took a few hours to look into adding this
interface, but will not have time to try to do so for a few months -- I have a
couple of hard and fast deadlines over the next couple of weeks.  So, I would
like to make the suggestion and participate in the dialog.

  Best regards,

  EBo --

ps: if someone can suggest how to hand code
"t.test(x,conf.level=(1-p))$conf.int[2]" so I can embedd it I would be
most
appreciative.

On Wed, 3 Sep 2008, EBo wrote:
>
> While doing some embedded programming and trying to figure out how to
generate
> a hand coded SEXP equivalent of the line
> "t.test(x,conf.level=(1-p))$conf.int[2]" I had an idea for an
addition to the
> embedded API.
>
> There are a number of hidden or static parse functions (R_ParseBuffer,
> R_Parse1Buffer, etc.) which take an IoBuffer* and returns a parsed tree. 
If
> one or more of these functions were exported to the Rembedded.h API we
could
> do something like the following:
>
>  R_Expr = R_Parse1Buffer(&R_ConsoleIob, 0, &status);
>  if (PARSE_OK==status) {
>    ...
>    value = eval(R_CurrentExpr, rho);
>    ...
>  }
We definitely do NOT want this frozen into the public API.
>
> or possibly simplifying the interface to take the CMDL string:
>
>  R_Expr = R_Parse1Line("t.test(x,conf.level=(1-p))$conf.int[2]",
&status);
>
> I think this would be a useful addition to the embedding interface, and
> hopefully not difficult to incorporate by someone more experienced with the
> internals than I currently am.  I took a few hours to look into adding this
> interface, but will not have time to try to do so for a few months -- I
have a
> couple of hard and fast deadlines over the next couple of weeks.  So, I
would
> like to make the suggestion and participate in the dialog.
Something along these lines should be feasible in principle, though
care may be needed in handlign of errors.

luke
>
>  Best regards,
>
>  EBo --
>
> ps: if someone can suggest how to hand code
> "t.test(x,conf.level=(1-p))$conf.int[2]" so I can embedd it I
would be most
> appreciative.
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:      luke at stat.uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

Luke Tierney <luke at stat.uiowa.edu> said:
> ...
> > do something like the following:
> >
> >  R_Expr = R_Parse1Buffer(&R_ConsoleIob, 0, &status);
> >  if (PARSE_OK==status) {
> >    ...
> >    value = eval(R_CurrentExpr, rho);
> >    ...
> >  }
> 
> We definitely do NOT want this frozen into the public API.
What is your objection with making something like this a part of the public
API?  I understand that having to use the IOBuffer seems a bit much, but I do
not understand your concern.
> > or possibly simplifying the interface to take the CMDL string:
> >
> >  R_Expr =
R_Parse1Line("t.test(x,conf.level=(1-p))$conf.int[2]", &status);
> 
> Something along these lines should be feasible in principle, though
> care may be needed in handlign of errors.
Agreed.  I would also add that R_Parse1Line should also not modify any global
variables so that it basicall functions independently and does not violate the
law of least surprise.  This imples that an environment would also be passed in:

  R_Expr=R_Parse1Line("t.test(x,conf.level=(1-p))$conf.int[2]", rho,
&status);

  Thanks and best regards,

  EBo --

On Wed, 3 Sep 2008, EBo wrote:
> Luke Tierney <luke at stat.uiowa.edu> said:
>
>> ...
>>> do something like the following:
>>>
>>>  R_Expr = R_Parse1Buffer(&R_ConsoleIob, 0, &status);
>>>  if (PARSE_OK==status) {
>>>    ...
>>>    value = eval(R_CurrentExpr, rho);
>>>    ...
>>>  }
>>
>> We definitely do NOT want this frozen into the public API.
>
> What is your objection with making something like this a part of the public
> API?  I understand that having to use the IOBuffer seems a bit much, but I
do
> not understand your concern.
We need the freedom to completely change these internals if doing so
proves useful.

luke
>>> or possibly simplifying the interface to take the CMDL string:
>>>
>>>  R_Expr =
R_Parse1Line("t.test(x,conf.level=(1-p))$conf.int[2]", &status);
>>
>> Something along these lines should be feasible in principle, though
>> care may be needed in handlign of errors.
>
> Agreed.  I would also add that R_Parse1Line should also not modify any
global
> variables so that it basicall functions independently and does not violate
the
> law of least surprise.  This imples that an environment would also be
passed in:
>
>  R_Expr=R_Parse1Line("t.test(x,conf.level=(1-p))$conf.int[2]",
rho, &status);
>
>  Thanks and best regards,
>
>  EBo --
>
>
-- 
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:      luke at stat.uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

Luke Tierney <luke at stat.uiowa.edu> said:
> On Wed, 3 Sep 2008, EBo wrote:
> 
> > Luke Tierney <luke at stat.uiowa.edu> said:
> >
> >> ...
> >>> do something like the following:
> >>>
> >>>  R_Expr = R_Parse1Buffer(&R_ConsoleIob, 0, &status);
> >>>  if (PARSE_OK==status) {
> >>>    ...
> >>>    value = eval(R_CurrentExpr, rho);
> >>>    ...
> >>>  }
> >>
> >> We definitely do NOT want this frozen into the public API.
> >
> > What is your objection with making something like this a part of the
public
> > API?  I understand that having to use the IOBuffer seems a bit much,
but I do
> > not understand your concern.
> 
> We need the freedom to completely change these internals if doing so
> proves useful.
Ah, that makes perfect sense.

Thanks,

  EBo --

I stumbled onto a near trivial solution... here is some example code:

  EBo --

#include <Rembedded.h>
#include <Rdefines.h>
#include <Rinternals.h>
#include <R_ext/Parse.h>


SEXP
LineEval (char *cmd)
{
    SEXP ans;
    int error;

    ans = R_tryLineEval (cmd, R_GlobalEnv, &error);

    if (error)
    {
        fprintf (stderr, "Error evaluating line \"%s\"\n",
cmd);
        return R_NilValue;
    }

    return ans;
}


SEXP
R_tryLineEval (char *cmd, SEXP rho, int *error)
{
    SEXP cmdSexp, cmdexpr, ans = R_NilValue;
    int i;
    ParseStatus status;

    *error = 0;

    // parse the R epression
    PROTECT(cmdSexp = allocVector(STRSXP, 1));
    SET_STRING_ELT(cmdSexp, 0, mkChar(cmd));
    cmdexpr = PROTECT(R_ParseVector(cmdSexp, -1, &status, R_NilValue));
    if (status != PARSE_OK) {
        UNPROTECT(2);
        return R_NilValue;
    }
    // Loop is needed here as EXPSEXP will be of length > 1 
    for(i = 0; i < length(cmdexpr); i++)
    {
        ans = R_tryEval(VECTOR_ELT(cmdexpr, i), R_GlobalEnv, error);
        if (*error) {
            UNPROTECT(2);
            return R_NilValue;
        }
    }
    UNPROTECT(2);

    return ans;
}


int
main (int argc, char *argv[])
{
    char *cmd[]         {"t.test(x,conf.level=0.67)",
         "t.test(x,conf.level=0.67)$conf.int[2]",
         "xyz=c(9,8,7,6); xyz=median(x)",
         "print(x)",
         NULL
        };
    SEXP ans;
    int  i;

    char *args[] = {"bla", "--gui=none",
"--silent", "--no-save"};
    Rf_initEmbeddedR (4, args);

    // set the variable "x" to the dataset.
    SEXP value;
    value = NEW_NUMERIC(11);
    for (i=10; 0<=i; i--)
        NUMERIC_DATA(value)[i] = i;
    PROTECT(value);
    setVar(install("x"), value, R_GlobalEnv);
 
    
    // spin through several R expressions and evaluate each.
    for (i=0; cmd[i]; i++)
    {
        ans = LineEval (cmd[i]);
        if (R_NilValue != ans)
        {
            printf ("cmd = \"%s\"\n", cmd[i]);
            if (IS_NUMERIC(ans))
                printf ("  ans is %f\n", REAL(ans)[0]);
            else 
                PrintValue(ans);
            printf ("#############\n\n");
        }
    }

    return 0;
}






EBo <ebo at sandien.com> said:
> Luke Tierney <luke at stat.uiowa.edu> said:
> 
> > On Wed, 3 Sep 2008, EBo wrote:
> > 
> > > Luke Tierney <luke at stat.uiowa.edu> said:
> > >
> > >> ...
> > >>> do something like the following:
> > >>>
> > >>>  R_Expr = R_Parse1Buffer(&R_ConsoleIob, 0,
&status);
> > >>>  if (PARSE_OK==status) {
> > >>>    ...
> > >>>    value = eval(R_CurrentExpr, rho);
> > >>>    ...
> > >>>  }
> > >>
> > >> We definitely do NOT want this frozen into the public API.
> > >
> > > What is your objection with making something like this a part of
the public
> > > API?  I understand that having to use the IOBuffer seems a bit
much, but
I do
> > > not understand your concern.
> > 
> > We need the freedom to completely change these internals if doing so
> > proves useful.
> 
> Ah, that makes perfect sense.
> 
> Thanks,
> 
>   EBo --
> 


--

On Sep 3, 2008, at 9:51 , EBo wrote:
>
> While doing some embedded programming and trying to figure out how  
> to generate
> a hand coded SEXP equivalent of the line
> "t.test(x,conf.level=(1-p))$conf.int[2]" I had an idea for an  
> addition to the
> embedded API.
>
> There are a number of hidden or static parse functions (R_ParseBuffer,
> R_Parse1Buffer, etc.) which take an IoBuffer* and returns a parsed  
> tree.  If
> one or more of these functions were exported to the Rembedded.h API  
> we could
> do something like the following:
>
>  R_Expr = R_Parse1Buffer(&R_ConsoleIob, 0, &status);
>  if (PARSE_OK==status) {
>    ...
>    value = eval(R_CurrentExpr, rho);
>    ...
>  }
>
> or possibly simplifying the interface to take the CMDL string:
>
>  R_Expr = R_Parse1Line("t.test(x,conf.level=(1-p))$conf.int[2]",
> &status);
>
Why do you think is R_ParseVector not sufficient for this? That is  
what most of use use to achieve exactly what you describe...
For something that even mimics the continuation behavior of the R  
console have a look at parseString function in Rserve.

Cheers,
Simon

> I think this would be a useful addition to the embedding interface,  
> and
> hopefully not difficult to incorporate by someone more experienced  
> with the
> internals than I currently am.  I took a few hours to look into  
> adding this
> interface, but will not have time to try to do so for a few months  
> -- I have a
> couple of hard and fast deadlines over the next couple of weeks.   
> So, I would
> like to make the suggestion and participate in the dialog.
>
>  Best regards,
>
>  EBo --
>
> ps: if someone can suggest how to hand code
> "t.test(x,conf.level=(1-p))$conf.int[2]" so I can embedd it I
would
> be most
> appreciative.
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

Simon Urbanek <simon.urbanek at r-project.org> said:
> Why do you think is R_ParseVector not sufficient for this? That is  
> what most of use use to achieve exactly what you describe...
> For something that even mimics the continuation behavior of the R  
> console have a look at parseString function in Rserve.
Thank you for the reply.  

I am new to R, and struggling to learn most of it's aspects and embedding
issues in particular.  When I first posted my questions and comments on the
IRC channel and here I asked how I would go about about achieving this
functionality.  Last night I stumbled onto an example of what R_ParseVector
actually does I was finially able to get it working.  Part of my post was
intended to say I have found a solution, and part of it to show how my
original thought would be done.  There is only on reason to add the API is
code clarity and convenience.  Actually a few additional sentences in the
embedded documentation better explaining what R_ParseVector does would
havekept me from creating this thread and a couple of days of pain.  As for
parseString, this is the first I have read of it, so will now check into it.

Thanks and best regards,

  EBo --

Jeffrey Horner <jeff.horner at vanderbilt.edu> said:
> Also, study the source code in the littler project. As it is a simple 
> command line alternative to the R shell script and executable, it may 
> bring you up to speed on simple embedding and parsing; another example 
> at least.
> 
> http://biostat.mc.vanderbilt.edu/LittleR

Jeff,

Thank you for the pointer.  Tis is exactly the kind of examples I had
originally asked for and had hoped to find.

  EBo --