R devel - Dec 2019 - Inconsistent behavior for the C AP's R_ParseVector() ?

Hi again,

Beside R_ParseVector()'s possible inconsistent behavior, R's handling of
zero-length named elements does not seem consistent either:

```
> lst <- list()
> lst[[""]] <- 1
> names(lst)
[1] ""
> list("" = 1)
Error: attempt to use zero-length variable name
```

Should the parser be made to accept as valid what is otherwise possible
when using `[[<` ?


Best,

Laurent



Le sam. 30 nov. 2019 ? 17:33, Laurent Gautier <lgautier at gmail.com> a
?crit :
> I found the following code comment in `src/main/gram.c`:
>
> ```
>
> /* Memory leak
>
> yyparse(), as generated by bison, allocates extra space for the parser
> stack using malloc(). Unfortunately this means that there is a memory
> leak in case of an R error (long-jump). In principle, we could define
> yyoverflow() to relocate the parser stacks for bison and allocate say on
> the R heap, but yyoverflow() is undocumented and somewhat complicated
> (we would have to replicate some macros from the generated parser here).
> The same problem exists at least in the Rd and LaTeX parsers in tools.
> */
>
> ```
>
> Could this be related to be issue ?
>
> Le sam. 30 nov. 2019 ? 14:04, Laurent Gautier <lgautier at gmail.com>
a
> ?crit :
>
>> Hi,
>>
>> The behavior of
>> ```
>> SEXP R_ParseVector(SEXP, int, ParseStatus *, SEXP);
>> ```
>> defined in `src/include/R_ext/Parse.h` appears to be inconsistent
>> depending on the string to be parsed.
>>
>> Trying to parse a string such as `"list(''=1+"` sets
the
>> `ParseStatus` to incomplete parsing error but trying to parse
>> `"list(''=123"` will result in R sending a message to
the console (followed but a crash):
>>
>> ```
>> R[write to console]: Error: attempt to use zero-length variable
nameR[write to console]: Fatal error: unable to initialize the JIT*** stack
smashing detected ***: <unknown> terminated
>> ```
>>
>> Is there a reason for the difference in behavior, and is there a
workaround ?
>>
>> Thanks,
>>
>>
>> Laurent
>>
>>
	[[alternative HTML version deleted]]

Dear Laurent,

could you please provide a complete reproducible example where parsing 
results in a crash of R? Calling parse(text="list(''=123")
from R works
fine for me (gives Error: attempt to use zero-length variable name).

I don't think the problem you observed could be related to the memory 
leak. The leak is on the heap, not stack.

Zero-length names of elements in a list are allowed. They are not the 
same thing as zero-length variables in an environment. If you try to 
convert "lst" from your example to an environment, you would get the 
error (attempt to use zero-length variable name).

Best
Tomas


On 11/30/19 11:55 PM, Laurent Gautier wrote:
> Hi again,
>
> Beside R_ParseVector()'s possible inconsistent behavior, R's
handling of
> zero-length named elements does not seem consistent either:
>
> ```
>> lst <- list()
>> lst[[""]] <- 1
>> names(lst)
> [1] ""
>> list("" = 1)
> Error: attempt to use zero-length variable name
> ```
>
> Should the parser be made to accept as valid what is otherwise possible
> when using `[[<` ?
>
>
> Best,
>
> Laurent
>
>
>
> Le sam. 30 nov. 2019 ? 17:33, Laurent Gautier <lgautier at gmail.com>
a ?crit :
>
>> I found the following code comment in `src/main/gram.c`:
>>
>> ```
>>
>> /* Memory leak
>>
>> yyparse(), as generated by bison, allocates extra space for the parser
>> stack using malloc(). Unfortunately this means that there is a memory
>> leak in case of an R error (long-jump). In principle, we could define
>> yyoverflow() to relocate the parser stacks for bison and allocate say
on
>> the R heap, but yyoverflow() is undocumented and somewhat complicated
>> (we would have to replicate some macros from the generated parser
here).
>> The same problem exists at least in the Rd and LaTeX parsers in tools.
>> */
>>
>> ```
>>
>> Could this be related to be issue ?
>>
>> Le sam. 30 nov. 2019 ? 14:04, Laurent Gautier <lgautier at
gmail.com> a
>> ?crit :
>>
>>> Hi,
>>>
>>> The behavior of
>>> ```
>>> SEXP R_ParseVector(SEXP, int, ParseStatus *, SEXP);
>>> ```
>>> defined in `src/include/R_ext/Parse.h` appears to be inconsistent
>>> depending on the string to be parsed.
>>>
>>> Trying to parse a string such as `"list(''=1+"`
sets the
>>> `ParseStatus` to incomplete parsing error but trying to parse
>>> `"list(''=123"` will result in R sending a
message to the console (followed but a crash):
>>>
>>> ```
>>> R[write to console]: Error: attempt to use zero-length variable
nameR[write to console]: Fatal error: unable to initialize the JIT*** stack
smashing detected ***: <unknown> terminated
>>> ```
>>>
>>> Is there a reason for the difference in behavior, and is there a
workaround ?
>>>
>>> Thanks,
>>>
>>>
>>> Laurent
>>>
>>>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

Thanks for the quick response Tomas.

The same error is indeed happening when trying to have a zero-length
variable name in an environment. The surprising bit is then "why is this
happening during parsing" (that is why are variables assigned to an
environment) ?

We are otherwise aware that the error is not occurring in the R console,
but can be traced to a call to R_ParseVector() in R's C API:(
https://github.com/rpy2/rpy2/blob/master/rpy2/rinterface_lib/_rinterface_capi.py#L509
).

Our specific setup is calling an embedded R from Python, using the cffi
library. An error on end was the first possibility considered, but the
puzzling specificity of the error (as shown below other parsing errors are
handled properly) and the difficulty tracing what is in happening in
R_ParseVector() made me ask whether someone on this list had a suggestion
about the possible issue"

```
>>> import rpy2.rinterface as ri>>> ri.initr()>>> e =
ri.parse("list(''=1+")
---------------------------------------------------------------------------RParsingError
Traceback (most recent call last)
>>> e = ri.parse("list(''=123") R[write to console]:
Error: attempt to use zero-length variable name
R[write to console]: Fatal error: unable to initialize the JIT

*** stack smashing detected ***: <unknown> terminated
```


Le lun. 2 d?c. 2019 ? 06:37, Tomas Kalibera <tomas.kalibera at gmail.com>
a
?crit :
> Dear Laurent,
>
> could you please provide a complete reproducible example where parsing
> results in a crash of R? Calling
parse(text="list(''=123") from R works
> fine for me (gives Error: attempt to use zero-length variable name).
>
> I don't think the problem you observed could be related to the memory
> leak. The leak is on the heap, not stack.
>
> Zero-length names of elements in a list are allowed. They are not the
> same thing as zero-length variables in an environment. If you try to
> convert "lst" from your example to an environment, you would get
the
> error (attempt to use zero-length variable name).
>
> Best
> Tomas
>
>
> On 11/30/19 11:55 PM, Laurent Gautier wrote:
> > Hi again,
> >
> > Beside R_ParseVector()'s possible inconsistent behavior, R's
handling of
> > zero-length named elements does not seem consistent either:
> >
> > ```
> >> lst <- list()
> >> lst[[""]] <- 1
> >> names(lst)
> > [1] ""
> >> list("" = 1)
> > Error: attempt to use zero-length variable name
> > ```
> >
> > Should the parser be made to accept as valid what is otherwise
possible
> > when using `[[<` ?
> >
> >
> > Best,
> >
> > Laurent
> >
> >
> >
> > Le sam. 30 nov. 2019 ? 17:33, Laurent Gautier <lgautier at
gmail.com> a
> ?crit :
> >
> >> I found the following code comment in `src/main/gram.c`:
> >>
> >> ```
> >>
> >> /* Memory leak
> >>
> >> yyparse(), as generated by bison, allocates extra space for the
parser
> >> stack using malloc(). Unfortunately this means that there is a
memory
> >> leak in case of an R error (long-jump). In principle, we could
define
> >> yyoverflow() to relocate the parser stacks for bison and allocate
say on
> >> the R heap, but yyoverflow() is undocumented and somewhat
complicated
> >> (we would have to replicate some macros from the generated parser
here).
> >> The same problem exists at least in the Rd and LaTeX parsers in
tools.
> >> */
> >>
> >> ```
> >>
> >> Could this be related to be issue ?
> >>
> >> Le sam. 30 nov. 2019 ? 14:04, Laurent Gautier <lgautier at
gmail.com> a
> >> ?crit :
> >>
> >>> Hi,
> >>>
> >>> The behavior of
> >>> ```
> >>> SEXP R_ParseVector(SEXP, int, ParseStatus *, SEXP);
> >>> ```
> >>> defined in `src/include/R_ext/Parse.h` appears to be
inconsistent
> >>> depending on the string to be parsed.
> >>>
> >>> Trying to parse a string such as
`"list(''=1+"` sets the
> >>> `ParseStatus` to incomplete parsing error but trying to parse
> >>> `"list(''=123"` will result in R sending a
message to the console
> (followed but a crash):
> >>>
> >>> ```
> >>> R[write to console]: Error: attempt to use zero-length
variable
> nameR[write to console]: Fatal error: unable to initialize the JIT*** stack
> smashing detected ***: <unknown> terminated
> >>> ```
> >>>
> >>> Is there a reason for the difference in behavior, and is there
a
> workaround ?
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Laurent
> >>>
> >>>
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
>
	[[alternative HTML version deleted]]