R devel - Jun 2008 - Embedded R, last errormessage, and stack smashing

Dear list,

I am having an embedded R, dying with
*** stack smashing detected *** in one specific case.

My code is such as I evaluate R expression with C code like

res = R_tryEval(expr, env, &error);

and in case of error, get the error message (usually sucessfully) with
code like below:

SEXP geterrmessage = findVar(install("geterrmessage"),
R_BaseNamespace);
PROTECT(expr = allocVector(LANGSXP, 1));
SETCAR(expr, geterrmessage);
PROTECT(res = Rf_eval(expr, R_GlobalEnv));
// ---> *** stack smashing detected ***

The call to Rf_eval does not return as the stack smashing error stops the show.

Tracing with gdb where the problem occurs, I follow the path:
eval -> R_CheckStack in main/error.c -> errorcall in main/error.c


In errorcall, I narrow down the origin of the problem around the lines:
658    va_start(ap, format);
659    verrorcall_dflt(call, format, ap);
660    va_end(ap);
and add a breakpoint on each one of those lines.


My gdb session till the stack smashing crashing is then looking like:

Breakpoint 7, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87698 "C
stack usage is too close to the limit")
    at errors.c:658
658	    va_start(ap, format);
(gdb) p ap
$2 = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x41c22f60,
reg_save_area = 0x41c22e70}}
(gdb) p format
$3 = 0x7f4dbdc87698 "C stack usage is too close to the limit"
(gdb) continue
Continuing.

Breakpoint 8, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87698 "C
stack usage is too close to the limit")
    at errors.c:659
659	    verrorcall_dflt(call, format, ap);
(gdb) continue
Continuing.

Breakpoint 6, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406 "%s")
at errors.c:648
648	    if (R_ErrorHook != NULL) {
(gdb) continue
Continuing.

Breakpoint 7, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406 "%s")
at errors.c:658
658	    va_start(ap, format);
(gdb) continue
Continuing.

Breakpoint 8, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406 "%s")
at errors.c:659
659	    verrorcall_dflt(call, format, ap);
(gdb) continue
Continuing.
*** stack smashing detected ***: /usr/bin/python terminated


My understanding is that there is a C stack problem about to happen R
tries to report, but in the process of reporting it
causes a C stack-related crash. Is there something odd with R handling
the situation, or should I look for the cause
of the problem elsewhere ?

Thanks,


L.

The only way to overcome the problem I can find is to tweak the
R_CStackLimit with:

R_CStackLimit = (uintptr_t) -1;

The question I am having now is: what are the implications of doing
so, that is what are the potential problems ?

The R-extensions manual says:
" Stack-checking can be disabled by seting R_CStackLimit (uintptr_t)-1, but
it is better to
if possible set appropriate values. (What these are and how to
determine them are OS-specific,
and the stack size limit may differ for secondary threads. If you have
a choice of stack size, at
least 8Mb is recommended.)"
I am not very sure of how an appropriate value can be determined.

I looked around, and the JRI (Java/R Interface) is just disabling
stack checking for example.


Thanks,


Laurent


2008/6/30 Laurent Gautier <lgautier at gmail.com>:
> Dear list,
>
> I am having an embedded R, dying with
> *** stack smashing detected *** in one specific case.
>
> My code is such as I evaluate R expression with C code like
>
> res = R_tryEval(expr, env, &error);
>
> and in case of error, get the error message (usually sucessfully) with
> code like below:
>
> SEXP geterrmessage = findVar(install("geterrmessage"),
R_BaseNamespace);
> PROTECT(expr = allocVector(LANGSXP, 1));
> SETCAR(expr, geterrmessage);
> PROTECT(res = Rf_eval(expr, R_GlobalEnv));
> // ---> *** stack smashing detected ***
>
> The call to Rf_eval does not return as the stack smashing error stops the
show.
>
> Tracing with gdb where the problem occurs, I follow the path:
> eval -> R_CheckStack in main/error.c -> errorcall in main/error.c
>
>
> In errorcall, I narrow down the origin of the problem around the lines:
> 658    va_start(ap, format);
> 659    verrorcall_dflt(call, format, ap);
> 660    va_end(ap);
> and add a breakpoint on each one of those lines.
>
>
> My gdb session till the stack smashing crashing is then looking like:
>
> Breakpoint 7, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87698 "C
> stack usage is too close to the limit")
>    at errors.c:658
> 658         va_start(ap, format);
> (gdb) p ap
> $2 = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x41c22f60,
> reg_save_area = 0x41c22e70}}
> (gdb) p format
> $3 = 0x7f4dbdc87698 "C stack usage is too close to the limit"
> (gdb) continue
> Continuing.
>
> Breakpoint 8, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87698 "C
> stack usage is too close to the limit")
>    at errors.c:659
> 659         verrorcall_dflt(call, format, ap);
> (gdb) continue
> Continuing.
>
> Breakpoint 6, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406
"%s")
> at errors.c:648
> 648         if (R_ErrorHook != NULL) {
> (gdb) continue
> Continuing.
>
> Breakpoint 7, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406
"%s")
> at errors.c:658
> 658         va_start(ap, format);
> (gdb) continue
> Continuing.
>
> Breakpoint 8, Rf_errorcall (call=0x828ef8, format=0x7f4dbdc87406
"%s")
> at errors.c:659
> 659         verrorcall_dflt(call, format, ap);
> (gdb) continue
> Continuing.
> *** stack smashing detected ***: /usr/bin/python terminated
>
>
> My understanding is that there is a C stack problem about to happen R
> tries to report, but in the process of reporting it
> causes a C stack-related crash. Is there something odd with R handling
> the situation, or should I look for the cause
> of the problem elsewhere ?
>
> Thanks,
>
>
> L.
>