R devel - Aug 2018 - SIGSEGV in R_RunWeakRefFinalizer, object allocated with Rcpp

Hi all,

I'm not sure if I'm not supposed to do the following (the dyn.unload
part, I mean) or this could be a bug (in R or Rcpp):

```
Rcpp::sourceCpp(code='
  #include <Rcpp.h>
  class Object {};
  //[[Rcpp::export]]
  SEXP new_object() {
    return Rcpp::XPtr<Object>(new Object());
  }'
)

new_object()
dyn.unload(list.files(tempdir(), ".(so|dll)$", recursive=TRUE,
full.names=TRUE))
gc() # segfault in R_RunWeakRefFinalizer

message("This is not printed")
```

This is the backtrace I get with R 3.5.1:

#0  0x61ec4fd0 in ?? ()
#1  0x6ca1cafc in R_RunWeakRefFinalizer (w=0xc786a98) at memory.c:1393
#2  0x6ca1cdba in RunFinalizers () at memory.c:1459
#3  0x6ca1d024 in R_RunPendingFinalizers () at memory.c:1495
#4  R_gc () at memory.c:2893
#5  do_gc (call=0x1d45b88, op=0x15241d0, args=0x1d454b8,
rho=0x1d45318) at memory.c:2013
#6  0x6c9db60f in bcEval (body=body at entry=0x1d45a88,
rho=rho at entry=0x1d45318, useCache=useCache at entry=TRUE)
    at eval.c:6781
#7  0x6c9ecfb2 in Rf_eval (e=0x1d45a88, rho=0x1d45318) at eval.c:624
#8  0x6c9ee6f1 in R_execClosure (call=call at entry=0x0,
newrho=<optimized out>, sysparent=<optimized out>,
    rho=0x15a3370, arglist=0x18c2498, op=0x1d45968) at eval.c:1773
#9  0x6c9ef605 in Rf_applyClosure (call=0x15a3370,
call at entry=0x1d45828, op=0x18c2498, op at entry=0x1d45968,
    arglist=0x1d45968, rho=rho at entry=0x15a3370,
suppliedvars=0x18c2498) at eval.c:1701
#10 0x6c9ecf78 in Rf_eval (e=e at entry=0x1d45828,
rho=rho at entry=0x15a3370) at eval.c:747
#11 0x6ca11170 in Rf_ReplIteration (rho=0x15a3370, savestack=0,
browselevel=0, state=0x142edec) at main.c:258
#12 0x6ca11567 in R_ReplConsole (rho=<optimized out>, savestack=0,
browselevel=0) at main.c:308
#13 0x6ca11604 in run_Rmainloop () at main.c:1082
#14 0x6ca11700 in Rf_mainloop () at main.c:1089
#15 0x00401836 in AppMain (argc=1, argv=0x15c16f8) at rterm.c:86
#16 0x00401649 in WinMain at 16 (Instance=0x400000, PrevInstance=0x0,
CmdLine=0x1904797 "", CmdShow=10)
    at graphappmain.c:23
#17 0x00402a8d in main ()

Any ideas?

I?aki

R's dyn.unload() will unconditionally unload the given shared object; it 
does not check whether there is any object (external pointer or weak 
reference) with a C finalizer pointing into the space of the shared 
object being unloaded. So it is expected that R will segfault later when 
such finalizer is run.

Currently there is no other way than to handle this on the side of the 
shared library/package, e.g. as Luke does in his simplemmap package. A 
library can declare R_unload_XXX function (see Writing R Extensions) 
that will be called before it is unloaded. So, one can keep say a list 
of objects with finalizers and clear/run the finalizers in R_unload_XXX.

I don't think we could do this automatically on the R's side with the 
current API. I don't think there is a portable way to check if a given C 
pointer is from a given loaded shared object (non-portable way on Linux 
might be scanning the maps in the /proc filesystem). Also, doing this 
for all objects with finalizers when unloading any library would 
probably be slow. Adding some API to aid libraries in doing the 
housekeeping is possible in principle.

So to answer your original question, this could probably be handled in 
Rcpp, but in either case I would not use dyn.unload() in the first 
place. This problem may be just one of many.

Best
Tomas



On 6.8.2018 17:35, I?aki ?car wrote:
> Hi all,
>
> I'm not sure if I'm not supposed to do the following (the
dyn.unload
> part, I mean) or this could be a bug (in R or Rcpp):
>
> ```
> Rcpp::sourceCpp(code='
>    #include <Rcpp.h>
>    class Object {};
>    //[[Rcpp::export]]
>    SEXP new_object() {
>      return Rcpp::XPtr<Object>(new Object());
>    }'
> )
>
> new_object()
> dyn.unload(list.files(tempdir(), ".(so|dll)$", recursive=TRUE,
full.names=TRUE))
> gc() # segfault in R_RunWeakRefFinalizer
>
> message("This is not printed")
> ```
>
> This is the backtrace I get with R 3.5.1:
>
> #0  0x61ec4fd0 in ?? ()
> #1  0x6ca1cafc in R_RunWeakRefFinalizer (w=0xc786a98) at memory.c:1393
> #2  0x6ca1cdba in RunFinalizers () at memory.c:1459
> #3  0x6ca1d024 in R_RunPendingFinalizers () at memory.c:1495
> #4  R_gc () at memory.c:2893
> #5  do_gc (call=0x1d45b88, op=0x15241d0, args=0x1d454b8,
> rho=0x1d45318) at memory.c:2013
> #6  0x6c9db60f in bcEval (body=body at entry=0x1d45a88,
> rho=rho at entry=0x1d45318, useCache=useCache at entry=TRUE)
>      at eval.c:6781
> #7  0x6c9ecfb2 in Rf_eval (e=0x1d45a88, rho=0x1d45318) at eval.c:624
> #8  0x6c9ee6f1 in R_execClosure (call=call at entry=0x0,
> newrho=<optimized out>, sysparent=<optimized out>,
>      rho=0x15a3370, arglist=0x18c2498, op=0x1d45968) at eval.c:1773
> #9  0x6c9ef605 in Rf_applyClosure (call=0x15a3370,
> call at entry=0x1d45828, op=0x18c2498, op at entry=0x1d45968,
>      arglist=0x1d45968, rho=rho at entry=0x15a3370,
> suppliedvars=0x18c2498) at eval.c:1701
> #10 0x6c9ecf78 in Rf_eval (e=e at entry=0x1d45828,
> rho=rho at entry=0x15a3370) at eval.c:747
> #11 0x6ca11170 in Rf_ReplIteration (rho=0x15a3370, savestack=0,
> browselevel=0, state=0x142edec) at main.c:258
> #12 0x6ca11567 in R_ReplConsole (rho=<optimized out>, savestack=0,
> browselevel=0) at main.c:308
> #13 0x6ca11604 in run_Rmainloop () at main.c:1082
> #14 0x6ca11700 in Rf_mainloop () at main.c:1089
> #15 0x00401836 in AppMain (argc=1, argv=0x15c16f8) at rterm.c:86
> #16 0x00401649 in WinMain at 16 (Instance=0x400000, PrevInstance=0x0,
> CmdLine=0x1904797 "", CmdShow=10)
>      at graphappmain.c:23
> #17 0x00402a8d in main ()
>
> Any ideas?
>
> I?aki
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

On 9 August 2018 at 20:37, Tomas Kalibera wrote:
| So to answer your original question, this could probably be handled in 
| Rcpp,

Hm. Why do you say that / what did you have in mind?

Recall that we do not alter SEXPs or introduce additional additional
reference counters -- because we do not think that altering the basic R API
for such calls would be a wise strategy.  So we do more or less what is done
in C for R, with some additional hand-holding which circumvents a number of
common errors.

| but in either case I would not use dyn.unload() in the first 
| place. This problem may be just one of many.

I think I'd second that. I never had much unloading packages or dynamic
libraries and tend to "just say no". Both short-lived processes (ie
via 'r')
as well as long sessions (ie R via ESS, running for weeks) work for my
workflows.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org