R devel - Aug 2018 - SIGSEGV in R_RunWeakRefFinalizer, object allocated with Rcpp

On 9 August 2018 at 20:37, Tomas Kalibera wrote:
| So to answer your original question, this could probably be handled in 
| Rcpp,

Hm. Why do you say that / what did you have in mind?

Recall that we do not alter SEXPs or introduce additional additional
reference counters -- because we do not think that altering the basic R API
for such calls would be a wise strategy.  So we do more or less what is done
in C for R, with some additional hand-holding which circumvents a number of
common errors.

| but in either case I would not use dyn.unload() in the first 
| place. This problem may be just one of many.

I think I'd second that. I never had much unloading packages or dynamic
libraries and tend to "just say no". Both short-lived processes (ie
via 'r')
as well as long sessions (ie R via ESS, running for weeks) work for my
workflows.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

On 9.8.2018 20:58, Dirk Eddelbuettel wrote:
> On 9 August 2018 at 20:37, Tomas Kalibera wrote:
> | So to answer your original question, this could probably be handled in
> | Rcpp,
>
> Hm. Why do you say that / what did you have in mind?
>
> Recall that we do not alter SEXPs or introduce additional additional
> reference counters -- because we do not think that altering the basic R API
> for such calls would be a wise strategy.  So we do more or less what is
done
> in C for R, with some additional hand-holding which circumvents a number of
> common errors.
Well if you wanted to support unloading - and I am not at all asking for 
that - you could keep R objects (weak references, external pointers) 
with C finalizers in some list and then unconditionally run or clear the 
finalizers in your R_unload, something like Luke does in his example 
package.

https://github.com/ltierney/Rpkg-simplemmap/blob/master/src/simplemmap.c

Best
Tomas
>
> | but in either case I would not use dyn.unload() in the first
> | place. This problem may be just one of many.
>
> I think I'd second that. I never had much unloading packages or dynamic
> libraries and tend to "just say no". Both short-lived processes
(ie via 'r')
> as well as long sessions (ie R via ESS, running for weeks) work for my
> workflows.
>
> Dirk
>

On Thu, 9 Aug 2018, Dirk Eddelbuettel wrote:
>
> On 9 August 2018 at 20:37, Tomas Kalibera wrote:
> | So to answer your original question, this could probably be handled in
> | Rcpp,
>
> Hm. Why do you say that / what did you have in mind?
We say it because it is true. Rcpp registers C finalizers and running
them after unloading will segfault. For now it would be better for Rcpp
(and everyone else) to explicitly discourage unloading as it is
unreliable on many levels.

What Rcpp could do to avoid segfaulting is to keep a weak list of all
objects to which it attaches C finalizers and arrange for those to be
cleaned up in an R_unload_<dllname> routine. Not clear it is worth the
trouble. At the R level we could provide more support for this since
we already have a weak list of objects with finalizers, but again not
clear it is worth the trouble.

Best,

luke
> Recall that we do not alter SEXPs or introduce additional additional
> reference counters -- because we do not think that altering the basic R API
> for such calls would be a wise strategy.  So we do more or less what is
done
> in C for R, with some additional hand-holding which circumvents a number of
> common errors.
>
> | but in either case I would not use dyn.unload() in the first
> | place. This problem may be just one of many.
>
> I think I'd second that. I never had much unloading packages or dynamic
> libraries and tend to "just say no". Both short-lived processes
(ie via 'r')
> as well as long sessions (ie R via ESS, running for weeks) work for my
> workflows.
>
> Dirk
>
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

On 9 August 2018 at 14:13, luke-tierney at uiowa.edu wrote:
| On Thu, 9 Aug 2018, Dirk Eddelbuettel wrote:
| 
| >
| > On 9 August 2018 at 20:37, Tomas Kalibera wrote:
| > | So to answer your original question, this could probably be handled in
| > | Rcpp,
| >
| > Hm. Why do you say that / what did you have in mind?
| 
| We say it because it is true. Rcpp registers C finalizers and running
| them after unloading will segfault. For now it would be better for Rcpp
| (and everyone else) to explicitly discourage unloading as it is
| unreliable on many levels.
| 
| What Rcpp could do to avoid segfaulting is to keep a weak list of all
| objects to which it attaches C finalizers and arrange for those to be
| cleaned up in an R_unload_<dllname> routine. Not clear it is worth the
| trouble. At the R level we could provide more support for this since
| we already have a weak list of objects with finalizers, but again not
| clear it is worth the trouble.

Ah, noted, thanks for the explainer.

As I remain in camp "don't try unloading" I won't make this a
priority
myself, but as always open to credible PRs. Could start with an issue ticket
and discussion if anybody is so inclined.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Thanks, Tomas, Luke, for the clarifications. Then, I have another question.

But first, let me introduce how I ended up here, because obviously I
just don't go around dyn.unloading things that I've just compiled. I
was testing a package with valgrind. Everything ok, no leaks. Great.
But I'm always suspicious (probably unjustifiably) of all the memory
that is reported as "still reachable", so I wanted to check whether
there was any difference if I detach(unload=TRUE) the package after
all the tests.

In a nutshell, I ended up discovering that the following code:

```
library(simmer)
simmer() # allocates a C++ object, as in my initial example
detach("package:simmer", unload=TRUE)
```

segfaults on Windows, but not on Linux (then I built the example in my
initial email to confirm it wasn't simmer's fault). So given that,
from your explanation, I should expect a segfault here, the question
is: what on Earth does (or does not) R on Linux do to avoid
segfaulting compared to Windows? :) And a corolary would be, can't R
on Windows do the same?

Regards,
I?aki

El jue., 9 ago. 2018 a las 21:13, <luke-tierney at uiowa.edu>
escribi?:
>
> On Thu, 9 Aug 2018, Dirk Eddelbuettel wrote:
>
> >
> > On 9 August 2018 at 20:37, Tomas Kalibera wrote:
> > | So to answer your original question, this could probably be handled
in
> > | Rcpp,
> >
> > Hm. Why do you say that / what did you have in mind?
>
> We say it because it is true. Rcpp registers C finalizers and running
> them after unloading will segfault. For now it would be better for Rcpp
> (and everyone else) to explicitly discourage unloading as it is
> unreliable on many levels.
>
> What Rcpp could do to avoid segfaulting is to keep a weak list of all
> objects to which it attaches C finalizers and arrange for those to be
> cleaned up in an R_unload_<dllname> routine. Not clear it is worth
the
> trouble. At the R level we could provide more support for this since
> we already have a weak list of objects with finalizers, but again not
> clear it is worth the trouble.
>
> Best,
>
> luke
>
> > Recall that we do not alter SEXPs or introduce additional additional
> > reference counters -- because we do not think that altering the basic
R API
> > for such calls would be a wise strategy.  So we do more or less what
is done
> > in C for R, with some additional hand-holding which circumvents a
number of
> > common errors.
> >
> > | but in either case I would not use dyn.unload() in the first
> > | place. This problem may be just one of many.
> >
> > I think I'd second that. I never had much unloading packages or
dynamic
> > libraries and tend to "just say no". Both short-lived
processes (ie via 'r')
> > as well as long sessions (ie R via ESS, running for weeks) work for my
> > workflows.
> >
> > Dirk
> >
> >
>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa                  Phone:             319-335-3386
> Department of Statistics and        Fax:               319-335-3017
>     Actuarial Science
> 241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
> Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu


--
I?aki ?car
http://www.enchufa2.es
@Enchufa2