"Jens Oehlschlägel"
2007-Aug-03 13:19 UTC
[R] How to properly finalize external pointers?
Dear R .Call() insiders,
Can someone enlighten me how to properly finalize external pointers in C code
(R-2.5.1 win)? What is the relation between R_ClearExternalPtr and the finalizer
set in R_RegisterCFinalizer?
I succeeded registering a finalizer that works when an R object containing an
external pointer is garbage collected. However, I have some difficulties
figuring out how to do that in an explicit closing function.
I observed that
- calling R_ClearExternalPtr does not trigger the finalizer and is dangerous
because it removes the pointer before the finalizer needs it at
garbage-collection-time (no finalization = memory leak)
- calling the finalizer directly ensures finalization but now the finalizer is
called twice (once again at garbage collection time, and I did not find
documentation how to unregister the finalizer)
- It works to delete the SEXP external pointer object but only if not calling
R_ClearExternalPtr (but why then do we need it?) Furthermore it is unfortunate
to delay freeing the external pointers memory if I know during runtime that it
can be done immediately.
Shouldn't R_ClearExternalPtr call the finalizer and then unregister it?
(this would also work when removing the SEXP external pointer object is
difficult because it was handed over to the closing function directly as a
parameter)
Best regards
Jens Oehlschl?gel
// C-code
static void rindex_finalize(SEXP extPtr){
pINT ptr = R_ExternalPtrAddr(extPtr);
if(ptr){
Free(ptr);
Rprintf("finalized\n");
}else{
Rprintf("nothing to finalize\n");
}
return;
}
SEXP rindex_open(
SEXP Sn
){
int i,n = INTEGER(Sn)[0];
pINT ptr = Calloc(sizeof(INT)*n, INT);
SEXP extPtr, ret;
for (i=0;i<n;i++){
ptr[i] = i;
}
extPtr = R_MakeExternalPtr(ptr, install("Rindex_extPtr"),
R_NilValue);
R_RegisterCFinalizer(extPtr, rindex_finalize);
PROTECT(ret = allocVector(VECSXP, 1));
SET_VECTOR_ELT(ret,0,extPtr);
UNPROTECT(1);
return ret;
}
SEXP rindex_close(
SEXP obj
){
int i, n= 10;
SEXP ret, extPtr=VECTOR_ELT(obj, 0);
pINT p, ptr = R_ExternalPtrAddr(extPtr);
PROTECT(ret = allocVector(INTSXP, n));
p = INTEGER(ret);
for (i=0;i<n;i++){
Rprintf("ptri=%d\n",ptr[i]);
p[i] = ptr[i];
}
/* this does finalize immediately but at next garbage collection again
rindex_finalize(extPtr);
*/
/* this must not called otherwise the pointer is gone at garbage collection
time
R_ClearExternalPtr(extPtr);
*/
/* this triggers the finalizer but only at next garbage collection */
SET_VECTOR_ELT(obj,0,R_NilValue);
UNPROTECT(1);
return ret;
}
# R-Code
initRindex <- function(){
dyn.load(file.path(.libPaths(), "rindex", "libs",
paste("rindex", .Platform$dynlib.ext, sep = "")))
}
doneRindex <- function(){
dyn.unload(file.path(.libPaths(), "rindex", "libs",
paste("rindex", .Platform$dynlib.ext, sep = "")))
}
openRindex <- function(n=10){
.Call("rindex_open", as.integer(n))
}
closeRindex <- function(extPtr){
.Call("rindex_close", extPtr)
}
if (FALSE){
# now try it
require(rindex)
initRindex()
extPtr <- openRindex()
extPtr
extPtr2 <- closeRindex(extPtr)
extPtr2
gc()
extPtr <- openRindex()
extPtr2 <- extPtr
extPtr2
rm(extPtr)
gc()
extPtr2
rm(extPtr2)
gc()
extPtr2
doneRindex()
}
> version
_
platform i386-pc-mingw32
arch i386
os mingw32
system i386, mingw32
status
major 2
minor 5.1
year 2007
month 06
day 27
svn rev 42083
language R
version.string R version 2.5.1 (2007-06-27)
--
Psssst! Schon vom neuen GMX MultiMessenger geh?rt?
Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
On 8/3/2007 9:19 AM, Jens Oehlschl?gel wrote:> Dear R .Call() insiders, > > Can someone enlighten me how to properly finalize external pointers in C code (R-2.5.1 win)? What is the relation between R_ClearExternalPtr and the finalizer set in R_RegisterCFinalizer? > > I succeeded registering a finalizer that works when an R object containing an external pointer is garbage collected. However, I have some difficulties figuring out how to do that in an explicit closing function. > > I observed that > - calling R_ClearExternalPtr does not trigger the finalizer and is dangerous because it removes the pointer before the finalizer needs it at garbage-collection-time (no finalization = memory leak) > - calling the finalizer directly ensures finalization but now the finalizer is called twice (once again at garbage collection time, and I did not find documentation how to unregister the finalizer) > - It works to delete the SEXP external pointer object but only if not calling R_ClearExternalPtr (but why then do we need it?) Furthermore it is unfortunate to delay freeing the external pointers memory if I know during runtime that it can be done immediately. > > Shouldn't R_ClearExternalPtr call the finalizer and then unregister it? (this would also work when removing the SEXP external pointer object is difficult because it was handed over to the closing function directly as a parameter)I think we want R_ClearExternalPtr to work even if the finalizer would fail (e.g. to clean up when there was an error when trying to build the external object). So I'd suggest that when you want to get rid of an external object immediately, you call the finalizer explicitly, then call R_ClearExternalPtr. The documentation doesn't address the question of whether this will clear the registered finalizer so I don't know if you'll get a second call to the finalizer during garbage collection, but even if you do, isn't it easy enough to do nothing when you see the null ptr, as you do below? By the way, questions about programming at this level are better asked in the R-devel group. Duncan Murdoch> Best regards > > > Jens Oehlschl?gel > > > // C-code > > static void rindex_finalize(SEXP extPtr){ > pINT ptr = R_ExternalPtrAddr(extPtr); > if(ptr){ > Free(ptr); > Rprintf("finalized\n"); > }else{ > Rprintf("nothing to finalize\n"); > } > return; > } > > > SEXP rindex_open( > SEXP Sn > ){ > int i,n = INTEGER(Sn)[0]; > pINT ptr = Calloc(sizeof(INT)*n, INT); > SEXP extPtr, ret; > for (i=0;i<n;i++){ > ptr[i] = i; > } > extPtr = R_MakeExternalPtr(ptr, install("Rindex_extPtr"), R_NilValue); > R_RegisterCFinalizer(extPtr, rindex_finalize); > > PROTECT(ret = allocVector(VECSXP, 1)); > SET_VECTOR_ELT(ret,0,extPtr); > UNPROTECT(1); > return ret; > } > > SEXP rindex_close( > SEXP obj > ){ > int i, n= 10; > SEXP ret, extPtr=VECTOR_ELT(obj, 0); > pINT p, ptr = R_ExternalPtrAddr(extPtr); > > PROTECT(ret = allocVector(INTSXP, n)); > p = INTEGER(ret); > for (i=0;i<n;i++){ > Rprintf("ptri=%d\n",ptr[i]); > p[i] = ptr[i]; > } > > /* this does finalize immediately but at next garbage collection again > rindex_finalize(extPtr); > */ > > /* this must not called otherwise the pointer is gone at garbage collection time > R_ClearExternalPtr(extPtr); > */ > > /* this triggers the finalizer but only at next garbage collection */ > SET_VECTOR_ELT(obj,0,R_NilValue); > > UNPROTECT(1); > return ret; > } > > > # R-Code > initRindex <- function(){ > dyn.load(file.path(.libPaths(), "rindex", "libs", paste("rindex", .Platform$dynlib.ext, sep = ""))) > } > > doneRindex <- function(){ > dyn.unload(file.path(.libPaths(), "rindex", "libs", paste("rindex", .Platform$dynlib.ext, sep = ""))) > } > > > openRindex <- function(n=10){ > .Call("rindex_open", as.integer(n)) > } > > closeRindex <- function(extPtr){ > .Call("rindex_close", extPtr) > } > > if (FALSE){ > # now try it > require(rindex) > > initRindex() > > extPtr <- openRindex() > extPtr > extPtr2 <- closeRindex(extPtr) > extPtr2 > gc() > > extPtr <- openRindex() > extPtr2 <- extPtr > extPtr2 > rm(extPtr) > gc() > extPtr2 > rm(extPtr2) > gc() > extPtr2 > > doneRindex() > } > > >> version > _ > platform i386-pc-mingw32 > arch i386 > os mingw32 > system i386, mingw32 > status > major 2 > minor 5.1 > year 2007 > month 06 > day 27 > svn rev 42083 > language R > version.string R version 2.5.1 (2007-06-27) > > >
On Fri, 3 Aug 2007, "Jens Oehlschl?gel" wrote:> Dear R .Call() insiders, > > Can someone enlighten me how to properly finalize external pointers in C code (R-2.5.1 win)? What is the relation between R_ClearExternalPtr and the finalizer set in R_RegisterCFinalizer?There is none. R_ClearExternalPtr(s) is equivalent to R_SetExternalPtrAddr(s, NULL);> I succeeded registering a finalizer that works when an R object containing an external pointer is garbage collected. However, I have some difficulties figuring out how to do that in an explicit closing function. > > I observed that > - calling R_ClearExternalPtr does not trigger the finalizer and is dangerous because it removes the pointer before the finalizer needs it at garbage-collection-time (no finalization = memory leak) > - calling the finalizer directly ensures finalization but now the finalizer is called twice (once again at garbage collection time, and I did not find documentation how to unregister the finalizer) > - It works to delete the SEXP external pointer object but only if not calling R_ClearExternalPtr (but why then do we need it?) Furthermore it is unfortunate to delay freeing the external pointers memory if I know during runtime that it can be done immediately. > > Shouldn't R_ClearExternalPtr call the finalizer and then unregister it? (this would also work when removing the SEXP external pointer object is difficult because it was handed over to the closing function directly as a parameter)No. External pointers are often used without finalization. Usually one would chose either an explicit release or finalization via GC but not both. If you want to use explicit release with finalization as a backup you need to write your C code accordingly. I would do this by setting the pointer to NULL with R_ClearExternalPtr in the explicit close routine and checking whether it has already been set to NULL in the finalizer. Best, luke> > Best regards > > > Jens Oehlschl?gel > > > // C-code > > static void rindex_finalize(SEXP extPtr){ > pINT ptr = R_ExternalPtrAddr(extPtr); > if(ptr){ > Free(ptr); > Rprintf("finalized\n"); > }else{ > Rprintf("nothing to finalize\n"); > } > return; > } > > > SEXP rindex_open( > SEXP Sn > ){ > int i,n = INTEGER(Sn)[0]; > pINT ptr = Calloc(sizeof(INT)*n, INT); > SEXP extPtr, ret; > for (i=0;i<n;i++){ > ptr[i] = i; > } > extPtr = R_MakeExternalPtr(ptr, install("Rindex_extPtr"), R_NilValue); > R_RegisterCFinalizer(extPtr, rindex_finalize); > > PROTECT(ret = allocVector(VECSXP, 1)); > SET_VECTOR_ELT(ret,0,extPtr); > UNPROTECT(1); > return ret; > } > > SEXP rindex_close( > SEXP obj > ){ > int i, n= 10; > SEXP ret, extPtr=VECTOR_ELT(obj, 0); > pINT p, ptr = R_ExternalPtrAddr(extPtr); > > PROTECT(ret = allocVector(INTSXP, n)); > p = INTEGER(ret); > for (i=0;i<n;i++){ > Rprintf("ptri=%d\n",ptr[i]); > p[i] = ptr[i]; > } > > /* this does finalize immediately but at next garbage collection again > rindex_finalize(extPtr); > */ > > /* this must not called otherwise the pointer is gone at garbage collection time > R_ClearExternalPtr(extPtr); > */ > > /* this triggers the finalizer but only at next garbage collection */ > SET_VECTOR_ELT(obj,0,R_NilValue); > > UNPROTECT(1); > return ret; > } > > > # R-Code > initRindex <- function(){ > dyn.load(file.path(.libPaths(), "rindex", "libs", paste("rindex", .Platform$dynlib.ext, sep = ""))) > } > > doneRindex <- function(){ > dyn.unload(file.path(.libPaths(), "rindex", "libs", paste("rindex", .Platform$dynlib.ext, sep = ""))) > } > > > openRindex <- function(n=10){ > .Call("rindex_open", as.integer(n)) > } > > closeRindex <- function(extPtr){ > .Call("rindex_close", extPtr) > } > > if (FALSE){ > # now try it > require(rindex) > > initRindex() > > extPtr <- openRindex() > extPtr > extPtr2 <- closeRindex(extPtr) > extPtr2 > gc() > > extPtr <- openRindex() > extPtr2 <- extPtr > extPtr2 > rm(extPtr) > gc() > extPtr2 > rm(extPtr2) > gc() > extPtr2 > > doneRindex() > } > > >> version > _ > platform i386-pc-mingw32 > arch i386 > os mingw32 > system i386, mingw32 > status > major 2 > minor 5.1 > year 2007 > month 06 > day 27 > svn rev 42083 > language R > version.string R version 2.5.1 (2007-06-27) > > > >-- Luke Tierney Chair, Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke at stat.uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu