R devel - Jan 2010 - R-devel Digest, Vol 83, Issue 2

[Disclaimer: what is below reflects my understanding from reading the R 
source, others will correct where deemed necessary]

On 1/2/10 12:00 PM, r-devel-request at r-project.org
wrote:
>
> Hello,
>
> We are currently making lots of changes to Rcpp (see the open Rcpp
> mailing list if interested [1] in the details).
>
> We are now using [2] R_PreserveObject and R_ReleaseObject to manage
> garbage collection instead of the PROTECT/UNPROTECT dance. This seems to
> work well, but I was wondering if there was documentation about it.
The most precise technical documentation is in memory.c
PROTECT is an alias for Rf_protect, itself an alias for
SEXP protect(SEXP s);
and uses a stack (R_PPStack) to store protected objects.
> In particular, if we preserve the same SEXP twice (or more), should we
> implement some sort of reference counting ?
This depends on the requirements for your system.

For example, in rpy2 I added a reference counting layer(*) because I 
wanted to allow several Python objects to share the same underlying R 
object, but that's not currently(*) counting how many times an object 
should be freed.
(*: imperfect, but currently doing a very decent job - details upon 
request).

That kind of feature could be provided by R's C-level API, since this 
could be seen of general use as well as give an opportunity to improve 
the performances of the R_PreservedObject/R_ReleaseObject duo whenever a 
lot of objects are protected and/or external code is 
protecting/releasing objects through a FIFO proxy.

> Reading the source (below, from memory.c) I think not, but some
> confirmation would help.
I understand the code in memory.c like an object preserved twice needs 
to be freed twice: R_PreciousList is just a (linked) list, and 
"R_PreserveObject(object)" adds the object to the beginning of the
list
while "R_ReleaseObject(object)" removes the first "object"
found from
the list.


> void R_PreserveObject(SEXP object)
> {
>       R_PreciousList = CONS(object, R_PreciousList);
> }
>
> static SEXP RecursiveRelease(SEXP object, SEXP list)
> {
>       if (!isNull(list)) {
> 	if (object == CAR(list))
> 	    return CDR(list);
> 	else
> 	    CDR(list) = RecursiveRelease(object, CDR(list));
>       }
>       return list;
> }
>
> void R_ReleaseObject(SEXP object)
> {
>       R_PreciousList =  RecursiveRelease(object, R_PreciousList);
> }
>
>
> I'd also be interested if there is some ideas on the relative
efficiency
> of the preserve/release mechanism compared to PROTECT/UNPROTECT.
PROTECT/UNPROTECT is trading granularity for speed. It is a stack with 
only tow operations possible:
- push 1 object into the stack
- pull (unprotect) N last objects from the stack


HTH,


L.



> Thanks,
>
> Romain
>
> [1]http://lists.r-forge.r-project.org/pipermail/rcpp-devel/
> [2]
>
http://r-forge.r-project.org/plugins/scmsvn/viewcvs.php/pkg/src/RObject.cpp?rev=255&root=rcpp&view=markup
>
> -- Romain Francois Professional R Enthusiast +33(0) 6 28 91 30 30
> http://romainfrancois.blog.free.fr |- http://tr.im/IW9B : C++ exceptions
> at the R level |- http://tr.im/IlMh : CPP package: exposing C++ objects
> `- http://tr.im/HlX9 : new package : bibtex

Thanks.

On 01/02/2010 05:36 PM, Laurent Gautier wrote:
>
> [Disclaimer: what is below reflects my understanding from reading the R
> source, others will correct where deemed necessary]
>
> On 1/2/10 12:00 PM, r-devel-request at r-project.org wrote:
>>
>> Hello,
>>
>> We are currently making lots of changes to Rcpp (see the open Rcpp
>> mailing list if interested [1] in the details).
>>
>> We are now using [2] R_PreserveObject and R_ReleaseObject to manage
>> garbage collection instead of the PROTECT/UNPROTECT dance. This seems
to
>> work well, but I was wondering if there was documentation about it.
>
> The most precise technical documentation is in memory.c
> PROTECT is an alias for Rf_protect, itself an alias for
> SEXP protect(SEXP s);
> and uses a stack (R_PPStack) to store protected objects.
>
>> In particular, if we preserve the same SEXP twice (or more), should we
>> implement some sort of reference counting ?
>
> This depends on the requirements for your system.
We wrap up SEXP into a C++ class that in particular manages itself 
preserving and releasing the object to garbage collection.
> For example, in rpy2 I added a reference counting layer(*) because I
> wanted to allow several Python objects to share the same underlying R
> object, but that's not currently(*) counting how many times an object
> should be freed.
> (*: imperfect, but currently doing a very decent job - details upon
> request).
>
> That kind of feature could be provided by R's C-level API, since this
> could be seen of general use as well as give an opportunity to improve
> the performances of the R_PreservedObject/R_ReleaseObject duo whenever a
> lot of objects are protected and/or external code is
> protecting/releasing objects through a FIFO proxy.
>
>
>> Reading the source (below, from memory.c) I think not, but some
>> confirmation would help.
>
> I understand the code in memory.c like an object preserved twice needs
> to be freed twice: R_PreciousList is just a (linked) list, and
> "R_PreserveObject(object)" adds the object to the beginning of
the list
> while "R_ReleaseObject(object)" removes the first
"object" found from
> the list.
That's what I understand of it too. It fits nicely into C++ assignment 
operator and copy constructor.
>> void R_PreserveObject(SEXP object)
>> {
>> R_PreciousList = CONS(object, R_PreciousList);
>> }
>>
>> static SEXP RecursiveRelease(SEXP object, SEXP list)
>> {
>> if (!isNull(list)) {
>> if (object == CAR(list))
>> return CDR(list);
>> else
>> CDR(list) = RecursiveRelease(object, CDR(list));
>> }
>> return list;
>> }
>>
>> void R_ReleaseObject(SEXP object)
>> {
>> R_PreciousList = RecursiveRelease(object, R_PreciousList);
>> }
>>
>>
>> I'd also be interested if there is some ideas on the relative
efficiency
>> of the preserve/release mechanism compared to PROTECT/UNPROTECT.
>
> PROTECT/UNPROTECT is trading granularity for speed. It is a stack with
> only tow operations possible:
> - push 1 object into the stack
> - pull (unprotect) N last objects from the stack
>
>
> HTH,
>
>
> L.
>
>
>
>
>> Thanks,
>>
>> Romain
>>
>> [1]http://lists.r-forge.r-project.org/pipermail/rcpp-devel/
>> [2]
>>
http://r-forge.r-project.org/plugins/scmsvn/viewcvs.php/pkg/src/RObject.cpp?rev=255&root=rcpp&view=markup
-- 
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://tr.im/IW9B : C++ exceptions at the R level
|- http://tr.im/IlMh : CPP package: exposing C++ objects
`- http://tr.im/HlX9 : new package : bibtex

On 02/01/2010 11:36 AM, Laurent Gautier wrote:
> [Disclaimer: what is below reflects my understanding from reading the R 
> source, others will correct where deemed necessary]
> 
> On 1/2/10 12:00 PM, r-devel-request at r-project.org wrote:
>> Hello,
>>
>> We are currently making lots of changes to Rcpp (see the open Rcpp
>> mailing list if interested [1] in the details).
>>
>> We are now using [2] R_PreserveObject and R_ReleaseObject to manage
>> garbage collection instead of the PROTECT/UNPROTECT dance. This seems
to
>> work well, but I was wondering if there was documentation about it.
> 
> The most precise technical documentation is in memory.c
> PROTECT is an alias for Rf_protect, itself an alias for
> SEXP protect(SEXP s);
> and uses a stack (R_PPStack) to store protected objects.
> 
>> In particular, if we preserve the same SEXP twice (or more), should we
>> implement some sort of reference counting ?
> 
> This depends on the requirements for your system.
> 
> For example, in rpy2 I added a reference counting layer(*) because I 
> wanted to allow several Python objects to share the same underlying R 
> object, but that's not currently(*) counting how many times an object 
> should be freed.
> (*: imperfect, but currently doing a very decent job - details upon 
> request).
> 
> That kind of feature could be provided by R's C-level API, since this 
> could be seen of general use as well as give an opportunity to improve 
> the performances of the R_PreservedObject/R_ReleaseObject duo whenever a 
> lot of objects are protected and/or external code is 
> protecting/releasing objects through a FIFO proxy.
> 
> 
>> Reading the source (below, from memory.c) I think not, but some
>> confirmation would help.
> 
> I understand the code in memory.c like an object preserved twice needs 
> to be freed twice: R_PreciousList is just a (linked) list, and 
> "R_PreserveObject(object)" adds the object to the beginning of
the list
> while "R_ReleaseObject(object)" removes the first
"object" found from
> the list.
> 
> 
> 
>> void R_PreserveObject(SEXP object)
>> {
>>       R_PreciousList = CONS(object, R_PreciousList);
>> }
>>
>> static SEXP RecursiveRelease(SEXP object, SEXP list)
>> {
>>       if (!isNull(list)) {
>> 	if (object == CAR(list))
>> 	    return CDR(list);
>> 	else
>> 	    CDR(list) = RecursiveRelease(object, CDR(list));
>>       }
>>       return list;
>> }
>>
>> void R_ReleaseObject(SEXP object)
>> {
>>       R_PreciousList =  RecursiveRelease(object, R_PreciousList);
>> }
>>
>>
>> I'd also be interested if there is some ideas on the relative
efficiency
>> of the preserve/release mechanism compared to PROTECT/UNPROTECT.
> 
> PROTECT/UNPROTECT is trading granularity for speed. It is a stack with 
> only tow operations possible:
> - push 1 object into the stack
> - pull (unprotect) N last objects from the stack
UNPROTECT_PTR is also possible, which does a linear search through the 
stack and unprotects something possibly deep within it.  There is also 
REPROTECT which allows you to replace an entry within the stack.

I would guess that UNPROTECT_PTR is more efficient than RecursiveRelease 
because it doesn't use so much stack space when it needs to go deep into 
the stack to release, but it is possible the compiler recognizes the 
tail recursion and RecursiveRelease is implemented efficiently.  In that 
case it could be more efficient than UNPROTECT_PTR, which has to move 
all the other entries down to fill the newly vacated space.  Really the 
only reliable way to answer efficiency questions like this is to try
both ways and see which works better in your application.

Another possibility is to maintain your own list or environment of 
objects, and just protect/preserve the list as a whole.

Duncan Murdoch
> 
> 
> HTH,
> 
> 
> L.
> 
> 
> 
> 
>> Thanks,
>>
>> Romain
>>
>> [1]http://lists.r-forge.r-project.org/pipermail/rcpp-devel/
>> [2]
>>
http://r-forge.r-project.org/plugins/scmsvn/viewcvs.php/pkg/src/RObject.cpp?rev=255&root=rcpp&view=markup
>>
>> -- Romain Francois Professional R Enthusiast +33(0) 6 28 91 30 30
>> http://romainfrancois.blog.free.fr |- http://tr.im/IW9B : C++
exceptions
>> at the R level |- http://tr.im/IlMh : CPP package: exposing C++ objects
>> `- http://tr.im/HlX9 : new package : bibtex
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

On 1/2/10 5:50 PM, Romain Francois wrote:
> Thanks.
>
> On 01/02/2010 05:36 PM, Laurent Gautier wrote:
>>
>> [Disclaimer: what is below reflects my understanding from reading the R
>> source, others will correct where deemed necessary]
>>
>> On 1/2/10 12:00 PM, r-devel-request at r-project.org wrote:
(...)
>>
>>> In particular, if we preserve the same SEXP twice (or more), should
we
>>> implement some sort of reference counting ?
>>
>> This depends on the requirements for your system.
>
> We wrap up SEXP into a C++ class that in particular manages itself
> preserving and releasing the object to garbage collection.
If you do not allow several C++ instances to share the same SEXP, you 
should not need reference counting. In the case of rpy2, this was made 
necessary for allowing inheritance (as nested calls to constructors can 
cause a transient sharing of the given SEXP across several Python objects).


(...)



L.