R devel - Nov 2017 - Truncating vectors by reference in C-backend

Dear all,

I want to create a function shrinkVector(x) that takes x and truncates 
it to the first element without copy.
With SETLENGTH and SET_TRUELENGTH, I can achieve that.  You can find a 
reproducible example below.
But the memory that was freed is not available for other objects 
afterwards, except if x is a list (VECSXP). Any suggestions?

library(inline)
## define the shrinking function
shrinkVector <- cfunction(signature(x = "ANY"),
                           body = paste0("SETLENGTH(x, 1);",
                                                   "SET_TRUELENGTH(x, 
1);",
                                                   
"return(R_NilValue);"))
## create a large vector that only fits into memory once
x <- 1 : 2e9
## shrink it
shrinkVector(x)
## shrinking seems to have worked
print(length(x) == 1)
# [1] TRUE
print(object.size(x))
# 48 bytes
## but I can't reuse the memory for a large x2:
x2 <- 1 : 2e8
# Error: cannot allocate vector of size 762.9 Mb
## if I remove x, it works
rm(x)
gc()
x2 <- 1 : 2e8

Thank you very much for your help.

Kind regards,
Markus Bonsch

DO NOT DO THIS!!! SETLENGTH and SET_TRUELENGTH are NOT part of the API
and are for internal use only.

Tho proper way to do this is to copy the data you want into a smaller
vector and allow the larger one to be garbage collected.

[VECSXPs are like all other vectors in that they are contigously
allocated by malloc, and there is no mprtable way to return part
a memory region allocated by malloc back to malloc].

There is experimental INTERNAL support for growable vectors that is
managed with INTERNALLY with TRUELENGTH and a bit setting, but this is
NOT, at leat not yet, intended for use by packages. The ALTREP
framework that should be available in the next release might allow
something like this to be done, but care is needed to make sure R's
copying semantics are adhered to.

Best,

luke

On Tue, 21 Nov 2017, mbonsch at posteo.de wrote:
> Dear all,
>
> I want to create a function shrinkVector(x) that takes x and truncates it 
> to the first element without copy.
> With SETLENGTH and SET_TRUELENGTH, I can achieve that.  You can find a 
> reproducible example below.
> But the memory that was freed is not available for other objects 
> afterwards, except if x is a list (VECSXP). Any suggestions?
>
> library(inline)
> ## define the shrinking function
> shrinkVector <- cfunction(signature(x = "ANY"),
>                          body = paste0("SETLENGTH(x, 1);",
>                                                  "SET_TRUELENGTH(x,
1);",
>                                                 
"return(R_NilValue);"))
> ## create a large vector that only fits into memory once
> x <- 1 : 2e9
> ## shrink it
> shrinkVector(x)
> ## shrinking seems to have worked
> print(length(x) == 1)
> # [1] TRUE
> print(object.size(x))
> # 48 bytes
> ## but I can't reuse the memory for a large x2:
> x2 <- 1 : 2e8
> # Error: cannot allocate vector of size 762.9 Mb
> ## if I remove x, it works
> rm(x)
> gc()
> x2 <- 1 : 2e8
>
> Thank you very much for your help.
>
> Kind regards,
> Markus Bonsch
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu