similar to: Any plans for ALTREP lists (VECSXP)?

Hi Kylie, Is it a list with only numerics in it? (I only see REALSXPs there, but obviously inspect isn't showing all of them). If so, you could load it up into one big vector and then also keep partitioning information around. Bioconductor does this (see ?IRanges::CompressedList ). The potential benefit here being that the underlying large vector could then be a big out-of-memory altrep. How

I can work on this. Thanks Luke. ~G On Wed, Jul 24, 2019 at 8:25 AM Tierney, Luke <luke-tierney at uiowa.edu> wrote: > If one of you wanted to try to create a patch to support ALTREP > generic vectors here are some notes: > > The main challenge I am aware of (there might be others): Allowing > DATAPTR to return a writable pointer would be too dangerous because > the GC

Hi Kylie, As an alternative in the short term, you could consider deriving from S4Vector's List class, implementing the getListElement() method to lazily create the objects. Michael On Tue, Jul 23, 2019 at 9:09 AM Bemis, Kylie <k.bemis at northeastern.edu> wrote: > > Hello, > > I was wondering if there were any plans for ALTREP lists (VECSXP)? > > It seems to me that

Hello, I?m experimenting with ALTREP and was wondering if there is a preferred way to create an ALTREP wrapper vector without using .Internal(wrap_meta(?)), which R CMD check doesn?t like since it uses an .Internal() function. I was trying to create a factor that used an ALTREP integer, but attempting to set the class and levels attributes always ended up duplicating and materializing the

If one of you wanted to try to create a patch to support ALTREP generic vectors here are some notes: The main challenge I am aware of (there might be others): Allowing DATAPTR to return a writable pointer would be too dangerous because the GC write barrier needs to see all mutations. So it would be best if Dataptr and Dataptr_or_null methods were not allowed to be defined. The default methods in

Hi Jiefei and Kylie, Great to see people engaging with the ALTREP framework and identifying places we may need more tooling. Comments inline. On Thu, Jul 18, 2019 at 12:22 PM King Jiefei <szwjf08 at gmail.com> wrote: > > If that is the case and you are 100% sure the reference number should be 1 > for your variable *y*, my solution is to call *SET_NAMED *in C++ to reset > the

Hi Kylie, For your question, I don't think a wrapper can completely solve your problem. The duplication occurs since your variable y has more than 1 reference number( Please see highlighted), so even you have a wrapper, any changes on the value of the wrapper still can trigger the duplication. > .Internal(inspect(y)) > @7fb0ce78c0f0 13 INTSXP g0c0 *[NAM(7)]* matter vector (mode=3,

On Mon, 21 Jan 2019, Martin Maechler wrote: >>>>>> Travers Ching >>>>>> on Tue, 15 Jan 2019 12:50:45 -0800 writes: > > > I have a toy alt-rep string package that generates > > randomly seeded strings. example: library(altstringisode) > > x <- altrandomStrings(1e8) head(x) [1] > >

I have a toy alt-rep string package that generates randomly seeded strings. example: library(altstringisode) x <- altrandomStrings(1e8) head(x) [1] "2PN0bdwPY7CA8M06zVKEkhHgZVgtV1" "5PN2qmWqBlQ9wQj99nsQzldVI5ZuGX" ... etc object.size(1e8) Object.size will call the set_altstring_Elt_method for every single element, materializing (slowly) every element of the vector. This

Hello Luke and Gabriel, Thank you very much for your quick responses. The explanation of STDVEC is very helpful and I appreciate it! For the wrapper, I have a few new questions. 1. Like Luke said a mutable object is not possible. However, I noticed that there is one extra argument *deep* in the function duplicate. I've googled all the available documentation for ALTREP but I did not find

A useful extension of ALTREP is having two new string methods which return the number of characters of a given string element and to return a substring of an element. Having these methods would allow retrieving these values without needing to create a CHARSXP for the full element data, which could potentially be costly for long elements. For example say you have an ALTREP altstring vector where

I ran into strange behavior when removing names. Two ways of removing names: i <- rep(1:4, length.out=20000) k <- c(a=1, b=2, c=3, d=4) x1 <- unname(k[i]) x2 <- k[i] x2 <- unname(x2) Are they identical? identical(x1,x2) # TRUE but no identical(serialize(x1,NULL),serialize(x2,NULL)) # FALSE But problem is with serialization type 3, cause:

Hi devel team, I'm working on C/C++ level ALTREP compatibility for a package. The package previously used pointers to access the data of a SEXP, so it would not work for some ALTREP objects which do not have a pointer. I plan to rewrite the code and use functions like get_elt, get_region, and get_subset to access the values of a vector, so I have a few questions for ALTREP: 1. Since an

On 1/22/19 6:17 PM, Kevin Ushey wrote: > I think that object.size() is most commonly used to answer the question, > "what R objects are consuming the most memory currently in my R session?" > and for that reason I think returning the size of the internal > representations of objects (for e.g. ALTREP objects; unevaluated promises) > is the right default behavior. I

> On Friday, May 15, 2020, 12:13:04 PM EDT, Dirk Eddelbuettel <edd at debian.org> wrote: > On 15 May 2020 at 15:41, Martin Maechler wrote: > | <whining> > | > |??? Why does nobody anymore? help R development by working with > |??? "R-devel", or at least then the alpha, beta and the "RC" > |??? (Release Candidate) versions that we release daily

Sorry for post a lot of things, for the first part of code, I copied my C++ iter macro by mistake(and you can see an explicit type casting). Here is the macro definition from R_exts/Itermacros.h #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ strt, nfull, expr) do { \ * const** etype *px = DATAPTR_OR_NULL(sx); *

Inside of the anyNA() function, it will use the legacy any(is.na()) code if x is an OBJECT(). If x is a vector of POSIXct, it will be an OBJECT(), but it is also TYPEOF(x) == REALSXP. Therefore, it will skip the faster ITERATE_BY_REGION, which is typically 5x faster in my testing. Is the OBJECT() condition really necessary, or could it be moved after the switch() for the individual TYPEOF(x)

Hi all, I'm wondering if there is any way to define a `SET_ELT` function for an ALTREP class? I see there are ` ALTINTEGER_SET_ELT` etc. functions exported in Rinternal.h, but there is no corresponding ALTREP APIs to define them. The only way to set the value of an ALTREP is through a pointer, which will require that the ALTREP data is in memory. Is it on purpose? Will there be any plan to

Hello, I have encountered two bugs when using ALTREP APIs. 1. STDVEC_DATAPTR >From RInternal.h file it has a comment: /* ALTREP support */ > void *(STDVEC_DATAPTR)(SEXP x); However, this comment might not be true, the easiest way to verify it is to define a C++ function: void C_testFunc(SEXP a) > { > STDVEC_DATAPTR(a); > } and call it in R via > a=1:10 > >

Hi Bob, Thanks for sending around the link to that. It looks mostly right and looks like a useful onramp. There are a few things to watch out for though (I've cc'ed Romain so he's aware of these comments). @romain I hope you taake the following comments as they are intended, as help rather than attacks. The largest issue I see is that the contract for Get_region is that it *populates