similar to: ALTREP: Design concept of alternative string

Hi Gabriel, Thanks for your explanation, I totally understand that it is almost impossible to change the data structure of STRSXP. However, what I'm proposing is not about changing the internal representation, but rather about how we design and use the ALTREP API. I might do not state the workarounds clearly as English is not my first language. Please let me explain them again in detail.

Hi Jiefei, The issue here is that while the memory consequences of what you're describing may be true, this is simply how R handles character vector (what you're calling string) values internally. It doesn't actually have anything to do with ALTREP. Standard character vector SEXPs have an array of CHARSXP pointers in their payload (what is returned by DATAPTR) as well. As far as I

On Fri, 10 May 2019, ??? wrote: > Hi Gabriel, > > Thanks for your explanation, I totally understand that it is almost > impossible to change the data structure of STRSXP. However, what I'm > proposing is not about changing the internal representation, but rather > about how we design and use the ALTREP API. > > I might do not state the workarounds clearly as English is

Hello, I?m experimenting with ALTREP and was wondering if there is a preferred way to create an ALTREP wrapper vector without using .Internal(wrap_meta(?)), which R CMD check doesn?t like since it uses an .Internal() function. I was trying to create a factor that used an ALTREP integer, but attempting to set the class and levels attributes always ended up duplicating and materializing the

Hi Jiefei and Kylie, Great to see people engaging with the ALTREP framework and identifying places we may need more tooling. Comments inline. On Thu, Jul 18, 2019 at 12:22 PM King Jiefei <szwjf08 at gmail.com> wrote: > > If that is the case and you are 100% sure the reference number should be 1 > for your variable *y*, my solution is to call *SET_NAMED *in C++ to reset > the

Hi Bob, Thanks for sending around the link to that. It looks mostly right and looks like a useful onramp. There are a few things to watch out for though (I've cc'ed Romain so he's aware of these comments). @romain I hope you taake the following comments as they are intended, as help rather than attacks. The largest issue I see is that the contract for Get_region is that it *populates

Sorry for post a lot of things, for the first part of code, I copied my C++ iter macro by mistake(and you can see an explicit type casting). Here is the macro definition from R_exts/Itermacros.h #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ strt, nfull, expr) do { \ * const** etype *px = DATAPTR_OR_NULL(sx); *

Benjamin, You happened to send a link which points to the OP's own package :) I think Jiefei would like to know how one can "officially" determine if an arbitrary ALTERP object belongs to a class that he owns. Regards, Denes On 10/19/20 10:22 AM, Benjamin Christoffersen wrote: > It seems as if you can you use the ALTREP macro as done in this > package:

Hi devel team, I'm working on C/C++ level ALTREP compatibility for a package. The package previously used pointers to access the data of a SEXP, so it would not work for some ALTREP objects which do not have a pointer. I plan to rewrite the code and use functions like get_elt, get_region, and get_subset to access the values of a vector, so I have a few questions for ALTREP: 1. Since an

Hi all, I would like to determine if an ALTREP object is from my package, I see there is a function `ALTREP_CLASS` defined in RInternal.h but its return value is neither a `R_altrep_class_t` object nor an STRSXP representing a class name. I do not know how to correctly use it. Any suggestions? Thanks, Jiefei [[alternative HTML version deleted]]

Thank Denes for the clarification, glad to see my package got one citation from my own question:) Also, thank Benjamin for sending many useful documents. Actually, the question is related to the SharedObject package that Benjamin has pointed to. I wanna avoid sharing an object that has already been shared, so I need to check whether an object is an ALTREP that is defined in my package. Since the

Hi, I am curious about whether there exist thread-safe functions in `Rinternals.h`. I know that R is single-threaded designed, but for the simple and straightforward functions like `DATAPTR` and `INTEGER_GET_REGION`, are these functions safe to call in a multi-thread environment? Best, Jiefei [[alternative HTML version deleted]]

I have a toy alt-rep string package that generates randomly seeded strings. example: library(altstringisode) x <- altrandomStrings(1e8) head(x) [1] "2PN0bdwPY7CA8M06zVKEkhHgZVgtV1" "5PN2qmWqBlQ9wQj99nsQzldVI5ZuGX" ... etc object.size(1e8) Object.size will call the set_altstring_Elt_method for every single element, materializing (slowly) every element of the vector. This

Jiefei, Beyond the general response that Luke gave, to be a bit more specific to what you said, DATAPTR and INTEGER_GET_REGION involve ALTREP method execution (for ALTREP objects, obviously) so even they are not as simple and straightforward as they were a couple years ago. They should not (any longer) be thought of as being guaranteed to be essentially bare metal data retrieval from memory.

A useful extension of ALTREP is having two new string methods which return the number of characters of a given string element and to return a substring of an element. Having these methods would allow retrieving these values without needing to create a CHARSXP for the full element data, which could potentially be costly for long elements. For example say you have an ALTREP altstring vector where

Hello Luke and Gabriel, Thank you very much for your quick responses. The explanation of STDVEC is very helpful and I appreciate it! For the wrapper, I have a few new questions. 1. Like Luke said a mutable object is not possible. However, I noticed that there is one extra argument *deep* in the function duplicate. I've googled all the available documentation for ALTREP but I did not find

Hi all, The coercion function defined for the ALTREP object will not be called by R when an assignment operation implicitly introduces coercion for a large ALTREP object. For example, If I create a vector of length 10, the ALTREP coercion function seems to work fine. ``` > x <- 1:10 > y <- wrap_altrep(x) > .Internal(inspect(y)) @0x000000001f9271c0 13 INTSXP g0c0 [REF(2)] I am

thanks for clearing that up, so these methods are actually not meant to be exported on Windows and OSX? Some of the ALTREP methods that now use 'attribute_hidden' would be very useful to packages that aim to be ALTREP aware, should the currently (exported) API be considered final? thanks for your time & best, Mark On Tue, Jun 4, 2019 at 6:52 PM Tierney, Luke <luke-tierney at

Hi Gabriel, here is a simple package for reproducing the problem. https://github.com/Jiefei-Wang/testPkg Best, Jiefei On Thu, Oct 8, 2020 at 5:04 AM Gabriel Becker <gabembecker at gmail.com> wrote: > Jiefei, > > Where does the code for your altrep class live? > > Thanks, > ~G > > On Wed, Oct 7, 2020 at 4:25 AM Jiefei Wang <szwjf08 at gmail.com> wrote: >

Hi Gabriel, thanks for your detailed explanation, that definitely clarifies the design choices that were made in setting up the ALTREP framework and I can see how those choices make sure existing code won't break. My specific use-case for wanting to check whether a vector is an ALTREP is the following: the fst package wraps an external C++ library (fstlib, independent from R) that was made