search for: dataptr_or_nul

...ffer with a copy of the data. *That buffer is expected to be safe to destructively modify, shuffle, etc though I don't know if we are actually doing that anywhere. As such, if I understand his C++ correctly, that Get_region method is not safe and shouldn't be used. The other point is that Dataptr_or_null is not actually *guaranteed *not to allocate. The default method returns NULL, but we have no way of preventing an allocation in a user-defined method, and probably (?) no easy way of detecting that it is occurring before it causes a bug. That said, Romain is correct that when you are writing Data...

...I copied my C++ iter macro by mistake(and you can see an explicit type casting). Here is the macro definition from R_exts/Itermacros.h #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ strt, nfull, expr) do { \ * const** etype *px = DATAPTR_OR_NULL(sx); * \ if (px != NULL) { \ R_xlen_t __ibr_n__ = strt + nfull; \ R_xlen_t nb = __ibr_n__; \ for (R_xlen_t idx = strt; idx < __ibr_n__;...

...upport ALTREP > generic vectors here are some notes: > > The main challenge I am aware of (there might be others): Allowing > DATAPTR to return a writable pointer would be too dangerous because > the GC write barrier needs to see all mutations. So it would be best > if Dataptr and Dataptr_or_null methods were not allowed to be > defined. The default methods in altrep.c should do the right think. > > A reasonable name for the abstract class would be 'altlist'. > > 'altrep' methods that a class can provide: > > Unserialize or UnserializeEX > Seri...

...roposing is not about changing the internal representation, but rather about how we design and use the ALTREP API. I might do not state the workarounds clearly as English is not my first language. Please let me explain them again in detail. 1. Update the existing R functions. When the ALTREP API Dataptr_or_null returns NULL, use get_element instead(or as best as we can). I have seen this pattern for some R functions, but somehow there are still some functions left that do not follow this rule. For example, print function will blindly call Dataptr (It even did not call Dataptr_or_null first) and forces me...

...a minimum example: C++ code ``` #include "R_ext/Itermacros.h" #define GET_REGION_BUFSIZE 2 //Redefine the macro since C++ is not happy with the implicit type conversion #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ strt, nfull, expr) do { \ const etype *px = (etype*)DATAPTR_OR_NULL(sx); \ if (px != NULL) { \ R_xlen_t __ibr_n__ = strt + nfull; \ R_xlen_t nb = __ibr_n__; \ for (R_xlen_t idx = strt; idx < __ibr_n__; idx += nb) { \ expr \ } \ } \ else ITERATE_BY_REGION_PARTIAL0(sx, px, idx, nb, etype, vtype, \ strt, nfull, expr); \ } while (0) // [[Rcpp::expo...

...termacros.h" >> #define GET_REGION_BUFSIZE 2 >> //Redefine the macro since C++ is not happy with the implicit type >> conversion >> #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ >> ? strt, nfull, expr) do { \ >> const etype *px = (etype*)DATAPTR_OR_NULL(sx); \ >> if (px != NULL) { \ >> ??? R_xlen_t __ibr_n__ = strt + nfull; \ >> ??? R_xlen_t nb = __ibr_n__; \ >> ??? for (R_xlen_t idx = strt; idx < __ibr_n__; idx += nb) { \ >> expr \ >> ???? } \ >> } \ >> else ITERATE_BY_REGION_PARTIAL0(sx, px, id...

...to try to create a patch to support ALTREP generic vectors here are some notes: The main challenge I am aware of (there might be others): Allowing DATAPTR to return a writable pointer would be too dangerous because the GC write barrier needs to see all mutations. So it would be best if Dataptr and Dataptr_or_null methods were not allowed to be defined. The default methods in altrep.c should do the right think. A reasonable name for the abstract class would be 'altlist'. 'altrep' methods that a class can provide: Unserialize or UnserializeEX Serialized_state Duplicate or Duplicat...

...ing the internal representation, but rather > about how we design and use the ALTREP API. > > I might do not state the workarounds clearly as English is not my first > language. Please let me explain them again in detail. > > 1. Update the existing R functions. When the ALTREP API Dataptr_or_null > returns NULL, use get_element instead(or as best as we can). I have seen > this pattern for some R functions, but somehow there are still some > functions left that do not follow this rule. For example, print function > will blindly call Dataptr (It even did not call Dataptr_or_null...

Hi Kylie, Is it a list with only numerics in it? (I only see REALSXPs there, but obviously inspect isn't showing all of them). If so, you could load it up into one big vector and then also keep partitioning information around. Bioconductor does this (see ?IRanges::CompressedList ). The potential benefit here being that the underlying large vector could then be a big out-of-memory altrep. How

...ake(and you can see an explicit type casting). Here is > the macro definition from R_exts/Itermacros.h > > #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ > > strt, nfull, expr) do { \ > > * const** etype *px = DATAPTR_OR_NULL(sx); * \ > > if (px != NULL) { \ > > R_xlen_t __ibr_n__ = strt + nfull; \ > > R_xlen_t nb = __ibr_n__; \ > > for (R_xle...

...; #include "R_ext/Itermacros.h" > #define GET_REGION_BUFSIZE 2 > //Redefine the macro since C++ is not happy with the implicit type > conversion > #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ > strt, nfull, expr) do { \ > const etype *px = (etype*)DATAPTR_OR_NULL(sx); \ > if (px != NULL) { \ > R_xlen_t __ibr_n__ = strt + nfull; \ > R_xlen_t nb = __ibr_n__; \ > for (R_xlen_t idx = strt; idx < __ibr_n__; idx += nb) { \ > expr \ > } \ > } \ > else ITERATE_BY_REGION_PARTIAL0(sx, px, idx, nb, etype, vtype, \ > strt...

Hi devel team, I'm working on C/C++ level ALTREP compatibility for a package. The package previously used pointers to access the data of a SEXP, so it would not work for some ALTREP objects which do not have a pointer. I plan to rewrite the code and use functions like get_elt, get_region, and get_subset to access the values of a vector, so I have a few questions for ALTREP: 1. Since an

Hello from Bioconductor, I'm developing a package to share R objects across clusters using boost library. The concept is similar to mmap package: https://cran.r-project.org/web/packages/mmap/index.html . However, I have a problem when I was trying to write Dataptr_method for the alternative string. Based on my understanding, the return value of the Dataptr_method function should be a vector

...; #define GET_REGION_BUFSIZE 2 > >> //Redefine the macro since C++ is not happy with the implicit type > >> conversion > >> #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ > >> strt, nfull, expr) do { \ > >> const etype *px = (etype*)DATAPTR_OR_NULL(sx); \ > >> if (px != NULL) { \ > >> R_xlen_t __ibr_n__ = strt + nfull; \ > >> R_xlen_t nb = __ibr_n__; \ > >> for (R_xlen_t idx = strt; idx < __ibr_n__; idx += nb) { \ > >> expr \ > >> } \ > >> } \ > >> el...

...ems not happy with it. Is it possible to add an explicit type casting so that it can be compatible with both language? #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype, \ strt, nfull, expr) do { \ *const etype *px = (const** etype *)DATAPTR_OR_NULL(sx); * \ if (px != NULL) { \ R_xlen_t __ibr_n__ = strt + nfull; \ R_xlen_t nb = __ibr_n__; \ for (R_xlen_t idx = strt; idx < __ibr_n__; idx += nb) { \...