similar to: some questions about R internal SEXP types

Hello, I am writing an R/Go interoperability tool[1] that work similarly to Rcpp; the tool takes packages written in Go and performs the necessary Go type analysis to wrap the Go code with C and R shims that allow the Go code to then be called from R. The system is largely complete (with the exception of having a clean approach to handling generalised attributes in the easy case[2] - the less

The general principle is that R packages are only allowed to use what is documented in the R help (? command) and in Writing R Extensions. The former covers what is allowed from R code in extensions, the latter mostly what is allowed from C code in extensions (with some references to Fortran). If you are implementing a Go interface for writing R packages, such Go interface should thus only

On 9/8/20 11:47 AM, Dan Kortschak wrote: > Thanks, Tomas. > > This is unfortunate. Calling between Go and C is not cheap; the gc > implementation of the Go compiler (as opposed to gccgo) uses different > calling conventions from C and there are checks to ensure that Go > allocated memory pointers do not leak into C code. For this reason I > wanted to avoid these if at all

Hi all, I believe I have found a bug (or perhaps just an oversight) with the ALTREP wrapper classes. The short form of this is that I believe that the wrapper classes need to override the default ALTREP `Extract_subset_method()` with a method that calls `Extract_subset()` on the "wrapped" object. I have a patch prepared here: https://github.com/DavisVaughan/r-source/pull/1 There is

If one of you wanted to try to create a patch to support ALTREP generic vectors here are some notes: The main challenge I am aware of (there might be others): Allowing DATAPTR to return a writable pointer would be too dangerous because the GC write barrier needs to see all mutations. So it would be best if Dataptr and Dataptr_or_null methods were not allowed to be defined. The default methods in

Experimenting with altrep objects and v3 serialization, I discovered a possible bug. Calling DATAPTR on a compact_intseq object returns a pointer to the expanded integer sequence in memory. If you modify this data, the object values appear to be changed. However, if the compact_intseq object is then serialized (with version=3), only the original integer sequence info is written. For example,

I can work on this. Thanks Luke. ~G On Wed, Jul 24, 2019 at 8:25 AM Tierney, Luke <luke-tierney at uiowa.edu> wrote: > If one of you wanted to try to create a patch to support ALTREP > generic vectors here are some notes: > > The main challenge I am aware of (there might be others): Allowing > DATAPTR to return a writable pointer would be too dangerous because > the GC

Hello, I have two related ALTREP questions. It seems like there is no way to assign attributes to an ALTREP vector without using C++ code. To be more specifically, I want to make an ALTREP matrix, I have tried the following R code but none of them work. ``` .Internal(inspect(1:6)) .Internal(inspect(matrix(1:6, 2,3))) .Internal(inspect(as.matrix(1:6))) .Internal(inspect(structure(1:6, dim =

Hi Jiefei, The meanings of the return values for sortedness can be found in RInternals.h, and are as follows: /* ALTREP sorting support */ enum {SORTED_DECR_NA_1ST = -2, SORTED_DECR = -1, UNKNOWN_SORTEDNESS = INT_MIN, /*INT_MIN is NA_INTEGER! */ SORTED_INCR = 1, SORTED_INCR_NA_1ST = 2, KNOWN_UNSORTED = 0}; The default value there is NA_INTEGER (ie INT_MIN),

Hi Andreas, note that with the introduction of ALTREP, as far as I understand, calls as "simple" as DATAPTR can execute arbitrary code (R or native). Even without ALTREP, if you execute user-provided R code via Rf_eval and such on some custom thread, you may end up executing native code of some package, which may assume it is executed only from the R main thread. Could you (1)

Hi Gabriel, Thanks for your answer and future update plan. Somehow this email has been delayed for a week, so there might be a wired reply from me saying that I have found the answer from the R source code, it was sent from me last week. Hopefully, this reply will not cost another week to post:) As a side note, I like the idea that defining a macro for sortedness, and I can see why we can only

For using R's .Call interface to C functions, all the examples I've seen have the C function return type SEXP. Why? What does R actually do with this return type? What happens if I *don't* return a SEXP? Reason I ask, is I've written some R code which allocates two long lists, and then calls a C function with .Call. My C code writes to those two pre-allocated lists, thus, I

Jiefei, Inline. On Thu, May 16, 2019 at 2:30 PM ??? <szwjf08 at gmail.com> wrote: > Hello Luke and Gabriel, > > Thank you very much for your quick responses. The explanation of STDVEC is > very helpful and I appreciate it! For the wrapper, I have a few new > questions. > > > 1. Like Luke said a mutable object is not possible. However, I noticed > that there is

Stepan, Andreas gave a lot more thought into what you question in your reply. His question was how you can avoid what you where proposing and have proper threading under safe conditions. Having dealt with this before, I think Andreas' write up is pretty much the most complete analysis I have seen. I'd wait for Luke to chime in as the ultimate authority if he gets to it. The

On 9/8/20 1:13 PM, Dan Kortschak wrote: > On Tue, 2020-09-08 at 12:08 +0200, Tomas Kalibera wrote: >> I am not sure if I understand correctly, but if you were accessing >> directly the memory of SEXPs from Go implementation instead of >> calling >> through exported access functions documented in WRE, that would be a >> really bad idea. Of course fine for research

Hi Jiefei, Thanks for tryingout the ALTREP stuff and letting us know how it is going. That said I don't think either of these are bugs, per se, but rather a misunderstanding of the API. Details inline. On Thu, May 16, 2019 at 11:57 AM ??? <szwjf08 at gmail.com> wrote: > Hello, > > I have encountered two bugs when using ALTREP APIs. > > 1. STDVEC_DATAPTR > > From

I found a bug in one of the fourteen speed patches I posted, namely in patch-vec-subset. I've fixed this (I now see one does need to duplicate index vectors sometimes, though one can avoid it most of the time). I also split this patch in two, since it really has two different and independent parts. The patch-vec-subset patch now has only some straightforward (locally-checkable) speedups for

Hi Mark, So depending pretty strongly on what you mean by "ALTREP aware", packages aren't necessarily supposed to be ALTREP aware. What I mean by this is that as of right now, ALTREP objects are designed to be interacted with by non-ALTREP-implementing package code, *more-or-less *exactly as standard (non-AR) SEXPs are: via the published C API. The more or less comes from the fact

Hi Gabriel, I have tried the macro and found a small issue, it seems like the macro is written in C and does an implicit type conversion(const void * to const int *), see below. While it is allowed in C, C++ seems not happy with it. Is it possible to add an explicit type casting so that it can be compatible with both language? #define ITERATE_BY_REGION_PARTIAL(sx, px, idx, nb, etype, vtype,

Not sure if you're using just C++ or Rcpp for C++ access but https://purrple.cat/blog/2018/10/14/altrep-and-cpp/ has some tips on using C++ w/ALTREP. > On Sep 23, 2019, at 3:17 PM, Wang Jiefei <szwjf08 at gmail.com> wrote: > > Sorry for post a lot of things, for the first part of code, I copied my C++ > iter macro by mistake(and you can see an explicit type casting). Here