similar to: external equiv to R_serialize()?

I''m currently using ruby-postgres (the compiled interface to PostgreSQL), and I''m seeing a huge performance hit with larger :binary fields in ActiveRecord subclasses owing, it appears, to the use of the bytea field type and the fact that using that datatype necessitates an in-memory "unescape" conversion. The best I can tell from looking at the Postgres Active

Is there a built-in function that handles the problem of bitstreams in postgresql? The problem occurs when the stream contains for example the " character. Postgres will think that the string has ended with that character. Is there a function that changes all such characters into "\[ASCII-number]"? And the other way around when getting the stream back from the database? References:

Hi! I''m trying to insert binary data into a PostgreSQL database where the field is of type ''bytea''. This works fine if a do Base64 encoding but I just want to do character escaping. How do I do this????. What I get is this: ActiveRecord::StatementInvalid in Init_test#save_file ArgumentError: Invalid Ruby/cstring: INSERT INTO

Hi, sorry if this question is trivial or unclear, this is my first venture into mixed C/R programming (I am reasonably experienced in each separately). I am trying to write a serialization function for a format called typedbytes, which is used as an interchange format in Hadoop circles. Since I would need to serialize according to the internal R format many small R objects I looked at the c

Full_Name: Murray Stokely Version: 2.10.1 OS: Linux Submission from: (NULL) (216.239.45.4) The Defn.h header includes limits.h for PATH_MAX and then checks if it hasn't been defined and if not sets something manually. Some of the R code uses PATH_MAX but a lot of other functions in unix/sys-unix.c and main/startup.c just hardcodes a limit of 256 characters. In my environment this is not

Hello, I'm developing a package (lazyvec) that makes full use of the ALTREP framework (R >= 3.6.0). One application of the package is to wrap existing ALTREP vectors in a new ALTREP vector and pass all calls from R to the contained object. The purpose of this is to provide a diagnostic framework for working with ALTREP vectors and show information about internal calls. The package builds

I was trying to find source for optimize and I ran across function (f, interval, ..., lower = min(interval), upper = max(interval), maximum = FALSE, tol = .Machine$double.eps^0.25) { if (maximum) { val <- .Internal(fmin(function(arg) -f(arg, ...), lower, upper, tol)) list(maximum = val, objective = f(val, ...)) } else { val <-

thanks for clearing that up, so these methods are actually not meant to be exported on Windows and OSX? Some of the ALTREP methods that now use 'attribute_hidden' would be very useful to packages that aim to be ALTREP aware, should the currently (exported) API be considered final? thanks for your time & best, Mark On Tue, Jun 4, 2019 at 6:52 PM Tierney, Luke <luke-tierney at

Hi, I've been doing some tests using serialize() to a raw vector: df <- data.frame(runif(50e6,1,10)) ser <- serialize(df,NULL) In this example the data frame and the serialized raw vector occupy ~400MB each, for a total of ~800M. However the memory peak during serialize() is ~1.2GB: $ cat /proc/15155/status |grep Vm ... VmHWM: 1207792 kB VmRSS: 817272 kB We work with very

Currently the function Rstd_Busy() does nothing (src/unix/sys-std.c): void attribute_hidden Rstd_Busy(int which) { } The function is called through a pointer and R interfaces can change this pointer and, thus, use a different function. I don't plan to create a whole new interface to R, but I maintain a package whose aim is to provide a new feature to R when it's running in a

Hello all, I'd like to utilize the R_Sock* functions from R_ext/R-ftp-http.h in my R package. The intent is to use these in conjunction with R_serialize() to store R objects in a remote data store. I'm aware that version 2.2.1 of "Writing R extensions" explains that these may be undocumented and unstable, but I have a couple of questions: 1) are they platform independent?

Hi, Short version of my question: Rf_applyClosure was marked attribute_hidden in Oct 2023, and I am curious why and if there is an alternative interface to it planned. Long version: I have been toying with building a package that makes it easier to do non-standard evaluation directly using promises, rather than wrapping these in a custom type (like e.g. rlang does). The advantage of this

Hello all, I don't know if it is possible, but I would like to use do.call in C code in my package. The function do.call is defined as > do.call function (what, args, quote = FALSE, envir = parent.frame()) { if (!is.list(args)) stop("second argument must be a list") if (quote) { enquote <- function(x) as.call(list(as.name("quote"),

Hi Gabriel, thanks for your detailed explanation, that definitely clarifies the design choices that were made in setting up the ALTREP framework and I can see how those choices make sure existing code won't break. My specific use-case for wanting to check whether a vector is an ALTREP is the following: the fst package wraps an external C++ library (fstlib, independent from R) that was made

Presumably one could stream over the data twice, the first to get the size, without storing the data. Slower but more memory efficient, unless I'm missing something. Michael On Tue, Mar 17, 2015 at 2:03 PM, Simon Urbanek <simon.urbanek at r-project.org> wrote: > Jorge, > > what you propose is not possible because the size of the output is > unknown, that's why a

On Fri, 10 Feb 2023 23:38:55 -0600 Spencer Graves <spencer.graves at prodsyse.com> wrote: > I have a 4.54 GB file that I'm trying to read in chunks using > "scan(..., skip=__)". It works as expected for small values of > "skip" but goes into an infinite loop for "skip=1e11" and similar > large values of skip: I cannot even interrupt it; I

When large calls cause errors R may stall for extended periods.? This is particularly likely to happen with `do.call`, as in this example with a 24 second stall: ??? x <- runif(1e7) ??? system.time(do.call(paste0, list(abs, x)))? # intentional error ??? ## Error in (function (..., collapse = NULL)? : ??? ##?? cannot coerce type 'builtin' to vector of type 'character' ??? ##

I've been inspired to look at the R source code by some strange timing results that I wrote about on my blog at radfordneal.wordpress.com (see the posts on "Speeding up parentheses..." and "Two surprising things...". I discovered that the strange speed advantage of curly brackets over parentheses is partially explained by an inefficiency in the evalList and

Function factor() in the current development version (2009-05-22) guarantees that levels are different character strings. However, they may represent the same decimal number. The following example is derived from a posting by Stavros Macrakis in thread "Match .3 in a sequence" in March nums <- 0.3 + 2e-16 * c(-2,-1,1,2) f <- factor(nums) levels(f) # [1]

Hi, All I have an encountered an issue where the args field is not saved correctly to the database. I encounter an error like this: ActiveRecord::StatementInvalid (RuntimeError: ERROR C22021 M invalid byte sequence for encoding "UTF8": 0xcb3a H This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by