similar to: RData File Specification?

Currently, I load the RData file then ls() and str(). But loading the file takes too long if the file is big. Most of the time, I only interested what the variables are in the the file and the attributes of the variables (like if it is a data.frame, matrix, what are the colnames/rownames, etc.) I'm wondering if there is any facility in R to help me avoid loading the whole file.

Are the functions 'R_Unserialize' and 'R_InitFileInPStream' allowed to be used in R packages? I guess I'm just not clear on the implications of this comment in 'Rinternals.h': /* The connection interface is not yet available to packages. To allow limited use of connection pointers this defines the opaque pointer type. */ I have a function in the

Hi, sorry if this question is trivial or unclear, this is my first venture into mixed C/R programming (I am reasonably experienced in each separately). I am trying to write a serialization function for a format called typedbytes, which is used as an interchange format in Hadoop circles. Since I would need to serialize according to the internal R format many small R objects I looked at the c

Hi, While debugging a network server I'm developing I noticed something unusual - call to unserialize() resulted in an error about loading a namespace. I was a bit taken back by this - why should unserializing an object cause a namespace lookup? Are there any other side-effects of unserialize() that I should be cautious about? I've been digging through the R_Unserialize() call, I

On 8/6/2009 4:11 PM, Marilyn & Rich Short wrote: > Hello, > > I'm having a problem in R. The R GUI is crashing with a message to > contact Microsoft for the solution. I've contacted Microsoft and they > are of no help. Below is a distilled set of code that will cause the > crash. As you will see, there are two do-loops within which is a "load" >

I noticed an issue where readChar does not return the correct value after a call to readline. It appears that readChar is not aware of the buffering, so it reads from the end of the buffer, rather than the current position in the file. This is a significant change of behavior from R-3.4.4. Below is a test case that I used to home in on the problem. --- p<-"test2.txt"

Hello, I am trying to read in elements out of a very large binary file ... the total file is 4 gigs. I want to select rows out of the file, and the current procedure I run works but is prohibitively slow (takes more than a day to run and still won't complete). Is there any faster way to accomplish this? My current procedure looks like this: readHH <- function(file_name,

Hi, Here's a patch to the readChar manual page (R-trunk as of today) that better clarifies readChar's return value. It could use some work as I'd also like to add some text about using nchar() to find the length of the string that readchar() returns, but I'm unsure which of type="bytes" or type="chars" to mention. Is it type="chars"? Index:

On Fri, 26 Jan 2024, Michael Chirico wrote: > I am curious why readLines() has a default (n=-1L) to read the full > file while readChar() has no default for nchars= (i.e., readChar(file) > is an error). Is there a technical reason for this? > > I often[1] see code like paste(readLines(f), collapse="\n") which > would be better served by readChar(), especially given

My opinion is that the proposed feature would be greatly appreciated by users. I had always wondered if I was the only one doing paste(readLines(f), collapse="\n") all the time. It would be great to have the proposed, more straightforward way to read the whole file as a string: readChar("my_file.txt", -1) or even better readChar("my_file.txt") Thanks for your detailed

I am curious why readLines() has a default (n=-1L) to read the full file while readChar() has no default for nchars= (i.e., readChar(file) is an error). Is there a technical reason for this? I often[1] see code like paste(readLines(f), collapse="\n") which would be better served by readChar(), especially given issues with the global string cache I've come across[2]. But lacking the

Tomas, Thank you for the explanation. I see in the documentation: "These functions are intended to be used with binary-mode connections." So I see how using it on a text connection is undefined, and not a bug. An error or warning when attempting to use a it on a text connection would be helpful considering how the behavior has changed in R-3.5. On Tue, May 29, 2018 at 3:09 AM, Tomas

I have R code to read a binary header file, consisting of several readBin() and readChar() statements. I am currently using version R-2.2.1 (on x86_64, RH EL4) and have received the following error Error in readChar(fid, n = 1) : invalid UTF-8 input in readChar() This is strange because (a) I was able to read in this file successfully in previous versions of R (circa 2005) and (b) it

Hi, I was trying to have R read files faster with readChar(). That was before I noticed that readChar() is not that bad! In any case, below I suggest a few simple changes that will make readChar slightly faster. I followed readChar(useBytes=T), and tried to identify all O(N) operations, where N is the size of the file. The assumption is that for LARGE files we want to avoid any O(N) operations,

Is this possible? I've tried both writeChar() and writeBin() to no avail. My goal is to serialize(ascii=FALSE) an object to a connection but determine the size of the serialized object before hand: sobject <- serialize(object,NULL,ascii=FALSE) len <- nchar(sobject) # # run some code here to notify listener on other end of connection # how many bytes I'm getting ready to send #

Hi all, Since I was having several problems trying to work with a large matrix I started to use the package R.huge but I'm having the following problem > x<-FileByteMatrix("covtype.data",nrow=581012,ncol=55) Error: cannot allocate vector of size 770.8 Mb In addition: Warning messages: 1: Reached total allocation of 447Mb: see help(memory.size) in: readChar(con = con, nchar =

Full_Name: Hugh C. Pumphrey Version: 1.4.1 OS: Linux (Debian Woody) Submission from: (NULL) (129.215.133.170) The function readChar() appears to have some type of problem with memory allocation. I don't know if "memory leak" is the correct term but if one uses readChar() many times, the R binary grows in size until it eats all your memory and swap space. The code enclosed below

Dear All: I am trying to load an R data set, but I got the following message. Please see below. The file is there. setwd("F:/Fall_2017/5-STA574/2-Notes/1-R/1-R_new/chapter4-Entering_Data") datahs0csv <- read.table("hs0.csv", header=T, sep=",") attach(datahs0csv) detach(datahs0csv) rm(list=ls()) Then I tried to reload the data, but I got this error message. I

Hi, I am looking for a way to search a file for position of some expression, from within R. My current code: sha1Pos = gregexpr("<sha1>", readChar(filename, file.info(filename)$size))[[1]] Works fine for small files, but text files I will be working with might get up to Gb range, so I was trying to accomplish the same without loading the whole file into R. I realize this is

Hi R gurus, I'm trying to use a ReSTful web service from within R. Specifically, I need to make HTTP PUT requests. I'm able to make the request and that goes well enough, but I'm having trouble properly consuming the HTTP response. The problem comes in when I'm trying to parse out the response body. I get the length of the response body from the Content-Length header. I then try