similar to: Slow IO: was [R] naive question

Hi, To speed up reading of large (few million lines) CSV files I am writing custom read functions (in C). By timing various approaches I figured out that one of the bottlenecks in reading character fields is the mkChar() function which on each call incurs a lot of garbage-collection-related overhead. I wonder if there is a "vectorized" version of mkChar, say mkChar2(char **, int

Hi, I am trying to figure a way to allocate a string SEXP so that gc() won't ever collect it. Here is a little bit of a background. Suppose I want to write a .Call-callable function that upon each call returns the same value, say mkChar("foo"): SEXP getFoo() { return mkChar("foo"); } The above implementation doesn't take advantage of the fact that

Hi, As was discussed earlier in another thread and as documented in R-exts .Call() should not be interruptible by Ctrl-C. However the following code, which spends most of its time inside mkChar, turned out to be interruptible on RH-7.3 R-1.8.1 gcc-2.96: #include <Rinternals.h> #include <R.h> SEXP foo0(const SEXP nSexp) { int i, n; SEXP resSexp; if (!isInteger(nSexp))

Hi, I am looking for an efficient way of skipping big chunks of lines on a connection (not necessarily at the beginning of the file). One way is to use read lines, e.g. readLines(1e6), but a) this incurs the overhead of construction of the return char vector and b) has a (fairly remote) potential to blow up the memory. Another way would be to use scan(), e.g. scan(con, skip=1e6, nmax=0)

I have a 100Mb comma-separated file, and R takes several minutes to read it (via read.table()). This is R 1.9.0 on a linux box with a couple gigabytes of RAM. I am conjecturing that R is gc-ing, so maybe there is some command-line arg I can give it to convince it that I have a lot of space, or?! Thanks! Igor

Hi, Is it possible to modify an object in the parent.env (as opposed to re-bind)? Here is what I tried: > x = 1:3 # try to modify the first element of x from within a new environment > local(get("x", parent.env(environment()))[1] <- NA) Error in eval(expr, envir, enclos) : Target of assignment expands to non-language object # On the other hand retrieval works just fine >

Hi, I thought it would be convenient if the check.names argument to read.table, which currently can only be TRUE/FALSE, could take a function value as well. If the function is supplied it should be used instead of the default make.names. Here is an example where it can come in handy. I tend to keep my data in coma-separated files with a header line. The header line is prefixed with a comment

Hi, What is the reason for seq.Date to require the 'by' argument and not to default it to 1 in the example below? > seq(from=as.Date("1996-01-01"), to=as.Date("1996-12-01")) Error in seq.Date(from = as.Date("1996-01-01"), to = as.Date("1996-12-01")) : exactly two of `to', `by' and `length.out' / `along.with' must be specified

Dear R-devel, Some time ago I started a thread that boiled down to clean-up actions after non-local exits in R, see below. I wonder if there has been any progress on this? R-ext 2.6.1 doesn't say much on the subject. How, for example, do people deal with a situation where their C (C++) function opens a file and then receives a signal or longjump-s on error(), how do they make sure the

Hi, I tried to write the dim method for the list class, but R doesn't seem to dispatch to it: > dim.list = function(x) c(length(x[[1]]), length(x)) > dim(list(1)) NULL > dim.list(list(1)) [1] 1 1 What is the correct way of registering dim.list with .Primitive("dim")? Thanks, Vadim [[alternative HTML version deleted]]

Hi, It seems that a formal function argument can not default to an "outer" variable of the same name: > x <- "foo" > ff <- function(x=x) x > ff() Error in ff() : recursive default argument reference > Is this intentional? Why? I use R-1.9.1. Thanks, Vadim

Hi, I am building r-patch from the sources (rsync-ed today). make check produced the following message: running tests of Internet and socket functions expect some differences make[3]: Entering directory `/usr/evahome/vograno/R/tests' running code in 'internet.R' ... OK comparing 'internet.Rout' to './internet.Rout.save' ...18c18 < Content type `text/plain;

Dear R-Users, I have a data file with timestamps and I wanted to use POSIXct time data type to represent the respective column. I played around with the type and found a couple of issues: * there seems to be no direct way of reading datetimes into a variable. Let's say this is my file "1992-02-27 23:03:20 PST" "1992-02-27 22:29:56 PST" "1992-01-14 01:03:30 PST"

Hi, This is not an R question per ce, but I feel like this is a right community to ask it. As a part of our work we run a lot of non-interactive computational jobs. To increase the throughput we would like to distribute the load over the entire network and we are looking at Linux network as a platform. Ideally we would like to be able to submit a job to the network, rather than to a computer, and

Hi, I am writing C++ functions that are to be called via .Call() interface. I'd been using error() (from R.h) to return to R if there is an error, but then I realized that this might be not safe as supposedly error() doesn't throw an exception and therefore some destructors do not get called and some memory may leak. Here is a simple example extern "C" void foo() { string

Hi, I am looking for a way to extract an "interesting" part of the match to a regular expression. For example the pattern "[./](*.)" matches a substring that begins with either "." or "/" followed by anything. I am interested in this "anything" w/o the "." or "/" prefix. If say I match the pattern against "abc/foo" I

Hi, I have two data sources. One records time in PST time zone, the other in GMT. I want to compute the difference between the two, but don't see how. Here is an example where I compute time difference between identical times each (meant to be) relative to its time zone. > as.POSIXlt("2000-05-10 10:15:00", "PST") - as.POSIXlt("2000-05-10 10:15:00",

Hi, Don't know if it belongs to r-devel or r-help, but since I am planning to alter some of R's internal code I am sending it here. The existing implementation of the sample() function, when the optional 'prob' argument is given, is quite inefficient. The complexity is O(sampleSize * universeSize), see ProbSampleReplace() and ProbSampleNoReplace() in random.c. This makes the

Hi, As far as I can tell data.frame class adds two features to those of lists: * matrix structure via [,] and [,]<- operators (well, I know these are actually "["(i, j, ...), not "[,]"). * row names attribute. It seems that the overhead of the support for the row names, both computational and RAM-wise, is rather non-trivial. I frequently subscript from a data.frame,

Hi, Is it possible to redirect the staff that normally goes to the R console window into a file. sink() does this for sdterr and stdout. But I need something that redirects "everything" that appear in the console window (including the top-level commands). I want to achieve the same effect as that of the following command under sh: R < foo.R &> foo.Rt only I want the