similar to: why is nrow() so slow?

dear R experts--- I am experimenting with multicore processing, so far with pretty disappointing results. Here is my simple example: A <- 100000 randvalues <- abs(rnorm(A)) minfn <- function( x, i ) { log(abs(x))+x^3+i/A+randvalues[i] } ?## an arbitrary function ARGV <- commandArgs(trailingOnly=TRUE) if (ARGV[1] == "do-onecore") { ?library(foreach) ?discard <-

dear R experts: ?apologies for all my speed and memory questions. ?I have a bet with my coauthors that I can make R reasonably efficient through R-appropriate programming techniques. this is not just for kicks, but for work. for benchmarking, my [3 year old] Mac Pro has 2.8GHz Xeons, 16GB of RAM, and R 2.13.1. right now, it seems that 'split()' is why I am losing my bet. ?(split is an

Dear R wizards: is there a clean way to assign to elements in a list? what I would like to do, in pseudo R+perl notation is f <- function(a,b) list(a+b,a-b) (c,d) <- f(1,2) and have c be assigned 1+2 and d be assigned 1-2. right now, I use the clunky x <- f(1,2) c <- x[[1]] d <- x[[2]] rm(x) which seems awful. is there a nicer syntax? regards, /iaw ---- Ivo Welch

This email is intended for R users that are not that familiar with R internals and are searching google about how to speed up R. Despite common misperception, R is not slow when it comes to iterative access. R is fast when it comes to matrices. R is very slow when it comes to iterative access into data frames. Such access occurs when a user uses "data$varname[index]", which is a very

Dear R Wizards---is it possible to get R to show its current call stack (sys.calls()) upon an error abort? I don't use ESS for execution, and it is often not obvious how to locate how I triggered an error in an R internal function. Seeing the call stack would make this easier. (right now, I sprinkle "cat" statements everywhere, just to locate the line where the error appears.) Of

dear R experts: has someone written a function that returns the results of by() as a data frame? ??of course, this can work only if the output of the function that is an argument to by() is a numerical vector. presumably, what is now names(byobject) would become a column in the data frame, and the by object's list elements would become columns. it's a little bit like flattening the by()

dear R wizards:? I am wrestling with reshape.? I have a long data set that I want to convert into a wide data set, in which rows are firms and columns are years. > summary(rin) firm fyear sim1 Min. :1004.00 Min. :1964.0 Min. : -1.00000 1st Qu.:1010.00 1st Qu.:1979.0 1st Qu.: -0.14334 Median :1016.00 Median :1986.0 Median : 0.00116 Mean

dear R wizards: I have a large data frame, a million rows, 40 columns. In this data frame, there are some (about 100,000) rows which I want to recompute (update), while I want to leave others just as is. this is based on a condition that I need to compute, based on what is in a few of the columns. what is the right R way to do this? I could subset out the rows that I want to recompute into a

Dear R developers---I just spent half a day debugging an R program, which had two bugs---I selected the wrongly named variable, which turns out to have been a scalar, which then happily multiplied as if it was a matrix; and another wrongly named variable from a data frame, that triggered no error when used as a[["name"]] or a$name . there should be an option to turn on that throws an

Dear R wizards: I am playing (and struggling) with the example in the constrOptim function. simple example. let's say I want to constrain my variables to be within -1 and 1. I believe I want a whole lot of constraints where ci is -1 and ui is either -1 or 1. That is, I have 2*N constraints. Should the following work? N=10 x= rep(1:N) ci= rep(-1, 2*N) ui= c(rep(1, N), rep(-1, N))

dear R wizards---more ignorance on my part, exacerbated by too few examples in the function documentations. > d <- data.frame( id=rep(1:3,3), x=rnorm(9), y=rnorm(9)) Question 1: how do I work with the output of "by"? for example, > b <- by( d, d$id, function(x) coef(lm( y ~ x, data=x ) )) > b d$id: 1 (Intercept) x 0.2303 0.3618

dear R wizards: an operation I execute often is the deletion of all observations (in a matrix or data set) that have at least one NA. (I now need this operation for kde2d, because its internal quantile call complains; could this be considered a buglet?) usually, my data sets are small enough for speed not to matter, and there I do not care whether my method is pretty inefficient (ok, I

dear R experts---I am using the coef() function to pick off the coefficients from an lm() object. alas, I also need the standard errors and I need them fast. I know I can do a "summary()" on the object and pick them off this way, but this computes other stuff I do not need. Or, I can compute (X' X)^(-1) s^2 myself. Has someone written a fast se() function? incidentally, I think

dear R experts---now I have a case where I want to estimate very large regression models with many fixed effects---not just the mean type, but cross-fixed effects---years, months, locations, firms. Many millions of observations, a few thousand variables (most of these variables are interaction fixed effects). could someone please point me to packages, if any, that would help me estimate such

this is not an important question, but I wonder why lm returns an error, and whether this can be shut off. it would seem to me that returning NA's would make more sense in some cases---after all, the problem is clearly that coefficients cannot be computed. I know that I can trap the lm.fit() error---although I have always found this to be quite inconvenient---and this is easy if I have only

is it possible to throw a stop() that is so hard that it will escape even tryCatch? /iaw ---- Ivo Welch (ivo.welch at gmail.com)

dear r experts---Is there a multicore equivalent of by(), just like mclapply() is the multicore equivalent of lapply()? if not, is there a fast way to convert a data.table into a list based on a column that lapply and mclapply can consume? advice appreciated...as always. regards, /iaw ---- Ivo Welch (ivo.welch at gmail.com)

dear R experts---is there a function that prints a data frame to a string? cat() cannot handle lists, so I cannot write cat("your data frame is:\n", df, "\n"). regards, /iaw ---- Ivo Welch (ivo.welch@gmail.com) [[alternative HTML version deleted]]

Dear R experts--- Has anyone written parallel versions of "by" (i.e., mcby) and "ave" (i.e. mcave) ? I did ask a question like this a year ago, and then the answer was no. for those who are googling the group for the answer to this question, in the meantime, the poor man's version of "by" is mclapply( split( ds, factor ), FUN ) I don't know the poor

Dear R experts--- after many years, I am planning to give in and write my first R package. I want to combine my collection of collected useful utility routines. as my guide, I am planning to use Friedrich Leisch's "Creating R Packages: A Tutorial" from Sep 2009. Is there a newer or better tutorial? this one is 4 years old. I also plan on one change---given that the