similar to: parallelized version of "by" and "ave"

dear r experts---Is there a multicore equivalent of by(), just like mclapply() is the multicore equivalent of lapply()? if not, is there a fast way to convert a data.table into a list based on a column that lapply and mclapply can consume? advice appreciated...as always. regards, /iaw ---- Ivo Welch (ivo.welch at gmail.com)

Dear R wizards--- I have a wrapper on mclapply() that makes it a little easier for me to do multiprocessing. (Posting this may make life easier for other googlers.) I pass a data frame, a vector that tells me what rows should be recomputed, and the function; and I get back a vector or matrix of answers. d <- data.frame( id=1:6, val=11:16 ) loc <- c(TRUE,TRUE,FALSE,TRUE,FALSE,TRUE)

dear R experts: ?apologies for all my speed and memory questions. ?I have a bet with my coauthors that I can make R reasonably efficient through R-appropriate programming techniques. this is not just for kicks, but for work. for benchmarking, my [3 year old] Mac Pro has 2.8GHz Xeons, 16GB of RAM, and R 2.13.1. right now, it seems that 'split()' is why I am losing my bet. ?(split is an

Dear R experts: could someone please point me to a page that explains how to set up more than 1 machine for library parallel (which is quickly becoming my favorite!) my dream setup would be a design where I just pass a list of hostnames:user:password to my parallel master, and then start R listener processes on each of my slaves by hand. R would start slave processes automatically on each slave

dear R experts---I am looking at a fairly uninformative error in my program: Error in mclapply(1:nrow(opts), solveme) : (converted from warning) all scheduled cores encountered errors in user code the doc on ?mclapply tells me that In addition, each process is running the job inside try(..., silent=TRUE) so if error occur they will be stored as try-error objects in the list. I looked up

dear R wizards---more ignorance on my part, exacerbated by too few examples in the function documentations. > d <- data.frame( id=rep(1:3,3), x=rnorm(9), y=rnorm(9)) Question 1: how do I work with the output of "by"? for example, > b <- by( d, d$id, function(x) coef(lm( y ~ x, data=x ) )) > b d$id: 1 (Intercept) x 0.2303 0.3618

The following program is whittled down from a much larger program that always works on Intel, and always works on AMD's threadripper with lapply but not mclappy. With mclapply on AMD, all processes go into "suspend" mode and the program then hangs. This bug is replicable on an AMD Ryzen Threadripper 2950X 16-Core Processor (128GB RAM), running latest ubuntu 18.04. The R version

Dear R developers: ... 7: lapply(seq_len(cores), inner.do) 8: FUN(1:3[[3]], ...) 9: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) Selection: .....................Error in sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) : long vectors not supported yet: memory.c:3100 admittedly, my outcome will be a very big list, with 30,000 elements, each containing data frames

? Mon, 30 Dec 2024 19:16:11 -0800 Ivo Welch <ivo.welch at gmail.com> ?????: > useless.function <- function( ) { > y <- rnorm(3); x <- rnorm(3) > summary( lm( y ~ x )) ## useless > NULL > } > > run30 <- function(i) { > message("run30=", i) > useless.function() > } > > run30( 0 ) > message("many mc")

dear R experts--- I am experimenting with multicore processing, so far with pretty disappointing results. Here is my simple example: A <- 100000 randvalues <- abs(rnorm(A)) minfn <- function( x, i ) { log(abs(x))+x^3+i/A+randvalues[i] } ?## an arbitrary function ARGV <- commandArgs(trailingOnly=TRUE) if (ARGV[1] == "do-onecore") { ?library(foreach) ?discard <-

sequoia, 15.2. R --vanilla : 4.4.2 (2024-10-31). I have the same basic setup on three macs: a macbook air, a mac pro m1, and a mac mini m4. The following code is running into a bug on the mac pro m1 and the mac mini, but works just fine on my macbook air. (of course, it doesn't do anything useful.) it's replicable! ``` $ R --vanilla > source("debug.R") ``` and (after

Dear R wizards: is there a clean way to assign to elements in a list? what I would like to do, in pseudo R+perl notation is f <- function(a,b) list(a+b,a-b) (c,d) <- f(1,2) and have c be assigned 1+2 and d be assigned 1-2. right now, I use the clunky x <- f(1,2) c <- x[[1]] d <- x[[2]] rm(x) which seems awful. is there a nicer syntax? regards, /iaw ---- Ivo Welch

dear R experts---now I have a case where I want to estimate very large regression models with many fixed effects---not just the mean type, but cross-fixed effects---years, months, locations, firms. Many millions of observations, a few thousand variables (most of these variables are interaction fixed effects). could someone please point me to packages, if any, that would help me estimate such

dear R experts---I am using the coef() function to pick off the coefficients from an lm() object. alas, I also need the standard errors and I need them fast. I know I can do a "summary()" on the object and pick them off this way, but this computes other stuff I do not need. Or, I can compute (X' X)^(-1) s^2 myself. Has someone written a fast se() function? incidentally, I think

this is not an important question, but I wonder why lm returns an error, and whether this can be shut off. it would seem to me that returning NA's would make more sense in some cases---after all, the problem is clearly that coefficients cannot be computed. I know that I can trap the lm.fit() error---although I have always found this to be quite inconvenient---and this is easy if I have only

Thanks, and all well taken. But are my beautiful GPUs (with integrated memory architecture) really nothing more than a cooling area for the chip? On Fri, Nov 15, 2024 at 6:06?AM Martin Maechler <maechler at stat.math.ethz.ch> wrote: > >>>>> Andrew Robinson via R-help > >>>>> on Thu, 14 Nov 2024 12:45:44 +0000 writes: > > > Not a direct

is it possible to throw a stop() that is so hard that it will escape even tryCatch? /iaw ---- Ivo Welch (ivo.welch at gmail.com)

dear R experts---is there a function that prints a data frame to a string? cat() cannot handle lists, so I cannot write cat("your data frame is:\n", df, "\n"). regards, /iaw ---- Ivo Welch (ivo.welch@gmail.com) [[alternative HTML version deleted]]

dear R experts--- I have (many) unidimensional root problems. think loc.of.root <- uniroot( f= function(x,a) log( exp(a) + a) + a, c(.,9e10), a=rnorm(1) ) $root (for some coefficients a, there won't be a solution; for others, it may exceed the domain. implied volatilities in various Black-Scholes formulas and variant formulas are like this, too.) except I don't need 1 root, but a

Dear R Wizards---is it possible to get R to show its current call stack (sys.calls()) upon an error abort? I don't use ESS for execution, and it is often not obvious how to locate how I triggered an error in an R internal function. Seeing the call stack would make this easier. (right now, I sprinkle "cat" statements everywhere, just to locate the line where the error appears.) Of