similar to: multicore by(), like mclapply?

Dear R wizards--- I have a wrapper on mclapply() that makes it a little easier for me to do multiprocessing. (Posting this may make life easier for other googlers.) I pass a data frame, a vector that tells me what rows should be recomputed, and the function; and I get back a vector or matrix of answers. d <- data.frame( id=1:6, val=11:16 ) loc <- c(TRUE,TRUE,FALSE,TRUE,FALSE,TRUE)

dear R experts--- I am experimenting with multicore processing, so far with pretty disappointing results. Here is my simple example: A <- 100000 randvalues <- abs(rnorm(A)) minfn <- function( x, i ) { log(abs(x))+x^3+i/A+randvalues[i] } ?## an arbitrary function ARGV <- commandArgs(trailingOnly=TRUE) if (ARGV[1] == "do-onecore") { ?library(foreach) ?discard <-

dear R experts---I am looking at a fairly uninformative error in my program: Error in mclapply(1:nrow(opts), solveme) : (converted from warning) all scheduled cores encountered errors in user code the doc on ?mclapply tells me that In addition, each process is running the job inside try(..., silent=TRUE) so if error occur they will be stored as try-error objects in the list. I looked up

The following program is whittled down from a much larger program that always works on Intel, and always works on AMD's threadripper with lapply but not mclappy. With mclapply on AMD, all processes go into "suspend" mode and the program then hangs. This bug is replicable on an AMD Ryzen Threadripper 2950X 16-Core Processor (128GB RAM), running latest ubuntu 18.04. The R version

Dear R experts--- Has anyone written parallel versions of "by" (i.e., mcby) and "ave" (i.e. mcave) ? I did ask a question like this a year ago, and then the answer was no. for those who are googling the group for the answer to this question, in the meantime, the poor man's version of "by" is mclapply( split( ds, factor ), FUN ) I don't know the poor

dear R experts: ?apologies for all my speed and memory questions. ?I have a bet with my coauthors that I can make R reasonably efficient through R-appropriate programming techniques. this is not just for kicks, but for work. for benchmarking, my [3 year old] Mac Pro has 2.8GHz Xeons, 16GB of RAM, and R 2.13.1. right now, it seems that 'split()' is why I am losing my bet. ?(split is an

Dear R experts: could someone please point me to a page that explains how to set up more than 1 machine for library parallel (which is quickly becoming my favorite!) my dream setup would be a design where I just pass a list of hostnames:user:password to my parallel master, and then start R listener processes on each of my slaves by hand. R would start slave processes automatically on each slave

Dear R developers: ... 7: lapply(seq_len(cores), inner.do) 8: FUN(1:3[[3]], ...) 9: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) Selection: .....................Error in sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) : long vectors not supported yet: memory.c:3100 admittedly, my outcome will be a very big list, with 30,000 elements, each containing data frames

Dear R experts--- after many years, I am planning to give in and write my first R package. I want to combine my collection of collected useful utility routines. as my guide, I am planning to use Friedrich Leisch's "Creating R Packages: A Tutorial" from Sep 2009. Is there a newer or better tutorial? this one is 4 years old. I also plan on one change---given that the

Dear R wizards: is there a clean way to assign to elements in a list? what I would like to do, in pseudo R+perl notation is f <- function(a,b) list(a+b,a-b) (c,d) <- f(1,2) and have c be assigned 1+2 and d be assigned 1-2. right now, I use the clunky x <- f(1,2) c <- x[[1]] d <- x[[2]] rm(x) which seems awful. is there a nicer syntax? regards, /iaw ---- Ivo Welch

I'm running r 2. on a mac running 10.6.4 and a dual-core macbook pro. I'm having a funny time with multicore. When I run it with 2 cores, mclapply, R borks with the following error. The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()

is it possible to throw a stop() that is so hard that it will escape even tryCatch? /iaw ---- Ivo Welch (ivo.welch at gmail.com)

every time I read the R release notes for the next release, I see many functions that I had forgotten about and many functions that I never knew existed to begin with. (who knew there were bibtex facilities in R? obviously, everyone except me.) I wonder whether there is a complete list of all R commands (incl the standard packages) somewhere, preferably each with its one-liner AND

dear R experts---I am using the coef() function to pick off the coefficients from an lm() object. alas, I also need the standard errors and I need them fast. I know I can do a "summary()" on the object and pick them off this way, but this computes other stuff I do not need. Or, I can compute (X' X)^(-1) s^2 myself. Has someone written a fast se() function? incidentally, I think

this is not an important question, but I wonder why lm returns an error, and whether this can be shut off. it would seem to me that returning NA's would make more sense in some cases---after all, the problem is clearly that coefficients cannot be computed. I know that I can trap the lm.fit() error---although I have always found this to be quite inconvenient---and this is easy if I have only

dear R experts: has someone written a function that returns the results of by() as a data frame? ??of course, this can work only if the output of the function that is an argument to by() is a numerical vector. presumably, what is now names(byobject) would become a column in the data frame, and the by object's list elements would become columns. it's a little bit like flattening the by()

dear R experts---is there a function that prints a data frame to a string? cat() cannot handle lists, so I cannot write cat("your data frame is:\n", df, "\n"). regards, /iaw ---- Ivo Welch (ivo.welch@gmail.com) [[alternative HTML version deleted]]

dear R experts---I am trying to replicate a perl feature. I want to be able to embed R commands inside a character string, and have the string be printed with the command executed. my perl equivalent is my $a=10; my $teststring = "the expression, $a+1, is ::$a+1::, but add one more for ::$a+2::\n"; $teststring =~ s/::(.*?)::/$1/gee; print $teststring; of course, R does not use

dear R wizards: an operation I execute often is the deletion of all observations (in a matrix or data set) that have at least one NA. (I now need this operation for kde2d, because its internal quantile call complains; could this be considered a buglet?) usually, my data sets are small enough for speed not to matter, and there I do not care whether my method is pretty inefficient (ok, I

dear R experts---now I have a case where I want to estimate very large regression models with many fixed effects---not just the mean type, but cross-fixed effects---years, months, locations, firms. Many millions of observations, a few thousand variables (most of these variables are interaction fixed effects). could someone please point me to packages, if any, that would help me estimate such