Displaying 20 results from an estimated 8000 matches similar to: "SLOW split() function"
2011 Oct 10
5
multicore by(), like mclapply?
dear r experts---Is there a multicore equivalent of by(), just like
mclapply() is the multicore equivalent of lapply()?
if not, is there a fast way to convert a data.table into a list based
on a column that lapply and mclapply can consume?
advice appreciated...as always.
regards,
/iaw
----
Ivo Welch (ivo.welch at gmail.com)
2012 Dec 24
2
parallelized version of "by" and "ave"
Dear R experts---
Has anyone written parallel versions of "by" (i.e., mcby) and "ave"
(i.e. mcave) ? I did ask a question like this a year ago, and then
the answer was no.
for those who are googling the group for the answer to this question,
in the meantime, the poor man's version of "by" is mclapply( split(
ds, factor ), FUN )
I don't know the poor
2011 Jul 02
5
%dopar% parallel processing experiment
dear R experts---
I am experimenting with multicore processing, so far with pretty
disappointing results. Here is my simple example:
A <- 100000
randvalues <- abs(rnorm(A))
minfn <- function( x, i ) { log(abs(x))+x^3+i/A+randvalues[i] } ?## an
arbitrary function
ARGV <- commandArgs(trailingOnly=TRUE)
if (ARGV[1] == "do-onecore") {
?library(foreach)
?discard <-
2013 Feb 06
5
First R Package --- Advice?
Dear R experts---
after many years, I am planning to give in and write my first R
package. I want to combine my collection of collected useful utility
routines.
as my guide, I am planning to use Friedrich Leisch's "Creating R
Packages: A Tutorial" from Sep 2009. Is there a newer or better
tutorial? this one is 4 years old.
I also plan on one change---given that the
2012 Mar 26
1
assigning vector or matrix sparsely (for use with mclapply)
Dear R wizards---
I have a wrapper on mclapply() that makes it a little easier for me to
do multiprocessing. (Posting this may make life easier for other
googlers.) I pass a data frame, a vector that tells me what rows
should be recomputed, and the function; and I get back a vector or
matrix of answers.
d <- data.frame( id=1:6, val=11:16 )
loc <- c(TRUE,TRUE,FALSE,TRUE,FALSE,TRUE)
2012 Mar 30
4
list assignment syntax?
Dear R wizards: is there a clean way to assign to elements in a list?
what I would like to do, in pseudo R+perl notation is
f <- function(a,b) list(a+b,a-b)
(c,d) <- f(1,2)
and have c be assigned 1+2 and d be assigned 1-2. right now, I use the clunky
x <- f(1,2)
c <- x[[1]]
d <- x[[2]]
rm(x)
which seems awful. is there a nicer syntax?
regards, /iaw
----
Ivo Welch
2011 Jul 24
2
split data frame temporary and work with only part of it?
dear R wizards: I have a large data frame, a million rows, 40
columns. In this data frame, there are some (about 100,000) rows
which I want to recompute (update), while I want to leave others just
as is. this is based on a condition that I need to compute, based on
what is in a few of the columns. what is the right R way to do this?
I could subset out the rows that I want to recompute into a
2013 Feb 07
4
Hard Stop?
is it possible to throw a stop() that is so hard that it will escape
even tryCatch?
/iaw
----
Ivo Welch (ivo.welch at gmail.com)
2013 Apr 04
6
categorized complete list of R commands?
every time I read the R release notes for the next release, I see many
functions that I had forgotten about and many functions that I never knew
existed to begin with. (who knew there were bibtex facilities in R?
obviously, everyone except me.)
I wonder whether there is a complete list of all R commands (incl the
standard packages) somewhere, preferably each with its one-liner AND
2010 Jan 08
4
fast lm se?
dear R experts---I am using the coef() function to pick off the coefficients
from an lm() object. alas, I also need the standard errors and I need them
fast. I know I can do a "summary()" on the object and pick them off this
way, but this computes other stuff I do not need. Or, I can compute (X'
X)^(-1) s^2 myself. Has someone written a fast se() function?
incidentally, I think
2010 Jun 11
3
lm without error
this is not an important question, but I wonder why lm returns an
error, and whether this can be shut off. it would seem to me that
returning NA's would make more sense in some cases---after all, the
problem is clearly that coefficients cannot be computed.
I know that I can trap the lm.fit() error---although I have always
found this to be quite inconvenient---and this is easy if I have only
2010 Aug 30
4
different interface to by (tapply)?
dear R experts:
has someone written a function that returns the results of by() as a
data frame? ??of course, this can work only if the output of the
function that is an argument to by() is a numerical vector.
presumably, what is now names(byobject) would become a column in the
data frame, and the by object's list elements would become columns.
it's a little bit like flattening the by()
2012 May 31
2
print.data.frame to string?
dear R experts---is there a function that prints a data frame to a string?
cat() cannot handle lists, so I cannot write cat("your data frame is:\n",
df, "\n").
regards, /iaw
----
Ivo Welch (ivo.welch@gmail.com)
[[alternative HTML version deleted]]
2012 Apr 18
1
multi-machine parallel setup?
Dear R experts:
could someone please point me to a page that explains how to set up
more than 1 machine for library parallel (which is quickly becoming my
favorite!)
my dream setup would be a design where I just pass a list of
hostnames:user:password to my parallel master, and then start R
listener processes on each of my slaves by hand. R would start slave
processes automatically on each slave
2013 Feb 09
2
character strings with embedded commands: perl "/gee" ?
dear R experts---I am trying to replicate a perl feature. I want to be
able to embed R commands inside a character string, and have the
string be printed with the command executed. my perl equivalent is
my $a=10;
my $teststring = "the expression, $a+1, is ::$a+1::, but add one
more for ::$a+2::\n";
$teststring =~ s/::(.*?)::/$1/gee;
print $teststring;
of course, R does not use
2004 Jul 07
3
fast NA elimination ?
dear R wizards: an operation I execute often is the deletion of all
observations (in a matrix or data set) that have at least one NA. (I
now need this operation for kde2d, because its internal quantile call
complains; could this be considered a buglet?) usually, my data sets
are small enough for speed not to matter, and there I do not care
whether my method is pretty inefficient (ok, I
2012 May 09
2
big quasi-fixed effects OLS model
dear R experts---now I have a case where I want to estimate very large
regression models with many fixed effects---not just the mean type, but
cross-fixed effects---years, months, locations, firms. Many millions of
observations, a few thousand variables (most of these variables are
interaction fixed effects). could someone please point me to packages, if
any, that would help me estimate such
2010 Aug 22
2
on abort error, always show call stack?
Dear R Wizards---is it possible to get R to show its current call
stack (sys.calls()) upon an error abort? I don't use ESS for
execution, and it is often not obvious how to locate how I triggered
an error in an R internal function. Seeing the call stack would make
this easier. (right now, I sprinkle "cat" statements everywhere, just
to locate the line where the error appears.) Of
2004 Mar 26
8
stop() vs. error() ?
Why does stop("we are done") print
"Error in eval.with.vis(expr, envir, enclos) :" ?
It would seem to me that a plain stop() is not an error, and that it
would make more sense to have an error() function that is different from
a stop(). Is there a rationale here that I am missing?
sincerely, /iaw
2013 Aug 20
7
Extending suggestion for stopifnot
I am using a variant of stopifnot a lot. can I suggest that base R
extends its functionality? I know how to do this for myself. this is
a suggestion for beginners and students. I don't think it would break
anything.
first, I think it would be more useful if it had an optional character
string, so users could write
stopifnot( is.matrix(m), "m is not a matrix" )
this would