search for: speedglm

...39;m finding that the time spent on garbage collection often exceeds the time spent on the regression itself. Consider this test program which I'm running on an Intel Haswell i7-4470 processor under Linux 3.13 using R 3.1.2 compiled with ICPC 14.1: nate at haswell:~$ cat > gc.R library(speedglm) createData <- function(n) { int <- -5 x <- rnorm(n, 50, 7) e <- rnorm(n, 0, 1) y <- int + (1.2 * x) + e return(data.frame(y, x)) } gc.time() data <- createData(500000) data.y <- as.matrix(data[1]) data.x <- model.matrix(y ~ ., data) fo...

hi, I have a dataset (the netflix dataset) which is basically ~18k columns and well variable number of rows but let's assume 25 thousand for now. The dataset is very sparse. I was wondering how to do kmeans/nearest neighbors or kernel density estimation on it. I tired using the spMatrix function in "Matrix" package. I think I'm able to create the matrix but as soon as I pass

...lm.fit, but as far as I understood, for that to be useful the computation of the design matrices and all of that other stuff that I frankly don't understand, needs to be the same for each model, which is not the case for my analysis, although perhaps I am wrong about this. I also took a look at speedglm and it seems to be for the analysis of large datasets. I think the problem is that, in effect there are many datasets, so not sure if this package can help. Any ideas on how to make this run faster would be greatly appreciated. I am planning on using parallelization to run the analysis in the end...

CRAN (and crantastic) updates this week New packages ------------ * bdoc (1.0) Michael Anderson http://crantastic.org/packages/bdoc This package contains a function that will classify DNA barcodes as well as a few test and reference data sets. * bdsmatrix (1.0) Terry Therneau http://crantastic.org/packages/bdsmatrix This is a special case of sparse matrices, used by coxme and