similar to: difficulty with sanitizer using bigmemory

Hello, good mornig, i have one questions, anybody know how to calculate svd of a matrix of the library bigmatrix with library irlba? thanks [[alternative HTML version deleted]]

Hello I'm a population ecologist and use R for all my stats and modelling. Recently I have been using R to numerically solve integral projection models. This involves constructing several large matrices. The current code by Easterling (Size-specific sensitivity: Applying a new structured population model. Ecology, 2000, 81, 694-708) uses nested loops to construct the matrices. To speed up the

Hi all, I have a vector that contains the row numbers of data taken from several filtering operations performed on a large data frame (20,000rows x 500cols). In order to output this subset of data, I've been looping through the vector containing the row numbers (keepRows). output <- data.frame(row.names = rownames(bigMatrix)) for(i in keepRows) { output <- rbind(output,

Hi, I have read several threads about memory issues in R and I can't seem to find a solution to my problem. I am running a sort of LASSO regression on several subsets of a big dataset. For some subsets it works well, and for some bigger subsets it does not work, with errors of type "cannot allocate vector of size 1.6Gb". The error occurs at this line of the code: example <-

Dear R-help list members, I am hoping to get you help in reproducing a problem I am having That is only reproducible on a large-memory machine. Whenever I run the following lines, get a segfault listed below: *** caught segfault *** address 0x7f092cc46e40, cause 'invalid permissions' Traceback: 1: La.svd(x, nu, nv) 2: svd(X) 3: ginv(bigmatrix) Here is the code that I run:

We are building a package, and want to create S4 methods for both head and mean for our own BigMatrix class. Following the recommendation in "Writing R Extensions" we use exportMethods instead of export in NAMESPACE (this is described as being "clearer"). This works for head, but not for mean. Obviously we importFrom(utils, head), but don't need to do this for mean,

Does anyone know a way to do the following: Save a large number of R objects to a file (like load() does) but then read back only a small named subset of them . As far as I can see, load() reads back everything. The context is: I have an application which will generate a large number of large matrices (approx 15000 matrices each of dimension 2000*30). I can generate these matrices using an

I have a large file backed big. matrix, with millions of rows and 20 columns. The columns contain data that I simply need to tabulate. There are a few dozen unique values. and I just want a frequency count Test code with a small "big" matrix. library(bigmemory) library(bigtabulate) test <- big.matrix(nrow = 100, ncol = 10) test[,1:3]<- sample(150) test[,4:6]<-

Dear all, I have a large dataset (N=100,000 with 89 variables per subject). This dataset is stored in a 100.000 x 89 matrix where each row describes one individual and each column one variable. What is the easiest way of selecting a subset of let's say 1.000 individuals out of that whole matrix? Thanks, Michael Michael Haenlein Associate Professor of Marketing ESCP-EAP European School of

Good morning, I have currently 63 .csv files most of which have lines which look like 01/06/05,23445 Though some files have two numbers beside each date. There are missing values, and currently the longest file has 318 rows. (merge() is losing the head and doing runaway memory allocation - but thats another question - I'm still trying to pin that issue down and make a small repeatable

Hello All, I tried to intall the bigmemory package from a CRAN mirror site and received the following output while installing. Any idea what's going on and how to fix it? The system details are provided below. --------------------- begin error messages ----------------------- * installing *source* package 'bigmemory' ... checking for Sun Studio compiler...no checking for

Hello, I have a rather large set of data I need to analyze, currently I need to work with a 200000 by 200000 matrix, I'm using the package bigmemory but so far I can only allocate a 66000 by 66000 matrix, when I increase those values I get the following error:   > AdjMat <- big.matrix(nrow=68000,ncol=68000) Cannot allocate memory BigMatrix.cpp line 225 Error in big.matrix(nrow = 68000,

I work with problems that have rather large data requirements -- typically a bunch of multigig arrays. Given how generous R is with using memory, the only way for me to work with R has been to use bigmatrices from bigmemory package. One thing that is missing a bit is interoperability of bigmatrices with C++ and possibly java. What i mean by that is API that would allow read and write filebacked