search for: bigmemory

I am trying to add the bigmemory packages but I get the following error message: In file included from bigmemory.cpp:14:0: ../inst/include/bigmemory/isna.hpp: In function 'bool neginf(double)': ../inst/include/bigmemory/isna.hpp:22:57: error: 'isinf' was not declared in this scope make: *** [bigmemory.o] Error 1 E...

Hello All, I tried to intall the bigmemory package from a CRAN mirror site and received the following output while installing. Any idea what's going on and how to fix it? The system details are provided below. --------------------- begin error messages ----------------------- * installing *source* package 'bigmemory' ... chec...

Hi, I am trying to run the bigmemory example provided on the http://www.bigmemory.org/ The example runs on the "airline data" and generates summary of the csv files:- library(bigmemory) library(biganalytics) x <- read.big.matrix("2005.csv", type="integer", header=TRUE, backingfile="airline.bin&q...

Dear helpers, Does anyone have information on the status of bigmemory and R3.0? Will it just take time for the devs to re-code for the new environment? Or is there an alternative for this new version? Thanks Ben Caldwell [[alternative HTML version deleted]]

...ata management in SAS, and Stata, mostly identifying spells, counting events in intervals, and a like, but I would like to do the data management-and fitting my models-in R. Though R can't handle the data in a normal R-way, it's simply too big. So I thought of trying either filehash, bigmemory or some other similar package I haven't heard of (yet). In the documentation to 'bigmemory' is says that the package is capable of ``basic manipulation '' on ``manageable subsets of the data '', but what does that actually mean? Since learning this in R is a ra...

...t file is, but it doesn't appear to have been detected as a standard file (like a CSV file) or -- perhaps -- doesn't even exist (or doesn't exist in your current directory)? Next: > In addition, I am planning to do a multiple imputation with MICE package > using the data read by bigmemory package. > So usually, the multiple imputation code is like this: > imp=mice(data.frame,m=50,seed=1234,print=F) > the data.frame is required. How can I change the big.matrix class > generated by bigmemory package to a data.frame? Please read the help files for bigmemory -- only matrix...

Dear all, bigKRLS, which has been on CRAN for a couple of years, had to be pulled recently due to what seems to be a sanitizer issue stemming from its use of bigmemory. bigKRLS works fine (we?ve used it ourselves on many different platforms and have had over 15,000 downloads without an end user reporting difficulties because of this issue). Unfortunately, we have been unable to reproduce the UndefinedBehaviorSanitizer error locally, and are unsure how to proceed....

I work with problems that have rather large data requirements -- typically a bunch of multigig arrays. Given how generous R is with using memory, the only way for me to work with R has been to use bigmatrices from bigmemory package. One thing that is missing a bit is interoperability of bigmatrices with C++ and possibly java. What i mean by that is API that would allow read and write filebacked matrices from C++, and ideally java without being called from R. Having ability to save armadillo matrices into filebacked ma...

Hi, Is it possible for me to read data from shared memory created by a vc++ program into R using bigmemory? [[alternative HTML version deleted]]

...environment for R, usable both on a multicore machine and across a network of multiple machines. The package gives the illusion of shared memory, again even across multiple machines on a network. This is based on a similar package I wrote for Perl back in 2002. Rdsm should complement the bigmemory package written by Mike Kane and Jay Emerson. As noted above and in my July announcement for my prototype, Rdsm works across a network of machines, while bigmemory is limited to multicore platforms. On the other hand, bigmemory is capable of handling huge data structures, which Rdsm is not. Rdsm...

I have pretty big data sizes, like matrices of .5 to 1.5GB so once i need to juggle several of them i am in need of disk cache. I am trying to use bigmemory package but getting problems that are hard to understand. I am getting seg faults and machine just hanging. I work by the way on Red Hat Linux, 64 bit R version 10. Simplest problem is just saving matrices. When i do something like r<-matrix(rnorm(100), nr=10); librarybigmemory) for(i in 1:3...

I recently learned about the bigmemory and foreach packages and am trying to use them to help me create a very large matrix. Without those packages, I can create the type of matrix that I want with 10 columns and 5e6 rows. I would like to be able to scale up to 5e9 rows, or more, if possible. I have created a simplified example o...

Hi, With reference to the mail below, I have large datasets, coming from various different sources, which I can read into filebacked big.matrix using library bigmemory. I want to merge them all into one 'big.matrix' object. (Later, I want to run regression using library 'biglm'). I am unsuccessfully trying to do this from quite some time now. Can you please suggest some way? Am I missing some already available function? Even a function...

Hi, I'm playing around with the 'bigmemory' package, and I have finally managed to create some really big matrices. However, only now I realize that there may not be functions made for what I want to do with the matrices... I would like to perform a cluster analysis based on a big.matrix. Googling around I have found indications that a...

...isn't a platform we can test and support. We believe there are problems with BOOST library compatibilities. We'll try (again) to clear up the other warnings in the logs, though. !-) We should also revisit the possibility of a CRAN BOOST library for use by a small group of packages (like bigmemory) which might make patches to BOOST easier to track and maintain. This might improve things in the long run. Jay -- John W. Emerson (Jay) Associate Professor of Statistics Department of Statistics Yale University http://www.stat.yale.edu/~jay [[alternative HTML version deleted]]

...I run and save models that keep copies of the modeled data inside the model objects as well (mle2 objects, to be specific). I'm searching for a way to cache some of these dataframes and objects to virtual memory (I think I'm using the right terminology...). I've read around, and while bigmemory and ff and the like would likely suit my purposes if I were just dealing with numeric matricies, I'm dealing with dataframes and objects. Any thoughts would be greatly appreciated! Thanks, Allie

...x <- big.matrix(nrow=20000,ncol=500000,type='short',init=0,dimnames=list(1:20000,1:500000)) #Gets around the 2^31 issue - yeah! in Unix, when I hit the "top" command, I see R is taking up about 18Gb RAM, even though the object x is 0 bytes in R. That's fine: that's how bigmemory is supposed to work I guess. My question is how do I return that RAM to the system once I don't want to use x any more? E.g., rm(x) then "top" in Unix, I expect that my RAM footprint is back ~0, but it remains at 18Gb. How do I return RAM to the system? Thanks, Matt -- Matthew...

Package "bigmemory" is now available on CRAN. A brief abstract follows: Multi-gigabyte data sets challenge and frustrate R users even on well-equipped hardware. C/C++ and Fortran programming can be helpful, but is cumbersome for interactive data analysis and lacks the flexibility and power of R's rich stat...

The re-engineered bigmemory package is now available (Version 3.5 and above) on CRAN. We strongly recommend you cease using the older versions at this point. bigmemory now offers completely platform-independent support for the big.matrix class in shared memory and, optionally, as filebacked matrices for larger-than-RAM appl...

The long-promised revision to bigmemory has arrived, with package 4.2.3 now on CRAN. The mutexes (locks) have been extracted and will be available through package synchronicity (on R-Forge, soon to appear on CRAN). Initial versions of packages biganalytics and bigtabulate are on CRAN, and new versions which resolve the warnings and hav...