similar to: huge data?

We are building a package, and want to create S4 methods for both head and mean for our own BigMatrix class. Following the recommendation in "Writing R Extensions" we use exportMethods instead of export in NAMESPACE (this is described as being "clearer"). This works for head, but not for mean. Obviously we importFrom(utils, head), but don't need to do this for mean,

I was surprised to observe the following difference between 2.4.1 and 2.6.0 after a long overdue upgrade a few months ago of our departmental server. It wasn't a bug fix, but a subtle improvement. Here's the simplest example I could create. The size is excessive, on the order of the Netflix Competition data. The integer matrix is about 1.12 GB, and if coerced to numeric it is 2.24 GB.

To answer your first question about read.big.matrix(), we don't know what your acc3.dat file is, but it doesn't appear to have been detected as a standard file (like a CSV file) or -- perhaps -- doesn't even exist (or doesn't exist in your current directory)? Next: > In addition, I am planning to do a multiple imputation with MICE package > using the data read by bigmemory

I didn't see anything on this in the bug reports, and a search of the archives had lots of false positives when searching on "split" to be helpful. I don't view this as particularly interesting or useful, but wanted to report it because I stumbled on it (and don't remember ever seeing "invalid permissions" as part of a segfault).? Yes, I realize this is a silly

Sorry, gmail seemed to have made an attachment out of my first attempted post. Trying again: ------------------------------ At the encouragement of many at UseR, I'm trying to build my first real package. I have no C/Fortran code, just plain old R code, so it should be rocket science. On a Linux box, I used package.skeleton() to create a basic package containing just one "hello

Hi -- I've found a need for an additional option to mosaicplot(), to suppress the labels. It's not difficult, obviously, a minor thing. Would you like me to submit my revised code (I'll use your code rather than my original source code which was adapted for S-Plus and R)? Or it might be a 5-minute change for the appropriate person. No problem either way, just let me know. Cheers,

foreach (or virtually anything you might use for concurrent programming) only really makes sense if the work the "clients" are doing is substantial enough to overwhelm the communication overhead. And there are many ways to accomplish the same task more or less efficiently (for example, doing blocks of tasks in chunks rather than passing each one as an individual job). But more to the

Hi, I have used the heckit function in micEcon. Now I would like to evaluate the fit of the probit part of the model but when I enter AIC(sk$probit) I get this error Error in logLik(object) : no applicable method for "logLik" How can I then get the AIC for this model? Side question: If you know - from the top of your head - some link to readings dealing with evaluating the

I apologize in advance for probably missing something obvious, but if someone could point me in the right direction I'd be grateful. This NOTE is not unique to our package (I list a few others, below). Package bcp has several Suggests (strucchange, for example). Then in an Rd file, we have if (require(strucchange)) { # Doing some examples making use of strucchange } The CRAN check

Hi, I'm playing around with the 'bigmemory' package, and I have finally managed to create some really big matrices. However, only now I realize that there may not be functions made for what I want to do with the matrices... I would like to perform a cluster analysis based on a big.matrix. Googling around I have found indications that a certain kmeans.big.matrix() function should

At the encouragement of many at UseR, I'm trying to build my first real package. I have no C/Fortran code, just plain old R code, so it should be rocket science. On a Linux box, I used package.skeleton() to create a basic package containing just one "hello world" type of function. I edited the DESCRIPTION file, changin the package name appropriately. I edited the hello.Rd file.

I am using the library(bigmemory) to handle large datasets, say 1 GB, and facing following problems. Any hints from anybody can be helpful. _Problem-1: _ I am using "read.big.matrix" function to create a filebacked big matrix of my data and get the following warning: > x = read.big.matrix("/home/utkarsh.s/data.csv",header=T,type="double",shared=T,backingfile

Version 2.2.0 of package bcp is now available.? It replaces the suggests of NetWorkSpaces (previously used for optional parallel MCMC) with the dependency on package foreach, giving greater flexibility and supporting a wider range of parallel backends (see doSNOW, doMC, etc...). For those unfamiliar with foreach (thanks to Steve Weston for this contribution), it's a beautiful and highly

Version 2.2.0 of package bcp is now available.? It replaces the suggests of NetWorkSpaces (previously used for optional parallel MCMC) with the dependency on package foreach, giving greater flexibility and supporting a wider range of parallel backends (see doSNOW, doMC, etc...). For those unfamiliar with foreach (thanks to Steve Weston for this contribution), it's a beautiful and highly

The re-engineered bigmemory package is now available (Version 3.5 and above) on CRAN. We strongly recommend you cease using the older versions at this point. bigmemory now offers completely platform-independent support for the big.matrix class in shared memory and, optionally, as filebacked matrices for larger-than-RAM applications. We're working on updating the package vignette, and a

Version 2.2.0 of package bcp is now available. It replaces the suggests of NetWorkSpaces (previously used for optional parallel MCMC) with the dependency on package foreach, giving greater flexibility and supporting a wider range of parallel backends (see doSNOW, doMC, etc...). For those unfamiliar with foreach (thanks to Steve Weston for this contribution), it's a beautiful and highly

Version 2.2.0 of package bcp is now available. It replaces the suggests of NetWorkSpaces (previously used for optional parallel MCMC) with the dependency on package foreach, giving greater flexibility and supporting a wider range of parallel backends (see doSNOW, doMC, etc...). For those unfamiliar with foreach (thanks to Steve Weston for this contribution), it's a beautiful and highly

The long-promised revision to bigmemory has arrived, with package 4.2.3 now on CRAN. The mutexes (locks) have been extracted and will be available through package synchronicity (on R-Forge, soon to appear on CRAN). Initial versions of packages biganalytics and bigtabulate are on CRAN, and new versions which resolve the warnings and have streamlined CRAN-friendly configurations will appear

The re-engineered bigmemory package is now available (Version 3.5 and above) on CRAN. We strongly recommend you cease using the older versions at this point. bigmemory now offers completely platform-independent support for the big.matrix class in shared memory and, optionally, as filebacked matrices for larger-than-RAM applications. We're working on updating the package vignette, and a

The long-promised revision to bigmemory has arrived, with package 4.2.3 now on CRAN. The mutexes (locks) have been extracted and will be available through package synchronicity (on R-Forge, soon to appear on CRAN). Initial versions of packages biganalytics and bigtabulate are on CRAN, and new versions which resolve the warnings and have streamlined CRAN-friendly configurations will appear