similar to: [Fwd: adding more columns in big.matrix object of bigmemory package]

I am using the library(bigmemory) to handle large datasets, say 1 GB, and facing following problems. Any hints from anybody can be helpful. _Problem-1: _ I am using "read.big.matrix" function to create a filebacked big matrix of my data and get the following warning: > x = read.big.matrix("/home/utkarsh.s/data.csv",header=T,type="double",shared=T,backingfile

Hi, I am trying to run the bigmemory example provided on the http://www.bigmemory.org/ The example runs on the "airline data" and generates summary of the csv files:- library(bigmemory) library(biganalytics) x <- read.big.matrix("2005.csv", type="integer", header=TRUE, backingfile="airline.bin", descriptorfile="airline.desc",

I recently learned about the bigmemory and foreach packages and am trying to use them to help me create a very large matrix. Without those packages, I can create the type of matrix that I want with 10 columns and 5e6 rows. I would like to be able to scale up to 5e9 rows, or more, if possible. I have created a simplified example of what I'm trying to do, below. The first part of the

I work with problems that have rather large data requirements -- typically a bunch of multigig arrays. Given how generous R is with using memory, the only way for me to work with R has been to use bigmatrices from bigmemory package. One thing that is missing a bit is interoperability of bigmatrices with C++ and possibly java. What i mean by that is API that would allow read and write filebacked

I have pretty big data sizes, like matrices of .5 to 1.5GB so once i need to juggle several of them i am in need of disk cache. I am trying to use bigmemory package but getting problems that are hard to understand. I am getting seg faults and machine just hanging. I work by the way on Red Hat Linux, 64 bit R version 10. Simplest problem is just saving matrices. When i do something like

Hi all, Is there any way I can add more columns to an existing filebacked big.matrix object. In general, I want a way to modify an existing big.matrix object, i.e., add rows/columns, rename colnames, etc. I tried the following: > library(bigmemory) > x = read.big.matrix("test.csv",header=T,type="double",shared=T,backingfile="test

Hi All, I have vector of length 52, say, x=sample(30,52,replace=T). I want to sort x and split into five *nearly equal groups*. Note that the observations are repeated in x so in case of a tie I want both the observations to fall in same group. This seems a very common task to do, but still I couldn't find an R function to do this. Any help would be highly appreciated. Regards Utkarsh

Hi all, I am performing a stepwise regression by running the "step" function on an "lm" object. Now I want to save the intermediate iterations. I know the argument trace=T will print it on the console, but I rather want to assign it to some R object or may be output it in a CSV or text file. Any help will be appreciated. Regards Utkarsh

Dear all, bigKRLS, which has been on CRAN for a couple of years, had to be pulled recently due to what seems to be a sanitizer issue stemming from its use of bigmemory. bigKRLS works fine (we?ve used it ourselves on many different platforms and have had over 15,000 downloads without an end user reporting difficulties because of this issue). Unfortunately, we have been unable to reproduce the

Hi All, I am currently using R version 2.8.1 on linux cent os 4.4 (i386) and want to upgrade to version 2.9.1. It seems to me that version-2.9.1 is it not for my OS. Am I right? Regards Utkarsh [[alternative HTML version deleted]]

Hi All, I am currently doing the following to compute summary statistics of aggregated data: a = aggregate(warpbreaks$breaks, warpbreaks[,-1], mean) b = aggregate(warpbreaks$breaks, warpbreaks[,-1], sum) c = aggregate(warpbreaks$breaks, warpbreaks[,-1], length) ans = cbind(a, b[,3], c[,3]) This seems unnecessarily complex to me so I tried > aggregate(warpbreaks$breaks, warpbreaks[,-1],

hi, the sas universal viewer might be a free, non-R way to convert a sas7bdat file to non-proprietary formats, not sure if it's windows-only. those other formats should be easier to import only a subset of columns into R.. https://support.sas.com/downloads/browse.htm?fil=&cat=74 On Thu, Aug 10, 2017 at 7:42 AM, peter dalgaard <pdalgd at gmail.com> wrote: > I had a look at this

I had a look at this a while back and it didn't seem to be easy. The path of least resistance would seem to be to use SAS itself to create a data set with fewer columns, but of course that requires you to get access to SAS. Otherwise, I think you'd have to modify sas7bdat::read.sas7bdat to drop unselected columns. That function is pure R code, so it might not be quite as hard as it

I never understood that why is the value returned by as.date function in the library(survival) never matches with the description given in the help file: Following is the extract from ?as.date Description: Converts any of the following character forms to a Julian date: 8/31/56, 8-31-1956, 31 8 56, 083156, 31Aug56, or August 31 1956. Usage: as.date(x, order = "mdy", ...)

I define the following function: (Please don't wonder about the use of this function, this is just a simplified version of my actual function. And please don't spend your time in finding an alternate way of doing the same as the following does not exactly represent my function. I am only interested in a good explanation) > f1 =

Hello everyone, I want to import data from huge sas files with 100s of columns. The good thing is that I am only interested in a few selected columns. Is there any way to do that without loading the full dataset. I have tried two functions: (1) read.sas7bdat *[from library 'sas7bdat']*, and (2) read_sas *[from library 'haven']. *But couldn't find what I am looking for. Best

Hi, I am using rpart package to fit classification trees. library(rpart) fit <- rpart(Kyphosis ~ Age + Number + Start, data=kyphosis) plot(fit,uniform=T) text(fit, use.n=TRUE) But I am unable to label the branches (not the nodes) of the tree. Can somebody help me out in this? Thank you, Regards Utkarsh Singhal | Amba Research Ph +91 80 3980 8017 | Mob +91 99 0295 8815

Hi All, I want to "merge" two datasets by column "ID" and I don't want the result to be sorted by "ID". I am doing the following: > z = merge(x, y, by = "ID", sort=F) The result is not sorted by "ID". But (as oppose to what I expected) it is not even in the original order of either "x" or "y". Can

Hi R, I am using nnet function of nnet package to fit neural networks. Now I want to get a unique solution every time I run the function for the same data. If I give rang=0, it solves my problem but I am not sure whether I am doing the right thing. Any comments are welcome. Thank you, Regards Utkarsh Singhal | Amba Research Ph +91 80 3980 8017 | Mob +91 99 0295 8815 Bangalore * Colombo

Hi All, There are several replies to the question below, but I think there must exist a better way of doing so. I just want to check whether all the elements of a vector are same. My vector has one million elements and it is highly likely that there are distinct elements in the first few itself. For example: > x = c(1,2,rep(1,100000)) I want the answer as FALSE, which is clear from the