similar to: bigmemory doubt

Hello All, I tried to intall the bigmemory package from a CRAN mirror site and received the following output while installing. Any idea what's going on and how to fix it? The system details are provided below. --------------------- begin error messages ----------------------- * installing *source* package 'bigmemory' ... checking for Sun Studio compiler...no checking for

Hi, I am trying to run the bigmemory example provided on the http://www.bigmemory.org/ The example runs on the "airline data" and generates summary of the csv files:- library(bigmemory) library(biganalytics) x <- read.big.matrix("2005.csv", type="integer", header=TRUE, backingfile="airline.bin", descriptorfile="airline.desc",

To answer your first question about read.big.matrix(), we don't know what your acc3.dat file is, but it doesn't appear to have been detected as a standard file (like a CSV file) or -- perhaps -- doesn't even exist (or doesn't exist in your current directory)? Next: > In addition, I am planning to do a multiple imputation with MICE package > using the data read by bigmemory

Hi, With reference to the mail below, I have large datasets, coming from various different sources, which I can read into filebacked big.matrix using library bigmemory. I want to merge them all into one 'big.matrix' object. (Later, I want to run regression using library 'biglm'). I am unsuccessfully trying to do this from quite some time now. Can you please

I work with problems that have rather large data requirements -- typically a bunch of multigig arrays. Given how generous R is with using memory, the only way for me to work with R has been to use bigmatrices from bigmemory package. One thing that is missing a bit is interoperability of bigmatrices with C++ and possibly java. What i mean by that is API that would allow read and write filebacked

I recently learned about the bigmemory and foreach packages and am trying to use them to help me create a very large matrix. Without those packages, I can create the type of matrix that I want with 10 columns and 5e6 rows. I would like to be able to scale up to 5e9 rows, or more, if possible. I have created a simplified example of what I'm trying to do, below. The first part of the

I have pretty big data sizes, like matrices of .5 to 1.5GB so once i need to juggle several of them i am in need of disk cache. I am trying to use bigmemory package but getting problems that are hard to understand. I am getting seg faults and machine just hanging. I work by the way on Red Hat Linux, 64 bit R version 10. Simplest problem is just saving matrices. When i do something like

Hi all, I'm on a Linux server with 48Gb RAM. I did the following: x <- big.matrix(nrow=20000,ncol=500000,type='short',init=0,dimnames=list(1:20000,1:500000)) #Gets around the 2^31 issue - yeah! in Unix, when I hit the "top" command, I see R is taking up about 18Gb RAM, even though the object x is 0 bytes in R. That's fine: that's how bigmemory is supposed to

I am using the library(bigmemory) to handle large datasets, say 1 GB, and facing following problems. Any hints from anybody can be helpful. _Problem-1: _ I am using "read.big.matrix" function to create a filebacked big matrix of my data and get the following warning: > x = read.big.matrix("/home/utkarsh.s/data.csv",header=T,type="double",shared=T,backingfile

Hi, I have a preallocated dataframe to which I have to add a list whose vectors will become rows in the dataframe. This is done iteratively. How can I do this? I'm dealing with very large numbers...list of 1000 vectors to a dataframe of 200000 iteratively for e.g. my list is as follows ll<-list(c("1","2","3"),c("2","3","4")) I

Hi, I have a vector who contents should look like this, "c" "d" NULL "e" "f" etc or 4 5 6 NULL 7 8 9 how can I represent the null value? [[alternative HTML version deleted]]

Hi, I need to check if a string "<rh>a,b,c,d<rh>" is delimited by two "<rh>" 's as efficiently as possible(I need to do this a lot of times) and return TRUE. Can someone suggest a good technique? [[alternative HTML version deleted]]

Hi, I have a dataframe column of the form v<-c("Fri Feb 05 20:00:01.43000 2010","Fri Feb 05 20:00:02.274000 2010","Fri Feb 05 20:00:02.274000 2010","Fri Feb 05 20:00:06.34000 2010") I need to convert this to datetime form. I did the following.. lapply(v,function(x){strptime(x, "%a %b %d %H:%M:%OS %Y")}) This gives me a list that looks like

Hi, I'm playing around with the 'bigmemory' package, and I have finally managed to create some really big matrices. However, only now I realize that there may not be functions made for what I want to do with the matrices... I would like to perform a cluster analysis based on a big.matrix. Googling around I have found indications that a certain kmeans.big.matrix() function should

Hi, I have a .Call in my R function in a loop that repeats a certain number of times. Each time, the .Call returns a list. So, when I say something like, y<-func() would y be a list of lists?(as many as the number of loops?) [[alternative HTML version deleted]]

Hi, Is it possible for me to store a list of vectors of 1 million entries? like, cc<-list(c(1,2,........1million),c(1,2,........1million)....) also what is the length of the longest string in R? I keep getting info from a socket and keep pasting on a string...when will this start becoming a problem? [[alternative HTML version deleted]]

Hi, I have a loop as follows, dataStr <- character(0) repeat{ fstr<-read.socket(sockfd) if(fstr=="") break dataStr<-paste(dataStr,fstr) } at what point does dataStr stop accepting(gets full)? I'm sending millions of records over the socket and need to know if all of it can go into dataStr. Also, Incase all of it cannot go into dataStr, I need to parse each

Hi, is there a queue implementation in R? [[alternative HTML version deleted]]

Hi, Is it possible to uncompress gzipped data coming over a socket? [[alternative HTML version deleted]]

Hi, I have a list which looks like this... > str(y) List of 10 $ : chr [1:4] "ABCD" "5" "0" "1" $ : chr [1:4] "DEF" "15" "1" "16" $ : chr [1:4] "AAA" "2" "17" "8" $ : chr [1:4] "SSS" "15" "25" "1" $ : chr [1:4] "III"