similar to: Appending objects created using filehash package

Dear R users, I try to use a very large file (~3 Gib) with the filehash package. The length of the dataset is around 4,000,000 obs. I get this message from R while trying to "load" the dataset (named "cc084.csv"): > dumpDF(read.csv("cc084.csv", header=T), dbName="db01") Erreur : impossible d'allouer un vecteur de taille 15.6 Mo (French) Error:

Dear Experienced R Practitioners, I have 4GB .txt data called "dataset.txt" and have attempted to use *ff, bigmemory, filehash and sqldf *packages to import it, but have had no success. The readLines output of this data is: readLines("dataset.txt",n=20) [1] " "

Trying to install package 'filehash' I get the following error on FreeBSD 9.0-CURRENT (amd64) with R version 2.11.0 (2010-01-15 r50990): ----------------------------------- R CMD INSTALL filehash_2.0-1.tar.gz * installing to library '/usr/local/lib/R/library' * installing *source* package 'filehash' ... ** libs gcc -std=gnu99 -I/usr/local/lib/R/include

Hi (Roger), I saw the announcement of filehash v2.0 and the sentence "This development has lead to better file locking for concurrent access and faster reading and writing of data in general" caught my attention. What kind of file locking do you refer to here? I am looking for a mechanism that can be used to lock files for reading and/or writing, and I'd love to have a cross

I have just uploaded to CRAN version 2.0 of the 'filehash' package. This version contains a major rewriting of many of the internals (much rewritten in C) for the DB1 format, which is the default. This development has lead to better file locking for concurrent access and faster reading and writing of data in general. In addition to rewriting the internals, I have added two modules for a

I have just uploaded to CRAN version 2.0 of the 'filehash' package. This version contains a major rewriting of many of the internals (much rewritten in C) for the DB1 format, which is the default. This development has lead to better file locking for concurrent access and faster reading and writing of data in general. In addition to rewriting the internals, I have added two modules for a

Hi all, I am trying to use the filehash library to analyze a 5M by 20 matrix with both double and string data types. After consulting a few tutorials online, it seems as though one needs to first read the data into R; then create an R object; and then assign that object a location in my computer via filehash. It seems like the benefit of this is minimizing memory allocation when running

Hi, I have been using filehash for a while. It has performed very well. However, recently I found filehash gives an error when I need to do something like db[c("a", "b")] when the db is in RDS format. Does any one know a way to get around that? The code below reproduces the error thanks Jeff filehashOption(defaultType = "DB1") dbCreate("mydb3", type =

Dear R-list I'm on my way to start a new project on a rather big panel, consisting of approximately 8 million observations in 30 waves of data and about 15 variables. I have a similar data set that is approximately 7 gigabytes in size. Until now I have done my data management in SAS, and Stata, mostly identifying spells, counting events in intervals, and a like, but I would like to

Hello, I'm using filehash on the windows XP and it has been working fine with the newest R version 2.6.2. However, on the windows vista, when I ran the same code, I got the following error: > dbCreate("simdb") #create simdb database [1] TRUE > db<-dbInit("simdb") #initiate an object of database Error in sprintf(gettext(fmt, domain = domain), ...) : object

I'm running into some hard-to-understand behavior with the evaluation environment when NextMethod is used. I'm using square-bracket indexing into objects, and the evaluation environment of the expression inside the square brackets seems to change depending on what kind of comparison operators are used. This behavior happens when the following conditions are met (this is what I've

Dear R experts, I remember a similar function existed and have been mentioned in R-help before. I tried my best to search but I really can't find it out. suppose I have an data frame like this: > somedata <- data.frame(age.min = 1, age.max = 1.5, male = TRUE, l = -1.013, m=16.133, s=0.07656) In order to back up the data and I don't want to use write.table(), I would like to back

Hi All, I have a few small questions about the usage of the C findVar( ) function when used in C code called with '.Call'. In my case I create an R function with an argument. This function calls some C code in which I use findVar( ) to retrieve the values from the argument. Ofcourse normally I would just give the values as argument to .Call, but in my project I need to use findVar for

Hi All, I have a very (very) large tab-delimited text file without headers. There are only 8 columns and millions of rows. I want to make numerous pieces of this file by sub-setting it for individual stations. Station is given as in the first column. I am trying to learn and use sqldf package for this but am stuck in a couple of places. To simulate my requirement, I have taken iris dataset as an

Dear R helpers: I am a newbie to R and have a question related to cleaning large data frames in R. So far, I have been using SAS for data cleaning because my data sets are relatively large (handling multiple files, each could be as large as 5-10 G). I am not a fan of SAS at all and am eager to move data cleaning tasks into R completely. Seems to me, there are 3 options. Using SQL, ff or

Dear R'ers, I have trouble installing tikzDevice in Ubuntu.  When I use install.packages("tikzDevice"), it gives error message: ERROR: dependency ‘filehash’ is not available for package ‘tikzDevice’ * removing ‘/usr/local/lib/R/site-library/tikzDevice’ Then I tried filehash installation, I get the message: "package ‘filehash’ is not available (for R version 2.13.1)"

Hi everybody! I'm trying to install the tikzDevice package, and I keep on getting the > ERROR: dependency ‘filehash’ is not available for package ‘tikzDevice’ I tried install.packages('filehash') and I get > package ‘filehash’ is not available Does anybody have the same problem or any hint? thank youhelena [[alternative HTML version deleted]]

Hello, I have just installed v 2.6.2 on a new computer running Windows XP and tried to perform 'update packages' via the menu option on the R console. Any advice on the following problem is much appreciated. Bob Below are the warning and error messages received. A search of the hard drive does not reveal any file including "RtmpgMMu03/libloc" . >

Hi all, Have a problem. Trying to read in a data set that has about 112,000,000 rows and 8 columns and obviously enough it was too big for R to handle. The columns are mode up of 2 integer columns and 6 logical columns. The text file is about 4.2 Gb in size. Also I have 4 Gb of RAM and 218 Gb of available space on the hard drive. I tried the dumpDF function but it was too big. Also tried bring in

served piping hot fresh from my svn repo: http://svn.techno-weenie.net/projects/mongrel/mongrel_send_file/README == mongrel_send_file GemPlugin This is a simple plugin to handle the sending of secure files from a rails app. Here''s the typical process: - Rails app authorizes user to download file - Rails app sets file info in the session, redirects to custom URL like