similar to: Any way to get read.table.ffdf() (in the ff package) to pass colClasses or comment.char parameters through to read.fwf() ?

*Dear R users, Ive just started using the ff package. There is a csv file (~4Gb) with 7 columns and 6e+7 rows. I want to read only column from the file, skipping the first 100 rows. Below Ive provided different outcomes, which will clarify my problem * > sessionInfo() R version 2.14.2 (2012-02-29) Platform: x86_64-pc-mingw32/x64 (64-bit) locale: ... attached base packages: [1] tools

Hi, The read.csv.ffdf function in package ff will create the ff object physical file in the default directories, I am trying to let the files created in the paths users specify, I think the point is to make use of the asffdf_args parameter, I have a test CSV file named D:\rtemp\fftest.csv, the content of the file is as following: col1,col2,col3 1,"amber",2.4 2,"linda",4.5

Hello fellow R users, I am trying to read a 6.9 million row text file with 26 columns separated by spaces into R using ff. When I specify a small number for first.rows, next.rows and nrows it is read with no issue. However, when I try to specify larger next.rows values and no nrows parameter to read the entire file, I keep getting errors. Please see code below. I am trying to this on a m1.large

I've spent some time trying to wrap my head around reading in large csv files with the ff-package. I think I know how to do it, but am bumping into some problems. I've tried to recreate the issues as best as I can with a smaller example and maybe someone can help explain the problems. The following code just creates a csv file with an integer column, character column and logical column.

Hello. I'm using as.ffdf(mydataframe) to create ffdf objects inside an lapply loop and returning that. I then use crbind to combine the lapply results into allData. So...simplified flow looks like this. res <- lapply(1:nchunks, function(n) { blah blah with nth chunk mydataframe <- data.frame(blah blah) dat <-

I have a big data set that includes character variables of many different values. I'm trying to use ff to read the data and then use biglm.big.matrix to build linear models. However, since big.matrix will convert all character vectors to factors and the character labels will be lost. I decided to create a lookup table outside of R for my character columns and use numbers to represent different

Hello everyone, I need to create and work with some big matrices that actually have somewhat over 2 million columns and 117 rows. To do some calculations on such big matrices R just needs too much memory for my PC (4GB installed). So I need a solution to work with large datasets. I'm trying to use the ff-package but I don't think I really understand the whole functionality of the

Hi My aim is to read a large .csv file into R. I ran the following code and am using R version 10.1 on Windows. >library(ff) > read.csv.ffdf(x=NULL,"file.csv",fileEncoding="",nrows=-1,first.rows=NULL,next.rows=NULL,levels=NULL,appendLevels=TRUE,FUN="read.table",transFUN=NULL,asffdf_args=list(),BATCHBYTES=getOption("ffbatchbytes"),VERBOSE=FALSE)

Full_Name: Shigeru Mase Version: R 1.9.1 and 2.0.0 OS: Debian Linux Submission from: (NULL) (219.35.150.115) I tried to read the following text file "test.txt" with 5 lines using read.fwf() function: # comment 1 1234567 # comment 2 1 234567 # comment 3 12345 67 # comment 4 # comment 5 In R 1.9.1, I got the following result (since I have R 2.0.0 installed now I cannot reproduce

# Your mailer is set to "none" (default on Windows), # hence we cannot send the bug report directly from R. # Please copy the bug report (after finishing it) to # your favorite email program and send it to # # r-bugs@r-project.org # ###################################################### Is this intended behavior for the read.fwf(na.strings="-999")? I anticipated that

Hello, I am having a problem with write.fwf in Windows. I wrote a code to ingest a number of text files with weather data in them, process them, and then output a text file with two parts: 1) a set of column names, 2) the processed data table. I wrote and tested the program on my Mac, and it worked fine. However, on the windows machine, where I intend the work to be done, when I run the

Full_Name: Giovanni Bonafe' Version: 2.2.0 OS: Linux Submission from: (NULL) (195.62.164.225) If the file "example.dat" is like this: aaa bbb ccc 3.4 1.2 5.6 4.6 10 32 667 343 1.7 With the older 1.9.1, as expected: > data<-read.fwf(file = "example.dat",widths=c(3,4,4),header=TRUE) > data aaa bbb ccc 1 3.4 1.2 5.6 2 4.6 10.0 32.0 3 667.0 343.0

Is there a way to handle the widths of values being written to a file using wrtite.fwf() ? For example, I used read.fwf(file, width.vector) to read a file. After making the necessary data manipulation, I want to write the data to a new file in the same width.vector format. Is there a way to specify this? Thanks in Advance Aparna [[alternative HTML version deleted]]

Hi, I am using version 2.1-1 of the ff package. I have a data set with 80 million rows and I need to create a new ffdf object, subseting by values in one of the original ffdf's columns. Here is my code: bigData <- read.table.ffdf(file="/data/demodata/data/smallData.txt", next.rows=1e5, head=TRUE, sep="|") dim(bigData) N <- nrow(bigData);N select <- ff(

Here is an example of the data I'm reading in using read.fwf: 5342.0 5450.4 0.9200 0.4506 34.7030 27.8411 37.1306 46.0034 0.0000 5438.0 5549.6 0.9300 0.4477-34.7280 27.8616 37.1506 46.0234 0.0000 And the commands I'm using to read it in: widths <- c(7,7,8,8,8,8,8,8,8) filedata <- read.fwf(fileopen, widths,sep="",dec=".") The problem is with

Running R 1.6.1 Linux Slackware 8.1 233MHZ AMD-K6 96MB RAM Using read.fwf, I tried to open a fixed-width file that of about 4 MB residing in the working directory, using the command below: dat<-read.fwf("sc01aai.dat", widths=fields$length) where fields$lengths is a vector of column widths, 28 to be exact. The data are a mix of character, text, and factor variables. R started

I'm having trouble getting ffdfindexget to work right in Windows. Even the most trivial of examples gives me problems. > myVec = ff(1:5) > another = ff(10:14) > littleFrame = ffdf(myVec, another) > posVec = ff(c(2, 4), vmode = 'integer') > ffdfindexget(littleFrame, posVec) Error in if (any(B < 1)) stop("B too small") : missing value where TRUE/FALSE

Full_Name: Emmanuel Paradis Version: 2.1.1 OS: Linux Submission from: (NULL) (193.49.41.105) read.fwf(..., header = TRUE) does not work properly since: 1/ the original header is printed on the console and not in FILE; 2/ the different 'parts' of the header should be separated with tabs to work with the call to read.table. Here is a suggested fix for src/library/utils/R/read.fwf.R:

Hi there, I am having trouble subsetting a data frame by a conditional via one column (of many). I read the file into R through "read.fwf," where I specified column widths. Original data is .DAT. I then utilized "names" function to read in column headings. For one column, PRVDR_NUM, I wish to further amend the entire data set, but only have PRVDR_NUM == 050108. This is

Dear R-list This is just message to inform that the there is an issue with write.fwf in the gdata library (from version 2.5.0 on). It does not seem to accept further arguments to write.table like "eol" as the help file indicates as it stops when executing tmp <- lapply(x, format.info, ...). Great package though - I use it a lot except for this function :) See example below. >