search for: vmodes

...tching here: library(ff) my.as.ffdf.data.frame <- function (x, vmode = NULL, col_args = list(), ...) { rnam <- attr(x, "row.names") if (is.integer(rnam)) { if (all(rnam == seq_along(rnam))) rnam <- NULL else rnam <- as.character(rnam) } x <- as.list(x) vmodes <- vector("list", length(x)) if (!is.null(vmode)) { nam <- names(x) if (is.list(vmode)) { vnam <- names(vmode) i <- match(vnam, .vmode[.vimplemented]) if (any(is.na(i))) stop("vmodes not implemented: ", paste("'", vnam[is.n...

Dear list, I am trying to understand and use the ff package. As I had some problems saving some ff objects, and as I did not fully manage to understand the whole concept of *.ff, *.ffData and *.RData with the help of the documentation, I tried to reproduce the examples from the help of ffsave. When I ran, however : (copied from the help) message("let's create some ff objects")

Dear R friends. After having some troubles learning how to create a ffdf object, now I find myself having problems saving it. this is the data i´d like to save: str(DATA) List of 3 $ virtual: 'data.frame': 6 obs. of 7 variables: .. $ VirtualVmode : chr "double" "short" "integer" "integer" ... .. $ AsIs : logi FALSE FALSE FALSE

Dear R Helpers, I would like to report on an error in the ff package here. The ff package is an R package which enables us to store large data on disk systematically and have fast access to the database. I used the package in Linux as a guest OS of VirtualBox, and executed the following commands. library(ff)

Hi, I am running the examples in page 70 of the ff package document, but it failed with the following error > cat("let's create some ff objects\n") let's create some ff objects > n <- 8e3 > a <- ff(sample(n, n, TRUE), vmode="integer", length=n, filename="d:/tmp/a.ff") > b <- ff(sample(255, n, TRUE), vmode="ubyte", length=n,

Hi, I have two questions on using the ff package and wonder if anyone who used ff can share some thoughts. I need to save a matrix as a memory-mapped file and load it back later. To save the matrix, I use mat = matrix(1:20, 4, 5) matFF = ff(mat, dim=dim(mat), filename="~/a.mat", overwrite=TRUE, dimnames = dimnames(mat)) To load it back, I use matFF2 = ff(vmode = "double",

I've spent some time trying to wrap my head around reading in large csv files with the ff-package. I think I know how to do it, but am bumping into some problems. I've tried to recreate the issues as best as I can with a smaller example and maybe someone can help explain the problems. The following code just creates a csv file with an integer column, character column and logical column.

Hello, I frequently have to export a large quantity of data from some source (for example, a database, or a hand-written perl script) and then read it into R. This occasionally takes a lot of time; I'm usually using read.table("filename",comment.char="",quote="") to read the data once it is written to disk. However, I *know* that the program that generates

Hello to everyone. I'm trying to use the %in% to match to vectors in ff format. a<-as.ff(data[,1]) %in% fire$fecha > aff (open) logical length=3653 (3653) [1] [2] [3] [4] [5] [6] [7] [8] [3646] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE : FALSE [3647] [3648] [3649] [3650] [3651] [3652] [3653] FALSE FALSE FALSE FALSE FALSE FALSE

Hi, I am using version 2.1-1 of the ff package. I have a data set with 80 million rows and I need to create a new ffdf object, subseting by values in one of the original ffdf's columns. Here is my code: bigData <- read.table.ffdf(file="/data/demodata/data/smallData.txt", next.rows=1e5, head=TRUE, sep="|") dim(bigData) N <- nrow(bigData);N select <- ff(

I'm having trouble getting ffdfindexget to work right in Windows. Even the most trivial of examples gives me problems. > myVec = ff(1:5) > another = ff(10:14) > littleFrame = ffdf(myVec, another) > posVec = ff(c(2, 4), vmode = 'integer') > ffdfindexget(littleFrame, posVec) Error in if (any(B < 1)) stop("B too small") : missing value where TRUE/FALSE

I've noticed that ff uses the unzip utility available on it's host OS to load datasets via ffload. It seems to work fine in linux, but when I try to use the package in Windows (hence dling Windows unzip utils) I get 2 errors, one telling me that the options aren't being passed to unzip correctly and another indicating that my filename is incorrect. First, does anyone know of an unzip

Hi, The read.csv.ffdf function in package ff will create the ff object physical file in the default directories, I am trying to let the files created in the paths users specify, I think the point is to make use of the asffdf_args parameter, I have a test CSV file named D:\rtemp\fftest.csv, the content of the file is as following: col1,col2,col3 1,"amber",2.4 2,"linda",4.5

I'm trying to import a table into R the file is about 700MB. Here's my first try: > DD<-read.table("01uklicsam-20070301.dat",header=TRUE) Error: cannot allocate vector of size 15.6 Mb In addition: Warning messages: 1: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : Reached total allocation of 1535Mb: see help(memory.size) 2: In scan(file, what,

Dear R community, ff Version 2.1.1 is available on CRAN. It now supports large data.frames, csv import/export, packed atomic datatypes and bit filtering from package 'bit' on which it depends from now. Some performance results in seconds from test data with 78 mio rows and 7 columns on a 3 GB notebook: sequential reading 1 mio rows: csv = 32.7 ffdf = 1.3 sequential writing 1 mio

Dear R community, ff Version 2.1.1 is available on CRAN. It now supports large data.frames, csv import/export, packed atomic datatypes and bit filtering from package 'bit' on which it depends from now. Some performance results in seconds from test data with 78 mio rows and 7 columns on a 3 GB notebook: sequential reading 1 mio rows: csv = 32.7 ffdf = 1.3 sequential writing 1 mio

Dear R community, The next release of package ff is available on CRAN. With kind help of Brian Ripley it now supports the Win64 and Sun versions of R. It has three major functional enhancements: a) new fast in-memory sorting and ordering functions (single-threaded) b) ff now supports on-disk sorting and ordering of ff vectors and ffdf dataframes c) ff integer vectors now can be used as

Dear R community, The next release of package ff is available on CRAN. With kind help of Brian Ripley it now supports the Win64 and Sun versions of R. It has three major functional enhancements: a) new fast in-memory sorting and ordering functions (single-threaded) b) ff now supports on-disk sorting and ordering of ff vectors and ffdf dataframes c) ff integer vectors now can be used as

Greetings useRs, Can anyone provide an example how to use ff to feed a very large data frame to glm? The data.frame cannot be loaded in R using conventional read.csv as it is too big. glm(...,data=ff.file) ?? Thank you Stephen B

Hello everyone, I need to create and work with some big matrices that actually have somewhat over 2 million columns and 117 rows. To do some calculations on such big matrices R just needs too much memory for my PC (4GB installed). So I need a solution to work with large datasets. I'm trying to use the ff-package but I don't think I really understand the whole functionality of the