similar to: New version of package ff

Dear R friends. After having some troubles learning how to create a ffdf object, now I find myself having problems saving it. this is the data i´d like to save: str(DATA) List of 3 $ virtual: 'data.frame': 6 obs. of 7 variables: .. $ VirtualVmode : chr "double" "short" "integer" "integer" ... .. $ AsIs : logi FALSE FALSE FALSE

Dear list, I am trying to understand and use the ff package. As I had some problems saving some ff objects, and as I did not fully manage to understand the whole concept of *.ff, *.ffData and *.RData with the help of the documentation, I tried to reproduce the examples from the help of ffsave. When I ran, however : (copied from the help) message("let's create some ff objects")

I use the ff-package for large data and have good success in session. To ensure my data is available for the next R session, I save my workspace and use the /ffsave/ command to archive my large data objects. Syntax is simple: > ffsave(file="~/path/archiveName", list="objectName") when I have multiple files to save, the list is expanded using c(). When I start a new R

Hi My aim is to read a large .csv file into R. I ran the following code and am using R version 10.1 on Windows. >library(ff) > read.csv.ffdf(x=NULL,"file.csv",fileEncoding="",nrows=-1,first.rows=NULL,next.rows=NULL,levels=NULL,appendLevels=TRUE,FUN="read.table",transFUN=NULL,asffdf_args=list(),BATCHBYTES=getOption("ffbatchbytes"),VERBOSE=FALSE)

Hi, I am running the examples in page 70 of the ff package document, but it failed with the following error > cat("let's create some ff objects\n") let's create some ff objects > n <- 8e3 > a <- ff(sample(n, n, TRUE), vmode="integer", length=n, filename="d:/tmp/a.ff") > b <- ff(sample(255, n, TRUE), vmode="ubyte", length=n,

Dear R community, The next release of package ff is available on CRAN. With kind help of Brian Ripley it now supports the Win64 and Sun versions of R. It has three major functional enhancements: a) new fast in-memory sorting and ordering functions (single-threaded) b) ff now supports on-disk sorting and ordering of ff vectors and ffdf dataframes c) ff integer vectors now can be used as

Dear R community, The next release of package ff is available on CRAN. With kind help of Brian Ripley it now supports the Win64 and Sun versions of R. It has three major functional enhancements: a) new fast in-memory sorting and ordering functions (single-threaded) b) ff now supports on-disk sorting and ordering of ff vectors and ffdf dataframes c) ff integer vectors now can be used as

Lawrence, My understanding is that only a minor change is needed in ff's C++ layer in order to remove the 64bit compiler warnings/errors. The C++ layer is maintained by Daniel Adler, who can give you an outlook if/when he plans to attack this. Until a 64bit version of ff is available, you might consider using the 32bit win version of R and ff on a 64bit win machine: while 32bit R itself

Hi, I have been using ffsave.image() to save mixture of ff and normal objects in my workspace. e.g. ffsave.image(file = "C:\output\saveobjects", rootpath = "D:\fftempdir", safe = TRUE) It works fine but once my workspace has large (~4GB) objects, I get the error: Error in ffsave.image(file = "C:\output\savedobjects", rootpath = "D:\fftempdir", safe =

Dear R community, ff Version 2.1.1 is available on CRAN. It now supports large data.frames, csv import/export, packed atomic datatypes and bit filtering from package 'bit' on which it depends from now. Some performance results in seconds from test data with 78 mio rows and 7 columns on a 3 GB notebook: sequential reading 1 mio rows: csv = 32.7 ffdf = 1.3 sequential writing 1 mio

Dear R community, ff Version 2.1.1 is available on CRAN. It now supports large data.frames, csv import/export, packed atomic datatypes and bit filtering from package 'bit' on which it depends from now. Some performance results in seconds from test data with 78 mio rows and 7 columns on a 3 GB notebook: sequential reading 1 mio rows: csv = 32.7 ffdf = 1.3 sequential writing 1 mio

I'm having trouble getting ffdfindexget to work right in Windows. Even the most trivial of examples gives me problems. > myVec = ff(1:5) > another = ff(10:14) > littleFrame = ffdf(myVec, another) > posVec = ff(c(2, 4), vmode = 'integer') > ffdfindexget(littleFrame, posVec) Error in if (any(B < 1)) stop("B too small") : missing value where TRUE/FALSE

Hi All, My script generates a mixture of normal and ff objects. I need to run the script for different parameter settings. Very often I would like to save the workspace for each parameter setting, so that I can get back to it later on. Is there an easy way to do this, instead of needing to save individual ff objects separately? I've tried the naive way of just saving the workspace, only to

I've noticed that ff uses the unzip utility available on it's host OS to load datasets via ffload. It seems to work fine in linux, but when I try to use the package in Windows (hence dling Windows unzip utils) I get 2 errors, one telling me that the options aren't being passed to unzip correctly and another indicating that my filename is incorrect. First, does anyone know of an unzip

Greetings useRs, Can anyone provide an example how to use ff to feed a very large data frame to glm? The data.frame cannot be loaded in R using conventional read.csv as it is too big. glm(...,data=ff.file) ?? Thank you Stephen B

Hi, After reading the slices from ff project site I know ff can be used with snowfall to achieve parallel processing, I want to know is it the only way, can we combine package foreach and ff, especially in a SMP server, with doMC and multicore behind foreach. Xiaobo.Gu [[alternative HTML version deleted]]

Dear R community, ff Version 2.0 is available on CRAN. Based on paging concepts from version 1.0, 2.0 is a major redesign of this package for handling large datasets. We have implemented numerous enhancements and performance improvements to make this package suitable as a 'base' package for large data processing. The ff package provides atomic data structures that are stored on disk

Dear R community, ff Version 2.0 is available on CRAN. Based on paging concepts from version 1.0, 2.0 is a major redesign of this package for handling large datasets. We have implemented numerous enhancements and performance improvements to make this package suitable as a 'base' package for large data processing. The ff package provides atomic data structures that are stored on disk

Hi all, I am having trouble loading a ff object previously saved in a different computer. I have both files .ffData and .RData, and the first of them is 13Mb large from which I know the data is therein. But when I try to ffload it, checkdir error: cannot create /home/_myUser_ Permission denied unable to process home/_myUser_/Rtempdir/ff1a831d500b8d.ff. and

Hello everyone, I need to create and work with some big matrices that actually have somewhat over 2 million columns and 117 rows. To do some calculations on such big matrices R just needs too much memory for my PC (4GB installed). So I need a solution to work with large datasets. I'm trying to use the ff-package but I don't think I really understand the whole functionality of the