search for: vmode

Really old subject?, so, all my apologizes for digging up but, since I also ran into this? maybe this hack can be useful to someone I propose monkey patching here: library(ff) my.as.ffdf.data.frame <- function (x, vmode = NULL, col_args = list(), ...) { rnam <- attr(x, "row.names") if (is.integer(rnam)) { if (all(rnam == seq_along(rnam))) rnam <- NULL else rnam <- as.character(rnam) } x <- as.list(x) vmodes <- vector("list", length(x)) if (!is.null(vmode)) {...

...erstand the whole concept of *.ff, *.ffData and *.RData with the help of the documentation, I tried to reproduce the examples from the help of ffsave. When I ran, however : (copied from the help) message("let's create some ff objects") n <- 8e3 a <- ff(sample(n, n, TRUE), vmode="integer", length=n, filename="d:/tmp/a.ff") b <- ff(sample(255, n, TRUE), vmode="ubyte", length=n, filename="d:/tmp/b.ff") x <- ff(sample(255, n, TRUE), vmode="ubyte", length=n, filename="d:/tmp/x.ff") y <- ff(sample(255, n...

Dear R friends. After having some troubles learning how to create a ffdf object, now I find myself having problems saving it. this is the data i´d like to save: str(DATA) List of 3 $ virtual: 'data.frame': 6 obs. of 7 variables: .. $ VirtualVmode : chr "double" "short" "integer" "integer" ... .. $ AsIs : logi FALSE FALSE FALSE FALSE FALSE FALSE .. $ VirtualIsMatrix : logi FALSE FALSE FALSE FALSE FALSE FALSE .. $ PhysicalIsMatrix : logi FALSE FALSE FALSE FALSE FALSE FALSE .. $...

...error in the ff package here. The ff package is an R package which enables us to store large data on disk systematically and have fast access to the database. I used the package in Linux as a guest OS of VirtualBox, and executed the following commands. library(ff) x<-ff(1:5,length=5,update=TRUE,vmode="double",filename="header.ff",overwrite=TRUE) Afterwards, I got the following error message. *** caught segfault *** address 0xffffffff, cause 'memory not mapped' Traceback: 1: .Call("get_vector", .ffmode[vmode], attr(x, "physical"), index, nr...

Hi, I am running the examples in page 70 of the ff package document, but it failed with the following error > cat("let's create some ff objects\n") let's create some ff objects > n <- 8e3 > a <- ff(sample(n, n, TRUE), vmode="integer", length=n, filename="d:/tmp/a.ff") > b <- ff(sample(255, n, TRUE), vmode="ubyte", length=n, filename="d:/tmp/b.ff") > x <- ff(sample(255, n, TRUE), vmode="ubyte", length=n, filename="d:/tmp/x.ff") > y <- ff(samp...

...who used ff can share some thoughts. I need to save a matrix as a memory-mapped file and load it back later. To save the matrix, I use mat = matrix(1:20, 4, 5) matFF = ff(mat, dim=dim(mat), filename="~/a.mat", overwrite=TRUE, dimnames = dimnames(mat)) To load it back, I use matFF2 = ff(vmode = "double", dim= ???, filename="~/a.mat", overwrite=F) However, I don't always know the dimension when loading the matrix back. If I miss the dim attributes, ff will return it as vector. Is there a way to load the matrix without specifying the dimension? The second questio...

...ot;'> data = read.csv.ffdf(x=NULL,file="data.csv",nrows=1001,first.rows = 500, next.rows = 1005,sep=",",colClasses = c("integer","character","logical"))Error in ff(initdata = initdata, length = length, levels = levels, ordered = ordered, : vmode 'character' not implemented> data = read.csv.ffdf(x=NULL,file="data.csv",nrows=1001,first.rows = 500, next.rows = 1005,sep=",",colClasses = rep("character",3))Error in ff(initdata = initdata, length = length, levels = levels, ordered = ordered, : vmode ...

Hello, I frequently have to export a large quantity of data from some source (for example, a database, or a hand-written perl script) and then read it into R. This occasionally takes a lot of time; I'm usually using read.table("filename",comment.char="",quote="") to read the data once it is written to disk. However, I *know* that the program that generates

...87 1966 Mensajes de aviso perdidosIn if (useNA == "no") c(NA, NaN) : la condición tiene longitud > 1 y sólo el primer elemento será usado in a regular data.frame I use data[a,] to extract the rows that a == TRUE, but when i do this in a ffdf i get this error: > data[a,]Error: vmode(index) == "integer" is not TRUE I'm just learning how to use the ff package so, obviously I'm missing something If any of you knows how to solve this, please teach me. Thank you so much. Lucas. [[alternative HTML version deleted]]

...d I need to create a new ffdf object, subseting by values in one of the original ffdf's columns. Here is my code: bigData <- read.table.ffdf(file="/data/demodata/data/smallData.txt", next.rows=1e5, head=TRUE, sep="|") dim(bigData) N <- nrow(bigData);N select <- ff( vmode='logical', length=N, update=FALSE) for (i in chunk(bigData, from=1, to=N, by=1e5)) select[i] <- ifelse(bigData[i,]$C.2 > 0.5, TRUE, FALSE) bd <- bigData[select,] Error in UseMethod("as.hi") : no applicable method for 'as.hi' applied to an object of class...

I'm having trouble getting ffdfindexget to work right in Windows. Even the most trivial of examples gives me problems. > myVec = ff(1:5) > another = ff(10:14) > littleFrame = ffdf(myVec, another) > posVec = ff(c(2, 4), vmode = 'integer') > ffdfindexget(littleFrame, posVec) Error in if (any(B < 1)) stop("B too small") : missing value where TRUE/FALSE needed In addition: Warning message: In bbatch(n, as.integer(BATCHBYTES/theobytes)) : NAs introduced by coercion I'm not really sure that it...

...lename is incorrect. First, does anyone know of an unzip utility that works with ff or am I doing something wrong there? Second, why does ff seem to add .Rdata and .ffData when looking for files but adds no extension when saving them? Code below... library('ff') ffcolTest = ff(c(1:500), vmode = "integer", length = 500, filename = <some folder>\\ffTest') quit() library('ff') ffload("<same path as above>") Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection In addition: Warning messages: 1: running command 'unzip -Z -...

...ol3 to D:/a.f,D:/b.f,D:/c.f respectively fdf <- read.csv.ffdf(file="D:/rtemp/fftest.csv",asffdf_args = list( col_args = c(list(filename="D:/a.f"), list(filename="D:/b.f"), list(filename="D:/c.f")))) and the error message is : Error in as.ff.default(1:2, vmode = NULL, filename = "D:/a.f", filename = "D:/b.f", : formal argument "filename" matched by multiple actual arguments I also tried the following: > fdf <- read.csv.ffdf(file="D:/rtemp/fftest.csv",asffdf_args = list( col_args = list(filename=c("...

I'm trying to import a table into R the file is about 700MB. Here's my first try: > DD<-read.table("01uklicsam-20070301.dat",header=TRUE) Error: cannot allocate vector of size 15.6 Mb In addition: Warning messages: 1: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : Reached total allocation of 1535Mb: see help(memory.size) 2: In scan(file, what,

...ing 1 mio rows: csv = 32.7 ffdf = 1.3 sequential writing 1 mio rows: csv = 35.5 ffdf = 1.5 Examples of things you can do with ff and bit: - direct random access to rows of large data-frame instead of talking to SQL database (?ffdf) - store 4-level factor like A,T,G,C with 2bit instead of 32bit (?vmode) - fast chunked iteration (?chunk) - run linear model on large dataset using biglm (?chunk.ffdf) - handle boolean selections by factor 32 faster and less RAM consuming (?bit) - handle very skewed selections very fast (?bitwhich) - parallel access to large dataset just by sending ff's small meta...

...ing 1 mio rows: csv = 32.7 ffdf = 1.3 sequential writing 1 mio rows: csv = 35.5 ffdf = 1.5 Examples of things you can do with ff and bit: - direct random access to rows of large data-frame instead of talking to SQL database (?ffdf) - store 4-level factor like A,T,G,C with 2bit instead of 32bit (?vmode) - fast chunked iteration (?chunk) - run linear model on large dataset using biglm (?chunk.ffdf) - handle boolean selections by factor 32 faster and less RAM consuming (?bit) - handle very skewed selections very fast (?bitwhich) - parallel access to large dataset just by sending ff's small meta...

...idea to reduce 'ffmaxbytes' on slaves or avoid ff sorting there completely. o New generic 'pagesize' with method 'pagesize.ff' which returns the current pagesize as defined on opening the ff object. USER VISIBLE CHANGES o [.ff now returns with the same vmode as the ff-object o Certain operations are faster now because we worked around unnecessary copying triggered by many of R's assignment functions. For example reading a factor from a (well-cached) file is now 20% faster and thus as fast as just creating this factor in-RAM usin...

...idea to reduce 'ffmaxbytes' on slaves or avoid ff sorting there completely. o New generic 'pagesize' with method 'pagesize.ff' which returns the current pagesize as defined on opening the ff object. USER VISIBLE CHANGES o [.ff now returns with the same vmode as the ff-object o Certain operations are faster now because we worked around unnecessary copying triggered by many of R's assignment functions. For example reading a factor from a (well-cached) file is now 20% faster and thus as fast as just creating this factor in-RAM usin...

Greetings useRs, Can anyone provide an example how to use ff to feed a very large data frame to glm? The data.frame cannot be loaded in R using conventional read.csv as it is too big. glm(...,data=ff.file) ?? Thank you Stephen B

...ully someone can help me either with the ff-package or a different solution. I am saving some calculated matrices as ff-objects as follows: require(ff) nr <- 117; nc <- 50 dat <- sample(0:100, size=(nr*nc), replace=TRUE) a <- matrix(dat, nrow=nr) ncols <- (nc*(nc-1))/2 b <- ff(vmode="double", dim=c(nr, ncols)) namb <- vector(mode="character", length=ncols) x <- 1 for(i in 1:(nc-1)){ for(j in (i+1):nc){ b[,x] <- a[,i]+a[,j] namb[x] <- paste(i, "_", j, sep="") x <- x+1 } } dimnames(b)[[2]] <- namb After the above...