I use the ff-package for large data and have good success in session. To ensure my data is available for the next R session, I save my workspace and use the /ffsave/ command to archive my large data objects. Syntax is simple:> ffsave(file="~/path/archiveName", list="objectName")when I have multiple files to save, the list is expanded using c(). When I start a new R session, I load the ff-package again and use /ffload/ to retrieve my data. Syntax is simple:> ffload(file="~/path/archiveName", list="objectName")If my workspace is emply, I can see the object enter. However, when I wish to interrogate the object, it appears that I have no physical connection to the data:> objectNameff (deleted) double length=21024 (21024) dim=c(73,144,2) dimorder=c(1,2,3)> open(objectName)Error: file.access(filename, 0) == 0 is not TRUE In simple words, my data is no longer usable...WHY? This shouldn't be so hard! -- View this message in context: http://r.789695.n4.nabble.com/HELP-ff-package-tp4632949.html Sent from the R help mailing list archive at Nabble.com.
I see now the configurations required for ffsave and ffload to work properly, allowing the ff-package to do what it was designed to do: 1) file permanence ff objects are stored in a temp directory and are deleted on exiting R. This is true of both standard and ff objects (so dont blame the package). In my case, I lost large and critical data files when exiting the program because I didnt manage my temp directory and the options called fffinalize and finonexit. Use getOptions() to see your current setting for both of these values. Note: when you reassigning the temp directory to one you establish, fffinalize changes automatically from delete to close. 2) temp directory ff files are physically stored in a temp directory and you want absolute control over this location. Create your own temp directory. I also see the value of controlling the names of the files stored there. Care here is essential. Package documentation points to use of fucntion pattern() to control temp files. Use it! Otherwise, your temp file names are exclusively random in nature. More control is better, especially when working with large dataframes (ffdf), which have many temp files associated with them. 3) rootpath rootpath is an option for both ffsave and ffload. The two must be consistent with the temp directory you create. 4) External zip utility Package documentation makes clear you need to use the external utilities zip 3.0 and unzip 6.0 from http://www.info-zip.org/. Once installed, adjust the option(unzip="directory") to point to where the zip code is stored. ffsave confirms the external code is working by showing how much compression has been achieved in the value output. -- View this message in context: http://r.789695.n4.nabble.com/HELP-ff-package-tp4632949p4632968.html Sent from the R help mailing list archive at Nabble.com.