R help - Sep 2012 - unzipping with ff

I've noticed that ff uses the unzip utility available on it's host OS to
load datasets via ffload.  It seems to work fine in linux, but when I try
to use the package in Windows (hence dling Windows unzip utils) I get 2
errors, one telling me that the options aren't being passed to unzip
correctly and another indicating that my filename is incorrect.  First,
does anyone know of an unzip utility that works with ff or am I doing
something wrong there?  Second, why does ff seem to add .Rdata and .ffData
when looking for files but adds no extension when saving them?  Code
below...

library('ff')
ffcolTest = ff(c(1:500), vmode = "integer", length = 500, filename =
<some
folder>\\ffTest')
quit()

library('ff')

ffload("<same path as above>")

Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection
In addition: Warning messages:
1: running command 'unzip -Z -1 "<myfile>.ffData"' had
status 9
2: In readChar(con, 5L, useBytes = TRUE) :
  cannot open compressed file '<myfile>.RData', probable reason
'No such
file or directory'


What am I doing wrong here?  I have data sets of about 50MM rows and 50
columns and I really don't want to use SAS.  Is ff the package that suits
me best or should I consider another package?  Thanks for your time.

Dave Mitchell

	[[alternative HTML version deleted]]

Hi Dave,

You need to save the ff file before you quit R.
As in ffsave(ffcolTest, file = <somefolder>\\ffTest')
When you reopen R, you can now use ffload("<same path as
above>").

Regarding your question on if ff suits your needs. This is probably the
case, I'm regularly using it for data size a lot bigger than yours.
Remark that ffbase also contains a lot of the basic data handling
functionalities you are used to when working with R and which allow you
to do data manipulations on ff vectors or ffdf objects.

good luck on trying out ff,
Jan

-----Oorspronkelijk bericht-----
Van: Dave Mitchell [mailto:dmmtchll at gmail.com] 
Verzonden: donderdag 13 september 2012 0:21
Aan: r-help at r-project.org
Onderwerp: [R] unzipping with ff

I've noticed that ff uses the unzip utility available on it's host OS to
load datasets via ffload.  It seems to work fine in linux, but when I
try
to use the package in Windows (hence dling Windows unzip utils) I get 2
errors, one telling me that the options aren't being passed to unzip
correctly and another indicating that my filename is incorrect.  First,
does anyone know of an unzip utility that works with ff or am I doing
something wrong there?  Second, why does ff seem to add .Rdata and
.ffData
when looking for files but adds no extension when saving them?  Code
below...

library('ff')
ffcolTest = ff(c(1:500), vmode = "integer", length = 500, filename
<some
folder>\\ffTest')
quit()

library('ff')

ffload("<same path as above>")

Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection
In addition: Warning messages:
1: running command 'unzip -Z -1 "<myfile>.ffData"' had
status 9
2: In readChar(con, 5L, useBytes = TRUE) :
  cannot open compressed file '<myfile>.RData', probable reason
'No such
file or directory'


What am I doing wrong here?  I have data sets of about 50MM rows and 50
columns and I really don't want to use SAS.  Is ff the package that
suits
me best or should I consider another package?  Thanks for your time.

Dave Mitchell

	[[alternative HTML version deleted]]