Jan Makes sense. Its just that I often receive large zip files that contain a variety of file types. Jef -----Original Message----- From: R-help <r-help-bounces at r-project.org> On Behalf Of Jan van der Laan Sent: Wednesday, June 9, 2021 12:56 PM To: r-help at r-project.org Subject: Re: [R] Read fst files read_fst is from the package fst. The fileformat fst uses is a binary format designed to be fast readable. It is a column oriented format and compressed. So, to be able to work fst needs access to the file itself and wont accept a file connection as functions like read.table an variants accept. Also, because it is a binary compressed format using a compression method that is fast to read, compressing also to zip seems to defeat the purpose of fst. HTH, Jan On 09-06-2021 15:28, Duncan Murdoch wrote:> On 09/06/2021 9:12 a.m., Jeff Reichman wrote: >> Duncan >> >> Yea that will work. It appears to be related to setting my working >> dir, for what ever reason neither seem to work >> (1) knitr::opts_knit$set(root.dir >> ="~/My_Reference_Library/Regression") # from R Notebook or >> (2) >> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") # >> from R chunk >> >> So it appears I can either (as you suggested) use two steps or combine >> but I need to enter the full path. Why other file types don't seem to >> need the full path ....????? > > You need to read the documentation for read_fst() to find what it needs. > If it doesn't explain this, then you should report the issue to its > author. > >> >> myObject <- >> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", >> filename = "myFile.fst")) >> >> Thank you. I guess just one of those R things > > No, it's a read_fst() thing. > > Duncan Murdoch > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
... but if you are receiving multiple-file zips then you should not be using unz() the way you are in your original post. I have to agree with other responders suggesting that you handle unzipping fst zips manually rather than as part of an R one-liner. On June 9, 2021 11:26:34 AM PDT, Jeff Reichman <reichmanj at sbcglobal.net> wrote:>Jan > >Makes sense. Its just that I often receive large zip files that >contain a variety of file types. > >Jef > >-----Original Message----- >From: R-help <r-help-bounces at r-project.org> On Behalf Of Jan van der >Laan >Sent: Wednesday, June 9, 2021 12:56 PM >To: r-help at r-project.org >Subject: Re: [R] Read fst files > > > >read_fst is from the package fst. The fileformat fst uses is a binary >format designed to be fast readable. It is a column oriented format >and >compressed. So, to be able to work fst needs access to the file itself >and wont accept a file connection as functions like read.table an >variants accept. > >Also, because it is a binary compressed format using a compression >method that is fast to read, compressing also to zip seems to defeat >the >purpose of fst. > >HTH, >Jan > > >On 09-06-2021 15:28, Duncan Murdoch wrote: >> On 09/06/2021 9:12 a.m., Jeff Reichman wrote: >>> Duncan >>> >>> Yea that will work. It appears to be related to setting my working >>> dir, for what ever reason neither seem to work >>> (1) knitr::opts_knit$set(root.dir >>> ="~/My_Reference_Library/Regression") # from R Notebook or >>> (2) >>> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") ># >>> from R chunk >>> >>> So it appears I can either (as you suggested) use two steps or >combine >>> but I need to enter the full path. Why other file types don't seem >to >>> need the full path ....????? >> >> You need to read the documentation for read_fst() to find what it >needs. >> If it doesn't explain this, then you should report the issue to its > >> author. >> >>> >>> myObject <- >>> >read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", > >>> filename = "myFile.fst")) >>> >>> Thank you. I guess just one of those R things >> >> No, it's a read_fst() thing. >> >> Duncan Murdoch >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > >______________________________________________ >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. > >______________________________________________ >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code.-- Sent from my phone. Please excuse my brevity.
Try using unzip(zipfile, files="desiredFile", exdir=tf<-tempfile()), not unz(zipfile, "desiredFile"), to copy the desired file from the zip file to a temporary location and use read_fst(tf) to read the desired file. -Bill On Wed, Jun 9, 2021 at 11:27 AM Jeff Reichman <reichmanj at sbcglobal.net> wrote:> Jan > > Makes sense. Its just that I often receive large zip files that contain a > variety of file types. > > Jef > > -----Original Message----- > From: R-help <r-help-bounces at r-project.org> On Behalf Of Jan van der Laan > Sent: Wednesday, June 9, 2021 12:56 PM > To: r-help at r-project.org > Subject: Re: [R] Read fst files > > > > read_fst is from the package fst. The fileformat fst uses is a binary > format designed to be fast readable. It is a column oriented format and > compressed. So, to be able to work fst needs access to the file itself > and wont accept a file connection as functions like read.table an > variants accept. > > Also, because it is a binary compressed format using a compression > method that is fast to read, compressing also to zip seems to defeat the > purpose of fst. > > HTH, > Jan > > > On 09-06-2021 15:28, Duncan Murdoch wrote: > > On 09/06/2021 9:12 a.m., Jeff Reichman wrote: > >> Duncan > >> > >> Yea that will work. It appears to be related to setting my working > >> dir, for what ever reason neither seem to work > >> (1) knitr::opts_knit$set(root.dir > >> ="~/My_Reference_Library/Regression") # from R Notebook or > >> (2) > >> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") # > >> from R chunk > >> > >> So it appears I can either (as you suggested) use two steps or combine > >> but I need to enter the full path. Why other file types don't seem to > >> need the full path ....????? > > > > You need to read the documentation for read_fst() to find what it needs. > > If it doesn't explain this, then you should report the issue to its > > author. > > > >> > >> myObject <- > >> > read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", > > >> filename = "myFile.fst")) > >> > >> Thank you. I guess just one of those R things > > > > No, it's a read_fst() thing. > > > > Duncan Murdoch > > > > ______________________________________________ > > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >[[alternative HTML version deleted]]