read_fst is from the package fst. The fileformat fst uses is a binary format designed to be fast readable. It is a column oriented format and compressed. So, to be able to work fst needs access to the file itself and wont accept a file connection as functions like read.table an variants accept. Also, because it is a binary compressed format using a compression method that is fast to read, compressing also to zip seems to defeat the purpose of fst. HTH, Jan On 09-06-2021 15:28, Duncan Murdoch wrote:> On 09/06/2021 9:12 a.m., Jeff Reichman wrote: >> Duncan >> >> Yea that will work. It appears to be related to setting my working >> dir, for what ever reason neither seem to work >> (1) knitr::opts_knit$set(root.dir >> ="~/My_Reference_Library/Regression") # from R Notebook or >> (2) >> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") # >> from R chunk >> >> So it appears I can either (as you suggested) use two steps or combine >> but I need to enter the full path. Why other file types don't seem to >> need the full path ....????? > > You need to read the documentation for read_fst() to find what it needs. > ?If it doesn't explain this, then you should report the issue to its > author. > >> >> myObject <- >> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", >> filename = "myFile.fst")) >> >> Thank you. I guess just one of those R things > > No, it's a read_fst() thing. > > Duncan Murdoch > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Jan Makes sense. Its just that I often receive large zip files that contain a variety of file types. Jef -----Original Message----- From: R-help <r-help-bounces at r-project.org> On Behalf Of Jan van der Laan Sent: Wednesday, June 9, 2021 12:56 PM To: r-help at r-project.org Subject: Re: [R] Read fst files read_fst is from the package fst. The fileformat fst uses is a binary format designed to be fast readable. It is a column oriented format and compressed. So, to be able to work fst needs access to the file itself and wont accept a file connection as functions like read.table an variants accept. Also, because it is a binary compressed format using a compression method that is fast to read, compressing also to zip seems to defeat the purpose of fst. HTH, Jan On 09-06-2021 15:28, Duncan Murdoch wrote:> On 09/06/2021 9:12 a.m., Jeff Reichman wrote: >> Duncan >> >> Yea that will work. It appears to be related to setting my working >> dir, for what ever reason neither seem to work >> (1) knitr::opts_knit$set(root.dir >> ="~/My_Reference_Library/Regression") # from R Notebook or >> (2) >> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") # >> from R chunk >> >> So it appears I can either (as you suggested) use two steps or combine >> but I need to enter the full path. Why other file types don't seem to >> need the full path ....????? > > You need to read the documentation for read_fst() to find what it needs. > If it doesn't explain this, then you should report the issue to its > author. > >> >> myObject <- >> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", >> filename = "myFile.fst")) >> >> Thank you. I guess just one of those R things > > No, it's a read_fst() thing. > > Duncan Murdoch > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
On 09/06/2021 1:55 p.m., Jan van der Laan wrote:> > > read_fst is from the package fst. The fileformat fst uses is a binary > format designed to be fast readable. It is a column oriented format and > compressed. So, to be able to work fst needs access to the file itself > and wont accept a file connection as functions like read.table an > variants accept.Thanks for the info. I think it is possible to handle such a file in a binary connection, but doing that in C/C++ would be kind of horrible, so I can understand your choice. Duncan Murdoch> > Also, because it is a binary compressed format using a compression > method that is fast to read, compressing also to zip seems to defeat the > purpose of fst. > > HTH, > Jan > > > On 09-06-2021 15:28, Duncan Murdoch wrote: >> On 09/06/2021 9:12 a.m., Jeff Reichman wrote: >>> Duncan >>> >>> Yea that will work. It appears to be related to setting my working >>> dir, for what ever reason neither seem to work >>> (1) knitr::opts_knit$set(root.dir >>> ="~/My_Reference_Library/Regression") # from R Notebook or >>> (2) >>> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") # >>> from R chunk >>> >>> So it appears I can either (as you suggested) use two steps or combine >>> but I need to enter the full path. Why other file types don't seem to >>> need the full path ....????? >> >> You need to read the documentation for read_fst() to find what it needs. >> ?If it doesn't explain this, then you should report the issue to its >> author. >> >>> >>> myObject <- >>> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", >>> filename = "myFile.fst")) >>> >>> Thank you. I guess just one of those R things >> >> No, it's a read_fst() thing. >> >> Duncan Murdoch >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >