R help - Nov 2018 - saveRDS() and readRDS() Why? [solved, pretty much anyway)

>>>>> Patrick Connolly 
>>>>>     on Thu, 8 Nov 2018 20:27:24 +1300 writes:
> Many thanks to Berwin, Eric, Robert, and Jan for their input.
> 
> I had hoped it was as simple as because I typed 
> 
> saveRDS("rawData", file = "rawData.rds") on the Windows
side.
> but that wasn't the case.
> 
> Robert Burbridge suggested:
> 
>  windows (not run)
> f <- file("rawData.rds", open="w")
> serialize(rawData, f, xdr = FALSE)
> close(f)
> 
> # linux
> rawData <- unserialize(file = "rawData.rds")
> 
> That didn't work: 
> Error in unserialize(file = "rawData.rds") : 
>   unused argument (file = "rawData.rds")
> (the argument isn't 'file')
> 
> Nor did 
> > rawData <- unserialize("rawData.rds")
> Error in unserialize("rawData.rds") : 
>   character vectors are no longer accepted by unserialize()
> 
> However 
> 
> readRDS(file = "rawData.rds") did!
> 
> So what I needed was serialize but not unserialize.
> 
> I still don't know Why, but I know How.
Hmm.. and nobody has been able to reproduce your problem, right?

IIUC, currently you are suggesting that [on Windows], if you do

      saveRDS(rawdata, file="rawdata.rds")

the resulting file is does not work with    readRDS()  on Linux.
What again are your R versions on the two platforms?

Could you  dput() -- provide a (short if possible) version of rawdata where
that problem occurs ?

Best,
Martin

> ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
>    ___    Patrick Connolly   
>  {~._.~}                   Great minds discuss ideas    
>  _( Y )_  	         Average minds discuss events 
> (:_~*~_:)                  Small minds discuss people  
>  (_)-(_)  	                      ..... Eleanor Roosevelt
> 	  
> ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

On Thu, 08-Nov-2018 at 11:06AM +0100, Martin Maechler wrote:

|> >>>>> Patrick Connolly 
|> >>>>>     on Thu, 8 Nov 2018 20:27:24 +1300 writes:

[...]

|> > 
|> > I still don't know Why, but I know How.
|> 
|> Hmm.. and nobody has been able to reproduce your problem, right?
|> 
|> IIUC, currently you are suggesting that [on Windows], if you do
|> 
|>       saveRDS(rawdata, file="rawdata.rds")
|> 
|> the resulting file is does not work with    readRDS()  on Linux.
|> What again are your R versions on the two platforms?


It's an old version on Windows.  I haven't used Windows R since then.

major          3                                          
minor          2.4                                        
year           2016                                       
month          03                                         
day            16                                         


I've tried R-3.5.0 and R-3.5.1 Linux versions.  The problem might be
entirely because of the ancient Windows version. 


|> 
|> Could you  dput() -- provide a (short if possible) version of rawdata
where
|> that problem occurs ?

I can't make a smaller version of rawdata which comes from scraping a
non-public web address, but next week when I'm back where those
machines are, I'll try it with a small data frame which is
reproducible.



|> 
|> Best,
|> Martin
|> 
|> 
|> >
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> >    ___    Patrick Connolly   
|> >  {~._.~}                   Great minds discuss ideas    
|> >  _( Y )_  	         Average minds discuss events 
|> > (:_~*~_:)                  Small minds discuss people  
|> >  (_)-(_)  	                      ..... Eleanor Roosevelt
|> > 	  
|> >
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> > 
|> > ______________________________________________
|> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
|> > https://stat.ethz.ch/mailman/listinfo/r-help
|> > PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
|> > and provide commented, minimal, self-contained, reproducible code.

-- 
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

The solution was very simple.  Don't use the same name for the rds
file  as used for the R object, viz a vie:

saveRDS(x, file = "x.rds")
and
x <- readRDS(file = "x.rds")

will not work; however

saveRDS(x, file = "y.rds")
and
x <- readRDS(file = "y.rds")
will work.

An undocumented feature?

Thanks to all who contributed.



On Sat, 10-Nov-2018 at 08:48PM +1300, Patrick Connolly wrote:

|> On Thu, 08-Nov-2018 at 11:06AM +0100, Martin Maechler wrote:
|> 
|> |> >>>>> Patrick Connolly 
|> |> >>>>>     on Thu, 8 Nov 2018 20:27:24 +1300 writes:
|> 
|> [...]
|> 
|> |> > 
|> |> > I still don't know Why, but I know How.
|> |> 
|> |> Hmm.. and nobody has been able to reproduce your problem, right?
|> |> 
|> |> IIUC, currently you are suggesting that [on Windows], if you do
|> |> 
|> |>       saveRDS(rawdata, file="rawdata.rds")
|> |> 
|> |> the resulting file is does not work with    readRDS()  on Linux.
|> |> What again are your R versions on the two platforms?
|> 
|> 
|> It's an old version on Windows.  I haven't used Windows R since
then.
|> 
|> major          3                                          
|> minor          2.4                                        
|> year           2016                                       
|> month          03                                         
|> day            16                                         
|> 
|> 
|> I've tried R-3.5.0 and R-3.5.1 Linux versions.  The problem might be
|> entirely because of the ancient Windows version. 
|> 
|> 
|> |> 
|> |> Could you  dput() -- provide a (short if possible) version of
rawdata where
|> |> that problem occurs ?
|> 
|> I can't make a smaller version of rawdata which comes from scraping a
|> non-public web address, but next week when I'm back where those
|> machines are, I'll try it with a small data frame which is
|> reproducible.
|> 
|> 
|> 
|> |> 
|> |> Best,
|> |> Martin
|> |> 
|> |> 
|> |> >
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> |> >    ___    Patrick Connolly   
|> |> >  {~._.~}                   Great minds discuss ideas    
|> |> >  _( Y )_  	         Average minds discuss events 
|> |> > (:_~*~_:)                  Small minds discuss people  
|> |> >  (_)-(_)  	                      ..... Eleanor Roosevelt
|> |> > 	  
|> |> >
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> |> > 
|> |> > ______________________________________________
|> |> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and
more, see
|> |> > https://stat.ethz.ch/mailman/listinfo/r-help
|> |> > PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
|> |> > and provide commented, minimal, self-contained, reproducible
code.
|> 
|> -- 
|> ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|>    ___    Patrick Connolly   
|>  {~._.~}                   Great minds discuss ideas    
|>  _( Y )_  	         Average minds discuss events 
|> (:_~*~_:)                  Small minds discuss people  
|>  (_)-(_)  	                      ..... Eleanor Roosevelt
|> 	  
|> ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> 
|> ______________________________________________
|> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
|> https://stat.ethz.ch/mailman/listinfo/r-help
|> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
|> and provide commented, minimal, self-contained, reproducible code.

-- 
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.

On 2018-11-12 22:49, peter dalgaard wrote:
> Er, where, what, how? I can't reproduce that, at least not on 3.5.1 on 
> MacOS:
> 
>> x <- airquality
>> saveRDS(x, file = "x.rds")
>> x <- NULL
>> x <- readRDS(file = "x.rds")
>> x
>     Ozone Solar.R Wind Temp Month Day
> 1      41     190  7.4   67     5   1
> 2      36     118  8.0   72     5   2
> 3      12     149 12.6   74     5   3
> ...
> 
> Looks fine to me.
> 
It seems to work fine using the same installation to read as used for 
the save.
But it's a different story if the save was done on a Windows 
installation and read on a Linux installation.

## On Windows 3.4.2
> x <- airquality
> saveRDS(x, file = "x.rds")
> saveRDS(x, file = "y.rds")
Files x.rds and y.rds are identical in size but utterly different in 
content.

## On Linux 3.5.1
> x <- readRDS(file =  "x.rds")
Error in readRDS(file = "x.rds") : error reading from
connection
> x <- readRDS(file =  "y.rds")
> head(x)
   Ozone Solar.R Wind Temp Month Day
1    41     190  7.4   67     5   1
2    36     118  8.0   72     5   2
3    12     149 12.6   74     5   3
4    18     313 11.5   62     5   4
5    NA      NA 14.3   56     5   5
6    28      NA 14.9   66     5   6

It might just be the age of the Windows installation.  I don't have much 
use for Windows, so I haven't had much inclination to install a newer 
version.

YMMV


> ?
> -pd
> 
> 
>> On 12 Nov 2018, at 08:28 , Patrick Connolly 
>> <p_connolly at slingshot.co.nz> wrote:
>> 
>> The solution was very simple.  Don't use the same name for the rds
>> file  as used for the R object, viz a vie:
>> 
>> saveRDS(x, file = "x.rds")
>> and
>> x <- readRDS(file = "x.rds")
>> 
>> will not work; however
>> 
>> saveRDS(x, file = "y.rds")
>> and
>> x <- readRDS(file = "y.rds")
>> will work.
>> 
>> An undocumented feature?
>> 
>> Thanks to all who contributed.
> 
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com

You wrote:
  ## On Windows 3.4.2
>   x <- airquality
>   saveRDS(x, file = "x.rds")
>   saveRDS(x, file = "y.rds")
>
  Files x.rds and y.rds are identical in size but utterly different in
content.

Wow!  Can you show us the results of
  x <- datasets::airquality
  saveRDS(x, file="x.rds")
  saveRDS(x, file="y.rds")
  tools::md5sum(c("x.rds", "y.rds"))
  dput(readBin("x.rds", what="raw",
n=file.size("x.rds")))
  dput(readBin("y.rds", what="raw",
n=file.size("y.rds")))

(Copy and paste, as text, from the R session, so we can see the input and
the output in context.





Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Mon, Nov 12, 2018 at 12:42 PM, p_connolly <p_connolly at
slingshot.co.nz>
wrote:
> On 2018-11-12 22:49, peter dalgaard wrote:
>
>> Er, where, what, how? I can't reproduce that, at least not on 3.5.1
on
>> MacOS:
>>
>> x <- airquality
>>> saveRDS(x, file = "x.rds")
>>> x <- NULL
>>> x <- readRDS(file = "x.rds")
>>> x
>>>
>>     Ozone Solar.R Wind Temp Month Day
>> 1      41     190  7.4   67     5   1
>> 2      36     118  8.0   72     5   2
>> 3      12     149 12.6   74     5   3
>> ...
>>
>> Looks fine to me.
>>
>>
> It seems to work fine using the same installation to read as used for the
> save.
> But it's a different story if the save was done on a Windows
installation
> and read on a Linux installation.
>
> ## On Windows 3.4.2
>
>> x <- airquality
>> saveRDS(x, file = "x.rds")
>> saveRDS(x, file = "y.rds")
>>
>
> Files x.rds and y.rds are identical in size but utterly different in
> content.
>
> ## On Linux 3.5.1
>
> x <- readRDS(file =  "x.rds")
>>
> Error in readRDS(file = "x.rds") : error reading from connection
>
>> x <- readRDS(file =  "y.rds")
>> head(x)
>>
>   Ozone Solar.R Wind Temp Month Day
> 1    41     190  7.4   67     5   1
> 2    36     118  8.0   72     5   2
> 3    12     149 12.6   74     5   3
> 4    18     313 11.5   62     5   4
> 5    NA      NA 14.3   56     5   5
> 6    28      NA 14.9   66     5   6
>
> It might just be the age of the Windows installation.  I don't have
much
> use for Windows, so I haven't had much inclination to install a newer
> version.
>
> YMMV
>
>
>
> ?
>> -pd
>>
>>
>> On 12 Nov 2018, at 08:28 , Patrick Connolly <p_connolly at
slingshot.co.nz>
>>> wrote:
>>>
>>> The solution was very simple.  Don't use the same name for the
rds
>>> file  as used for the R object, viz a vie:
>>>
>>> saveRDS(x, file = "x.rds")
>>> and
>>> x <- readRDS(file = "x.rds")
>>>
>>> will not work; however
>>>
>>> saveRDS(x, file = "y.rds")
>>> and
>>> x <- readRDS(file = "y.rds")
>>> will work.
>>>
>>> An undocumented feature?
>>>
>>> Thanks to all who contributed.
>>>
>>
>> --
>> Peter Dalgaard, Professor,
>> Center for Statistics, Copenhagen Business School
>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>> Phone: (+45)38153501
>> Office: A 4.23
>> Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com
>>
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posti
> ng-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
	[[alternative HTML version deleted]]