similar to: How to benchmark speed of load/readRDS correctly

Displaying 20 results from an estimated 3000 matches similar to: "How to benchmark speed of load/readRDS correctly"

2017 Aug 22
0
How to benchmark speed of load/readRDS correctly
The large value for maximum time may be due to garbage collection, which happens periodically. E.g., try the following, where the unlist(as.list()) creates a lot of garbage. I get a very large time every 102 or 51 iterations and a moderately large time more often mb <- microbenchmark::microbenchmark({ x <- as.list(sin(1:5e5)); x <- unlist(x) / cos(1:5e5) ; sum(x) }, times=1000)
2017 Aug 22
1
How to benchmark speed of load/readRDS correctly
Note that if you force a garbage collection each iteration the times are more stable. However, on the average it is faster to let the garbage collector decide when to leap into action. mb_gc <- microbenchmark::microbenchmark(gc(), { x <- as.list(sin(1:5e5)); x <- unlist(x) / cos(1:5e5) ; sum(x) }, times=1000, control=list(order="inorder")) with(mb_gc,
2017 Aug 22
0
How to benchmark speed of load/readRDS correctly
You need to study how reading files works in your operating system. This question is not about R. -- Sent from my phone. Please excuse my brevity. On August 22, 2017 5:53:09 AM PDT, raphael.felber at agroscope.admin.ch wrote: >Dear all > >I was thinking about efficient reading data into R and tried several >ways to test if load(file.Rdata) or readRDS(file.rds) is faster. The
2017 Aug 23
1
How to benchmark speed of load/readRDS correctly
Hi there Thanks for your answers. I didn't expect that this would be so complex. Honestly, I don't understand everything you wrote since I'm not an IT specialist. But I read something that reading *.rds files is faster than loading *.Rdata and I wanted to proof that for my system and R version. But thanks anyway for your time. Cheers Raphael > -----Urspr?ngliche Nachricht-----
2017 Aug 22
0
How to benchmark speed of load/readRDS correctly
Caching happens, both within the operating system and within the C standard library. Ostensibly the intent for those caches is to help performance, but you are right that different low-level caching algorithms can be a poor match for specific application level use cases such as copying files or parsing text syntax. However, the OS and even the specific file system drivers (e.g. ext4 on flash disk
2017 Aug 22
3
How to benchmark speed of load/readRDS correctly
Not convinced Jeff is completely right about this not concerning R, since I've found that the application language (R, perl, etc.) makes a difference in how files are accessed by/to OS. He is certainly correct that OS (and versions) are where the actual reading and writing happens, but sometimes the call to those can be inefficient. (Sorry, I've not got examples specifically for file
2017 Aug 02
3
Remove attribute from netcdf4 object
Dear all For a model I need to combine several netCDF files into one (which works fine). For better overview I'd like to delete/remove some of the attributes. Is there a simple way doing this? I'm using the package netcdf4, which creates an object of class(nc) = "ncdf4". It seems that for earlier versions of netcdf objects, there was the function att.delete.nc{RNetCDF}. But
2017 Aug 14
2
ncdf4: Why are NAs converted to _FillValue when saving?
Dear all I'm a newbie regarding netcdf data. Today I realized that I maybe do not understand some basics of the netcdf. I want to create a *.nc file containing three variables for Switzerland. All data outside of the country are NAs. The third variable is calculated from the first two variables. Basically there is no problem to do that. I copy the file with the data of the first variable,
2017 Aug 14
0
ncdf4: Why are NAs converted to _FillValue when saving?
On Mon, Aug 14, 2017 at 5:29 AM, <raphael.felber at agroscope.admin.ch> wrote: Dear all > > I'm a newbie regarding netcdf data. Today I realized that I maybe do not > understand some basics of the netcdf. I want to create a *.nc file > containing three variables for Switzerland. All data outside of the country > are NAs. The third variable is calculated from the first two
2017 Sep 07
1
extend limited dimension in netcdf
Dear all I have to combine 3D netCDF files (lon, lat, time). The files contain data of one month and I need a year file containing all the data. Because the attributes of all files are the same, I copied the first file and appended the data of the other months. This went well until the provider of the data changed the time-dimension from UNLIMITED to LIMITED. Is there a way to change the time
2011 Oct 18
9
readRDS and saveRDS
Hi all, Is there any chance that readRDS and saveRDS might one day become read.rds and write.rds? That would make them more consistent with the other reading and writing functions. Hadley -- Assistant Professor / Dobelman Family Junior Chair Department of Statistics / Rice University http://had.co.nz/
2012 Sep 15
2
Risk of readRDS() not detecting race conditions with parallel saveRDS()?
I hardly know anything about the format used in (non-compressed) serialization/RDS, but hoping someone with more knowledge could give me some feedback; Consider two R processes running in parallel on the same unknown file system. Both of them write and read to the same RDS file foo.rds (without compression) at random times using saveRDS(object, file="foo.rds", compress=FALSE) and
2018 Apr 22
2
Problem reading RDS files
Hi there, I faced a weird problem doing a seemingly simple task in R. Specifically, when trying for reading an RDS file from the working directory, the following error is appeared. Code: records <- readRDS("tweets.rds") Error: Error in readRDS("tweets.rds") : error reading from connection In addition: Warning message: In readRDS("tweets.rds") : invalid or
2018 Apr 02
3
recordPlot/replayPlot not working with saveRDS/readRDS
The documentation for recordPlot says the following: > As of R 3.3.0, it is possible (again) to replay a plot from another R session using, for example, saveRDS and readRDS. However, I haven't been able to save and restore a plot displaylist and have it work within the same R session, using R 3.4.3 or 3.3.3. Here's an example: # Save displaylist for a simple plot
2012 Jul 29
1
readRDS, In as.double.xts(fishReport$count) : NAs introduced by coercion
Hello, I looked in the R-help but could not find an archive addressing the following. I would like to convert a character to numeric after reading a file with RDS extension. After using as.numeric, I checked if it is numeric. It was not converted. Please help. Here is my code >Report <- readRDS(file="RDS/Report.RDS") > Report[1:2,] dive_id date
2018 Apr 22
0
Problem reading RDS files
Wouldn't the obvious problem be that your data file is corrupted or was never created using saveRDS in the first place? Can you show us a complete example of creating and attempting to read what was just created? On April 22, 2018 10:20:05 AM CDT, mohammad moradi <mri.moradi at gmail.com> wrote: >Hi there, > >I faced a weird problem doing a seemingly simple task in R.
2018 Apr 23
1
Problem reading RDS files
I've tried to re-experiment the tutorial presented at http://www.rdatamining.com/docs/twitter-analysis-with-r and specifically aimed to use rds files (tweet records) at http://www.rdatamining.com/data/. On Sun, Apr 22, 2018 at 9:16 PM, Jeff Newmiller <jdnewmil at dcn.davis.ca.us> wrote: > Wouldn't the obvious problem be that your data file is corrupted or was > never created
2006 Nov 02
3
CRAN task views work only once per session (PR#9330)
Cran task views seems to be a "once-per-session" process -- the first attempt to access views in a (RGui for Windows) session works, but subsequent attempts fail. There is a noticeably long pause before the failing call returns. Example session with two calls to "available.views" follows, but similar effects have been observed with two calls to "install.views" and
2012 Jan 09
1
serializing recordedplot object
I use recordPlot() to save plots to disk that I render later to a variety of formats. This works fine for base R plots and ggplot2 plots, and also used to work for lattice plots. However somewhere in version 2.14 things stopped working for lattice plots. Here is an example: library(lattice); histogram(rnorm(100)); x <- recordPlot(); saveRDS(x, "myplot.rds"); y <-
2018 Apr 03
1
[FORGED] recordPlot/replayPlot not working with saveRDS/readRDS
>>>>> Paul Murrell <paul at stat.auckland.ac.nz> >>>>> on Tue, 3 Apr 2018 09:41:56 +1200 writes: > Hi What you are doing "wrong" is loading a recordedplot > into the same session that it was created in. The > saveRDS()/readRDS() works if you save in one R session and > then read in a different R session. The