similar to: How to benchmark speed of load/readRDS correctly

The large value for maximum time may be due to garbage collection, which happens periodically. E.g., try the following, where the unlist(as.list()) creates a lot of garbage. I get a very large time every 102 or 51 iterations and a moderately large time more often mb <- microbenchmark::microbenchmark({ x <- as.list(sin(1:5e5)); x <- unlist(x) / cos(1:5e5) ; sum(x) }, times=1000)

Note that if you force a garbage collection each iteration the times are more stable. However, on the average it is faster to let the garbage collector decide when to leap into action. mb_gc <- microbenchmark::microbenchmark(gc(), { x <- as.list(sin(1:5e5)); x <- unlist(x) / cos(1:5e5) ; sum(x) }, times=1000, control=list(order="inorder")) with(mb_gc,

You need to study how reading files works in your operating system. This question is not about R. -- Sent from my phone. Please excuse my brevity. On August 22, 2017 5:53:09 AM PDT, raphael.felber at agroscope.admin.ch wrote: >Dear all > >I was thinking about efficient reading data into R and tried several >ways to test if load(file.Rdata) or readRDS(file.rds) is faster. The

Hi there Thanks for your answers. I didn't expect that this would be so complex. Honestly, I don't understand everything you wrote since I'm not an IT specialist. But I read something that reading *.rds files is faster than loading *.Rdata and I wanted to proof that for my system and R version. But thanks anyway for your time. Cheers Raphael > -----Urspr?ngliche Nachricht-----

Caching happens, both within the operating system and within the C standard library. Ostensibly the intent for those caches is to help performance, but you are right that different low-level caching algorithms can be a poor match for specific application level use cases such as copying files or parsing text syntax. However, the OS and even the specific file system drivers (e.g. ext4 on flash disk

Not convinced Jeff is completely right about this not concerning R, since I've found that the application language (R, perl, etc.) makes a difference in how files are accessed by/to OS. He is certainly correct that OS (and versions) are where the actual reading and writing happens, but sometimes the call to those can be inefficient. (Sorry, I've not got examples specifically for file

Dear all For a model I need to combine several netCDF files into one (which works fine). For better overview I'd like to delete/remove some of the attributes. Is there a simple way doing this? I'm using the package netcdf4, which creates an object of class(nc) = "ncdf4". It seems that for earlier versions of netcdf objects, there was the function att.delete.nc{RNetCDF}. But

Dear all I'm a newbie regarding netcdf data. Today I realized that I maybe do not understand some basics of the netcdf. I want to create a *.nc file containing three variables for Switzerland. All data outside of the country are NAs. The third variable is calculated from the first two variables. Basically there is no problem to do that. I copy the file with the data of the first variable,

On Mon, Aug 14, 2017 at 5:29 AM, <raphael.felber at agroscope.admin.ch> wrote: Dear all > > I'm a newbie regarding netcdf data. Today I realized that I maybe do not > understand some basics of the netcdf. I want to create a *.nc file > containing three variables for Switzerland. All data outside of the country > are NAs. The third variable is calculated from the first two

Dear all I have to combine 3D netCDF files (lon, lat, time). The files contain data of one month and I need a year file containing all the data. Because the attributes of all files are the same, I copied the first file and appended the data of the other months. This went well until the provider of the data changed the time-dimension from UNLIMITED to LIMITED. Is there a way to change the time

Hi all, Is there any chance that readRDS and saveRDS might one day become read.rds and write.rds? That would make them more consistent with the other reading and writing functions. Hadley -- Assistant Professor / Dobelman Family Junior Chair Department of Statistics / Rice University http://had.co.nz/

I hardly know anything about the format used in (non-compressed) serialization/RDS, but hoping someone with more knowledge could give me some feedback; Consider two R processes running in parallel on the same unknown file system. Both of them write and read to the same RDS file foo.rds (without compression) at random times using saveRDS(object, file="foo.rds", compress=FALSE) and

Hi there, I faced a weird problem doing a seemingly simple task in R. Specifically, when trying for reading an RDS file from the working directory, the following error is appeared. Code: records <- readRDS("tweets.rds") Error: Error in readRDS("tweets.rds") : error reading from connection In addition: Warning message: In readRDS("tweets.rds") : invalid or

The documentation for recordPlot says the following: > As of R 3.3.0, it is possible (again) to replay a plot from another R session using, for example, saveRDS and readRDS. However, I haven't been able to save and restore a plot displaylist and have it work within the same R session, using R 3.4.3 or 3.3.3. Here's an example: # Save displaylist for a simple plot

Hello, I looked in the R-help but could not find an archive addressing the following. I would like to convert a character to numeric after reading a file with RDS extension. After using as.numeric, I checked if it is numeric. It was not converted. Please help. Here is my code >Report <- readRDS(file="RDS/Report.RDS") > Report[1:2,] dive_id date

Wouldn't the obvious problem be that your data file is corrupted or was never created using saveRDS in the first place? Can you show us a complete example of creating and attempting to read what was just created? On April 22, 2018 10:20:05 AM CDT, mohammad moradi <mri.moradi at gmail.com> wrote: >Hi there, > >I faced a weird problem doing a seemingly simple task in R.

I've tried to re-experiment the tutorial presented at http://www.rdatamining.com/docs/twitter-analysis-with-r and specifically aimed to use rds files (tweet records) at http://www.rdatamining.com/data/. On Sun, Apr 22, 2018 at 9:16 PM, Jeff Newmiller <jdnewmil at dcn.davis.ca.us> wrote: > Wouldn't the obvious problem be that your data file is corrupted or was > never created

Cran task views seems to be a "once-per-session" process -- the first attempt to access views in a (RGui for Windows) session works, but subsequent attempts fail. There is a noticeably long pause before the failing call returns. Example session with two calls to "available.views" follows, but similar effects have been observed with two calls to "install.views" and

I use recordPlot() to save plots to disk that I render later to a variety of formats. This works fine for base R plots and ggplot2 plots, and also used to work for lattice plots. However somewhere in version 2.14 things stopped working for lattice plots. Here is an example: library(lattice); histogram(rnorm(100)); x <- recordPlot(); saveRDS(x, "myplot.rds"); y <-

>>>>> Paul Murrell <paul at stat.auckland.ac.nz> >>>>> on Tue, 3 Apr 2018 09:41:56 +1200 writes: > Hi What you are doing "wrong" is loading a recordedplot > into the same session that it was created in. The > saveRDS()/readRDS() works if you save in one R session and > then read in a different R session. The