similar to: Do environments make copies?

I'm trying to read in datasets with roughly 150,000 rows and 600 features. I wrote a function using scan() to read it in (I have a 4GB linux machine) and it works like a charm. Unfortunately, converting the scanned list into a datafame using as.data.frame() causes the memory usage to explode (it can go from 300MB for the scanned list to 1.4GB for a data.frame of 30000 rows) and it fails

Is it correct that there is no way of avoiding deep copying of data structures? Or asked from a different perspective, is it true that there are no pointers? :) (not that I am a fan of pointer, they just let me decide when to do deep copy on my own when the memory manager doesnt do it for me :) ) I was considering writing code in R that would need the internal representation of complex graph

I have a data set of roughly 700MB which during processing grows up to 2G ( I'm using a 4G linux box). After the work is done I clean up (rm()) and the state is returned to 700MB. Yet I find I cannot run the same routine again as it claims to not be able to allocate memory even though gcinfo() claims there is 1.1G left. At the start of the second time ===============================

I have a data set of roughly 700MB which during processing grows up to 2G ( I'm using a 4G linux box). After the work is done I clean up (rm()) and the state is returned to 700MB. Yet I find I cannot run the same routine again as it claims to not be able to allocate memory even though gcinfo() claims there is 1.1G left. At the start of the second time ===============================

Hello Everyone! Here are a couple of thought/questions on refClasses integration in R core functionality. First, coexistence with S4: > X <- setRefClass("classX", fields = c("a", "b"), + representation = list(c = "character")) > x <- X$new() > x at c <- "sss" > x An object of class "classX"

Good afternoon, earlier today I wrote a mail to the user mailing list about current status of the current development. As pointed out by Mr. Rücker, it seems to be good to also write to this list with a bit more technical details. Please note that this mail represents current master branch ("2.5.x"). We do recommend to use current stable branch ("2.4.x") for production.

On Fri, 5 Aug 2016, Winston Chang wrote: > My understanding is that R will not make copies of lists if there is > only one reference to the object. However, I've encountered a case > where R does make copies, even though (I think) there should be only > one reference to the object. I hope that someone could shed some light > on why this is happening. > > I'll start

Dear all, I have a data frame of 2160 rows and 4320 columns, which I hope to condense to a smaller dataset by finding averages of 6 by 6 blocks of values (to produce a data frame of 360 rows by 720 columns). How would I go about finding the mean of a 6 x 6 block, then find the mean of the next adjacent 6 x 6 block, and so on, until the whole data frame has been covered? One slight twist is that

My understanding is that R will not make copies of lists if there is only one reference to the object. However, I've encountered a case where R does make copies, even though (I think) there should be only one reference to the object. I hope that someone could shed some light on why this is happening. I'll start with a simple example. Below, x is a list with one element, and changing that

Dear all, I am working on a 64 bits Linux system. I issue the following R commands: > rm(list=ls()) # To remove all objects in the workspace. > gc() # To free memory. used (Mb) gc trigger (Mb) max used (Mb) Ncells 124250 6.7 350000 18.7 350000 18.7 Vcells 124547 1.0 786432 6.0 476934 3.7 > gc() # I had to do it again, don't know why! used (Mb) gc trigger (Mb) max used (Mb) Ncells

hello! ------ i use 32bit.Linux(SuSe)Server, so i'm limited with 3.5Gb of memory i demonstrate, that there is times to times a problem with allocating of objects of large size, for example 0.state (no objects yet created) ------------------------------------ > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 162070 4.4 350000 9.4 350000

Hi. As shown in the simplified example below, I'm having trouble allocating memory for large vectors, even though it would appear that there is more than enough memory available. That is, even with a memory limit of 1500 MB, R 2.6.1 (Win) will allocate memory for a first vector of 285 MB, but not for a second vector of the same size. Forcing garbage collection does not seem

As others already mentioned, in your example you are first creating an integer matrix and the coercing it to a double matrix by assigning (double) 1 to element [1,1]. However, even when correcting for this mistake, there is an extra copy created when using matrix(). Try this in a fresh vanilla R session: > print(gc()) used (Mb) gc trigger (Mb) max used (Mb) Ncells 136684 3.7

No. But it has not been posted either. You got that message because you sent your message to the wrong address. You should have sent it to r-help at r-project.org You had probably sent it to r-help-request at r-project.org which would have had the effect that the server would have tried to interpret the contents of you message as commands (e.g. to unsubscribe, change your subscription

Hello all. I have a large .txt file whose variables are fixed-columns, ie, variable V1 goes from columns 1 to 7, V2 from 8 to 23 etc. This is a 60GB file with 90 variables and 60 million observations. I'm working with a Pentium 4, 1GB RAM, Windows XP Pro. I tried the following code just to see if I could work with 2 variables but it seems not possible: R : Copyright 2005, The R Foundation

I was surprised to observe the following difference between 2.4.1 and 2.6.0 after a long overdue upgrade a few months ago of our departmental server. It wasn't a bug fix, but a subtle improvement. Here's the simplest example I could create. The size is excessive, on the order of the Netflix Competition data. The integer matrix is about 1.12 GB, and if coerced to numeric it is 2.24 GB.

Hi, when doing as.double() on an object that is already a double, the object seems to be copied internally, doubling the memory requirement. See example below. Same for as.character() etc. Is this intended? Example: % R --vanilla > x <- double(1e7) > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 234019 6.3 467875 12.5 350000 9.4 Vcells 10103774 77.1

Dear useRs? I have the following problem! I have a function that calls one or more functions, depending on the input parameters. I am searching for the fastest way to select and execute the selected functions and return their results in a list. The number of possible functions is 10, however usually only 2 are selected (although sometimes more, even all). For examples, if I have function

Dear All, I would like some advice on creating R packages, passing by reference and oo R. I have created a package that is neither elegant nor extensible and rather cumbersome (it works). I would like to re write the code to make the package distributable (should it be of interest) and easy to maintain. The package is for Bayesian model determination via a reversible jump algorithm and has

Hi there, I seek your expert opinion on the following memory related questions. The output below was gotten from R-2.6.2, compiled with --enable-memory-profiling on Ubuntu Linux. ======================================================================= >>> Code and output 1: > gc( ) used (Mb) gc trigger (Mb) max used (Mb) Ncells 131180 7.1 350000 18.7 350000 18.7