David L. Van Brunt, Ph.D.
2007-Mar-23 18:14 UTC
[R] memory, speed, and assigning results into new v. existing variable
I have a very large data frame, and I'm doing a conversion of all columns into factors. Takes a while (thanks to folks here though, for making faster!), but am wondering about optimization from a memory perspective... Internally, am I better off assigning into a new data frame, or doing one of these: dataframe<-someoperation(dataframe) It would seem that re-assigning into the same data frame *might* be more efficient in that I don't have to double the memory with 2 objects, but then again.. there'd have to be a "working copy" in memory until the operation was complete, right? Just curious about the inner workings of R. -- --------------------------------------- David L. Van Brunt, Ph.D. mailto:dlvanbrunt@gmail.com "If Tyranny and Oppression come to this land, it will be in the guise of fighting a foreign enemy." --James Madison [[alternative HTML version deleted]]
David L. Van Brunt, Ph.D.
2007-Mar-23 18:59 UTC
[R] memory, speed, and assigning results into new v. existing variable
Yes, at the end... but I'm getting memory allocation errors and outright crashes DURING the operation when assigning into the same variable... still waiting for the alternative to survive or not... On 3/23/07, Bert Gunter <gunter.berton@gene.com> wrote:> > Actually... there can be more than one "working copy". > > Clearly, at the end, when you assign back to the same object, you use less > memory. > > Cheers, > Bert Gunter > Genentech Nonclinical Statistics > South San Francisco, CA 94404 > 650-467-7374 > > > -----Original Message----- > From: r-help-bounces@stat.math.ethz.ch > [mailto:r-help-bounces@stat.math.ethz.ch] On Behalf Of David L. Van Brunt, > Ph.D. > Sent: Friday, March 23, 2007 11:15 AM > To: R-Help List > Subject: [R] memory, speed,and assigning results into new v. existing > variable > > I have a very large data frame, and I'm doing a conversion of all columns > into factors. Takes a while (thanks to folks here though, for making > faster!), but am wondering about optimization from a memory perspective... > > Internally, am I better off assigning into a new data frame, or doing one > of > these: > > dataframe<-someoperation(dataframe) > > It would seem that re-assigning into the same data frame *might* be more > efficient in that I don't have to double the memory with 2 objects, but > then > again.. there'd have to be a "working copy" in memory until the operation > was complete, right? > > Just curious about the inner workings of R. > > -- > --------------------------------------- > David L. Van Brunt, Ph.D. > mailto:dlvanbrunt@gmail.com > > "If Tyranny and Oppression come to this land, it will be in the guise of > fighting a foreign enemy." > --James Madison > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > >-- --------------------------------------- David L. Van Brunt, Ph.D. mailto:dlvanbrunt@gmail.com "If Tyranny and Oppression come to this land, it will be in the guise of fighting a foreign enemy." --James Madison [[alternative HTML version deleted]]
Possibly Parallel Threads
- replacing all NA's in a dataframe with zeros...
- How might I -remove- a tree from a random forest?
- randomForest gives different results for formula call v. x, y methods. Why?
- Repost: Examples of "classwt", "strata", and "sampsize" i n randomForest?
- Repost: Examples of "classwt", "strata", and "sampsize" in randomForest?