Hello,
I need to calculate a moving average and an exponentially weighted moving
average over a fairly large data set (500K rows).
Doing this in a for loop works nicely, but is slow.
ewma <- data$col[1]
N <- dim(data)[1]
for(i in 2:N){
data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
}
Since the moving average "accumulates" as we move through the data,
I'm not sure on the best/fastest way to do this.
Does anyone have any suggestions on how to avoid a loop doing this?
--
Noah Silverman
UCLA Department of Statistics
8117 Math Sciences Building #8208
Los Angeles, CA 90095
[[alternative HTML version deleted]]
Have you looked at SMA/EMA from the TTR package? That's a pretty quick implementation. runmean from caTools is even better for the SMA but I don't think there's an easy way to turn that into an EWMA. Hope this helps, Michael Weylandt On Fri, Sep 2, 2011 at 12:43 PM, Noah Silverman <noahsilverman@ucla.edu>wrote:> Hello, > > I need to calculate a moving average and an exponentially weighted moving > average over a fairly large data set (500K rows). > > Doing this in a for loop works nicely, but is slow. > > ewma <- data$col[1] > N <- dim(data)[1] > for(i in 2:N){ > data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i] > } > > > Since the moving average "accumulates" as we move through the data, I'm not > sure on the best/fastest way to do this. > > Does anyone have any suggestions on how to avoid a loop doing this? > > > > > -- > Noah Silverman > UCLA Department of Statistics > 8117 Math Sciences Building #8208 > Los Angeles, CA 90095 > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >[[alternative HTML version deleted]]
There is a recent blog post by Dirk Eddelbeutel on how to do something similar using his Rcpp package and C++, with massive time improvements. http://dirk.eddelbuettel.com/blog/ On 9/2/2011 12:43 PM, Noah Silverman wrote:> Hello, > > I need to calculate a moving average and an exponentially weighted moving average over a fairly large data set (500K rows). > > Doing this in a for loop works nicely, but is slow. > > ewma<- data$col[1] > N<- dim(data)[1] > for(i in 2:N){ > data$ewma<- alpha * data$ewma[i-1] + (1-alpha) * data$value[i] > } > > > Since the moving average "accumulates" as we move through the data, I'm not sure on the best/fastest way to do this. > > Does anyone have any suggestions on how to avoid a loop doing this? > > > > > -- > Noah Silverman > UCLA Department of Statistics > 8117 Math Sciences Building #8208 > Los Angeles, CA 90095 > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.[[alternative HTML version deleted]]