Hi all, If I use a moving average, it will smooth the choppy time series, but it will lead to lagging... How do I smooth timeseries without the lagging effect? Thanks!
losemind wrote:> > If I use a moving average, it will smooth the choppy time series, but > it will lead to lagging... >The lagging can only be removed if you look into the future, otherwise you run into causality problems. So the easiest way is to center your output data on half the window width. Not using a rectangular weighting is recommended. Dieter -- View this message in context: http://www.nabble.com/how-to-smooth-timeseries-without-the-lagging--tp24654666p24656137.html Sent from the R help mailing list archive at Nabble.com.
Gabor Grothendieck
2009-Jul-25 10:13 UTC
[R] how to smooth timeseries without the lagging?
On Sat, Jul 25, 2009 at 5:07 AM, Dieter Menne<dieter.menne at menne-biomed.de> wrote:> > > > losemind wrote: >> >> If I use a moving average, it will smooth the choppy time series, but >> it will lead to lagging... >> > > The lagging can only be removed if you look into the future, otherwise you > run into causality problems. > So the easiest way is to center your output data on half the window width. > Not using a rectangular weighting is recommended. >Note that rollmean/rollmax/rollmedian/rollapply in the zoo package centers by default and whether its centered or left and or right aligned is controlled by the align= argument.
Have you thought about fourier filtering? here is some code that may help. library(StreamMetabolism) library(mFilter) data(DOTemp) #this makes it the same everytime set.seed(100) #making a time series with the frequency = 96 #reading in one unit in other words this is 15min data #so there are 96 readings in one day this will make it #easier to interpret, but feel free to put it on any scale that you would like #this makes a time series with random noise added and this is what we will remove starting at zero x <- ts(rep(coredata(DOTemp[,"DO"]), 24)+rnorm(2304), frequency=96, start=0) #fourier filtering short.fft <- fft(x) #so look at this plot and you can see that most of the power is one day and below plot(Re(short.fft), xlim=c(0,10), ylim=c(-1000, 1000)) #so now you know that you need to chunk out the higher #frequencies to look at only the daily and longer signals, #but just to be safe lets include 0-3 where most of the power seems to be #the index values come from 3*96=288 and since the fourier transform #is symetrical then you have to leave 288 observations on the #other side also. So: #first index 3*69 #to the second index (see above) length(x)-288 short.fft[288:2016] = 0+0i short.ifft = fft(short.fft, inverse = TRUE)/length(short.fft) #so lets look at why we did this par(mfrow=c(2,1)) plot(x) plot(Re(short.ifft)) #and look at the spectrum #most of the high noise garbage is gone. If you want to #get rid of more then please help yourself. spectrum(short.ifft) hth Stephen Sefick On Fri, Jul 24, 2009 at 10:44 PM, Michael<comtech.usa at gmail.com> wrote:> Hi all, > > If I use a moving average, it will smooth the choppy time series, but > it will lead to lagging... > > How do I smooth timeseries without the lagging effect? > > Thanks! > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Stephen Sefick Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods. We are mammals, and have not exhausted the annoying little problems of being mammals. -K. Mullis