Hello,
I wrote the function, below, in the hope of _quickly_ generating a
sliding-window time series of the mean, sd, median, and mad values
from a matrix of data. The script below is anything but quick; it has
been working away on a 2500 x 2500 matrix with a sliding window of 100
x 100 for five days, with no end in sight. Obviously, I am not the
best of programmers. Can anyone offer suggestions as to how I might
optimize this script.
Thank you,
John
common.ground<-function(inMatrix,window){
cleanMatrix<-as.matrix(inMatrix)
diag(cleanMatrix)<-NA
outMatrix<-matrix(0,dim(inMatrix)[1]-window,4)
colnames(outMatrix)<-c("mean","SD",
"median","mad")
for(i in 1:dim(outMatrix)[1]){
for(j in window:dim(cleanMatrix)[2]){
outMatrix[i,1]<-mean(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE)
outMatrix[i,2]<-sd(c(cleanMatrix[c(i:j),c(i:j)]),na.rm=TRUE)
outMatrix[i,3]<-median(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE)
outMatrix[i,4]<-mad(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE)
}
}
return(outMatrix)
}
See: http://tolstoy.newcastle.edu.au/R/help/04/10/5161.html On 10/4/06, JOHN VOIKLIS <voiklis at gmail.com> wrote:> Hello, > > I wrote the function, below, in the hope of _quickly_ generating a > sliding-window time series of the mean, sd, median, and mad values > from a matrix of data. The script below is anything but quick; it has > been working away on a 2500 x 2500 matrix with a sliding window of 100 > x 100 for five days, with no end in sight. Obviously, I am not the > best of programmers. Can anyone offer suggestions as to how I might > optimize this script. > > Thank you, > > John > > common.ground<-function(inMatrix,window){ > > cleanMatrix<-as.matrix(inMatrix) > diag(cleanMatrix)<-NA > outMatrix<-matrix(0,dim(inMatrix)[1]-window,4) > colnames(outMatrix)<-c("mean","SD", "median","mad") > > for(i in 1:dim(outMatrix)[1]){ > for(j in window:dim(cleanMatrix)[2]){ > outMatrix[i,1]<-mean(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > outMatrix[i,2]<-sd(c(cleanMatrix[c(i:j),c(i:j)]),na.rm=TRUE) > outMatrix[i,3]<-median(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > outMatrix[i,4]<-mad(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > } > } > return(outMatrix) > } > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
Also see package caTools. On 10/4/06, Gabor Grothendieck <ggrothendieck at gmail.com> wrote:> See: > > http://tolstoy.newcastle.edu.au/R/help/04/10/5161.html > > On 10/4/06, JOHN VOIKLIS <voiklis at gmail.com> wrote: > > Hello, > > > > I wrote the function, below, in the hope of _quickly_ generating a > > sliding-window time series of the mean, sd, median, and mad values > > from a matrix of data. The script below is anything but quick; it has > > been working away on a 2500 x 2500 matrix with a sliding window of 100 > > x 100 for five days, with no end in sight. Obviously, I am not the > > best of programmers. Can anyone offer suggestions as to how I might > > optimize this script. > > > > Thank you, > > > > John > > > > common.ground<-function(inMatrix,window){ > > > > cleanMatrix<-as.matrix(inMatrix) > > diag(cleanMatrix)<-NA > > outMatrix<-matrix(0,dim(inMatrix)[1]-window,4) > > colnames(outMatrix)<-c("mean","SD", "median","mad") > > > > for(i in 1:dim(outMatrix)[1]){ > > for(j in window:dim(cleanMatrix)[2]){ > > outMatrix[i,1]<-mean(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > outMatrix[i,2]<-sd(c(cleanMatrix[c(i:j),c(i:j)]),na.rm=TRUE) > > outMatrix[i,3]<-median(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > outMatrix[i,4]<-mad(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > } > > } > > return(outMatrix) > > } > > > > ______________________________________________ > > R-help at stat.math.ethz.ch mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > >
On 10/4/06, Gabor Grothendieck <ggrothendieck at gmail.com> wrote:> Also see package caTools. > > On 10/4/06, Gabor Grothendieck <ggrothendieck at gmail.com> wrote: > > See: > > > > http://tolstoy.newcastle.edu.au/R/help/04/10/5161.html > > > > On 10/4/06, JOHN VOIKLIS <voiklis at gmail.com> wrote: > > > Hello, > > > > > > I wrote the function, below, in the hope of _quickly_ generating a > > > sliding-window time series of the mean, sd, median, and mad values > > > from a matrix of data. The script below is anything but quick; it has > > > been working away on a 2500 x 2500 matrix with a sliding window of 100 > > > x 100 for five days, with no end in sight. Obviously, I am not the > > > best of programmers. Can anyone offer suggestions as to how I might > > > optimize this script. > > > > > > Thank you, > > > > > > John > > > > > > common.ground<-function(inMatrix,window){ > > > > > > cleanMatrix<-as.matrix(inMatrix) > > > diag(cleanMatrix)<-NA > > > outMatrix<-matrix(0,dim(inMatrix)[1]-window,4) > > > colnames(outMatrix)<-c("mean","SD", "median","mad") > > > > > > for(i in 1:dim(outMatrix)[1]){ > > > for(j in window:dim(cleanMatrix)[2]){ > > > outMatrix[i,1]<-mean(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > > outMatrix[i,2]<-sd(c(cleanMatrix[c(i:j),c(i:j)]),na.rm=TRUE) > > > outMatrix[i,3]<-median(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > > outMatrix[i,4]<-mad(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > > } > > > } > > > return(outMatrix) > > > }Also you could look at rollmax, rollmean and rollmedian in the zoo package. rollapply in zoo with FUN = sd could be used for the sd.
Thank you, Gabor, for all your suggestions. Best, John On 10/5/06, Gabor Grothendieck <ggrothendieck at gmail.com> wrote:> On 10/4/06, Gabor Grothendieck <ggrothendieck at gmail.com> wrote: > > Also see package caTools. > > > > On 10/4/06, Gabor Grothendieck <ggrothendieck at gmail.com> wrote: > > > See: > > > > > > http://tolstoy.newcastle.edu.au/R/help/04/10/5161.html > > > > > > On 10/4/06, JOHN VOIKLIS <voiklis at gmail.com> wrote: > > > > Hello, > > > > > > > > I wrote the function, below, in the hope of _quickly_ generating a > > > > sliding-window time series of the mean, sd, median, and mad values > > > > from a matrix of data. The script below is anything but quick; it has > > > > been working away on a 2500 x 2500 matrix with a sliding window of 100 > > > > x 100 for five days, with no end in sight. Obviously, I am not the > > > > best of programmers. Can anyone offer suggestions as to how I might > > > > optimize this script. > > > > > > > > Thank you, > > > > > > > > John > > > > > > > > common.ground<-function(inMatrix,window){ > > > > > > > > cleanMatrix<-as.matrix(inMatrix) > > > > diag(cleanMatrix)<-NA > > > > outMatrix<-matrix(0,dim(inMatrix)[1]-window,4) > > > > colnames(outMatrix)<-c("mean","SD", "median","mad") > > > > > > > > for(i in 1:dim(outMatrix)[1]){ > > > > for(j in window:dim(cleanMatrix)[2]){ > > > > outMatrix[i,1]<-mean(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > > > outMatrix[i,2]<-sd(c(cleanMatrix[c(i:j),c(i:j)]),na.rm=TRUE) > > > > outMatrix[i,3]<-median(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > > > outMatrix[i,4]<-mad(cleanMatrix[c(i:j),c(i:j)],na.rm=TRUE) > > > > } > > > > } > > > > return(outMatrix) > > > > } > > Also you could look at rollmax, rollmean and rollmedian in the zoo package. > rollapply in zoo with FUN = sd could be used for the sd. >