On Fri, 2004-09-03 at 15:37, jmoreira at fe.up.pt wrote:> Hello to everybody,
>
> Does anyone has implemented a function for evaluating models using
windowing
> strategies, such as growing window or sliding window ones?
> The aim is to evaluate regression models on a time series data. I do not
use
> cross-validation once data sorted in a radom way does not make sense when
> evaluating time series.
>
I include two functions I've written that I think accomplish what you
want. They return a list with probably too many unnecessary components
for you (they were useful in the context I've used them), so you will
probably want to change that part.
Hope it helps,
Luis Torgo
#==================================================================# This
function allows the execution of sliding window tests using
# any algorithm.
# Example:
# > p.rt <-
sliding.window.testing(exp[1:800,],700,'rpart',learner.pars=list(fk5 ~
.))
# > p.nn <- sliding.window.testing(exp[1:800,],700,'nnet',
# learner.pars=list(fk5 ~ .,size=10,linout=T),
# relearn.step=7)
# Note: This last example only re-learns a new model every 7 cases
#---------------------------------------------------------------------
sliding.window.testing <- function(orig.data, window.size,
learner, learner.pars,
relearn.step=1, test.pos=window.size+1) {
init.test <- test.pos
n <- nrow(orig.data)
preds <- vector()
while (test.pos <= n) {
cat('*')
learner.pars$data <- orig.data[(test.pos-window.size):(test.pos-1),]
model <- do.call(learner,learner.pars)
preds <-
c(preds,predict(model,orig.data[test.pos:min(n,test.pos+relearn.step-1),]))
test.pos <- test.pos+relearn.step
}
cat('\n')
list(train.period=c(test.pos-relearn.step-window.size,test.pos-relearn.step-1),
model.call=list(func=learner,pars=learner.pars),
model=model,
test.period=c(init.test,n),
preds=preds,
preds.close=NULL,preds.ret=NULL,err.ret=NULL,weigh.preds=NULL)
}
# =====================================================================# This
function allows the execution of growing window tests using any
# algorithm.
# Example:
# > p.rt <-
growing.window.testing(exp[1:800,],700,'rpart',learner.pars=list(fk5 ~
.))
# > p.nn <- growing.window.testing(exp[1:800,],700,'nnet',
# learner.pars=list(fk5 ~ .,size=10,linout=T),
# relearn.step=7)
# Note: This last example only re-learns a new model every 7 cases
# ----------------------------------------------------------------------
growing.window.testing <- function(orig.data,
learner, learner.pars,
relearn.step=1, test.pos) {
init.test <- test.pos
n <- nrow(orig.data)
preds <- vector()
while (test.pos <= n) {
cat('*')
learner.pars$data <- orig.data[1:(test.pos-1),]
model <- do.call(learner,learner.pars)
preds <-
c(preds,predict(model,orig.data[test.pos:min(n,test.pos+relearn.step-1),]))
test.pos <- test.pos+relearn.step
}
cat('\n')
list(model.call=list(func=learner,pars=learner.pars),
model=model,
test.period=c(init.test,n),
preds=preds)
}
--
Luis Torgo
FEP/LIACC, University of Porto Phone : (+351) 22 607 88 30
Machine Learning Group Fax : (+351) 22 600 36 54
R. Campo Alegre, 823 email : ltorgo at liacc.up.pt
4150 PORTO - PORTUGAL WWW : http://www.liacc.up.pt/~ltorgo