Hi, I have two DLM model specifications (x[t] and y[t] are univariate): MODEL1: y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2) b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2) MODEL2: y[t] = a[t]+e[t], e[t] ~ N(0,v2^2) a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2) I run the filter through data recursively to obtain state variables for each model. However, how do I know if b[t]x[t] in MODEL1 is different from MODEL2? In other words, how do I know if x[t] makes a difference in explaining dynamic of y[t]? Another question is that how do I compare MODEL1 and MODEL2? From model specification point of view, how can one say that MODEL1 is better than MODEL2? Any suggestion/reference would be greatly appreciated. Thank you. ac
Erb Philipp (erbp)
2009-Oct-08 06:32 UTC
[R] [R-SIG-Finance] Evaluating/comparing dynamic linear model
What kind of filter are you using? Since your models are expressed in state space form I suggest that you fit your models by maximizing the log likelihood function of the Kalman filter output (see e.g. FKF-package). Using the obtained log likelihood values you might perform a likelihood ratio test to test the hypothesis whether model 1 explains yt "better" than model 2. HTH, Phil -----Urspr?ngliche Nachricht----- Von: r-sig-finance-bounces at stat.math.ethz.ch [mailto:r-sig-finance-bounces at stat.math.ethz.ch] Im Auftrag von R_help Help Gesendet: Donnerstag, 8. Oktober 2009 02:55 An: r-sig-finance at stat.math.ethz.ch; r-help at r-project.org Betreff: [R-SIG-Finance] Evaluating/comparing dynamic linear model Hi, I have two DLM model specifications (x[t] and y[t] are univariate): MODEL1: y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2) b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2) MODEL2: y[t] = a[t]+e[t], e[t] ~ N(0,v2^2) a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2) I run the filter through data recursively to obtain state variables for each model. However, how do I know if b[t]x[t] in MODEL1 is different from MODEL2? In other words, how do I know if x[t] makes a difference in explaining dynamic of y[t]? Another question is that how do I compare MODEL1 and MODEL2? From model specification point of view, how can one say that MODEL1 is better than MODEL2? Any suggestion/reference would be greatly appreciated. Thank you. ac _______________________________________________ R-SIG-Finance at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-finance -- Subscriber-posting only. -- If you want to post, subscribe first.
The standard asymptotic theory of likelihood ratio tests assumes that you are testing a submodel, which is not the case here. Moreover, even when testing submodels, there are other assumptions that often are not met in the case of DLMs - the typical example being hypothesised values on the boundary of the parameter space. An informal, but commonly used approach to compare DLMs is to compare their predictive performance. There are many ways to measure it. You can find some of them in the book "dynamic linear models with R". BTW, in the package dlm you find functions for MLE, forecasting, one-step-ahead forecast errors (which is what you need to compute predictive performance), and more... HTH, Giovanni> Date: Thu, 08 Oct 2009 08:32:21 +0200 > From: "Erb Philipp (erbp)" <erbp at zhaw.ch> > Sender: r-help-bounces at r-project.org > Precedence: list > Thread-topic: [R-SIG-Finance] Evaluating/comparing dynamic linear model > Thread-index: AcpHsn7VAX05YB13ShmB34Y7FCJyDAALTuuA > > What kind of filter are you using? Since your models are expressed > in state space form I suggest that you fit your models by maximizing > the log likelihood function of the Kalman filter output (see > e.g. FKF-package). Using the obtained log likelihood values you > might perform a likelihood ratio test to test the hypothesis whether > model 1 explains yt "better" than model 2. > > HTH, Phil > > > -----Urspr?ngliche Nachricht----- > Von: r-sig-finance-bounces at stat.math.ethz.ch [mailto:r-sig-finance-bounces at stat.math.ethz.ch] Im Auftrag von R_help Help > Gesendet: Donnerstag, 8. Oktober 2009 02:55 > An: r-sig-finance at stat.math.ethz.ch; r-help at r-project.org > Betreff: [R-SIG-Finance] Evaluating/comparing dynamic linear model > > Hi, > > I have two DLM model specifications (x[t] and y[t] are univariate): > > MODEL1: > y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2) > b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2) > > MODEL2: > y[t] = a[t]+e[t], e[t] ~ N(0,v2^2) > a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2) > > I run the filter through data recursively to obtain state variables > for each model. However, how do I know if b[t]x[t] in MODEL1 is > different from MODEL2? In other words, how do I know if x[t] makes a > difference in explaining dynamic of y[t]? > > Another question is that how do I compare MODEL1 and MODEL2? From > model specification point of view, how can one say that MODEL1 is > better than MODEL2? Any suggestion/reference would be greatly > appreciated. Thank you. > > ac > > _______________________________________________ > R-SIG-Finance at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. > -- If you want to post, subscribe first. > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > > >
Apologies for my overhasty and thus inaccurate reply. I of course assumed that model2 is a submodel of model1. But thanks to Giovanni: a closer look at your equations shows that this is only the case if at == bt (and of course same innovations/ disturbances et, etat) and xt e.g. something like xt == theta0 * zt + 1. In that case you are allowed to perform a likelihood ratio test to test the hypothesis theta0 == 0. If model2 is included in model1 likelihood ratio test might be more convenient than comparing the predictive performance. -----Original Message----- From: Giovanni Petris [mailto:GPetris at uark.edu] Sent: Thu 10/8/2009 3:55 PM To: Erb Philipp (erbp) Cc: rhelpacc at gmail.com; r-help at r-project.org Subject: Re: [R] Evaluating/comparing dynamic linear model The standard asymptotic theory of likelihood ratio tests assumes that you are testing a submodel, which is not the case here. Moreover, even when testing submodels, there are other assumptions that often are not met in the case of DLMs - the typical example being hypothesised values on the boundary of the parameter space. An informal, but commonly used approach to compare DLMs is to compare their predictive performance. There are many ways to measure it. You can find some of them in the book "dynamic linear models with R". BTW, in the package dlm you find functions for MLE, forecasting, one-step-ahead forecast errors (which is what you need to compute predictive performance), and more... HTH, Giovanni> Date: Thu, 08 Oct 2009 08:32:21 +0200 > From: "Erb Philipp (erbp)" <erbp at zhaw.ch> > Sender: r-help-bounces at r-project.org > Precedence: list > Thread-topic: [R-SIG-Finance] Evaluating/comparing dynamic linear model > Thread-index: AcpHsn7VAX05YB13ShmB34Y7FCJyDAALTuuA > > What kind of filter are you using? Since your models are expressed > in state space form I suggest that you fit your models by maximizing > the log likelihood function of the Kalman filter output (see > e.g. FKF-package). Using the obtained log likelihood values you > might perform a likelihood ratio test to test the hypothesis whether > model 1 explains yt "better" than model 2. > > HTH, Phil > > > -----Urspr?ngliche Nachricht----- > Von: r-sig-finance-bounces at stat.math.ethz.ch [mailto:r-sig-finance-bounces at stat.math.ethz.ch] Im Auftrag von R_help Help > Gesendet: Donnerstag, 8. Oktober 2009 02:55 > An: r-sig-finance at stat.math.ethz.ch; r-help at r-project.org > Betreff: [R-SIG-Finance] Evaluating/comparing dynamic linear model > > Hi, > > I have two DLM model specifications (x[t] and y[t] are univariate): > > MODEL1: > y[t] = b[t]x[t]+e[t], e[t] ~ N(0,v1^2) > b[t] = b[t-1]+eta[t], eta[t] ~ N(0,w1^2) > > MODEL2: > y[t] = a[t]+e[t], e[t] ~ N(0,v2^2) > a[t] = a[t-1]+eta[t], eta[t] ~ N(0,w2^2) > > I run the filter through data recursively to obtain state variables > for each model. However, how do I know if b[t]x[t] in MODEL1 is > different from MODEL2? In other words, how do I know if x[t] makes a > difference in explaining dynamic of y[t]? > > Another question is that how do I compare MODEL1 and MODEL2? From > model specification point of view, how can one say that MODEL1 is > better than MODEL2? Any suggestion/reference would be greatly > appreciated. Thank you. > > ac > > _______________________________________________ > R-SIG-Finance at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-sig-finance > -- Subscriber-posting only. > -- If you want to post, subscribe first. > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > > >