rapton
2009-Sep-04 16:10 UTC
[R] Using anova(f1, f2) to compare lmer models yields seemingly erroneous Chisq = 0, p = 1
Hello, I am using R to analyze a large multilevel data set, using lmer() to model my data, and using anova() to compare the fit of various models. When I run two models, the output of each model is generated correctly as far as I can tell (e.g. summary(f1) and summary(f2) for the multilevel model output look perfectly reasonable), and in this case (see below) predictor.1 explains vastly more variance in outcome than predictor.2 (R2 = 15% vs. 5% in OLS regression, with very large N). What I am utterly puzzled by is that when I run an anova comparing the two multilevel model fits, the Chisq comes back as 0, with p = 1. I am pretty sure that fit #1 (f1) is a much better predictor of the outcome than f2, which is reflected in the AIC, BIC , and logLik values. Why might anova be giving me this curious output? How can I fix it? I am sure I am making a dumb error somewhere, but I cannot figure out what it is. Any help or suggestions would be greatly appreciated! -Matt> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i)) > f2 <- (lmer(outcome ~ predictor.2 + (1 | person), data=i)) > anova(f1, f2)Data: i Models: f1: outcome ~ predictor.1 + (1 | person) f2: outcome ~ predictor.2 + (1 | person) Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) f1 6 45443 45489 -22715 f2 25 47317 47511 -23633 0 19 1 -- View this message in context: http://www.nabble.com/Using-anova%28f1%2C-f2%29-to-compare-lmer-models-yields-seemingly-erroneous-Chisq-%3D-0%2C-p-%3D-1-tp25297254p25297254.html Sent from the R help mailing list archive at Nabble.com.
Bert Gunter
2009-Sep-04 20:18 UTC
[R] Using anova(f1, f2) to compare lmer models yields seemingly erroneous Chisq = 0, p = 1
My guess would be: "Likelihood comparisons are not meaningful for objects fit using restricted maximum likelihood and with different fixed effects. " (from ?anova.lme in the nlme package). Are you using the REML = TRUE default? Bert Gunter Genentech Nonclinical Statistics -----Original Message----- From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of rapton Sent: Friday, September 04, 2009 9:10 AM To: r-help at r-project.org Subject: [R] Using anova(f1, f2) to compare lmer models yields seemingly erroneous Chisq = 0, p = 1 Hello, I am using R to analyze a large multilevel data set, using lmer() to model my data, and using anova() to compare the fit of various models. When I run two models, the output of each model is generated correctly as far as I can tell (e.g. summary(f1) and summary(f2) for the multilevel model output look perfectly reasonable), and in this case (see below) predictor.1 explains vastly more variance in outcome than predictor.2 (R2 = 15% vs. 5% in OLS regression, with very large N). What I am utterly puzzled by is that when I run an anova comparing the two multilevel model fits, the Chisq comes back as 0, with p = 1. I am pretty sure that fit #1 (f1) is a much better predictor of the outcome than f2, which is reflected in the AIC, BIC , and logLik values. Why might anova be giving me this curious output? How can I fix it? I am sure I am making a dumb error somewhere, but I cannot figure out what it is. Any help or suggestions would be greatly appreciated! -Matt> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i)) > f2 <- (lmer(outcome ~ predictor.2 + (1 | person), data=i)) > anova(f1, f2)Data: i Models: f1: outcome ~ predictor.1 + (1 | person) f2: outcome ~ predictor.2 + (1 | person) Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) f1 6 45443 45489 -22715 f2 25 47317 47511 -23633 0 19 1 -- View this message in context: http://www.nabble.com/Using-anova%28f1%2C-f2%29-to-compare-lmer-models-yield s-seemingly-erroneous-Chisq-%3D-0%2C-p-%3D-1-tp25297254p25297254.html Sent from the R help mailing list archive at Nabble.com. ______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Alain Zuur
2009-Sep-07 16:33 UTC
[R] Using anova(f1, f2) to compare lmer models yields seemingly erroneous Chisq = 0, p = 1
rapton wrote:> > Hello, > > I am using R to analyze a large multilevel data set, using > lmer() to model my data, and using anova() to compare the fit of various > models. When I run two models, the output of each model is generated > correctly as far as I can tell (e.g. summary(f1) and summary(f2) for the > multilevel model output look perfectly reasonable), and in this case (see > below) predictor.1 explains vastly more variance in outcome than > predictor.2 > (R2 = 15% vs. 5% in OLS regression, with very large N). What I am utterly > puzzled by is that when I run an anova comparing the two multilevel model > fits, the Chisq comes back as 0, with p = 1. I am pretty sure that fit #1 > (f1) is a much better predictor of the outcome than f2, which is reflected > in the AIC, BIC , and logLik values. Why might anova be giving me this > curious output? How can I fix it? I am sure I am making a dumb error > somewhere, but I cannot figure out what it is. Any help or suggestions > would > be greatly appreciated! > > -Matt > > >> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i)) >> f2 <- (lmer(outcome ~ predictor.2 + (1 | person), data=i)) >> anova(f1, f2) > > Data: i > Models: > f1: outcome ~ predictor.1 + (1 | person) > f2: outcome ~ predictor.2 + (1 | person) > Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) > f1 6 45443 45489 -22715 > f2 25 47317 47511 -23633 0 19 1 >Your models are nest nested....it doesn't make sense to do. Alain ----- -------------------------------------------------------------------- Dr. Alain F. Zuur First author of: 1. Analysing Ecological Data (2007). Zuur, AF, Ieno, EN and Smith, GM. Springer. 680 p. 2. Mixed effects models and extensions in ecology with R. (2009). Zuur, AF, Ieno, EN, Walker, N, Saveliev, AA, and Smith, GM. Springer. 3. A Beginner's Guide to R (2009). Zuur, AF, Ieno, EN, Meesters, EHWG. Springer Statistical consultancy, courses, data analysis and software Highland Statistics Ltd. 6 Laverock road UK - AB41 6FN Newburgh Email: highstat at highstat.com URL: www.highstat.com -- View this message in context: http://www.nabble.com/Using-anova%28f1%2C-f2%29-to-compare-lmer-models-yields-seemingly-erroneous-Chisq-%3D-0%2C-p-%3D-1-tp25297254p25333120.html Sent from the R help mailing list archive at Nabble.com.
Alain Zuur
2009-Sep-07 16:34 UTC
[R] Using anova(f1, f2) to compare lmer models yields seemingly erroneous Chisq = 0, p = 1
rapton wrote:> > Hello, > > I am using R to analyze a large multilevel data set, using > lmer() to model my data, and using anova() to compare the fit of various > models. When I run two models, the output of each model is generated > correctly as far as I can tell (e.g. summary(f1) and summary(f2) for the > multilevel model output look perfectly reasonable), and in this case (see > below) predictor.1 explains vastly more variance in outcome than > predictor.2 > (R2 = 15% vs. 5% in OLS regression, with very large N). What I am utterly > puzzled by is that when I run an anova comparing the two multilevel model > fits, the Chisq comes back as 0, with p = 1. I am pretty sure that fit #1 > (f1) is a much better predictor of the outcome than f2, which is reflected > in the AIC, BIC , and logLik values. Why might anova be giving me this > curious output? How can I fix it? I am sure I am making a dumb error > somewhere, but I cannot figure out what it is. Any help or suggestions > would > be greatly appreciated! > > -Matt > > >> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i)) >> f2 <- (lmer(outcome ~ predictor.2 + (1 | person), data=i)) >> anova(f1, f2) > > Data: i > Models: > f1: outcome ~ predictor.1 + (1 | person) > f2: outcome ~ predictor.2 + (1 | person) > Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) > f1 6 45443 45489 -22715 > f2 25 47317 47511 -23633 0 19 1 >** NOT ** nested ....sorry....the brain is going faster than the fingers. ----- -------------------------------------------------------------------- Dr. Alain F. Zuur First author of: 1. Analysing Ecological Data (2007). Zuur, AF, Ieno, EN and Smith, GM. Springer. 680 p. 2. Mixed effects models and extensions in ecology with R. (2009). Zuur, AF, Ieno, EN, Walker, N, Saveliev, AA, and Smith, GM. Springer. 3. A Beginner's Guide to R (2009). Zuur, AF, Ieno, EN, Meesters, EHWG. Springer Statistical consultancy, courses, data analysis and software Highland Statistics Ltd. 6 Laverock road UK - AB41 6FN Newburgh Email: highstat at highstat.com URL: www.highstat.com -- View this message in context: http://www.nabble.com/Using-anova%28f1%2C-f2%29-to-compare-lmer-models-yields-seemingly-erroneous-Chisq-%3D-0%2C-p-%3D-1-tp25297254p25333148.html Sent from the R help mailing list archive at Nabble.com.