R help - Feb 2013 - metafor - interpretion of QM in mixed-effects model with factor moderator

Hi,

I'm using metafor to perform a mixed-effects meta-analysis. I'd like to
test whether the effect is different for animals and plants/whether
"group"
(animal/plant) influences the effect size, but am having trouble
interpreting the results I get. I've read previous posts about QM in
metafor, but I'm still a bit confused. I've dummy-coded the factors:

MData$k.animal<-ifelse(MData$Type=="Animal",1,0)MData$k.plant<-ifelse(MData$Type=="Plant",
1,0)


I've then run the model with the following script:

MixModel<-rma(z, v.z., mods=cbind(k.animal, k.plant), intercept=FALSE,
data=MData, subset=(Response=="Tolerance"))MixModel


I receive the output:

Test of Moderators (coefficient(s) 1,2):
QM(df = 2) = 8.5758, p-val = 0.0137

Model Results:

          estimate      se    zval    pval    ci.lb   ci.ub
k.animal    0.4877  0.2703  1.8042  0.0712  -0.0421  1.0175  .
k.plant     0.5095  0.2209  2.3066  0.0211   0.0766  0.9424  *

Does this mean that, overall, there is a significant effect of "group"
on
the overall effect size (based on the QM test)? If I want to test whether
the effect size for plants is different to that for animals, can I use:

MixModel2<-rma(z, v.z., mods=cbind(k.plant), data=MData,
subset=(Response=="Tolerance")) ?


Here I get:

Test of Moderators (coefficient(s) 2):
QM(df = 1) = 0.0039, p-val = 0.9502

Model Results:

         estimate      se    zval    pval    ci.lb   ci.ub
intrcpt    0.4877  0.2703  1.8042  0.0712  -0.0421  1.0175  .
k.plant    0.0218  0.3491  0.0624  0.9502  -0.6624  0.7060


Does this mean that there is no significant difference in effect size
between plants and animals (now the intercept?) ? Or is this something
completely different?

I'd really appreciate any input,

Rachel Slatyer

	[[alternative HTML version deleted]]

Please see my comments below.

Best,
Wolfgang
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at
r-project.org]
> On Behalf Of Rachel Slatyer
> Sent: Wednesday, February 27, 2013 06:00
> To: R-help at r-project.org
> Subject: [R] metafor - interpretion of QM in mixed-effects model with
> factor moderator
> 
> Hi,
> 
> I'm using metafor to perform a mixed-effects meta-analysis. I'd
like to
> test whether the effect is different for animals and plants/whether
> "group"
> (animal/plant) influences the effect size, but am having trouble
> interpreting the results I get. I've read previous posts about QM in
> metafor, but I'm still a bit confused. I've dummy-coded the
factors:
> 
> MData$k.animal<-ifelse(MData$Type=="Animal",1,0)
> MData$k.plant<-ifelse(MData$Type=="Plant",1,0)
> 
> I've then run the model with the following script:
> 
> MixModel<-rma(z, v.z., mods=cbind(k.animal, k.plant), intercept=FALSE,
> data=MData, subset=(Response=="Tolerance"))
> 
> I receive the output:
> 
> Test of Moderators (coefficient(s) 1,2):
> QM(df = 2) = 8.5758, p-val = 0.0137
> 
> Model Results:
> 
>           estimate      se    zval    pval    ci.lb   ci.ub
> k.animal    0.4877  0.2703  1.8042  0.0712  -0.0421  1.0175  .
> k.plant     0.5095  0.2209  2.3066  0.0211   0.0766  0.9424  *
> 
> Does this mean that, overall, there is a significant effect of
"group" on
> the overall effect size (based on the QM test)? 
Not quite. Since you have removed the intercept, the two estimates reflect the
estimated effect sizes (or to be precise, the estimated average effect sizes)
for "animals" and "plants", respectively. The QM test is
testing these two coefficients simultaneously, so it is testing the null
hypothesis that the average effect sizes are both equal to 0. Since the test is
significant, we can reject that null hypothesis. However, this does not test
whether the effect is different for animals or plants.
> If I want to test whether
> the effect size for plants is different to that for animals, can I use:
> 
> MixModel2<-rma(z, v.z., mods=cbind(k.plant), data=MData,
> subset=(Response=="Tolerance")) ?
> 
> Here I get:
> 
> Test of Moderators (coefficient(s) 2):
> QM(df = 1) = 0.0039, p-val = 0.9502
> 
> Model Results:
> 
>          estimate      se    zval    pval    ci.lb   ci.ub
> intrcpt    0.4877  0.2703  1.8042  0.0712  -0.0421  1.0175  .
> k.plant    0.0218  0.3491  0.0624  0.9502  -0.6624  0.7060
> 
> Does this mean that there is no significant difference in effect size
> between plants and animals (now the intercept?) ? Or is this something
> completely different?
Correct. Now you are testing whether the difference in the average effect for
"plants" is different from the average effect for "animals".
The estimated difference is 0.0218 (so the estimated average effect for plants
is 0.4877 + 0.0218 = 0.5095 -- the same value as obtained from your first
model), which is not significantly different from zero.

This is a nice example to illustrate that the common practice of subgrouping
(whether it be in a meta-analysis or in a primary trial) and testing effects
within subgroups can be misleading. Note that your first example indicates a
significant effect for plants, but not for animals. If you were to fit a
random-effects model just within the subset of studies with animals and just
within the subset of studies with plants, you will probably obtain rather
similar results as in your first model (the difference only arising from the
fact that in your first model, tau^2 is estimated once, while two separate
models within those two subgroups will also give you two separate values for
tau^2). However, the difference in the effect is far from significant and that
is in fact the proper test of this interaction.
> I'd really appreciate any input,
> 
> Rachel Slatyer
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.