John Williams
2014-Apr-08 00:29 UTC
[R] {metafor} variance explaination for paired pre-test/posttest
In a previous post https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html <https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html> , the following calculation was given for imputing the variance of change scores for paired studies: // begin quote 2) Often, the dependent variable is not the same in each study. Then you will have to resort to a standardized outcome measure. There are two options: a) standardization based on the change score standard deviation Then yi = (m1i - m2i) / sdi with sampling variance vi = 1/ni + yi^2 / (2*ni). // end quote I used the sampling variance equation above in a paper that is being reviewed by a coauthor, who is a biostatistician. He commented that he has never seen this equation for variance before, and it looks strange to him. To put my knowledge into perspective, I am an undergraduate taking my first statistics course. I imputed the t-statistic from two-sided p-values reported in the paper, and used that to get the sdi (as in the previous post). I consulted the Cochrane Handbook and The Handbook of Research Syntheses and Meta-analysis 2nd Ed (Cooper, Hedges, Valentine 2009) and couldn't find that equation anywhere. Would Prof. Viechtbauer, or anyone else knowledgeable, mind explaining the sample variance above? I need to be able to defend my choice of equation. Since it's the only method that I found that doesn't rely on a correlation coefficient (which are not included in the papers), I'd like to be able to justify it and not redo calculations for 23 studies if possible. Thank you very much, John ~~~~ John Williams ALB Candidate, Harvard University (Expected May 2014) johnwilliams at fas.harvard.edu jawilliamsjr at gmail.com -- View this message in context: http://r.789695.n4.nabble.com/metafor-variance-explaination-for-paired-pre-test-posttest-tp4688365.html Sent from the R help mailing list archive at Nabble.com.
Viechtbauer Wolfgang (STAT)
2014-Apr-08 07:54 UTC
[R] {metafor} variance explaination for paired pre-test/posttest
The standardized mean change using 'change score standardization' is described in this article: Gibbons, R. D., Hedeker, D. R., & Davis, J. M. (1993). Estimation of effect size from a series of experiments involving paired comparisons. Journal of Educational Statistics, 18(3), 271-279. For a comparison of the standardized mean change using change versus raw score standardization, see: Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105-125. Viechtbauer, W. (2007). Approximate confidence intervals for standardized effect sizes in the two-independent and two-dependent samples design. Journal of Educational and Behavioral Statistics, 32(1), 39-60. These articles also provide equations for the sampling variance of the standardized mean change. The equation 1/ni + yi^2/(2*ni) is the estimate based on the asymptotic variance of the standardized mean change using change score standardization. Best, Wolfgang -- Wolfgang Viechtbauer, Ph.D., Statistician Department of Psychiatry and Psychology School for Mental Health and Neuroscience Faculty of Health, Medicine, and Life Sciences Maastricht University, P.O. Box 616 (VIJV1) 6200 MD Maastricht, The Netherlands +31 (43) 388-4170 | http://www.wvbauer.com> -----Original Message----- > From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] > On Behalf Of John Williams > Sent: Tuesday, April 08, 2014 02:30 > To: r-help at r-project.org > Subject: [R] {metafor} variance explaination for paired pre-test/posttest > > In a previous post > https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html > <https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html> , the > following calculation was given for imputing the variance of change > scores > for paired studies: > > // begin quote > > 2) Often, the dependent variable is not the same in each study. Then you > will have to resort to a standardized outcome measure. There are two > options: > > a) standardization based on the change score standard deviation > > Then yi = (m1i - m2i) / sdi with sampling variance vi = 1/ni + yi^2 / > (2*ni). > > // end quote > > I used the sampling variance equation above in a paper that is being > reviewed by a coauthor, who is a biostatistician. > > He commented that he has never seen this equation for variance before, > and > it looks strange to him. To put my knowledge into perspective, I am an > undergraduate taking my first statistics course. I imputed the t- > statistic > from two-sided p-values reported in the paper, and used that to get the > sdi > (as in the previous post). > > I consulted the Cochrane Handbook and The Handbook of Research Syntheses > and > Meta-analysis 2nd Ed (Cooper, Hedges, Valentine 2009) and couldn't find > that > equation anywhere. > > Would Prof. Viechtbauer, or anyone else knowledgeable, mind explaining > the > sample variance above? I need to be able to defend my choice of equation. > Since it's the only method that I found that doesn't rely on a > correlation > coefficient (which are not included in the papers), I'd like to be able > to > justify it and not redo calculations for 23 studies if possible. > > Thank you very much, > > John > > ~~~~ > John Williams > ALB Candidate, Harvard University (Expected May 2014) > johnwilliams at fas.harvard.edu > jawilliamsjr at gmail.com