R help - Apr 2014 - {metafor} variance explaination for paired pre-test/posttest

In a previous post
https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html
<https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html>  , the
following calculation was given for imputing the variance of change scores
for paired studies:

// begin quote

2) Often, the dependent variable is not the same in each study. Then you
will have to resort to a standardized outcome measure. There are two
options:

a) standardization based on the change score standard deviation

Then yi = (m1i - m2i) / sdi with sampling variance vi = 1/ni + yi^2 /
(2*ni).

// end quote

I used the sampling variance equation above in a paper that is being
reviewed by a coauthor, who is a biostatistician. 

He commented that he has never seen this equation for variance before, and
it looks strange to him. To put my knowledge into perspective, I am an
undergraduate taking my first statistics course. I imputed the t-statistic
from two-sided p-values reported in the paper, and used that to get the sdi
(as in the previous post). 

I consulted the Cochrane Handbook and The Handbook of Research Syntheses and
Meta-analysis 2nd Ed (Cooper, Hedges, Valentine 2009) and couldn't find that
equation anywhere. 

Would Prof. Viechtbauer, or anyone else knowledgeable, mind explaining the
sample variance above? I need to be able to defend my choice of equation.
Since it's the only method that I found that doesn't rely on a
correlation
coefficient (which are not included in the papers), I'd like to be able to
justify it and not redo calculations for 23 studies if possible.

Thank you very much,

John

~~~~
John Williams
ALB Candidate, Harvard University (Expected May 2014)
johnwilliams at fas.harvard.edu
jawilliamsjr at gmail.com



--
View this message in context:
http://r.789695.n4.nabble.com/metafor-variance-explaination-for-paired-pre-test-posttest-tp4688365.html
Sent from the R help mailing list archive at Nabble.com.

The standardized mean change using 'change score standardization' is
described in this article:

Gibbons, R. D., Hedeker, D. R., & Davis, J. M. (1993). Estimation of effect
size from a series of experiments involving paired comparisons. Journal of
Educational Statistics, 18(3), 271-279.

For a comparison of the standardized mean change using change versus raw score
standardization, see:

Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in
meta-analysis with repeated measures and independent-groups designs.
Psychological Methods, 7(1), 105-125.

Viechtbauer, W. (2007). Approximate confidence intervals for standardized effect
sizes in the two-independent and two-dependent samples design. Journal of
Educational and Behavioral Statistics, 32(1), 39-60.

These articles also provide equations for the sampling variance of the
standardized mean change. The equation 1/ni + yi^2/(2*ni) is the estimate based
on the asymptotic variance of the standardized mean change using change score
standardization.

Best,
Wolfgang

--   
Wolfgang Viechtbauer, Ph.D., Statistician   
Department of Psychiatry and Psychology   
School for Mental Health and Neuroscience   
Faculty of Health, Medicine, and Life Sciences   
Maastricht University, P.O. Box 616 (VIJV1)   
6200 MD Maastricht, The Netherlands   
+31 (43) 388-4170 | http://www.wvbauer.com   
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at
r-project.org]
> On Behalf Of John Williams
> Sent: Tuesday, April 08, 2014 02:30
> To: r-help at r-project.org
> Subject: [R] {metafor} variance explaination for paired pre-test/posttest
> 
> In a previous post
> https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html
> <https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html>  , the
> following calculation was given for imputing the variance of change
> scores
> for paired studies:
> 
> // begin quote
> 
> 2) Often, the dependent variable is not the same in each study. Then you
> will have to resort to a standardized outcome measure. There are two
> options:
> 
> a) standardization based on the change score standard deviation
> 
> Then yi = (m1i - m2i) / sdi with sampling variance vi = 1/ni + yi^2 /
> (2*ni).
> 
> // end quote
> 
> I used the sampling variance equation above in a paper that is being
> reviewed by a coauthor, who is a biostatistician.
> 
> He commented that he has never seen this equation for variance before,
> and
> it looks strange to him. To put my knowledge into perspective, I am an
> undergraduate taking my first statistics course. I imputed the t-
> statistic
> from two-sided p-values reported in the paper, and used that to get the
> sdi
> (as in the previous post).
> 
> I consulted the Cochrane Handbook and The Handbook of Research Syntheses
> and
> Meta-analysis 2nd Ed (Cooper, Hedges, Valentine 2009) and couldn't find
> that
> equation anywhere.
> 
> Would Prof. Viechtbauer, or anyone else knowledgeable, mind explaining
> the
> sample variance above? I need to be able to defend my choice of equation.
> Since it's the only method that I found that doesn't rely on a
> correlation
> coefficient (which are not included in the papers), I'd like to be able
> to
> justify it and not redo calculations for 23 studies if possible.
> 
> Thank you very much,
> 
> John
> 
> ~~~~
> John Williams
> ALB Candidate, Harvard University (Expected May 2014)
> johnwilliams at fas.harvard.edu
> jawilliamsjr at gmail.com