R help - Apr 2015 - 'metafor' - standardized mean difference in pre-post design studies

Hi, I have a quesite on meta-analysis with 'metafor'.
I would like to calculate the standardized mean difference (SMD), as
Hedges' g, in pre-post design studies.
I have data on baseline (sample size, mean and SD in both the experimental
and the control group) and at end of treatment (same as before).
The 'metafor' site report a calculation based on Morris (2008).
However, I would like to calculate the SMD as in Comprehensive
Meta-analysis according to Borenstein:

d = (mean.pre - mean.post) / SD_within

SD_within = SD.diff / square root (2(1-r)

r = correlation between pairs of observation (often it is not reported, and
suggestion is to use r = 0.70)

The variance of d (Vd) is calculated as (1/n + d^2/2n)2(1-r), where n number of
pairs

To derive Hedges' g from d, the correction 'J' is used:

J = 1 - (3/4df - 1), where df = degrees of freedom, which in two
independent groups is n1+n2-2

Essentially, J = 1 - (3/4*((n1+n2)-2) - 1)

Ultimately, g = J x d, and variance of g (Vg) = J^2 x Vd


I had some hint by Wolfgang Viechtbauer, but I'm stucked on here
(essentially, because my poor programming abilities)
I was stuck on applying the Viechtbauer's hint to my dataset.
Probably I'm doing something wrong. However, what I get it is not what I
found with Comprehensive Meta-Analysis.
In CMA I've found g = -0.49 (95%CI: -0.64 to -0.33).

Moreover, I do not know how to apply the J correction for calculating the
Hedges'g.
My request is: can anyone check the codes?
Can anyone help me in adding the J correction?
What should I multiply for J?
Should I use the final yi and vi as measures of d and Variance of d?


Thank you in advance,
Antonello Preti


This is my dataset (with imputed r = 0.70, put in the 'ri' variable):

##### the data

dat <- structure(list(study = structure(c(11L, 8L, 7L, 12L, 13L, 4L,
5L, 1L, 10L, 3L, 6L, 9L, 2L), .Label = c("Study A, 2012",
"Study B, 2013", "Study C, 2013", "Study D, 2010",
"Study E, 2012", "Study F, 2013", "Study G, 2006",
"Study H, 2005", "Study I, 2013", "Study L, 2012",
"Study M, 2003", "Study N, 2007", "Study P, 2007"
), class = "factor"), c_pre_mean = c(4.9, 15.18, 19.01, 5.1,
16.5, 27.35, 18.1, 2.4, 14.23, 0.08, 21.26, 21.5, 21.73), c_pre_sd = c(2.6,
2.21, 7.1, 1.5, 7.2, 13.92, 5.4, 0.13, 4.89, 0.94, 7.65, 5.22,
8.43), c_post_mean = c(6.1, 13.98, 18.5, 4.53, 15.9, 23, 16.9,
2.2, 16.58, -0.02, 16, 16.84, 23.54), c_post_sd = c(2.06, 3.24,
7, 2.06, 6.8, 12.06, 3.8, 0.13, 6.35, 0.88, 4.69, 4.64, 6.74),
    c_sample = c(14, 13, 19, 15, 34, 20, 24, 35, 31, 26, 49,
    21, 22), e_pre_mean = c(4.6, 13.81, 19.9, 5.3, 18.7, 22.71,
    19.2, 2.7, 15.97, -0.22, 20.9, 20.43, 21.94), e_pre_sd = c(2.1,
    6.64, 8.1, 2.9, 7.3, 7.82, 4.1, 0.13, 6.73, 0.93, 5.18, 4.87,
    7.02), e_post_mean = c(4.64, 15.86, 18.1, 4.33, 17.2, 24.89,
    17.6, 2.8, 13.6, 0.06, 17.41, 16.05, 19.29), e_post_sd = c(2.34,
    7.76, 7.8, 2.26, 7.4, 11.89, 3.7, 0.13, 5.79, 1.12, 5.16,
    4.17, 6.58), e_sample = c(14, 18, 16, 16, 33, 28, 29, 36,
    38, 27, 43, 25, 17), ri = c(.70,
.70,.70,.70,.70,.70,.70,.70,.70,.70,.70,.70,.70)), .Names = c("study",
"c_pre_mean", "c_pre_sd",
"c_post_mean", "c_post_sd", "c_sample",
"e_pre_mean", "e_pre_sd",
"e_post_mean", "e_post_sd", "e_sample",
"ri"), class = "data.frame",
row.names = c(NA,
-13L))


### check the data

dim(dat)
head(dat)
str(dat)

### to make easy the operations (I know it should'nt be done)

attach(dat)


### call the library

library('metafor')

### The hint by Viechtbauer is that CMA computes the d value for a pre-post
design in a slightly different way than metafor. CMA computes:

### d = (m_1 - m_2) / (SD_diff / sqrt(2*(1-r)))

### To do this, I should calculate something like this (first the control
group):

sd1i= c_pre_sd
sd2i = c_post_sd
ni = c_sample

dat$sdi_c <- with(dat, sqrt((sd1i^2 + sd2i^2 -
2*ri*sd1i*sd2i))/sqrt(2*(1-ri)))

### Then apply the usual calculation with escalc:

datC <- escalc(measure="SMCR", m1i=c_pre_mean, m2i=c_post_mean,
sd1i=sdi_c,
ni=ni, ri=ri, data=dat)
summary(datC)


#### The same on the experimental group

sd1i= e_pre_sd
sd2i = e_post_sd
ni = e_sample


dat$sdi_e <- with(dat, sqrt((sd1i^2 + sd2i^2 -
2*ri*sd1i*sd2i))/sqrt(2*(1-ri)))

datE <- escalc(measure="SMCR", m1i=e_pre_mean, m2i=e_post_mean,
sd1i=sdi_e,
ni=ni, ri=ri, data=dat)
summary(datE)


#### Computing the Difference in the Standardized Mean Change

datFin <- data.frame(yi = datE$yi - datC$yi, vi = datE$vi + datC$vi)
round(datFin, 2)



###############################################
#
# fixed-effects model
#
###############################################

model.FE <- rma(yi, vi, data=datFin, method="FE", digits=2)

summary(model.FE)

# plot globale

plot(model.FE, slab=paste(study))


###
###

	[[alternative HTML version deleted]]

> Hi, I have a quesite on meta-analysis with 'metafor'.
> I would like to calculate the standardized mean difference (SMD), as
> Hedges' g, in pre-post design studies.
> I have data on baseline (sample size, mean and SD in both the
> experimental
> and the control group) and at end of treatment (same as before).
> The 'metafor' site report a calculation based on Morris (2008).
> However, I would like to calculate the SMD as in Comprehensive
> Meta-analysis according to Borenstein:
> 
> d = (mean.pre - mean.post) / SD_within
> 
> SD_within = SD.diff / square root (2(1-r)
Note that this assumes that the SDs are the same at baseline and at the end of
the treatment. Also, it is not how d values for pre-post designs are typically
computed. There are several articles that describe various approaches, in
particular:

Becker, B. J. (1988). Synthesizing standardized mean-change measures. British
Journal of Mathematical and Statistical Psychology, 41(2), 257-278.

Gibbons, R. D., Hedeker, D. R., & Davis, J. M. (1993). Estimation of effect
size from a series of experiments involving paired comparisons. Journal of
Educational Statistics, 18(3), 271-279.

Morris, S. B. (2000). Distribution of the standardized mean change effect size
for meta-analysis on repeated measures. British Journal of Mathematical and
Statistical Psychology, 53(1), 17-29.

Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in
meta-analysis with repeated measures and independent-groups designs.
Psychological Methods, 7(1), 105-125.

Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control
group designs. Organizational Research Methods, 11(2), 364-386.

The two approaches that have been most thoroughly studied and described are:

d = (mean.pre - mean.post) / SD.diff

(standardization by the change score SD) and

d = (mean.pre - mean.post) / SD.pre

(standardization by the pre-test SD; one could also use the post-test SD).

The method described in the book is a bit of a juxtaposition, where SD.diff is
'corrected' by 1/sqrt(2*(1-r)), which is identical to SD.pre (or
SD.post) when SD.pre = SD.post. But that's never exactly the case. Plus I am
not aware of any proper derivations of the large-sample distribution of d
computed in this manner.
> r = correlation between pairs of observation (often it is not reported,
> and suggestion is to use r = 0.70)
Suggested where? I hope this is not a general "if you don't know the
correlation, just use .70" suggestion, because that would be nonsense. If
you don't know r for the sample, then you could try to make a reasonable
guess that is informed by the characteristic or attribute that is being measured
(some things are much more stable than other things) and the timelag between
baseline and the follow-up measurement. Also, if some treatment happens between
baseline and follow-up -- and some people are more likely to respond to the
treatment than others -- then this is likely to reduce the correlation to some
extent, depending on how much variability there is in treatment responses. These
are at least some of the considerations that should go into making a proper
guess about r.
> The variance of d (Vd) is calculated as (1/n + d^2/2n)2(1-r), where n >
number of pairs
As mentioned above, I am not aware of any derivation of that equation. One can
show that 2(1-r)/n + d^2/2n is an estimate of the asymptotic sampling variance
of d when d is computed as (mean.pre - mean.post) / SD.pre (or with SD.post).
So, when d is computed in the manner above, it is a bit like (mean.pre -
mean.post) / SD.pre -- except for the way that SD.pre is actually estimated. So,
if anything, the equation should look more like the one above and not the one in
the book. I have actually communicated with Michael and Larry (Hedges) about
this and Michael indicated that changes may need to be made to CMA.
> To derive Hedges' g from d, the correction 'J' is used:
> 
> J = 1 - (3/4df - 1), where df = degrees of freedom, which in two
> independent groups is n1+n2-2
> 
> Essentially, J = 1 - (3/4*((n1+n2)-2) - 1)
> 
> Ultimately, g = J x d, and variance of g (Vg) = J^2 x Vd
> 
> I had some hint by Wolfgang Viechtbauer, but I'm stucked on here
> (essentially, because my poor programming abilities)
> I was stuck on applying the Viechtbauer's hint to my dataset.
> Probably I'm doing something wrong. However, what I get it is not what
I
> found with Comprehensive Meta-Analysis.
> In CMA I've found g = -0.49 (95%CI: -0.64 to -0.33).
You won't get the same thing, as CMA does what is described in the book, but
that's not what metafor does (due to the reasons described above).
> Moreover, I do not know how to apply the J correction for calculating the
> Hedges'g.
> My request is: can anyone check the codes?
> Can anyone help me in adding the J correction?
> What should I multiply for J?
> Should I use the final yi and vi as measures of d and Variance of d?
Why don't you just use what escalc() gives you?
> Thank you in advance,
> Antonello Preti
[code snipped]

Best,
Wolfgang