Random Effects Methods

If the effect size parameters are not identical (or almost so) across studies, an alternative method for combining estimates across studies is the random effects model. In this model, studies are considered as a sample of possible studies and their effect size parameters are considered as a sample from a universe of possible effect size estimates. In this model the object is to estimate the mean \x and variance t2 of the population of effect sizes (the population of 0 values) from which the observed study effect sizes are sampled.

If the effect size parameters corresponding to the studies in our sample of studies (01, •••, 0k) were observed, we could simply compute their variance as a sample estimate of t2. Because they are not observed we must estimate their variance indirectly by noting that the variance of the observed effect size estimates (71, •••, Tk) depends partly on vi, which represent estimation d a

Table 58.3 Example data for computing fixed effects meta-analyses using the Mantel-Haenszel method

Control Treatment

Table 58.3 Example data for computing fixed effects meta-analyses using the Mantel-Haenszel method

Control Treatment

Study

a

b

c

d

N

P

Q

R

S

PR

PS

QR

QS

1

14

88

2

102

206

0.563

0.437

6.932

0.854

3.903

0.481

3.029

0.373

2

29

97

12

104

242

0.550

0.450

12.463

4.810

6.849

2.643

5.613

2.166

3

56

388

18

191

653

0.378

0.622

16.380

10.695

6.196

4.046

10.184

6.650

4

4

42

1

23

70

0.386

0.614

1.314

0.600

0.507

0.231

0.807

0.369

5

9

34

11

36

90

0.500

0.500

3.600

4.156

1.800

2.078

1.800

2.078

6

57

343

35

379

814

0.536

0.464

26.539

14.748

14.215

7.900

12.324

6.849

7

12

86

3

98

199

0.553

0.447

5.910

1.296

3.267

0.717

2.643

0.580

8

12

181

11

187

391

0.509

0.491

5.739

5.092

2.921

2.592

2.818

2.500

9

5

21

0

30

56

0.625

0.375

2.679

0.000

1.674

0.000

1.004

0.000

10

4

46

0

47

97

0.526

0.474

1.938

0.000

1.019

0.000

0.919

0.000

Totals

5.125

4.875

83.494

42.252

42.351

20.687

41.142

21.565

Note: There are no columns for ad/N or bc/N because ad/N = R and bc/N = S

Note: There are no columns for ad/N or bc/N because ad/N = R and bc/N = S

errors and partly on t2, which represents true heterogeneity among 0 i. The g-statistic used to test heterogeneity is a weighted sample variance that can be used to obtain an indirect estimate of t2. In particular,

if the quantity on the right-hand side of the equation is positive, and zero otherwise, where c is a normalizing constant given by k

Random effects methods compute the weighted mean effect size as

where w* = 1/v* = 1 /(vi + t2). This corresponds to weighting each effect size by the inverse of a new variance, v* = vi + t2, which includes a component of between-study variation. As in the fixed effect case, the weighted mean T, * is also normally distributed, the variance v* of T, * is the reciprocal of the sum of the weights v,*= £ w*

and a 95% confidence interval for the average effect size x is given by

A test of the hypothesis that 0 = 0 uses the test statistic

The level a two-tailed test rejects the null hypothesis when |Z| exceeds the 100a percent critical value of the standard normal distribution (e.g., 1.96 for a = 0.05).

The fixed and random effects weighted means are similar in form and differ only in the weights used to compute them. When T2 > 0, the w* are more similar to one another than the w. This means that studies receive more equal weights in the random effects weighted mean than in the fixed effects weighted mean, where one study can dominate (receive very large weight) if it has a very small variance (usually because it has a very large sample size). By contrast, in the random effects weighted mean, where the weight given to each study is more similar, no single study can completely dominate. Similarly, when T2 > 0, each wi* is larger than the corresponding Wi. Because the variance of the weighted mean is the inverse of the sum of the weights, this means that the variance v*. of the random effects weighted mean T* * is larger than the variance v* of the fixed effects weighted mean T* . One implication of this is that confidence intervals for the random effects weighted mean are longer than those of the fixed effects weighted mean.

Note that the test of the hypothesis that t2 = 0 in the random effects analysis is exactly the test of the hypothesis that 0i = ••• = 0k based on the g-statistics described in connection with the fixed effects analysis, since if t2 = 0, the effect size parameters will be identical.

A quantitative description of the amount of heterogeneity can be provided in either one of two ways. The estimate of t2 provides one such estimate. The square root of this estimate, T ,isan estimate of the standard deviation of the distribution of the effect size parameters across studies. An alternative way to characterize heterogeneity is to describe the proportion of variation in the observed effect size estimates that is due to t2. The estimate

g does just this. Because T describes the absolute amount of variation in 0s and I2 describes the amount of variation relative to the total variation c k of estimates (including the amount of variation due to both variation of 0 s and errors of estimation), both are complementary ways to describe variation in effect size parameters.

3.3.1 Example

Returning to our example of k = 10 studies of interventions to promote smoking cessation among pregnant women, we use the quantities in Table 58.2 to compute and give an estimate of the between-studies variance component (t2), the random effects weight w*, w*T, and their sums. First compute the normalizing constant c as c = 53.796 - 631.931/53.796 = 42.05, then use this quantity along with the Q-statistic computed in the fixed effects analysis (Q = 13.534) to compute the estimate of t2 as

This value is used to compute the w* values and the w*T values in Table 58.2, for example, the random effects weight in study 1 is w* = 1 /(0.593 + 0.108) = 1.427. Using these random effects weights, the random effects weighted mean of the log-odds ratios is

T. = 18.233/26.554 = 0.687 with a variance of v. = 1/26.554 = 0.038, which leads to the 95% confidence interval for the log-odds ratio ln(m) of

0.306 =0.687 - 1.96V0.038 < ln(m) < 0.687 + 1.96V0.038 = 1.067.

Converting these into the metric of (unlogged) odds ration m yields the estimate o = exp(0.687)

= 1.99 and the 95% confidence interval

Note that the point estimate of the odds ratio is slightly larger than that computed using the fixed effects model, and the variance of the log-odds ratio computed using the random effects model is twice as large as the value of 0.019 computed using the fixed effects model. Similarly the confidence interval for m computed using the random effects model is wider (1.36-2.91) than that computed using the fixed effects model (1.42-2.43).

Using the value of the Q statistic of Q = 13.534 computed in the example for the fixed effects analysis, the value of I2, representing the proportion of variance in the estimates that is due to variation in effect size parameters across studies is

0 0

Post a comment