**Meta-analysis** is the combination of data from several studies to
produce a single estimate. From the statistical point of view, meta-analysis
is a straightforward application of multifactorial methods. We have several
studies of the same thing, which might be clinical trials or epidemiological
studies, perhaps carried out in different countries. Each trial gives us
an estimate of an effect. We assume that these are estimates of the same
global population value. We check the assumptions of the analysis, and,
if these assumptions are satisfied, we combine the separate study estimates
to make a common estimate. This is a multifactorial analysis, where the
treatment or risk factor is one predictor variable and the study is another,
categorical, predictor variable.

The main problems of meta-analysis arise before we begin the analysis
of the data. First, we must have a clear definition of the question so
that we only include studies which address this. For example, if we want
to know whether lowering serum cholesterol reduces mortality from coronary
artery disease, we would not want to include a study where the attempt
to lower cholesterol failed. On the other hand, if we ask whether dietary
advice lowers mortality, we would include such a study. Which studies we
include may have a profound influence on the conclusions (Thompson 1993).
Second, we must have all the relevant studies. A simple literature search
is not enough. Not all studies which have been started are published; studies
which produce significant differences are more likely to be published than
those which do not (e.g. Pocock and Hughes 1990; Easterbrook *et al.
*1991). Within a study, results which are significant may be emphasized
and parts of the data which produce no differences may be ignored by the
investigators as uninteresting. Publication of unfavourable results may
be discouraged by the sponsors of research. Researchers who are not native
English speakers may feel that publication in the English language literature
is more prestigious as it will reach a wider audience, and so try there
first, only publishing in their own language if they cannot publish in
English. The English language literature may thus contain more positive
results than do other literatures. The phenomenon by which significant
and positive results are more likely to be reported, and reported more
prominently, than non-significant and negative ones is called **publication
bias**. Thus we must not only trawl the published literature for studies,
but use personal knowledge of ourselves and others to locate all the unpublished
studies. Only then should we carry out the meta-analysis.

When we have all the studies which meet the definition, we combine them to get a common estimate of the effect of the treatment or risk factor. We regard the studies as providing several observations of the same population value. There are two stages in meta-analysis. First we check that the studies do provide estimates of the same thing. Second, we calculate the common estimate and its confidence interval. To do this we may have the original data from all the studies, which we can combine into one large data file with study as one of the variables, or we may only have summary statistics obtained from publications.

If the outcome measure is continuous, such as mean fall in blood pressure,
we can check that subjects are from the same population by analysis of
variance, with treatment or risk factor, study, and interaction between
them in the model. Multiple regression can also be used, remembering that
study is a categorical variable and dummy variables are required. We test
the treatment times study interaction in the usual way. If the interaction
is significant this indicates that the treatment effect is not the same
in all studies, and so we cannot combine the studies. It is the interaction
which is important. It does not matter much if the mean blood pressure
varies from study to study. What matters is whether the effect of the treatment
on blood pressure varies more than we would expect. We may want to examine
the studies to see whether any characteristic of the studies explains this
variation. This might be a feature of the subjects, the treatment or the
data collection. If there is no interaction, then the data are consistent
with the treatment or risk factor effect being constant. This is called
a **fixed effects model** (see Section 10.12). We can drop the interaction
term from the model and the treatment or risk factor effect is then the
estimate we want. Its standard error and confidence interval are found
as described in Section 17.2. If there is an interaction, we cannot estimate
a single treatment effect. We can think of the studies as a random sample
of the possible trials and estimate the mean treatment effect for this
population. This is called the **random effects model** (Section 10.12).
The confidence interval is usually much wider than that found using the
fixed effect model.

If the outcome measure is dichotomous, such as survived or died, the estimate of the treatment or risk factor effect will be in the form of an odds ratio (Section 13.7). We can proceed in the same way as for a continuous outcome, using logistic regression (Section 17.8). Several other methods exist for checking the homogeneity of the odds ratios across studies, such as Woolf's test (see Armitage and Berry 1994) or that of Breslow and Day (1980). They all give similar answers, and, since they are based on different large-sample approximations, the larger the study samples the more similar the results will be. Provided the odds ratios are homogeneous across studies, we can then estimate the common odds ratio. This can be done using the Mantel-Haenszel method (see Armitage and Berry 1994) or by logistic regression.

For example, Glasziou and Mackerras (1993) carried out a meta-analysis of vitamin A supplementation in infectious disease. Their data for five community studies are shown in the Table:

Study

Dose regime | Vitamin A | Controls | |||
---|---|---|---|---|---|

deaths | number | deaths | number | ||

1. | 200,000 IU six-monthly | 101 | 12,991 | 130 | 12,209 |

2. | 200,000 IU six-monthly | 39 | 7,076 | 41 | 7,006 |

3. | 8,333 IU weekly | 37 | 7,764 | 80 | 7,755 |

4. | 200,000 IU four-monthly | 152 | 12,541 | 210 | 12,264 |

5. | 200,000 IU once | 138 | 3,786 | 167 | 3,411 |

We can obtain odds ratios and confidence intervals as described in Section 13.7:

Study | Odds ratio | 95% confidence interval |
---|---|---|

1 | 0.73 | 0.56 to 0.95 |

2 | 0.94 | 0.61 to 1.46 |

3 | 0.46 | 0.31 to 0.68 |

4 | 0.70 | 0.57 to 0.87 |

5 | 0.73 | 0.58 to 0.93 |

The common odds ratio can be found in several ways. To use logistic regression,
we regress the event of death on vitamin A treatment and study. I shall
treat the treatment as a dichotomous variable, set to 1 if treated with
vitamin A, 0 if control. Study is a categorical variable, so we create
dummy variables *study1 *to *study4*, which are set to one for
studies 1 to 4 respectively, and to zero otherwise. We test the interaction
by creating another set of variables, the products of *study1 *to
*study4 *and vitamin A. Logistic regression of death on vitamin A,
study and interaction gives a chi-squared statistic for the model of 496.99
with 9 degrees of freedom, which is highly significant. Logistic regression
without the interaction terms gives 490.33 with 5 degrees of freedom The
difference is 496.99 - 490.33 = 6.66 with 9-5 = 4 degrees of freedom, which
has P = 0.15, so we can drop the interaction from the model. The adjusted
odds ratio for vitamin A is 0.70, 95% confidence interval 0.62 to 0.79,
P<0.0001.

The odds ratios and their confidence intervals are shown in the following figure. The confidence interval is indicated by a line, the point estimate of the odds ratio by a circle.

In the above picture the most important trial appears to be Study 2, with the widest confidence interval. In fact, it is the study with the least effect on the whole estimate, because it is the study where the odds ratio is least well estimated. In the second picture, the odds ratio is indicated by the middle of a square.

The area of the square is proportional to the number of subjects in the study. This now makes Study 2 appear relatively unimportant, and makes the overall estimate stand out.

There many variants on this style of graph, which is sometimes called
a **forest **diagram. The graph is often shown with the studies on the
vertical axis and the odds ratio or difference in mean on the horizontal
axis:

The combined estimate of the effect may be shown as a lozenge or diamond shape and for odds ratios a logarithmic scale is often employed.

Armitage, P. and Berry, G. (1994) *Statistical Methods in Medical Research,
3rd Ed. *Blackwell, Oxford.

Breslow, N.E. and Day, N.E. (1987) *Statistical methods in cancer
research. Volume II - the design and analysis of cohort studies *IARC,
Lyon.

Easterbrook. P.J., Berlin, J.A., Gopalan, R., and Mathews, D.R. (1991)
Publication bias in clinical research. *Lancet ***337 **867-72.

Glasziou, P.P. and Mackerras, D.E.M. (1993) Vitamin A supplementation
in infectious disease: a meta-analysis. *British Medical Journal ***306
**366-70.

Pocock, S.J. and Hughes, M.D. (1990) Estimation issues in clinical trials
and overviews. *Statistics in Medicine ***9 **657-71.

Thompson, S.G. (1993) Controversies in meta-analysis: the case of the
trials of serum cholesterol reduction. *Statistical methods in medical
research ***2 **173-92.

Back to *An Introduction to Medical Statistics
*contents.

Back to Martin Bland's Home Page.

This page maintained by Martin Bland.

Last updated: 27 October, 2003.