These are two sections from Martin Bland's text book An Introduction to Medical Statistics, Third Edition. I hope that the topics will be useful in their own right, as well as giving a flavour of the book.
People who volunteer for new treatments and those who refuse them may be very different. An illustration is provided by the field trial of Salk poliomyelitis vaccine carried out in 1954 in the USA (Meier 1977). This was carried out using two different designs simultaneously, due to a dispute about the correct method. In some districts, second grade school-children were invited to participate in the trial, and randomly allocated to receive vaccine or an inert saline injection. In other districts, all second grade children were offered vaccination and the first and third grade left unvaccinated as controls. The argument against this `observed control' approach was that the groups may not be comparable, whereas the argument against the randomized control method was that the saline injection could provoke paralysis in infected children. The results are shown in the Table:
| Study group | Number in group | Paralytic Polio | ||
|---|---|---|---|---|
| Number of cases | Rate per 100,000 | |||
| Randomized control: | ||||
| Vaccinated: | 200745 | 33 | 16 | |
| Control | 201229 | 115 | 57 | |
| Not inoculated | 338778 | 121 | 36 | |
| Observed control: | ||||
| Vaccinated 2nd grade | 221998 | 38 | 17 | |
| Control 1st and 3rd grade | 725173 | 330 | 46 | |
| Unvaccinated 2nd grade | 123605 | 43 | 35 | |
In the randomized control areas the vaccinated group clearly experienced far less polio than the control group. Since these were randomly allocated, the only difference between them should be the treatment, which is clearly preferable to saline. However, the control group also had more polio than those who had refused to participate in the trial. The difference between the control and not inoculated group is in both treatment (saline injection) and selection; they are self-selected as volunteers and refusers. The observed control areas enable us to distinguish between these two factors. The polio rates in the vaccinated children are very similar in both parts of the study, as are the rates in the not inoculated second grade children. It is the two control groups which differ. These were selected in different ways: in the randomized control areas they were volunteers, whereas in the observed controls areas they were everybody eligible, both potential volunteers and potential refusers. Now suppose that the vaccine were saline instead, and that the randomized vaccinated children had the same polio experience as those receiving saline. We would expect 200,745 times 57 / 100,000 = 114 cases, instead of the 33 observed. The total number of cases in the randomized areas would be 114 + 115 + 121 = 350 and the rate per 100,000 would 47. This compares very closely with the rate of 46 in the observed control first and third grade group. Thus it seems that the principal difference between the saline control group of volunteers and the not inoculated group of refusers is selection, not treatment.
There is a simple explanation of this. Polio is a viral disease transmitted by the faecal-oral route. Before the development of vaccine almost everyone in the population was exposed to it at some time, usually in childhood. In the majority of cases, paralysis does not result and immunity is conferred without the child being aware of having been exposed to polio. In a small minority of cases, about one in 200, paralysis or death occurs and a diagnosis of polio is made. The older the exposed individual is, the greater the chance of paralysis developing. Hence, children who are protected from infection by high standards of hygiene are likely to be older when they are first exposed to polio than those children from homes with low standards of hygiene, and thus more likely to develop the clinical disease. There are many factors which may influence parents in their decision as to whether to volunteer or refuse their child for a vaccine trial. These may include education, personal experience, current illness, and others, but certainly include interest in health and hygiene. Thus in this trial the high risk children tended to be volunteered and the low risk children tended to be refused. The higher risk volunteer control children experienced 57 cases of polio per 100,000, compared to 36/100,000 among the lower risk refusers.
In most diseases, the effect of volunteer bias is opposite to this. Poor conditions are related both to refusal to participate and to high risk, whereas volunteers tend to be low risk. The effect of volunteer bias is then to produce an apparent difference in favour of the treatment. We can see that comparisons between volunteers and other groups can never be reliable indicators of treatment effects.
In the observed control areas of the Salk trial (Table~\ref{2tsalk}), quite apart from the non-random age difference, the vaccinated and control groups are not comparable. However, it is possible to make a reasonable comparison in this study by comparing all second grade children, both vaccinated and refused, to the control group. The rate in the second grade children is 23 per 100,000, which is less than the rate of 46 in the control group, demonstrating the effectiveness of the vaccine. The `treatment' which we are evaluating is not vaccination itself, but a policy of offering vaccination and treating those who accept. A similar problem can arise in a randomized trial, for example in evaluating the effectiveness of health checkups (South-east London Screening Study Group 1977). Subjects were randomized to a screening group or to a control group. The screening group were invited to attend for an examination, some accepted and were screened and some refused. When comparing the results in terms of subsequent mortality, it was essential to compare the controls to the screening groups containing both screened and refusers. For example, the refusers may have included people who were already too ill to come for screening. The important point is that the random allocation procedure produces comparable groups and it is these we must compare, whatever selection may be made within them. We therefore analyse the data according to the way we intended to treat subjects, not the way in which they were actually treated. This is analysis by intention to treat. The alternative, analysing by treatment actually received, is called on treatment analysis.
Analysis by intention to treat is not free of bias. As some patients may receive the other group's treatment, the difference may be smaller than it should be. We know that there is a bias and we know that it will make the treatment difference smaller, by an unknown amount. On treatment analyses, on the other hand, are biased in favour of showing a difference, whether there is one or not. Statisticians call methods which are biased against finding any effect conservative. If we must err, we like to do so in the conservative direction.
Meier, P. (1977) The biggest health experiment ever: the 1954 field trial of the Salk poliomyelitis vaccine. in Statistics: a Guide to the Biological and Health Sciences ed. J.M. Tanur et al. Holden-Day, San Francisco.
South-east London Screening Study Group. (1977) A controlled trial of multiphasic screening in middle-age: results of the South-east London Screening Study. International Journal of Epidemiology 6 357-63.
Back to An Introduction to Medical Statistics contents
Back to Martin Bland's Home Page
This page maintained by Martin Bland.
Last updated: 27 October, 2003.