For many statistical problems there are several possible solutions, just as for many diseases there are several treatments, similar perhaps in their overall efficacy but displaying variation in their side effects, in their interactions with other diseases or treatments and in their suitability for different types of patients. There is often no one right treatment, but rather treatment is decided on the prescriber's judgement of these effects, past experience and plain prejudice. Many problems in statistical analysis are like this. In comparing the means of two small groups, for instance, we could use a t test, a t test with a transformation, a Mann-Whitney U test, or one of several others. Our choice of method depends on the plausibility of Normal assumptions, the importance of obtaining a confidence interval, the ease of calculation, and so on. It depends on plain prejudice, too. Some users of statistical methods are very concerned about the implications of Normal assumptions and will advocate non-parametric methods wherever possible, while others are too careless of the errors that may be introduced when assumptions are not met.
I sometimes meet people who tell me that they have used non-parametric methods throughout their analysis as if this is some kind of badge of statistical purity. It is nothing of the kind. It may mean that their significance tests have less power than they might have, and that results are left as `not significant' when, for example, a confidence interval for a difference might be more informative.
On the other hand, such methods are very useful when the necessary assumptions of the t distribution method cannot be made, and it would be equally wrong to eschew their use. Rather, we should choose the method most suited to the problem, bearing in mind both the assumptions we are making and what we really want to know. We shall say more about what method to use when in Chapter 14.
There is a common misconception that when the number of observations is very small, usually said to be less than six, Normal distribution methods such as t tests and regression must not be used and that rank methods should be used instead. I have never seen any argument put forward in support of this, but inspection of the tables of the test statistics for rank methods will show that it is nonsense. For such small samples rank tests cannot produce any significance at the usual 5% level. Should one need statistical analysis of such small samples, Normal methods are required.
Back to An Introduction to Medical Statistics contents
Back to Martin Bland's Home Page
This page maintained by Martin Bland
Last updated: 10 November, 2003