Authors (in alphabetical order) JM Bland, BK Butland, JL Peacock, J Poloniecki, F Reid, P Sedgwick.

Edited by BK Butland, prepared for the World Wide Web by JM Bland.

We welcome comments. **Please include the words "comments on guide" in the
subject line.**

Email comments on content to Barbara Butland.

Email comments on Web implementation (broken links, etc.) to Martin Bland.

Department of Public Health Sciences

St George's Hospital Medical School

Cranmer Terrace

London SW17 0RE

This handbook was funded by the South East Regional Office (SERO) Research and Knowledge Management Directorate and is targeted at those applying for research funding, from any source.

A-1 Type of study

A-1.2 Combinations and sequences of studies

A-1.3 Cohort studies

A-1.4 Case-control studies

A-1.5 Cross-sectional studies

A-1.5b The estimation of sensitivity and specificity

A-1.5c When to calculate sensitivity and specificity

A-1.5d Cross-sectional ecological studies

A-1.5e Studies of measurement validity, reliability and agreement

A-1.8 Randomised controlled clinical trials

A-1.9 Pilot and Exploratory Studies

A-3 Study subjects

A-4 Types of variables

A-4.2 Types of data

A-4.3 Methods of data collection

A-4.4 Validity and reliability

A-4.4b Repeatability (test-re-test reliability)

A-4.4c Inter-rater reliability (inter-rater agreement)

B-2 Describing the intervention (s) / treatment (s)

B-3 Choice of control treatment / need for control group

B-4 Blindness

B-5.2 When might we use randomisation?

B-5.3 Why randomise?

B-5.4 What is not randomisation?

B-5.5 How do we randomise?

B-5.6 Randomisation in blocks

B-5.7 Randomisation in strata

B-5.8 Minimisation

B-5.9 Clusters

B-5.10 Trial designs

B-5.10b Crossover design

B-5.10c Within group comparisons

B-5.10d Sequential design

B-5.10e Factorial design

B-7 Data monitoring

B-7.2 When should a trial be stopped early?

B-8.2 Emergencies

B-8.3 Children

B-8.4 Mentally incompetent subjects

B-8.5 Cluster randomised designs

B-8.6 Random consent designs

B-10 Achieving the sample size

B-11 After the trial is over

C-1.2 Matching in case-control studies

C-3 Recall bias

C-4 Sample survey: selecting a representative sample

C-5 Generalisability and extrapolation of results

C-6 Maximising response rates to questionnaire surveys

D-2 Why is it important to consider sample size?

D-3 Information required to calculate a sample size

D-4 Explanation of statistical terms

D-4.2 Probability value (p-value)

D-4.3 Significance level

D-4.4 Power

D-4.5 Effect size of clinical importance

D-4.6 One-sided and two-sided tests of significance

D-6 Allowing for response rates and other losses to the sample

D-7 Consistency with study aims and statistical analysis

D-8 Three specific examples of sample size calculations & statements

D-8.2 Comparing two proportions

D-8.3 Comparing two means

E-1.2 Level of detail required

E-4 Assumptions

E-6 Hierarchical or multilevel data

E-7.2 Multiple testing: why is it a problem?

E-7.3 The Bonferroni correction

E-7.4 How to deal with multiple testing

E-7.4b More than one predictor measurement in an observational study

E-7.4c Measurements repeated over time (serial measurements)

E-7.4d Comparisons of more than two groups

E-7.4e Testing the study hypothesis within subgroups

E-7.4f Repeatedly testing the difference in a study as more patients are recruited

E-9 Intention to treat in clinical trials

E-10 Cluster randomised trials

E-11 Collapsing variables

E-12 Estimation and confidence intervals

F-2 Statistical software

F-3 Ethics

F-3.2 Critical appraisal

F-3.3 Studies involving human subjects

F-4.2 Data protection

Martin Bland PhD - Professor of Medical Statistics

Barbara Butland MSc - Lecturer in Medical Statistics

Janet Peacock PhD - Senior Lecturer in Medical Statistics

Jan Poloniecki DPhil - Senior Lecturer in Medical Statistics

Fiona Reid MSc - Lecturer in Medical Statistics

Philip Sedgwick PhD - Lecturer in Medical Statistics

All six authors routinely reviewed grant proposals for The South East Research and Development Project Grant Scheme. This was a responsive funding scheme for the South East Regional Office which ran for a number of years until it ended in October 2001. The scheme was responsible for spending of approximately £1.2 million each year on new and ongoing projects, all based on researchers' own ideas. Its primary objective was the production of new, high quality knowledge. It was open to anyone whose work was relevant to the UK National Health Service (NHS). Up to 150,000 pounds was available for each project, although the majority were smaller. The criteria stipulated that applications were to be relevant to the NHS, to follow a clear, well defined protocol, would withstand external peer review and that the findings would be generalisable to others in the NHS. Although academic advice was available to applicants through Research and Development Support Units (RDSU) in the Region and detailed guidance notes accompanied the application form, a number of common statistical problems consistently emerged. It is hoped that in summarising these statistical weak points the handbook will be of use to those applying for research funding from any source and in the long term reduce the number of proposals that are rejected by grant giving bodies purely on statistical grounds. The handbook was commissioned and initially funded by the South East Regional Office (SERO) Research and Knowledge Management Office Directorate who provided comments and encouragement throughout the writing process. In this respect the authors would particularly like to extend their thanks to John Newton and Lesley Elliott.

It is not envisaged that the handbook should be read from cover to cover but rather that the contents list or the checklist (Appendix 2) should be used to help the applicant navigate through the book ignoring sections that have no bearing on their own research. To facilitate this type of use the text is organised into many short reasonably self-contained paragraphs, each with its own index code e.g. A-1.1. It is hoped that these codes will be useful to reviewers, consulting statisticians and researchers alike. Each paragraph may contain links to other related paragraphs as well as to useful references in the literature and on the Web. The handbook is best suited to interactive use on the Web although it may also be used effectively in printed form.

a) the basic study design(s) to see whether the applicant needs to have included information on randomisation, confounding, hierarchical data etc and whether the design is appropriate to the study aims.

b) the type of data the study will generate as without this information the statistician cannot assess whether the applicant's sample size calculation and proposed methods of statistical analysis are appropriate.

c) the number of subjects that will be asked to take part in the study and the number it is anticipated will be recruited as the figure from the sample size calculation should match the latter and not the former.

d) the total number of outcome variables that the applicant plans to measure as this will highlight potential problems in terms of multiple testing.

d) whether the sample size calculation and the proposed statistical analysis are based on the same statistical tests. The applicant should therefore mention when reporting the sample size calculation(s) the test(s) on which it is based.

e) whether the proposed statistical analysis includes the calculation of confidence intervals as well as significance testing.

f) whether the applicants have the required statistical expertise for their proposed statistical analysis.

g) sufficient information to replicate and so check the applicant's sample size calculation.

Public Health Sciences Home Page

St. George's Hospital Medical School Home Page

This page is maintained by
Martin Bland

Last updated: [September 11, 2002]