Statistics Guide for Research Grant Applicants

Authors (in alphabetical order) JM Bland, BK Butland, JL Peacock, J Poloniecki, F Reid, P Sedgwick.

Edited by BK Butland, prepared for the World Wide Web by JM Bland.

We welcome comments. Please include the words "comments on guide" in the subject line.
Email comments on content to Barbara Butland.
Email comments on Web implementation (broken links, etc.) to Martin Bland.

Division of Community Health Sciences
St George's University of London
Cranmer Terrace
London SW17 0RE

This handbook was funded by the NHS South East Regional Office (SERO) Research and Knowledge Management Directorate and is targeted at those applying for research funding, from any source.

Brief Table of Contents

Detailed Table of Contents

Preface

Background

The aim of this handbook is to help applicants to appreciate some of the statistical pitfalls that await them when constructing a grant proposal. It was written by the following six statisticians based at St George's Hospital Medical School:

Martin Bland PhD - Professor of Medical Statistics
Barbara Butland MSc - Lecturer in Medical Statistics
Janet Peacock PhD - Senior Lecturer in Medical Statistics
Jan Poloniecki DPhil - Senior Lecturer in Medical Statistics
Fiona Reid MSc - Lecturer in Medical Statistics
Philip Sedgwick PhD - Lecturer in Medical Statistics

(Martin Bland is now Prof. of Health Statistics, University of York, and Janet Peacock is Prof. of Medical Statistics, University of Southampton.)

All six authors routinely reviewed grant proposals for The South East Research and Development Project Grant Scheme. This was a responsive funding scheme for the South East Regional Office which ran for a number of years until it ended in October 2001. The scheme was responsible for spending of approximately £1.2 million each year on new and ongoing projects, all based on researchers' own ideas. Its primary objective was the production of new, high quality knowledge. It was open to anyone whose work was relevant to the UK National Health Service (NHS). Up to 150,000 pounds was available for each project, although the majority were smaller. The criteria stipulated that applications were to be relevant to the NHS, to follow a clear, well defined protocol, would withstand external peer review and that the findings would be generalisable to others in the NHS. Although academic advice was available to applicants through Research and Development Support Units (RDSU) in the Region and detailed guidance notes accompanied the application form, a number of common statistical problems consistently emerged. It is hoped that in summarising these statistical weak points the handbook will be of use to those applying for research funding from any source and in the long term reduce the number of proposals that are rejected by grant giving bodies purely on statistical grounds. The handbook was commissioned and initially funded by the South East Regional Office (SERO) Research and Knowledge Management Office Directorate who provided comments and encouragement throughout the writing process. In this respect the authors would particularly like to extend their thanks to John Newton and Lesley Elliott.

Back to top

Content

The process of constructing the handbook began with a pilot exercise. All six authors retrieved their reviews for the last year to two years noting down recurring comments. Based on the results of the pilot work a list of topics was drawn up to form the main skeleton of the handbook. A few additional topics have since been added at the suggestion of Colin Cryer (South East Institute of Public Health) a fellow statistical reviewer for The Project Grant Scheme and Lyn Fletcher (Statistician, Oxford Research and Development Support Unit).

Back to top

Using the handbook

The handbook is not designed to teach statistics but to provide extra information to those who already have a basic statistical knowledge. For example it assumes some understanding of confidence intervals and significance testing but not statistical power or sample size calculation. To help the beginner there are references to standard Medical Statistics Textbooks. However, the handbook should not be viewed as an alternative to discussing your proposal with a statistician prior to submission. Such a discussion is strongly recommended. Rather, it is hoped that the handbook will make grant applicants more aware of the right questions to ask and the right information to take along to a statistical consultation and in addition, help them to understand any advice given. [Please note that although the authors would welcome any comments on the content of the guide, particularly correction of errors, they cannot give advice on projects]

It is not envisaged that the handbook should be read from cover to cover but rather that the contents list or the checklist (Appendix 2) should be used to help the applicant navigate through the book ignoring sections that have no bearing on their own research. To facilitate this type of use the text is organised into many short reasonably self-contained paragraphs, each with its own index code e.g. A-1.1. It is hoped that these codes will be useful to reviewers, consulting statisticians and researchers alike. Each paragraph may contain links to other related paragraphs as well as to useful references in the literature and on the Web. The handbook is best suited to interactive use on the Web although it may also be used effectively in printed form.

Back to top

Statistical review

Another aim of the handbook is to try and clarify the sort of checklist that a statistician might use in the process of reviewing a grant proposal. Most statisticians will not rigidly follow any such list but the sort of things that they will be trying to extract from the text of any grant proposal are as follows (this list is not exhaustive):

a) the basic study design(s) to see whether the applicant needs to have included information on randomisation, confounding, hierarchical data etc and whether the design is appropriate to the study aims.

b) the type of data the study will generate as without this information the statistician cannot assess whether the applicant's sample size calculation and proposed methods of statistical analysis are appropriate.

c) the number of subjects that will be asked to take part in the study and the number it is anticipated will be recruited as the figure from the sample size calculation should match the latter and not the former.

d) the total number of outcome variables that the applicant plans to measure as this will highlight potential problems in terms of multiple testing.

d) whether the sample size calculation and the proposed statistical analysis are based on the same statistical tests. The applicant should therefore mention when reporting the sample size calculation(s) the test(s) on which it is based.

e) whether the proposed statistical analysis includes the calculation of confidence intervals as well as significance testing.

f) whether the applicants have the required statistical expertise for their proposed statistical analysis.

g) sufficient information to replicate and so check the applicant's sample size calculation.

Back to top

Development

The handbook should be considered as a work in progress and the authors would welcome any comments on the content of the guide, particularly correction of errors.

Back to top


To St. George's Community Health Sciences Home Page

To St. George's University of London Home Page

Back to Martin Bland's home page.

This page is maintained by Martin Bland.

Last updated: 10 September, 2009.

Copyright St. George's University of London

Back to top