Question 6: The intraobserver repeatability coefficient was 3.8 and the interobserver repeatability coefficient was 2.1. Why should this be very surprising?
We might expect that measurements by different observers would vary by more than would measurements by the same observer. They should not vary by less. Hence if the inter-observer repeatability is different from the intra-observer repeatability, it should be larger, not smaller.
The explanation for this can only be found by reading the full paper and even there it is not clear. Although it is not mentioned in the abstract, the inter-observer statistics are for the mean of three observations by the same observer, whereas the intra-observer statistics are for a single observation by the observer.
For single observations, the inter-observer can be calculated as follows. The intra-observer standard deviation is 3.8/1.96 root 2 = 1.37. The variance is the square of this, 1.372 = 1.88. The inter-observer standard deviation 2.1/1.96 root 2 = 0.76. The variance is the square of this, 0.762 = 0.58. This is made up of two components: within observer and between observers (including heterogeneity). The within observers component is for the mean of three observations, so its variance is the variance of one observation (1.88) divided by 3: 1.88/3 = 0.63. The between observers component is the difference between the total and the within observer component: 0.58 – 0.63 = -0.05. A variance cannot be negative, so our best estimate is to set this to zero. Hence our best estimate is that using different observers has no effect on the variance. The variance for single observations by different observers on the same subject would be same as the variance for the same observer and the repeatability coefficients should be the same.
Back to question.
This page maintained by Martin Bland.
Last updated: 23 February, 2007.