Comparing within-subject variances in a study to compare two methods of measurement

This note is available in HTML format (below) or in an Adobe Acrobat PDF version. If you do not have Adobe Acrobat, you can download it free from this link: get Adobe Acrobat PDF Reader.

Download PDF version of "Comparing within-subject variances in a study to compare two methods of measurement".

The HTML version follows.

In the design for comparing two methods of measurement proposed by Bland and Altman (1986), two observations are made by each method on each subject. This design was use to compare a Wright peak flow meter and a min Wright peak flow meter. The following measurements of peak expiratory flow (litres/min) were obtained:

We recommended that the repeatability should be calculated for each method separately and compared. I was recently asked how we could carry out a statistical comparison of the two repeatabilities.

The problem is how to compare the within subject standard deviations in a matched sample.

Denote the pairs of measurements by the same method on subject i by x_i and y_i. The standard deviation for a single subject s_i is given by the following formula for variance, i.e. standard devation squared:

s_i² = {x_i² + y_i² - (x_i + y_i)²)/2} /(2-1)

= x_i²/2 + y_i²/2 - x_iy_i

= (x_i - y_i)²/2

Hence for each subject the squared difference (x_i - y_i)² is an estimate of the within-subject variance for that method of measurement times 2, and the absolute value |x_i - y_i| is an estimate of the within-subject standard deviation for that method of measurement times root 2. We can compare these estimates between the two methods of measurement using the two sample t method. It is usually preferable to compare variances rather than to compare standard deviations directly.

For the PEFR meter data, the squared differences are:

Squared differences For the PEFR meter data

Subject	Wright meter	Mini meter
1	16	169
2	4	225
3	16	144
4	1089	256
5	36	0
6	2916	625
7	4	9216
8	121	100
9	144	256
10	16	169
11	9	144
12	529	441
13	64	1089
14	196	100
15	169	81
16	2601	400
17	36	64

For the paired t method, the differences between the squared differences by the two methods should follow a Normal distribution and be unrelated to the average squared difference for the subject. This is clearly not the case here, as the graph shows:

D

The assumptions of the paired t method are clearly not met in this case and I suspect that this will always be so. A log transformation of the squared differences is quite effective:

D

One of the differences for the Wright meter was zero. It was replaced by half the next smallest value, 64, for this analysis.

Proceeding with the paired t test (Stata output) we get:

One-sample t test                                     Number of obs =       17

------------------------------------------------------------------------------
Variable |      Mean    Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
   lslws |  1.098972    .5972562   1.84003   0.0844      -.1671547    2.365098
------------------------------------------------------------------------------
Degrees of freedom: 16

Thus there is only very weak evidence that there is a difference between the within-subject variances. Antilogging the mean difference we get exp(1.098972) = 3.00, showing that the within-subject variance for the mini meter is estimated to be 3 times that for the Wright meter, but there is a vary wide confidence interval for this ratio, from exp(-0.1671547) = 0.85 to exp(2.365098) = 10.65.

The square root of the ratio will be the ratio of the within-subject standard deviations for the two methods of measurement.

Reference

Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; i: 307-10. Full text.

Back to frequently asked questions on the design and analysis of measurement studies.

Back to measurement studies menu.

Back to Martin Bland's home page.

This page maintained by Martin Bland.
Last updated: 4 January, 2010.

Back to top.