Question 2: The percentage agreement was between 67 and 76, but the corresponding kappa values were between 0.08-0.24. Why were the kappa values lower than the proportion showing agreement? What does the size of the difference suggest about the data?
Kappa was lower than percent agreement because it is calculated by subtracting the expected proportion agreeing from the observed proportion agreeing, then dividing by one minus the expected proportion agreeing. This is always lower than the proportion agreeing if the observed proportion is greater than the expected.
The size of the difference suggests that the proportions reported in one of the two categories were quite high. Kappas are often small under these circumstances.
Back to question.
This page maintained by Martin Bland.
Last updated: 1 March, 2007.