Question 2: What is meant by ‘weighted kappa = 0.91’? What does this tell us?
We are told that the gestalt assessment is categorically graded, i.e. the categories are ordered and there are more than two of them. Simple kappa treats all disagreements as the same, whether the categories are adjoining or at opposite ends of the order. Weighted kappa enables us to take the ordering into account. We can give greater weight to disagreements where the two observations are of categories at opposite ends of the scale than for disagreements where the categories are close together.
The weights are chosen by the researcher and will affect the value of weighted kappa. They should be chosen in advance. If the disagreements are all given equal weight, we get simple kappa.
Weighted kappa = 0.91 is usually regarded as very good agreement, being greater than 0.8.
Back to question.
This page maintained by Martin Bland.
Last updated: 1 March, 2007.