Exercise: observer agreement about sex, answer 1

Question

What is meant by “Agreement” and “Expected agreement”?

Suggested answer

Here agreement is the percentage of observations where both observers record the same answer. It is 100×(118+63)/183 = 98.91.

Expected agreement is the percentage of observations where we would expect both observers to record the same answer if their answers were not related. It is the sum of the frequencies which would be expected under the null hypothesis in a chi-squared test. These are found by row total multiplied by column total divided by grand total. For example, for the first cell, female and female, 119 × 119 / 183 = 77.4.

The expected frequencies are:

Sex recorded by first observer	Sex recorded by second observer		Total
	female	male
female	77.4	41.6	119.0
male	41.6	22.4	64.0
Total	119.0	64.0	183.0

Percentage expected agreement = 100×(77.4 + 22.4)/183 = 54.54. Note the slight rounding error.

Back to question.

To Martin Bland's home page.

This page maintained by Martin Bland.
Last updated: 21 July, 2008.

Back to top.