Exercise: observer agreement about sex, answer 1


What is meant by “Agreement” and “Expected agreement”?

Suggested answer

Here agreement is the percentage of observations where both observers record the same answer. It is 100×(118+63)/183 = 98.91.

Expected agreement is the percentage of observations where we would expect both observers to record the same answer if their answers were not related. It is the sum of the frequencies which would be expected under the null hypothesis in a chi-squared test. These are found by row total multiplied by column total divided by grand total. For example, for the first cell, female and female, 119 × 119 / 183 = 77.4.

The expected frequencies are:

Sex recorded by
first observer
Sex recorded by
second observer
female male
female 77.4 41.6 119.0
male 41.6 22.4 64.0
Total 119.0 64.0 183.0

Percentage expected agreement = 100×(77.4 + 22.4)/183 = 54.54. Note the slight rounding error.

Back to question.

To Martin Bland's home page.

This page maintained by Martin Bland.
Last updated: 21 July, 2008.

Back to top.