# Exercise: observer agreement about sex

This website is for students following the M.Sc. in Evidence Based Practice at the University of York.

183 students were observed twice by different student observers. These measured height (mm), arm circumference (mm), head circumference, and pulse (beats/min) and recorded sex and eye colour. They entered these into a computer file. Eye colour and sex were entered as numerical codes.

The following table shows sex as recorded by two observers:

Sex recorded by
first observer
Sex recorded by
second observer
Total
female male
female 118 1 119
male 1 63 64
Total 119 64 183

This is the output from SPSS 16, where kappa is a statistic available from crosstabs:

Symmetric measures

Value
Assym. Std.
Errora

Approx Tb

Approx. Sig.
Measure of agreement

Kappa 0.976 0.017

13.203

.000

N of Valid Cases   183
a: Not assuming the null hypothesis.
b: Using the asymptotic standard error assuming the null hypothesis.

This is the output from a Stata command for Cohen's kappa:

```. kap sex1 sex2

Expected
Agreement   Agreement     Kappa   Std. Err.         Z      Prob>Z
-----------------------------------------------------------------
98.91%      54.52%     0.9760     0.0739      13.20      0.0000
```

Question 1:

What is meant by “Agreement” and “Expected agreement”?

Question 2:

What does kappa mean and what can we conclude?

Question 3:

What is “Z”?

Question 4:

“Prob>Z” is the P value. What is it testing?

Question 5:

How is it possible for kappa to be less than 1.00 for sex?

The following is a data listing for the two students who appear to have changed sex:

Student A
First       Second
height     1858 1607
arm 234 258
pulse 66 72
eye hazel brown
sex male female

Student B
First       Second
height     1717 1711
arm 247 251
pulse 84 72
eye green green
sex female male

Question 6:

What do you think has happened in the case of Student A?

Question 7:

What do you think has happened in the case of Student B?