I have the formula for the probability of chance agreement where:

k people are giving ratings

n is the number of possible ratings to choose from

This is the probability for chance agreement where the ratings are allowed to differ by 2 (eg. 1 and 3 would be classed as in agreement).

$\displaystyle P = \frac{3^{k-2} (5n-6)+I \sum_{i=3}^k 3^{k-i} (n-3) 2^{i-1}}{n^k}$

I is an indicator function, $\displaystyle I = 1 \mbox{ if } k \ge 3, I = 0 \mbox{ if } k \le 2. $

Can anyone help me describe how this probability occurs. I was hoping for an explanation along the lines of, "No. of exact agreements + No. of agreements where ratings differ by 1 ..." etc.

I understand the way that the probability for chance agreement in the case where the ratings can differ by one works:

$\displaystyle P = \frac{{\left(n-1\right)}{\displaystyle\sum_{i=1}^{k-1} 2^{i}}+n}{n^k} = \frac{(n-1) \sum_{i=1}^{k-1} 2^{i}}{n^k}+ \frac{1}{n^{k-1}}$

The final term in the numerator is then the number of exact matches.

The (n-1) gives the number of adjacent pairs that would be classed as in agreement.

The summation can then be written as, $\displaystyle \displaystyle(\frac{2}{n})^k -2\displaystyle(\frac{1}{n})^k$ which is the matches differing by one minus the exact matches as they have already been counted.

I just can't work out the similar ideas for the probability where ratings can differ by 2!