Hello,
I am attempting to run a mixed ICC (3,k)/(3,1) with multiple raters
from surveys. I am assessing the reliability of 6 raters, 12 subjects,
and 27 items. I want to assess reliability for each survey item
between raters, a problem exists in that I have missing values for certain unrated subjects (the same 6 raters rated 8 of the subjects, 5 of them rated 1, 4 rated 1, and 3 rated 2).
(see below for a sample item to be assessed for reliability)
I ran preliminary ICCs without with the missing values and SPSS (version 20.0) runs it, but how does it deal with the missing values? Am I artificially
inflating my reliability results (which I don't want to do)? Does SPSS
ignore those subjects altogether? Muller and Buttner (1994) state that
missing values makes a mixed model ICC an invalid reliability test, is this correct?
If so, I was going to substitute the highest and lowest extreme values in order to underestimate the true reliability value, thus creating a more conservative reliability value, however, I would rather not do this as I want a 'true' value for reliability. I hope this all makes sense, I have tried to be as clear as possible.
I am really trying to move forward in
my data analysis, so help would be extremely appreciated.
Sincerely,
Alex
EXAMPLE:
A B C D E F
A 1 2 1 2 2 2
B 1 2 1 1 1 1
C 4 2 4 4 5 6
D 5 6 5 5 4 5
E 7 5 6 7 7 7
F 4 2 1 - - -
G 3 2 2 - - -
H 3 3 3 - 3 -
I 6 4 6 6 6 6
J 1 3 1 - 2 -
K 4 3 1 5 7 5
L 7 6 7 6 - 5