Hey RASimmons.

This model looks like a contigency/catiegorical data problem. You might want to model this as such and then do a statistical test for association for the rating for a particular word combination cell against the variable corresponding to a rater.

If you have evidence of an association, a chi-square analysis should pick it up. If you don't, then it is equivalent to rejecting the hypothesis that there is an assocation.