Thread: Which statistical test to use here?

OK, I have a bunch of data that I need help analyzing. I'm going to try and explain this as well as I can, but I might not do a very good job (if this problem belongs in the basic statistics board, I apologize)

I have a set of about 200 movies, along with a set of 100 judges. Each judge has chosen their 10 favorite movies and ranked them from #1 to #10. Also, they have chosen 5 movies that just missed the cut (I'll call them "honorary mention") These 5 movies are unranked. So for each of the 100 judges, there is a ranked list from #1 to #10, followed by an unranked list of 5. (10 ranked, 5 unranked)

I want to be able to take my data and rank all #200 movies as accurately as possible. So here are my questions.

1. What point values to I assign to each rank? The simplest would be 10 points for #1, 9 points for #2, 8 points for #3, and so on down to 1 point for the #10 ranked movie. so a movie with three #2 votes would get 27 points. Is this the best point values for each judges rank?

2. What do I do about the "honorary mentions"? Each judge's list will have 15 movies, and 5 of them will be unranked. For example, "Movie A" is on 3 top-10 lists, with rankings of #9, #8, and #5. "Movie B" doesn't appear on any top-10 lists, but appears as an "honorary mention" 9 times. Is there some way to accurately compare these two movies?

3. What actual statistical test do I use here?


Well, that's it. I probably could have worded all of that better, but if anybody knows what I'm talking about, I'd appreciate the help!

Give 1 point to each honorary mention. After that give the top 10 movies values >1 depending on the importance of being the best.

It is really up to you. You maybe want to give the top movie from each judge 20 points, then the 2nd 15 points.

The real question here is if the judges are showing a basis towards a particular movie. This can be judged using a two-way contingency table and a chi-square test.

Arrow's Impossibilty Theorem means that there is no "best" way for you to convert individual ranking preferences into a group preference.

What is the null hypothesis you wish to test? If it is that all movies will receive an equal number of points then you shoul use the chi squared test. Alternatively you can look at two movies and do a simple t test to see if they are significantly different.

Read a little about Arrow's Impossibility Theorem...and it was exactly what I was afraid of. Oh well...

But thanks to both you guys about the chi-square test advice. I'll fiddle around with the data a little more, see what happens. Looking good so far though. All things considered, the whole thing seems to be working out well now that I know what I can't do with the data.

I decided to go with the 20-11 points system. Seems to be working out better than the 10-1 system. At least for now.

So thanks guys, you pointed me in the right direction and kept me from looking for that "perfect" system that I never would have found...