Thus far, I've been unable to find guidance for my problem. It's psychology-related, but the application actually doesn't matter much, so I'll use a soda example.
I have Coke, Pepsi, Dr. Pepper, Mr. Pibb, Sprite, and Mountain Dew. I want to know which is the "best" soda. My way of determining this is by having people taste two of the sodas, and tell me which one they prefer. Each person only does one trial.
I have 300 taste-testers, and 15 comparison groups (e.g., Coke vs. Pepsi, Coke vs. Mountain Dew, etc.) Each comparison group has 20 taste-testers. For the sake of argument, pretend order doesn't matter (in each group, 10 had Soda 1 first, and 10 had Soda 2 first, and there were no order effects).
So, now I have a win-loss grid. Let's say Coke was favored 14 to 6 over Pepsi, Coke was favored 20-0 over Sprite, etc.
I'd like to do two things:
1) Analyze each comparison group to see if there was a significant difference in preference for one soda over the other, and
2) See if there are statistically significant differences in overall preference between the beverages, by totaling the wins for each soda and comparing against each other.
Is either 1) or 2) doable, statistically? I'm working with SPSS, but I have access to SAS and MATLAB. It seems like it should be chi-square related, but my scenario has restricted options (i.e., it's a choice of 1 out of 2, not 1 out of 6). Maybe a flavor of regression will do the job?
The real experiment can't be done within-subjects because rather than soda, I'm asking people to read relatively long info packets, and pick the packet they like better; 6 of these packets would be too confusing for them. And I'd rather not do a strict between-subjects design, because I'd lose the "winner/loser" aspect of this.
Thanks in advance,