Thread: How can I calculate sig differences in win-loss?

Greetings, all...

Thus far, I've been unable to find guidance for my problem. It's psychology-related, but the application actually doesn't matter much, so I'll use a soda example.

I have Coke, Pepsi, Dr. Pepper, Mr. Pibb, Sprite, and Mountain Dew. I want to know which is the "best" soda. My way of determining this is by having people taste two of the sodas, and tell me which one they prefer. Each person only does one trial.

I have 300 taste-testers, and 15 comparison groups (e.g., Coke vs. Pepsi, Coke vs. Mountain Dew, etc.) Each comparison group has 20 taste-testers. For the sake of argument, pretend order doesn't matter (in each group, 10 had Soda 1 first, and 10 had Soda 2 first, and there were no order effects).

So, now I have a win-loss grid. Let's say Coke was favored 14 to 6 over Pepsi, Coke was favored 20-0 over Sprite, etc.

I'd like to do two things:
1) Analyze each comparison group to see if there was a significant difference in preference for one soda over the other, and
2) See if there are statistically significant differences in overall preference between the beverages, by totaling the wins for each soda and comparing against each other.

Is either 1) or 2) doable, statistically? I'm working with SPSS, but I have access to SAS and MATLAB. It seems like it should be chi-square related, but my scenario has restricted options (i.e., it's a choice of 1 out of 2, not 1 out of 6). Maybe a flavor of regression will do the job?

The real experiment can't be done within-subjects because rather than soda, I'm asking people to read relatively long info packets, and pick the packet they like better; 6 of these packets would be too confusing for them. And I'd rather not do a strict between-subjects design, because I'd lose the "winner/loser" aspect of this.

Thanks in advance,
Rodney

1) if people choose randomly when indifferent; then under the null (that people are indifferent) the number of "wins" for each beverage will follow a B(20,0.5) distribution; so you can test directly the number of wins in each cell of your grid.

2) i dont really understand what your trying to do here.

For (2), I am trying to see if there's a significant difference in wins, overall, between the drinks. But at this point, I guess the same principle that you mentioned in (1) applies: each drink would be subject to 100 tests (20 per opponent). In this case, the wins for each beverage should follow a B(100,0.5) distribution, I think.