# Problem: Cumulative binomial/multinomial, signal detection?

• Dec 19th 2012, 06:13 AM
PsyCog
Problem: Cumulative binomial/multinomial, signal detection?
Dear all,

I have just arrived to the forum and really need some advice. Please consider the following problem:

In a psychological experiment participants are asked to intuitively decide if visual stimuli, a light spot, were produced by device “A”. A series of similar but different stimuli are presented and each time the participant as to decide whether it came from device “A” or not. I say intuitively because they have no real way to know if it came from “A” or some other device, but due to an experimental manipulation that we introduce we believe participants are able to intuitively identify above chance stimuli “A”.

Experiment 1:

Although participants are not told this, stimuli are produced by only two devices “A” and “B”. For each of these devices there are 10 different, but very similar, light spots. The difference between stimuli within-device type is considerably small, and even between devices is not very large, but there’s always some characteristics that distinguish them (between “A” and “B”). By the end of the experiment, participants vary in the degree of acquired awareness regarding these between-device differences. The vast majority believes there were around 4 different devices (in reality there are only 2). Stimuli “A” and “B” are presented an equal number of times (participants do not know this) and in randomized fashion. So probability of each type of stimuli to occur is 50%
The answer is binomial: yes –it is device “A” or No – it is not device “A”.

Question 1: What statistical test shall we use to evaluate if participants identify stimuli “A” above change?

We thought of :
1 - Signal detection theory
2 – Cumulative Binomial Probability

We think both are correct and equivalent, right?

However, our concern was that results could be explained by a rejection of “B” rather than a selection of “A”, thus we carried out a second experiment.

Experiment 2:

This experiment is very similar to the first experiment with the exception that now stimuli are produced from 4 different devices “A”, “B”, “C” and “D”. The probability of occurrence of each type of stimuli is the same, i.e. 25%. Again, participants are not told the number of different devices.
The answer is still binomial: yes –it is device “A” or No – it is not device “A”.

Question 2: What statistical test shall we use to evaluate if participants identify stimuli “A” above change? That is, the probability to chose “A” is bigger than the probability to chose the other devices?
Worth of noticing that the devices are different across participants so that the false-alarms of for example “B” cannot be “clustered” across participants and treated as a specific type of false alarm (the same for “C” and “D”). Our emphasis is always on finding out if the identification of “A” is higher than chance, both at participant level and group level

Thank you,
PsyCog
• Dec 19th 2012, 04:48 PM
abender
Re: Problem: Cumulative binomial/multinomial, signal detection?
Hi. Welcome to the forums.

Regarding your questions, I first need some clarification. Does the participant know what exactly "A" is? Is the participant shown "A" before the experiment, so he or she can acquire a baseline? Or, is "A" something that common folks understand? Example of this: If "A" is Nike and "B" is any other type of sneaker, and we assume wearing Nike shoes is something everyone has done at some point, then one has a reason to identify "A" or not. Otherwise, a participant is calling out "A" and "other" indiscriminately.

Do you understand what I am trying to ask?

-Andy

EDIT: I believe my question is answered in this line: '... but due to an experimental manipulation that we introduce we believe participants are able to intuitively identify above chance stimuli “A”.' My oversight.
• Dec 19th 2012, 05:20 PM
abender
Re: Problem: Cumulative binomial/multinomial, signal detection?
A few more things to clear up:

Quote:

Question 2: What statistical test shall we use to evaluate if participants identify stimuli “A” above change? That is, the probability to chose “A” is bigger than the probability to chose the other devices?
1) Are we testing for (correct) identification or simply what the participants' choose as their (possibly incorrect) answer?

2) What does 'identify stimuli "A" above change' mean? Do you mean 'chance' instead of 'change'? If not, then are we testing for CORRECT identification of "A" vs. correct identification of B, OR just choosing "A" vs. just choosing "other"?
• Dec 21st 2012, 08:13 AM
PsyCog
Re: Problem: Cumulative binomial/multinomial, signal detection?
Hi Abender,

Answering to your first thread: Yes, that's exactly what we are trying to test, if participants intuitively choose "A" more often than the others. They haven't seen "A" before, and there is nothing that can objectively (any cognitive process) help them on the choice, but we believe a kind of implicit knowledge would lead them towards choosing "A".