doesn't the law of large numbers apply here? for sifficiently large n the sameple average will almost surely match the population average of the relevent machine; and you will almost surely be able to guess correctly.
Two machines, A and B, both have a light bulb on them. For machine A, the light flashes red with .8 prob and blue with .2 prob. For machine B, .2 prob red and .8 prob blue. With .5 probability you are presented with machine A or B. Your task is to observe the color of its flashes, so as to make a best guess whether it's A or B.
Before you start to observe, however, you must make a decision as to how many times you want to observe its flash (say n times), and after n times, you make your guess.
Intuitively I thought the larger n is, the better our chances are. But when I calculated n=2, I found that you still have a .8 prob of being correct, the same as when you observe only once. Is it true that no matter how many times you observe, you can't do better than .8? I find it baffling.
Assuming you watch the light N times, the probability of exactly K results is just the binomial theorem. And since your 2 lights are symmetrical the chance for the 80% light for either of them is p = .8, and q = .2
-- Equation 1
The average number of times the 80% light will show up i this:
Equation 2
Note: this is our average if we run the simulation an infinite number of times. Very important to understand.
For n= 100 that (not surprisingly) works out to an average of 80 times.
Now the next step we want is what is called the standard deviation. This value represents how spread out the data points are. Or to put it another way... how likely are we to get a value that isn't 80. if the standard deviation is high.. then for example we might expect only 60 flashes to happen 10% of the time. But if the standard deviation is low.. 60 flashes might only happen less than 1% of the time.
Unfortunately, while standard deviations are simple for a set of points, its a little more complicated involving a continuous function like the one above. If your familiar with calculus and integrals, you should be able to search up standard deviation and see how to do it for continuous functions.
However, we don't need to be all that fancy to see. We know 80 is the average for 100. Lets just add up all the probabilites of getting EXACTLY 70 to 90 flashes. That is, plug in into Equation 1. (That will be 21 times we have to evauluate that function and add the results).
Using a quick computer program I obtained this value:
99.1607%
This means our standard deviation is really low for 100 "looks". We can say with very high certaintly which bulb we are looking at with 100 flashes to record.
Note, the number of flashes is very important though. For example lets say we only used 10 flashes:
Using equation 2, our average number of flashes is 8. This is exactly what we would predict. So it seems the number of Flashes doens't change our average. Again remember, this "average" is based on an infinite number of runs, not just 1 run.
So now, how likely are we to get our average. Lets plug in into equation 1. Only 3 values this time to figure out, but essential the same deviation as our 100 flash case. This time we only get a probablity of
77.1752%
So you can see the more flashes you see.. the more likely you are to actually hit 80% of the total flashes be the more likely color. Though even for a mere 10 flashes we can be reasonably sure which one we saw.
And all of the above is just a long winded way of saying what SpringFan25 said about the law of large numbers. The more flashes you see, the higher the chance is that you saw the exact average.
Thank you! I was misled by n=2. If we define random variable v=k/n, then var(v)=pq/n, which converges to 0 for large n.
But curiously (and counterintuitively!), when n=4, the result isn't better than n=3 either! So it seems one can't improve from n=2k-1 to n=2k. Do you think this is true? If so, is it just coincidence, or there are deeper reasons for this?
Well actually as n approaches 1, my quick and dirty technique fails. There aren't enough data points around the average to get a "feel" for what the standard deviation will be, so you have to use the actual integration to find it. I haven't really done it, but it seems that the best confidence you can get from 1 flash is 80%. You can never be more certain that that for 1 flash. The 77% i showed you is for 7 8 or 9. Not the likelihood that it would be 2, which would be much much lower. So the confidence that its the big one is probably slightly higher than 80% for every number after 1 and i suspect reaches 99%+ rather quickly.