Thread: Need help determing number of test questions necessary

I need to devise a test that can state with 95% confidence that the test taker knows x percentage of a body of knowledge questions if they are tested on a sampling of the questions and score x percentage.

There are 20,000 questions. Obviously I cannot test my students on all of them, but I want to know what number of questions I need to test them on to determine with 95% confidence that they know the same percentage of the 20,000 questions as they do the questions they were tested with.

It would be great if you could show me how you determined the answer. I may need to change some of the criteria later.

I would say 377 questions would be best, but thats still a lot.

Look here Sample Size Calculator - Confidence Level, Confidence Interval, Sample Size, Population Size, Relevant Population - Creative Research Systems

Enter 20,000 and the desired error, I put is 5% which gives 95% 5% confidence.

Great. Good advice and helpful link. One question: How did you come up with 5% or 5 for the confidence interval? What does confidence interval really mean in this context?