Hi all,
If i have two means with a margin of error (95% confidence intervals). how can I subtract one from the other, what will the computed margin of error be?
e.g. (10 +/-2) - (8 +/-1)
Many Thanks
The relative potential error in an estimate formed by the difference of two estimates of the same sign and roughly comparable magnitude is frequently very great.
Estimate of a is 1000 with a potential relative error of 1% so $990 \le a \le 1010.$
Estimate of b is 900 with a potential relative error of 1% so $891 \le b \le 909.$
So estimate of a - b is $1000 - 900 = 100.$ But
$990 -909 \le a -b \le 1010 - 891 \implies 81 \le a - b \le 119.$
The potential relative error is 19%. Once you understand this, you will never read financial statements the same way and will have a clue why accountants are obsessive.
If the first measurement is "a" with "r%" error then the largest it can be is (1+ r/100)a and the least it can be is (1- r/100)a. If the second measurement is "b" with "r%" error then the largest it can be is (1+ r/100)b and the least it can be is (1- r/100)b.
When we subtract the second measurement from the first, the largest value will be when we subtract the least value of the second from the largest value of the first: (1+ r/100)a- (1- r/100)b= (a- b)+ (r/100)(a+ b). The smallest value will be when we subtract the largest value of the second from the smallest value of the first: (1- r/100)a- (1+ r/100)b= (a- b)- (r/100)(a+ b).
I was introduced to analysis of errors in a course in numerical methods. Most of math is approached in a very Platonic spirit, where numbers are assumed to be known with complete certainty. A course in numerical methods stresses that the numbers used in many calculations are subject to potential error, and part of the purpose of the course is to minimize the effect of unknown but bounded potential errors.
The fact that numbers and therefore calculations are frequently imperfect does not mean that they are useless. Imperfect information is better than no information. What analysis of errors brought home to me was two things. First, just because data are numeric does not mean that they are meaningful. Second, an appreciation for why people like physical scientists and accountants are so concerned about preventing MATERIAL error in the numbers that are used for further calculations. Now let's take your example.
Your a had a potential relative error of 10%. Whether that is "good enough" depends on the situation. Your b had a potential relative error of 12.5%. Again, whether you want to use an estimate with that degree of potential error depends on what are the consequences of being wrong, the costs of getting better information, etc. For many purposes of practical life, potential errors of that magnitude are properly accepted. (Mostly because the world is filled with uncertainties and to obsess about those that at least have known bounds and ignore those that are completely unknown is silly.) But part of the situation is whether estimates are going to be used in calculating other estimates. In that case, errors build up. If what is important in your example is neither a nor b in isolation but rather their difference, then you have to look at how the potential errors propagate.
Let's apply to your example the formula for propagation of error when subtracting two estimates of the same sign but with independent errors:
$\dfrac{10 * 0.1 + 8 * 0.125}{10 - 8} = \dfrac{1 + 1}{2} = 100\%.$
Let's check to see that the formula works.
$9 \le a \le 11,\ estimate = 10.$
$7 \le b \le 9,\ estimate = 8 \implies -\ 9 \le - b \le -7 \implies 9 - 9 \le a - b \le 11 - 7 \implies$
$0 \le a - b \le 4,\ estimate = 2.$
Is that a good enough estimate? Perhaps. I know that the difference is non-negative. If negative is bad and positive is good, I can make the qualitative assertion that the situation is not bad, and that may be all I need to know.
In most practical cases of course, that purely qualitative assessment is not enough. You want or need much more accurate estimates. For a real life example, I chaired a committee that reviewed internal controls for compliance with the Sarbanes-Oxley Act. We decided that we wanted an error no greater than 1% in the difference a - b and worked backward to what degree of accuracy was then required in the estimates of a and b. Those obviously turned out to be much smaller than 1%.
I hope some others answer this question and give different viewpoints because yours was an excellent question.