help explaining unrounding concepts

Aug 2019
3
1
Colorado
We have some people trying to report four places to the right of the decimal with inputs that rounded to 2 and 3 decimal places. Can you please articulate why it is impossible to guarantee four decimal precision? I have been unsuccessful in my attempts and am looking for help from someone with a formal math background. Many thanks in advance.
 
Oct 2009
1,245
555
Brisbane
You can see why with a simple numerical example:

Suppose you have the two numbers 0.2549 and 0.8349 which could be rounded to 2 decimal places as 0.25 and 0.83 respectively.

0.2549 x 0.8349 = 0.2128 (correct to 4 decimal places)

However if the rounded values are inputted: 0.25 x 0.83 = 0.2075, so its not precise.
 
  • Like
Reactions: 1 person
Aug 2019
3
1
Colorado
Thanks for the reply. If I may ask a follow-up, say the inputs being multiplied have different number of places

0.23 / 0.834 = 0.275779376498801

Given the inputs, do you feel it is more appropriate to report 0.28 or 0.278?
 
Oct 2009
1,245
555
Brisbane
I don't think there's any hard and fast rule about this. I'd go with 3 dps if the input numbers are exact, probably 2 dps if the inputs are rounded.
(BTW 0.276 not 0.278)
 
Aug 2019
3
1
Colorado
I don't think there's any hard and fast rule about this. I'd go with 3 dps if the input numbers are exact, probably 2 dps if the inputs are rounded.
That sounds reasonable to me. Thanks again for your input.

(BTW 0.276 not 0.278)
Ha, good catch...too much jockeying around with the example and messed it up :)
 
  • Like
Reactions: 1 person