[G4] 2 + 2 = 4.000000001?

[Doug McNutt]

At 23:21 -0500 2/26/04, sr ferenczy wrote:
>521.82 + 4 comes up as 525.82 on the calculator, but the paper tape shows 525.8200000000001
>interesting.

525.8200000000001 has sixteen digits. Floating point operations handle only 15 digits when stored as 64 bit doubles. (Actually it's 52 * log[10](2) = 52 * 0.30103.)  The answer is correct within experimental error. Perhaps the calculator should always enforce a round to 15 digits but the answer is not incorrect.

Try 52182 + 400 followed by a divide by 100. All of those, but not the answer, can be expressed exactly as floats.

Take pencil and paper and, using base 10 arithmetic, write down a decimal number exactly equal to 1/7. Do not use more than 15 digits and do not add any . . .'s at the end. What is the exact price, to the penny, of one item when they are three for a buck? What do you pay if you buy three at the exact price in three separate transactions?

If the ancients had only ignored their thumbs when they learned to count we would have an octal number system and many of these problems would never bother users. Long live the the peck, the quart, and the fractional inch,.

-- 
--> Marriage and kilo are troubled words. Turmoil results when centuries-old usage is altered in specialized jargon <--.