[Ti] interesting calculator behavior!?

Sherman Gregory sherman at qualcomm.com
Tue Aug 26 11:12:16 PDT 2003


At 10:58 AM -0400 8/26/03, Steve Wozniak wrote:
>
>Your problem is that 4.6 (decimal) does not store perfectly in 
>binary. It repeats forever, like 1/3 does in decimal (.3333333...). 
>6/10 in binary is .10011001100... At some point you have to clip off 
>a dwindling amount, and perfect precision calculations from that 
>point on have "roundoff" errors.
>

Steve,
It is really interesting that you would be the one to reply and 
explain this.  You see, you indirectly taught me this very lesson 
back 1979 or 1980 when I discovered this when writing some accounting 
type software on my Apple ][+.  I remembering commenting about this 
at my local Apple retailer and thinking that this was a bug.

Also interesting that the 6502 had a BCD mode to help deal with this 
sort of thing, but I don't recall ever seeing that in any other 
processor.

      Sherman



More information about the Titanium mailing list