[Ti] interesting calculator behavior!?
Sherman Gregory
sherman at qualcomm.com
Tue Aug 26 11:12:16 PDT 2003
At 10:58 AM -0400 8/26/03, Steve Wozniak wrote:
>
>Your problem is that 4.6 (decimal) does not store perfectly in
>binary. It repeats forever, like 1/3 does in decimal (.3333333...).
>6/10 in binary is .10011001100... At some point you have to clip off
>a dwindling amount, and perfect precision calculations from that
>point on have "roundoff" errors.
>
Steve,
It is really interesting that you would be the one to reply and
explain this. You see, you indirectly taught me this very lesson
back 1979 or 1980 when I discovered this when writing some accounting
type software on my Apple ][+. I remembering commenting about this
at my local Apple retailer and thinking that this was a bug.
Also interesting that the 6502 had a BCD mode to help deal with this
sort of thing, but I don't recall ever seeing that in any other
processor.
Sherman
More information about the Titanium
mailing list