Classic Computer Magazine Archive Article from Compute! magazine

Faulty Math?
I have a Commodore 64, and there seems to be an error of some kind in the math logic. If you raise the number 5 to the power of 7 (PRINT 5↑7), you get the result of 78125.0001, which is incorrect.

    I have found this error with other numbers, and although these amounts are small, I'm afraid there are others which I haven't found yet which are causing errors in math programs. Could you tell me what causes this?
Bill Briggs

You have come across the phenomenon known as rounding error. Most home computers use the binary system for mathematical calculations. While some fractional numbers translate easily to binary, some do not. The same is true of decimal numbering, the system we're used to. The fraction 1/3 translates into 33 percent (after rounding off all those extra threes). And three times 33 percent comes to 99 percent. The result is off by 1 percent.

    You will never be able to translate 1/3 into a perfect percentage in decimal. And your computer will never be exactly accurate when it uses certain functions in binary. But how do fractions occur when raising one whole number to the power of another whole number?

    If you were trying to do the expression you mentioned on paper, you'd write 5 X 5 X 5 X 5 X 5 X 5 X 5. However, the computer uses a more sophisticated mathematical formula to avoid having to perform the repeated multiplications. To see how it works, try entering the following line in direct mode:


Notice that this gives the same answer as PRINT 5↑7. It's the fractions involved in evaluating the logarithm (LOG) and exponential (EXP) functions that lead to the rounding error. Other functions that suffer the same problem for a similar reason are SIN, COS, TAN, ATN, and SQR.
    For a thorough discussion on rounding errors, see "De-Bugging BASIC, Part 2," in the February 1985 issue of our sister publication, COMPUTE!'s GAZETTE.