float in C# & C++

The answer lies in understanding how computers represent floating point numbers in binary. The fractional components of floats (and doubles) are stored in binary fractional form, meaning it can precisely store binary fractions such as 1/2, 1/4, 1/8, 1/16, 1/32, 1/64, and so on. In a similar fashion of using combinations of powers of two to encode any integer, floats use combinations of these binary fractions to approximate any decimal number.

Notice I said approximate instead of encode.

There are some numbers which aren't representable in this way. 0.1 is one example. You can try all you like, but you will never find a combination of binary fractions that, when added together, equal exactly 0.1. You'll always be a tiny bit off.

So floats merely approximate the numbers they're trying to represent, meaning you should be careful when performing calculations with floats where absolute precision is necessary.

As for the two programs above giving different results, I'm not sure I can exactly answer it. All I know is that different systems will have different ways of making assumptions about the numbers their floats are approximating. Understanding the fundamental approximate nature of floats should make it no surprise that your two programs above disagree.

Take up a piece of paper and a pencil and give this exercise a try to see why this is so:

Binary fractions are fractions that are split in two each time. So that would be 1/2, 1/4, 1/8, 1/16, and so on. When writing floating point numbers in binary representation, each digit in the fractional component holds a place value of one of these binary fractions. Like so:


0 . 1 1 1 1 1 ...

    ^ ^ ^ ^ ^

    | | | | |

    | | | | \-- 1/32nd's place. Decimal value: 0.03125

    | | | |

    | | | \---- 1/16th's place. Decimal value: 0.0625

    | | |

    | | \------ 1/8th's place. Decimal value: 0.125

    | |

    | \-------- 1/4th's place. Decimal value: 0.25

    |

    \---------- 1/2's place. Decimal value: 0.5

Let's try to encode the value 0.75. Just like building a binary integer, we start with the place closest to the decimal and work our way away. The first place value after the decimal is 1/2, or 0.5. Ask yourself, is 0.5 less than or equal to 0.75 (my target number)? Yes it is, so we put a 1 in that place value and subtract 0.5 from our target value, resulting in 0.25 remaining. Our float that we're building now looks like this:

0.1

The next place value is 1/4, or 0.25. Ask your self, is 0.25 less than or equal to 0.25, my target remainder? Yes it is, so put a 1 in that place and subtract 0.25 from your target. The remainder is 0, so we're done. Our float now looks like this:

0.11

So you can see that 0.75 is comprised of 0.5 and 0.25. Now, all that's left is to put it into actual floating point notation along with an exponent. We move the decimal point to just after the greatest significant digit, and store the exponent.

1.1 x 2^-1

What this means is we take the value: 1.1 (or 1.5 in decimal format), and multiply it by 2 to the negative first power. This is how floats are stored. (Actually, since we always move the decimal to after the greatest significant digit, we can assume the first value will always be 1, so this 1 is not actually stored. It is implied. Also, since floats are always in binary, the 2 is also implied, so we just store the exponent, -1, in two's complement form.

So the way you store 0.75 in floating point notation is:

mantissa: 1
exponent: -1

Now, with all that out of the way, try the same exercise above, but with the target value 0.1.

Start with the first place value. Is 0.5 less than or equal to 0.1? No, so we put a zero there and move to the next decimal place. Is 0.25 less than or equal to 0.1? No, so put a zero in that place and go to the next. Is 0.125 less than or equal to 0.1? No, so put a zero and move on. Next digit: Is 0.0625 less than or equal to 0.1? Yes. Put a 1 there and subtract the place value to get the remainder. So now, it looks like this so far:

0.0001 (binary) remainder 0.0375 (decimal)

Our remainder isn't zero yet, so move on to the next position. Is 0.03125 less than or equal to 0.0375? Yes. Put a 1 there, subtract the place value (remainder is now 0.00625). So far, it looks like this:

0.00011 (binary) remainder 0.00625 (decimal)

Keep going. Next position: is 0.015625 less than or equal to 0.00625? No. Put a zero and move on. Next position: is 0.0078125 less than or equal to 0.00625? No. Put a zero and move on. Next position: is 0.00390625 less than or equal to 0.00625? Yes. Put a one, subtract the place value, and now it looks like this:

0.00011001 (binary) remainder 0.00234375

You can keep going all day long, but you'll soon find out that you will never encounter a place value which uses up the entire remainder. You'll always have a little remainder left each time, on until infinity. Computers are finite machines, meaning they don't have room to store infinitely many digits, so the computer has to call it quits somewhere and round off. That's where the margin of error is introduced.

Now, the reason two calculations like the ones in my last post aren't guaranteed to be exactly equivalent is because different numbers might get rounded off at slightly different places. Take, for example, this calculation:

0.45 + 0.45 = 0.9

If we were to represent these numbers as being rounded to the nearest tenth, we would have this:

0.5 + 0.5 = 1.0

Which is not the correct answer (though it is within a certain margin of error, so we accept it for now.)

Now take this calculation:

0.39 + 0.51 = 0.9

Again, our system can only handle one decimal place, so rounding this off to the nearest tenth, we get this:

0.4 + 0.5 = 0.9

So, by all accounts, (0.45 + 0.45) should be equivalent to (0.39 + 0.51). They both equal 0.9 exactly. But because of our margin of error and the rounding that takes place, they are not equivalent. One equals 1.0 and the other equals 0.9.

In order to test for equivalence in this system, we have to subtract one from the other, then test to see if the absolute value of the difference falls below a certain threshold--our "margin of error". In this particular example, we could use 0.2 as the threshold. If you subtract 0.9 from 1.0, you get 0.1. Is 0.1 less than 0.2? Yes it is, so we should assume the two calculations are equivalent.

As you might have guessed, this "approximation" may sometimes result in two calculations which are not equivalent getting treated as though they are. That's just the nature of the beast. Floats shouldn't be used where absolute precision is necessary. There are other data types which provide greater precision.

But I hope that demystifies how computers deal with floats.