Floating Point Arithmetic and The Descent into Madness

While I should confess upfront that I’ve always had a weaker command of the details of floating point arithmetic than I feel I ought to have, this sort of thing still blows my mind when I stumble upon it. These moments invariably make me realize that floating point math will simply never satisfy my naive hopes as a mathematician:

1
2
3
0.1 + 0.1 == 0.2 # True
0.1 + 0.1 + 0.1 == 0.3 # False
0.1 + 0.1 + 0.1 + 0.1 == 0.4 # True

On my Intel Core 2 Duo machine running OS X, those statements have the indicated truth values in all three of Julia, R and Python.

Consider this evidence for the truth of the combined propositions, “God created the integers. All else is the work of man,” and “Out of the crooked timber of humanity no straight thing was ever made.”

10 responses to “Floating Point Arithmetic and The Descent into Madness”

  1. Harlan

    Yep, binary representations of decimal numbers are weird. You probably know that Julia/R/Python have the same result because all are leveraging the IEEE floating-point logic that Intel has built into their chips since I think the mid to late 1980s. Julia has support for Rational types (fractions), and I’m sure Python has a library for that too. Both of those would get the equality right. And there are floating-point “approximately equal” methods in most languages. R has all.equal() with a default tolerance that’s good enough:

    > all.equal(0.1 + 0.1 + 0.1, .3)
    [1] TRUE

  2. Harlan

    Oh, in Julia it’s approx_eq(a,b). Strangely, it’s not in the documentation anywhere, I don’t think. I learned about it while looking at the test files to work on my (still half-finished) testing framework. I’ll add it to the docs…

  3. Harlan

    Ack! As it turns out, there isn’t an approx_eq() in base Julia! It is defined, however, in extras/test.jl. Now that I think about it, I’m going to submit a pull request to put that function in base. I’m also going to redefine it so that the tolerance is, by default, twice the max uncertainty of the numbers being compared: 2max(eps(a), eps(b)), using the handy eps() function that _is_ defined in base. I’d previously borrowed R’s fixed 1e-6, which is probably not a great idea.

    Sorry about the confusion!

  4. Walter

    MATLAB also handles it like the three mentioned (R, julia, python).
    Mathematica (Wolfram) gets it right …
    I guess Maple will also manage … though I haven’t tested it there …

  5. Harlan

    Ahah, to my surprise, there’s an isclose() function in extras/nearequal.jl! As you said, who knew?

    They really gotta get documentation for extras straightened out.

  6. Arnon

    God said: “you shall never ever use == on floating point numbers”

  7. Stefan Karpinski

    This floating point behavior is slightly less shocking if your language doesn’t lie to you when printing floats:

    R> 0.1 + 0.2
    [1] 0.3

    Matlab> 0.1 + 0.2

    ans =

    0.3000

    julia> 0.1 + 0.2
    0.30000000000000004

    In R and Matlab, the result of 0.1+0.2 *looks* like exactly 0.3 even though it’s not. Julia uses the excellent double-conversion library [http://code.google.com/p/double-conversion/] to efficiently print the shortest decimal representation that will exactly reproduce a float value. This is a surprisingly hard problem that has only recently been satisfactorily solved by Florian Loitisch [http://florian.loitsch.com/publications/dtoa-pldi2010.pdf], who also wrote the double-conversion library.

  8. Harlan

    Stefan, fascinating that it was solved so recently! Have printed that article and will read it soon!