Category Archives: Floating Point

Sometimes Floating Point Math is Perfect

I’ve written in the past about how to compare floating-point numbers for the common scenario where two results should be similar but may not be identical. In that scenario it is reasonable to use an AlmostEqual function for comparisons. But … Continue reading

Posted in Floating Point | Tagged | 2 Comments

Everything Old is New Again, and a Compiler Bug

“What’s an EXCEPTION_FLT_STACK_CHECK exception?” one of my coworkers said. I said “It’s a weird and rare crash. Why do you ask?” It turns out that one of these weird and rare crashes had started showing up in Chrome (M54 branch, … Continue reading

Posted in Debugging, Floating Point | Tagged | 28 Comments

Intel Underestimates Error Bounds by 1.3 quintillion

Intel’s manuals for their x86/x64 processor clearly state that the fsin instruction (calculating the trigonometric sine) has a maximum error, in round-to-nearest mode, of one unit in the last place. This is not true. It’s not even close. The worst-case … Continue reading

Posted in Floating Point, Investigative Reporting, Programming | Tagged , , | 131 Comments

Please Calculate This Circle’s Circumference

“Please write a C++ function that takes a circle’s diameter as a float and returns the circumference as a float.” It sounds like the sort of question you might get in the first week of a C++ programming class. And … Continue reading

Posted in Floating Point, Programming | Tagged , , , | 74 Comments

There are Only Four Billion Floats–So Test Them All!

A few months ago I saw a blog post touting fancy new SSE3 functions for implementing vector floor, ceil, and round functions. There was the inevitable proud proclaiming of impressive performance and correctness. However the ceil function gave the wrong … Continue reading

Posted in Floating Point | Tagged , | 39 Comments

Floating-Point Determinism

Is IEEE floating-point math deterministic? Will you always get the same results from the same inputs? The answer is an unequivocal “yes”. Unfortunately the answer is also an unequivocal “no”. I’m afraid you will need to clarify your question. My … Continue reading

Posted in Floating Point, Visual Studio | Tagged | 58 Comments

Float Precision Revisited: Nine Digit Float Portability

Last year I pointed out that float variables can be converted to text and then back to the same binary value using printf(“%1.8e”). You can also use %.9g for more compact results. I also supplied a test program that used … Continue reading

Posted in Floating Point | Tagged , , , , | 18 Comments