Category Archives: Floating Point

Floating Point in the Browser, Part 3: When x+y=x (y != 0)

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading

Posted in Chromium, Computers and Internet, Floating Point | Tagged , | 5 Comments

Floating Point in the Browser, Part 2: Bad Epsilon

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading

Posted in Chromium, Computers and Internet, Floating Point | Tagged , | 2 Comments

Floating Point in the Browser, Part 1: Impossible Expectations

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading

Posted in Chromium, Computers and Internet, Floating Point | Tagged , | 22 Comments

The Easy Ones – Three Bugs Hiding in the Open

I write a lot about investigations into tricky bugs – CPU defects, kernel bugs, transient 4-GB memory allocations – but most bugs are not that esoteric. Sometimes tracking down a bug is as simple as paying attention to server dashboards, … Continue reading

Posted in Bugs, Code analysis, Code Reliability, Debugging, Floating Point, Linux, Performance | Tagged | 23 Comments

Exercises in Emulation: Xbox 360’s FMA Instruction

Years ago I worked in the Xbox 360 group at Microsoft. We were thinking about releasing a new console, and we thought it would be nice if that console could run the games of the previous console. Emulation is always … Continue reading

Posted in Floating Point | Tagged , , , , | 23 Comments

Sometimes Floating Point Math is Perfect

I’ve written in the past about how to compare floating-point numbers for the common scenario where two results should be similar but may not be identical. In that scenario it is reasonable to use an AlmostEqual function for comparisons. But … Continue reading

Posted in Floating Point | Tagged | 20 Comments

Everything Old is New Again, and a Compiler Bug

“What’s an EXCEPTION_FLT_STACK_CHECK exception?” one of my coworkers said. I said “It’s a weird and rare crash. Why do you ask?” It turns out that one of these weird and rare crashes had started showing up in Chrome (M54 branch, … Continue reading

Posted in Debugging, Floating Point | Tagged | 28 Comments

Intel Underestimates Error Bounds by 1.3 quintillion

Intel’s manuals for their x86/x64 processor clearly state that the fsin instruction (calculating the trigonometric sine) has a maximum error, in round-to-nearest mode, of one unit in the last place. This is not true. It’s not even close. The worst-case … Continue reading

Posted in Floating Point, Investigative Reporting, Programming | Tagged , , | 152 Comments

Please Calculate This Circle’s Circumference

“Please write a C++ function that takes a circle’s diameter as a float and returns the circumference as a float.” It sounds like the sort of question you might get in the first week of a C++ programming class. And … Continue reading

Posted in Floating Point, Programming | Tagged , , , | 74 Comments

There are Only Four Billion Floats–So Test Them All!

A few months ago I saw a blog post touting fancy new SSE3 functions for implementing vector floor, ceil, and round functions. There was the inevitable proud proclaiming of impressive performance and correctness. However the ceil function gave the wrong … Continue reading

Posted in Floating Point | Tagged , | 52 Comments