
Recent Posts
Categories
 AltDevBlogADay
 Bugs
 Chromium
 Code analysis
 Code Reliability
 Commuting
 Computers and Internet
 Debugging
 Documentation
 Drinks
 Environment
 Floating Point
 Fractals
 Fun
 Gaming
 Investigative Reporting
 Linux
 Math
 memory
 Performance
 Programming
 Quadratic
 Rants
 Security
 Symbols
 Travel
 uiforetw
 Uncategorized
 Unicycling
 Visual Studio
 WLPG
 Xbox 360
 xperf
Follow me on Twitter
My TweetsMeta
Tag Archives: floating point
Exercises in Emulation: Xbox 360’s FMA Instruction
Years ago I worked in the Xbox 360 group at Microsoft. We were thinking about releasing a new console, and we thought it would be nice if that console could run the games of the previous console. Emulation is always … Continue reading
Sometimes Floating Point Math is Perfect
I’ve written in the past about how to compare floatingpoint numbers for the common scenario where two results should be similar but may not be identical. In that scenario it is reasonable to use an AlmostEqual function for comparisons. But … Continue reading
There are Only Four Billion Floats–So Test Them All!
A few months ago I saw a blog post touting fancy new SSE3 functions for implementing vector floor, ceil, and round functions. There was the inevitable proud proclaiming of impressive performance and correctness. However the ceil function gave the wrong … Continue reading
Float Precision Revisited: Nine Digit Float Portability
Last year I pointed out that float variables can be converted to text and then back to the same binary value using printf(“%1.8e”). You can also use %.9g for more compact results. I also supplied a test program that used … Continue reading
Posted in Floating Point
Tagged float, floating point, floating point precision, printf, scanf
17 Comments
Doubles are not floats, so don’t compare them
I’ve seen a few online discussions linking to my Comparing Floating Point Numbers page for misguided reasons and I wanted to discuss those reasons to help people understand why throwing epsilons at the problem without understanding the situation is a … Continue reading
Posted in Floating Point, Programming
Tagged base conversion, epsilon, float, floating point, precision
24 Comments
That’s Not Normal–the Performance of Odd Floats
Denormals, NaNs, and infinities round out the set of standard floatingpoint values, and these important values can sometimes cause performance problems. The good news is, it’s getting better, and there are diagnostics you can use to watch for problems. In … Continue reading
Posted in AltDevBlogADay, Floating Point, Performance, Programming
Tagged denormal, floating point, infinity, NaN, special numbers, specials
21 Comments
Exceptional Floating Point
Floatingpoint math has an answer for everything, but sometimes that’s not what you want. Sometimes instead of getting an answer to the question sqrt(1.0) (it’s NaN) it’s better to know that your software is asking imaginary questions. The IEEE standard … Continue reading
Posted in AltDevBlogADay, Floating Point, Programming
Tagged divide by zero, double, exceptions, float, floating point
24 Comments
Floatingpoint complexities
Binary floatingpoint math is complex and subtle. I’ve collected here a few of my favorite oddball facts about IEEE floatingpoint math, based on the articles so far in my floatingpoint series. The focus in this list is on float but … Continue reading
Posted in AltDevBlogADay, Floating Point, Programming
Tagged comparing, denormal, digits, float, floating point, floating point precision, floating point tricks, precision, printing
15 Comments
Intermediate FloatingPoint Precision
Riddle me this Batman: how much precision are these calculations evaluated at? If you answered ‘double’ and ‘float’ then you score one point for youthful idealism, but zero points for correctness. The correct answer, for zeroidealism points and forty two … Continue reading
Float Precision–From Zero to 100+ Digits
How much precision does a float have? It depends on the float, and it depends on what you mean by precision. Typical reasonable answers range from 69 decimal digits, but it turns out that you can make a case for … Continue reading
Posted in AltDevBlogADay, Floating Point, Programming
Tagged double, float, floating point, floating point precision, precision, significant figures
52 Comments