
Recent Posts
Categories
 AltDevBlogADay
 Bugs
 Chromium
 Code analysis
 Code Reliability
 Commuting
 Computers and Internet
 Debugging
 Documentation
 Drinks
 Environment
 Floating Point
 Fractals
 Fun
 Gaming
 Investigative Reporting
 Linux
 Math
 memory
 Performance
 Programming
 Quadratic
 Rants
 Security
 Symbols
 Travel
 uiforetw
 Uncategorized
 Unicycling
 Visual Studio
 WLPG
 Xbox 360
 xperf
Follow me on Twitter
My TweetsMeta
Tag Archives: precision
Floating Point in the Browser, Part 3: When x+y=x (y != 0)
A few years ago I did a lot of thinking and writing about floatingpoint math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hardearned knowledge. … Continue reading
Floating Point in the Browser, Part 2: Bad Epsilon
A few years ago I did a lot of thinking and writing about floatingpoint math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hardearned knowledge. … Continue reading
Floating Point in the Browser, Part 1: Impossible Expectations
A few years ago I did a lot of thinking and writing about floatingpoint math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hardearned knowledge. … Continue reading
Doubles are not floats, so don’t compare them
I’ve seen a few online discussions linking to my Comparing Floating Point Numbers page for misguided reasons and I wanted to discuss those reasons to help people understand why throwing epsilons at the problem without understanding the situation is a … Continue reading
Posted in Floating Point, Programming
Tagged base conversion, epsilon, float, floating point, precision
24 Comments
Floatingpoint complexities
Binary floatingpoint math is complex and subtle. I’ve collected here a few of my favorite oddball facts about IEEE floatingpoint math, based on the articles so far in my floatingpoint series. The focus in this list is on float but … Continue reading
Posted in AltDevBlogADay, Floating Point, Programming
Tagged comparing, denormal, digits, float, floating point, floating point precision, floating point tricks, precision, printing
15 Comments
Intermediate FloatingPoint Precision
Riddle me this Batman: how much precision are these calculations evaluated at? If you answered ‘double’ and ‘float’ then you score one point for youthful idealism, but zero points for correctness. The correct answer, for zeroidealism points and forty two … Continue reading
C++ 11 std::async for Fast Float Format Finding
After a recent post on float precision there was some debate about roundtripping of floats. My claim was that if you print a float with printf(“%1.8e”, f); and then scan it back in then you are guaranteed to get back … Continue reading
Posted in Floating Point, Programming, Visual Studio
Tagged float, precision, print, scanf
24 Comments
Float Precision–From Zero to 100+ Digits
How much precision does a float have? It depends on the float, and it depends on what you mean by precision. Typical reasonable answers range from 69 decimal digits, but it turns out that you can make a case for … Continue reading
Posted in AltDevBlogADay, Floating Point, Programming
Tagged double, float, floating point, floating point precision, precision, significant figures
52 Comments
Comparing Floating Point Numbers, 2012 Edition
This post is a more carefully thought out and peer reviewed version of a floatingpoint comparison article I wrote many years ago. This one gives solid advice and some surprising observations about the tricky subject of comparing floatingpoint numbers. A … Continue reading
Posted in AltDevBlogADay, Floating Point, Programming
Tagged comparing floats, double, epsilon, float, floating point precision, floating point tricks, precision
181 Comments
Don’t Store That in a Float
I promised in my last post to show an example of the importance of knowing how much precision a float has at a particular value. Here goes. As a general rule this type of data should never be stored in … Continue reading
Posted in AltDevBlogADay, Floating Point, Programming
Tagged double, float, floating point, game time, microseconds, precision, time resolution
27 Comments