Tag Archives: precision

Floating Point in the Browser, Part 3: When x+y=x (y != 0)

Posted on October 18, 2020 by brucedawson

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading →

Posted in Chromium, Computers and Internet, Floating Point | Tagged , | 5 Comments

Floating Point in the Browser, Part 2: Bad Epsilon

Posted on October 11, 2020 by brucedawson

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading →

Posted in Chromium, Computers and Internet, Floating Point | Tagged , | 2 Comments

Floating Point in the Browser, Part 1: Impossible Expectations

Posted on September 27, 2020 by brucedawson

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading →

Posted in Chromium, Computers and Internet, Floating Point | Tagged , | 21 Comments

Doubles are not floats, so don’t compare them

Posted on June 26, 2012 by brucedawson

I’ve seen a few online discussions linking to my Comparing Floating Point Numbers page for misguided reasons and I wanted to discuss those reasons to help people understand why throwing epsilons at the problem without understanding the situation is a … Continue reading →

Posted in Floating Point, Programming | Tagged , , , , | 24 Comments

Floating-point complexities

Posted on April 5, 2012 by brucedawson

Binary floating-point math is complex and subtle. I’ve collected here a few of my favorite oddball facts about IEEE floating-point math, based on the articles so far in my floating-point series. The focus in this list is on float but … Continue reading →

Posted in AltDevBlogADay, Floating Point, Programming | Tagged , , , , , , , , | 15 Comments

Intermediate Floating-Point Precision

Posted on March 21, 2012 by brucedawson

Riddle me this Batman: how much precision are these calculations evaluated at? If you answered ‘double’ and ‘float’ then you score one point for youthful idealism, but zero points for correctness. The correct answer, for zero-idealism points and forty two … Continue reading →

Posted in AltDevBlogADay, Floating Point, Programming, Visual Studio | Tagged , , , , , | 43 Comments

C++ 11 std::async for Fast Float Format Finding

Posted on March 11, 2012 by brucedawson

After a recent post on float precision there was some debate about round-tripping of floats. My claim was that if you print a float with printf(“%1.8e”, f); and then scan it back in then you are guaranteed to get back … Continue reading →

Posted in Floating Point, Programming, Visual Studio | Tagged , , , | 24 Comments

Float Precision–From Zero to 100+ Digits

Posted on March 8, 2012 by brucedawson

How much precision does a float have? It depends on the float, and it depends on what you mean by precision. Typical reasonable answers range from 6-9 decimal digits, but it turns out that you can make a case for … Continue reading →

Posted in AltDevBlogADay, Floating Point, Programming | Tagged , , , , , | 52 Comments

Comparing Floating Point Numbers, 2012 Edition

Posted on February 25, 2012 by brucedawson

This post is a more carefully thought out and peer reviewed version of a floating-point comparison article I wrote many years ago. This one gives solid advice and some surprising observations about the tricky subject of comparing floating-point numbers. A … Continue reading →

Posted in AltDevBlogADay, Floating Point, Programming | Tagged , , , , , , | 181 Comments

Don’t Store That in a Float

Posted on February 13, 2012 by brucedawson

I promised in my last post to show an example of the importance of knowing how much precision a float has at a particular value. Here goes. As a general rule this type of data should never be stored in … Continue reading →

Posted in AltDevBlogADay, Floating Point, Programming | Tagged , , , , , , | 27 Comments