TL;DR – the phrase “90% faster” is interpreted by people in two different ways:
- 90% less time
- The speed (ops/sec or km/h) is 90% greater
I strongly prefer #2 but due to the ambiguity this phrase should never be used. The preferred unambiguous phrase should be “1.9 times as fast” (or “ten times as fast”, if that is what you were trying to say). To do otherwise is to risk being misunderstood by a significant portion of your audience.
Describing performance improvements exists at the intersection of mathematics and linguistics. It is quite common to use incorrect math to describe performance improvements, and it is possible to use incorrect, misleading, or just sub-optimal rhetoric to describe your math.
Consider this hypothetical press release:
AirTrain Inc. is proud to announce the new AirTrain-8000. This revolutionary new plane can fly from London to Seattle at an average speed of 7,700 km/h – a huge improvement over the 770 km/h of other jets. This drops the travel time from ten hours to just one, making the AirTrain-8000 90% faster than our competitors.
A press release like this would never be released. The new plane is ten times as fast as previous planes (7,700 km/h divided by 770 km/h), and no marketing team should allow this improvement to be summarized as “90% faster”, which to many people means “almost twice as fast.” And yet, when talking about computers – where ten-times speedups happen fairly often – this mistake is made quite frequently.
This abuse of percentages has made them meaningless for describing optimizations – we need to stop using them. The AirTrain-8000 is ten times as fast, full stop.
In the AirTrain example above the elapsed time for the journey was reduced by 90%. It would be reasonable for the press release to say that. It would be accurate (but uselessly ambiguous) to call this plane “90% better.” But saying that a plane that goes 7,700 km/h instead of 770 km/h is “90% faster” means that many readers will interpret the new plane as being 1.9 times as fast. To my mind “90% faster” means that the speed of the new plane is 1.9 times greater (isn’t that what faster means?) but to some other smart people “90% faster” means “90% less elapsed time.” That interpretation mystifies me, but cannot be ignored.
You sit on a throne of lies
It’s quite easy to find examples of this rhetorical error. A google search for “90% faster” (with the quotes) finds many articles talking about reductions in elapsed time of 90% or more, where the new process is actually at least 900% faster. Here are just a few examples:
- 90% Faster Game Loads? – more than ten times as fast
- Get a business education 90% faster – more than thirteen times as fast
- 90% Faster Processing – fifteen times as fast
- Finds Files 90% Faster! – ten times as fast
- 67% faster than HTTP – 3.07 times as fast
I could go on.
My bold claim
If you optimize a task so that it takes 90% less time than before then that is a ten-times speedup, and it should be described as such. It doesn’t matter whether the task is flying a plane from London to Seattle, loading a game, or finding a file. A 90% reduction in time is a ten-times speedup, period, and the five examples above are all incorrect, or at least poor communication. Why would you want to describe your achievement in a way that makes it sound less significant to many people?
What 90% faster means
Let’s imagine that I ride my scooter at 10 m/s and a friend rides theirs at 19 m/s. Going 19 m/s instead of 10 m/s is, mathematically, 1.9-times the speed. We convert that to a percentage increase by going:
100% * 1.9 – 100% = 90%
Therefore their speed is 90% greater than mine. Another way to say 90% more speed is to say 90% faster.
Aside: the subtracting of 100% is because, by convention, a percentage change always refers to just the change, whereas in “1.9-times speedup” the “1.9” is a ratio of the two speeds. I don’t make the rules, but that is the linguistic/mathematical convention. Some people disagree and think that 1.9-times the speed should be called 190% faster but that is non-standard.
What 90% faster doesn’t mean
If I do some task in 100 s and a friend does it in 10 s then my friend is doing it in 90% less time. And many people seem content to say that in this situation my friend is 90% faster than I. But wait. What if the task that we are doing is riding our scooters 1,000 m to work. In that case I am riding at 10 m/s and my friend (propelled by a jet pack?) is riding at 100 m/s.
It seems to make little sense to use the term “90% faster” to mean “going 19 m/s instead of 10 m/s” and also to mean “going 100 m/s instead of 10 m/s”. It can’t mean both, and this second usage of 90% faster is wrong – in this case my friend is actually 900% faster. Some people call it 1,000% faster (see previous section), which is unfortunate.
Let’s look at one concrete example. In this article a gamer describes a technique they have found for getting a game to load in 50 s instead of 510 s. They then describe that as “90% Faster Game Loads?”.
This confusion seems to happen whenever an improvement is a reduction in how long it takes to do a task, so let’s go through the math.
Before: the game loaded in 510 s which means 0.00196 game loads per second. Alternately, you can load this game 7.06 times per hour.
After: with the suggested hack in use the game loads in 50 s, which means you can load the game 0.02 times per second, 72 times per hour.
The faster method works out to be a bit more than ten times the game loads per time period, regardless of what time period is chosen. It is 900% faster, not 90% faster.
The game loading example might “feel” weird because nobody is going to repeatedly load the game for an hour and be pleased that they could load it 72 times instead of 7.06 times. But that doesn’t change the fact that game loading is more than ten times as fast. Skydivers can freefall at about ten times the speed of casual cyclists, but we don’t discount this accomplishment just because they can only do it for a minute or so at a time. They travel about two miles in a minute, which is a speed of 120 miles per hour, and their failure to maintain that speed for an hour does not diminish that accomplishment.
If we have an old speed and a new speed then the speedup ratio can be calculated as:
speedup ratio = x = NewSpeed / OldSpeed
We can then say that the new speed is x times as fast as the old speed, ten times as fast in the case of the AirTrain-8000.
If we have old and new elapsed times for a fixed task then translating a reduction in time into an “x times as fast” sentence is just as easy. It’s as simple as
speedup ratio = x = OldTime / NewTime
It’s just as simple as calculating the speedup ratio from speed numbers, although it is old/new instead of new/old. This calculation works for any sort of fixed task that you speed up. You can work out this formula by calculating new and old speed with this formula:
speed = tasks/s = NumTasks / ElapsedTimeInSeconds
and then inserting the two speeds into the NewSpeed / OldSpeed formula until NumTasks cancels out. Simple algebra.
I originally wanted to recommend that people use “90% speedup” when they make something 1.9 times as fast. But my research made me realize that this is a terrible idea – the multiple incorrect usages are too pervasive and you simply can’t count on people understanding what you mean, with the possible exception of small percentages like “20% faster” where the ambiguity is modest.
Therefore, when you reduce how long a task takes – any task – you should calculate the speedup factor as NewSpeed / OldSpeed and use that speedup to describe your achievement. You should also share your before/after numbers so readers can check your math.
When you hear something described as a 90% to 99.9% speedup you should be wary. Unless the before and after data is shared you really have no idea what was achieved. It usually means “I’m good at optimizing but poor at communication.”
You should never describe something as being a “90% improvement” or “90% better” – these phrases are meaningless. Instead, embrace the big and accurate 10-times as fast number. It’s 10 times better!
Putting my money where my mouth is, when I reduced the render times for a particular fractal zoom movie from six months to 18 hours I didn’t call it 99.6% faster, I called it 240 times as fast. I’m glad that I preemptively followed my own advice.
Twitter discussion is here, reddit discussion is here.
Update: it was pointed out by a reader that “two times faster” and “two times as fast” mean different things. “two times faster” refers to the increase in speed and means that the speed ratio is 3:1, while “two times as fast” means that the speed ratio is 2:1. I’ve updated the article for clarity. At the very least “two times faster” is ambiguous and subject to misinterpretation.
Hear, hear! It’s not something I get worked up about, but I’m glad someone’s pointed it out.
I’m completely in agreement. Percentages are misunderstood, by both writers and readers. I try to avoid them and just use factors (e.g., “1.5x faster”) and share actual measurements. For speedups, the only place percentages make sense are in the “more than 1x, less than 2x” range, as in “30% faster” in place of “1.3x faster”. Above 2x they get confusing (3x faster is “200% faster”, and parsing things like “1700% faster” shouldn’t be necessary). I think reduction in something is harder to describe. Sometimes percentages make more sense (e.g., “30% fewer crashes” versus “0.7x as many crashes”). I hate it when people use “times” for reductions, as in “3 times less crashes” — three times what? Maybe they mean one third as many crashes, but who knows? A related issue is describing something (for example video encoding) relative to realtime. If something is “3x realtime” does that mean it takes one third the time (e.g, 20 minutes to encode an hours worth of content), or does it mean it takes three times the time (e.g., 3 hours to encode an hours worth of content)? Be specific or you risk having your reader think the opposite.
If you’re reporting a speedup as a ratio, then how about plotting it using a logarithmic scale? https://johnwickerson.wordpress.com/2016/08/06/ratios-on-logarithmic-scales/
That way, it’s very easy to spot if a speedup or slowdown has occurred.
YES YES YES!!! This is such a common mistake, it blights discourse about all things numerical. I’ve heard people who claim to be ECONOMISTS make exactly this error. I have entirely given up the use of percentages to describe change and only use them for describing portions of things, e.g. “that’s 80% of our income”, “that’s 140% of the original investment”. You’re on a road to ambiguity when you say things like “that’s 80% smaller” or “that’s 140% faster”.
Thanks for writing this. Now I have somewhere to point people to…
It makes me sad to have to give up on saying things like “50% faster” because I know what I mean and I am strongly believe that my interpretation is correct, but… language is about communication and if a significant percentage (!!!) of people will misinterpret or be unsure of my meaning then I must phrase things differently. 1.5 times faster it is.
Surely you mean 0.5 times faster?! When talking about a change, don’t you subtract one (just as you would subtract 100% if you were expressing it as a percentage)? 😉
Also, your press release didn’t use consistent units: The new plane goes 7700 km/h, but the old ones go 770 km. That is, the old ones have a distance limit rather than a speed limit. 😉
Good catches. I’ve fixed them both. The km versus km/h was just sloppy. The “x times faster” versus “x times as fast” issue caught me unawares. Apparently I am guilty of one of the sins of which I have accused others. I think that using “0.5 times faster” to mean “1.5 times as fast” is *wicked* confusing so I have resolved to always use the “x times as fast” phrasing in the future.
Your feedback has made this blog post up to 50% better, or more!
Rants are so much better when they have this much rigor.
How about using the term “order of magnitude”? While phraseology might sometimes make it unclear whether it’s a precise or imprecise measure, it has the advantage that it’s commutative. If X is an order of magnitude faster than Y, Y is an order of magnitude faster than X.
I assume that was supposed to end with “order of magnitude slower than X.”?
I do use that phrasing for huge changes. For instance, if something is a thousand times as fast I’ll say it’s ten orders of magnitude faster (base-2 orders of magnitude make all of your optimizations seem better).
If the new plain is 10 times faster (or 11 times as fast) its speed is 90% of the former but 1100%.
Where the hell is the confusion?
You are making your point too often in this blog post.
You’re not wrong, and it used to be even more redundant. My only comment in my defense is that the more I talked to people about this the more conflicting-and-contradictory viewpoints I found. I felt forced to refute some beliefs that you may not have encountered.
That is, the whole thing seems unnecessary, and yet, my search for “90% faster” found more misuses than correct uses.
What baffle me is most of these shouldn’t even be a problem in the first place.
As pointed out “two times faster” and “two times as fast” is another worrying example.
Seriously, what went wrong? It is not so much about English or Maths, but basic logic error.
Personally, anything that has to do with time, it is much better to write something like X times faster, the bigger the number, the better.
However for memory, storage space, or anything that has less to do with time, % reduction makes much more sense. E.g 80% less memory being used.
Instead of “X times faster” I would recommend “X times as fast” to avoid the 1.5/0.5 ambiguity. I agree that the logic used in some of these cases is surprising, but “human languages” and “precise statements” are often in opposition.
There is also a problem around language regarding improvement and degradation when you are referring to absolute quantities of time (not rates). I’m a performance engineer and I’ve tried to lay down the law at work regarding this; it isn’t easy. Suppose a benchmark normally takes 5s but now we experience a regression and the test takes 10s. One can say:
* The test takes “2x (or twice) as long”, which most people would understand as definitely bad.
* The test is “half as fast”, definitely nonstandard and just weird.
* The test is “2x slower”, technically wrong but most people would understand as bad (it’s only 1x “slower”). Using “2x (or twice) as slow” would probably work also but the word “slow” sort of implies that the normal execution time is not good either!
* We have a performance degradation of 100%. Yikes. It’s technically correct (10s – 5s)/5s x 100%. But as you’ve mentioned, percentages can cause more confusion.
Everything you and others have mentioned also applies to performance improvements, with one caveat that is unlikely to cause much worry for most of us: whereas the amount of degradation can increase practically without bound, the amount of improvement can only increase up to our ability to measure it, i.e., infinity is a lot further away than zero when you deal in micro- or nanoseconds resolution. The reason I say it’s not a problem in practice is that in most cases we can just add more input to make the tests run longer.
Use log scale, then zero is exactly the same distance away as infinity.
I’m not sure whether I’m serious or not…
Great blog Bruce! I learned a lot from it, so thanks for your efforts!
Re the post, recently I find myself avoiding people not capable of interpreting such basic things correctly. “90% faster” => is this really such a big challenge to understand? Come on! What achievements can we expect from such people in life? It’s a shame that a guy of your caliber has to spend time & write a post about it. Such problems should not even exist. Eh, me and my idealism…
What I Talk About When I Talk About Running is on my read list!
Pingback: A Real-World Benchmark of WebAssembly vs. ES6 - GistTree