Category Archives: Debugging

The Easy Ones – Three Bugs Hiding in the Open

I write a lot about investigations into tricky bugs – CPU defects, kernel bugs, transient 4-GB memory allocations – but most bugs are not that esoteric. Sometimes tracking down a bug is as simple as paying attention to server dashboards, … Continue reading

Posted in Bugs, Code analysis, Code Reliability, Debugging, Floating Point, Linux, Performance | Tagged | 23 Comments

GDI leaks and the importance of luck

In May 2019 I was asked to look at a potentially serious Chrome bug. I initially misdiagnosed it as unimportant, thus wasting two valuable weeks, and when I rejoined the investigation it was the number one browser-process crash in Chrome’s … Continue reading

Posted in Bugs, Chromium, Debugging, Programming | Tagged | 24 Comments

Creating a Public Symbol Server, Easily

I’ve been a big fan of symbol servers for years. They are a part of the Microsoft/Windows ecosystem that is far better than anything I have seen for other operating systems. With Microsoft’s and Chrome’s symbol servers configured I can … Continue reading

Posted in Debugging, Programming, Symbols, uiforetw | Tagged | 16 Comments

Compiler bug? Linker bug? Windows Kernel bug.

See the end of the post for an October 2018 bug fix update, or read the whole story: Flaky failures are the worst. In this particular investigation, which spanned twenty months, we suspected hardware failure, compiler bugs, linker bugs, and … Continue reading

Posted in Debugging, Investigative Reporting, Programming | Tagged , , , | 66 Comments

Zombie Processes are Eating your Memory

Zombies probably won’t consume 32 GB of your memory like they did to me, but zombie processes do exist, and I can help you find them and make sure that developers fix them. Tool source link is at the bottom. … Continue reading

Posted in Debugging, Investigative Reporting, Performance, Programming, Rants | Tagged , , | 81 Comments

Finding a CPU Design Bug in the Xbox 360

The recent reveal of Meltdown and Spectre reminded me of the time I found a related design bug in the Xbox 360 CPU – a newly added instruction whose mere existence was dangerous. Back in 2005 I was the Xbox … Continue reading

Posted in Debugging | Tagged , , , , | 76 Comments

Analyzing a Confusing Crash–Stack Walks Gone Bad

Part of my job always seems to include crash analysis. A program crashes on a customer’s machine, a minidump is uploaded to the cloud, and it might be my desk that it appears on when Monday morning rolls around. The … Continue reading

Posted in Debugging | Tagged , | 8 Comments

Everything Old is New Again, and a Compiler Bug

“What’s an EXCEPTION_FLT_STACK_CHECK exception?” one of my coworkers said. I said “It’s a weird and rare crash. Why do you ask?” It turns out that one of these weird and rare crashes had started showing up in Chrome (M54 branch, … Continue reading

Posted in Debugging, Floating Point | Tagged | 28 Comments

Thread Naming in Windows: Time for Something Better

Windows lets you give names to the threads in your process which can then be displayed by debuggers. This is a great idea, but the implementation is showing its age – it’s time for some fixes, or something better. Update, … Continue reading

Posted in Debugging, Programming, Visual Studio | Tagged , , , | 23 Comments