VC++ /analyze Bug Finder Bug Fixed

Last October I wrote about a crashing bug in the /analyze feature of Microsoft’s VC++ compiler – a use-after-free in their annotation parser.

It’s now fixed.

Last year I switched to using VS 2012 for doing static analysis, because it gives much more accurate warnings than VS 2010 does and it gives fewer false positives. This change definitely improved the quality of the static analysis reports but, as reported previously, the VS 2012 annotation parser had a serious bug that would cause it to crash occasionally. I eventually used App Verifier to show that it was a use-after-free bug (shown in the screen shot to the right). The awesome irony was that the use-after-free bug was in the annotation parser – code whose only purpose is to help find bugs.

(see You’ve Got a Bug in Your Bug (Finder) for more details)

Responsible crashing

When the VC++ compiler crashes it prints an Internal Compiler Error message that looks something like this:

(compiler file ‘msc1ast.cpp’, line 1443)
To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++
Help menu, or open the Technical Support help file for more information
Internal Compiler Error in C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\CL.exe.

That’s okay, I guess, but I have to say that clang handles compiler crashes better. When trying to reproduce a compiler bug the usual steps are to create a preprocessed file and send that in along with the compiler command line. Doing that in Visual Studio is tedious and error prone. Clang creates preprocessed source files and associated run scripts automatically upon a crash, making it easier to report bugs. Nice. Having that feature might have made this bug a lot easier to report. VS team – this is a feature request.

Making it easy to report bugs is particularly important because, as far as I can tell, the VC++ compiler traps crashes and prevents them from going through Windows Error Reporting or being caught in a debugger. This means that submitting crash dumps for VC++ compiler crashes is significantly challenging. Therefore it would also be nice if the compiler would save a minidump when it crashed instead of just trying to sweep the crash under the code-rearrangement rug.

Crash consequences

This bug was causing the compiler to crash about twenty times a day during /analyze runs on our build machines. These crashes were particularly annoying because our /analyze build system is based around reporting new warnings. Whenever the compiler crashed it would miss some warnings, and that would make those warnings appear as new warnings on the next build. I had to add all sorts of hacks to the analysis scripts in order to work around this problem. Roughly speaking, I had to configure our /analyze build system to only count a warning as new if it hadn’t been seen for the last three builds. I look forward to removing this hack.

I blogged about this problem and this caused Microsoft to reach out to me for more information. It took a bit of back and forth but they were eventually able to reproduce the crash by using my test project and App Verifier.

I was pretty pleased when I looked at the release notes for Visual Studio 2012 Update 3 RC 2 and saw, under the C++ category: “An internal compiler error occurs intermittently when you compile by using the “/analyze” option.” I quickly installed the update and since then I have seen no compiler crashes. Hurray!

If you use the VS 2012 version of /analyze (and you should) then you should install this update immediately. The updated version of cl.exe is:

Microsoft (R) C/C++ Optimizing Compiler Version 17.00.60521 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

Note: the final version of Visual Studio 2012 Update 3 shipped June 26, 2013. Get it.

Bonus improvement

Use-after-free can cause crashes, but it can also cause random undefined behavior – there’s no knowing what the state of the freed memory will be. I had long wondered whether the use-after-free in the annotation parser was causing any of the annotations to be incorrectly parsed. Therefore I wasn’t terribly surprised when the first build after installing the fixed compiler found a newly found bug. This bug had been present for as long as /analyze has been running but had never showed up before. Either the /analyze engine was improved in this update, or this bug was previously missed because of the annotation parsing bug.

Test your code with all available tools

Running /analyze for the last two years has convinced me that any responsible software developer should use static analysis. Programmers make mistakes, and static analysis finds many bugs for us every week, some of them quite serious and subtle. /analyze is available for VC++ 2012 professional, so there’s no excuse to not use it.

( see Try /analyze for Free for details on how to get /analyze)

Testing your code with App Verifier (or valgrind and Electric Fence on Linux) should also be a minimum requirement. These tools help find otherwise invisible or hard to find bugs, and not using them just makes your job harder. If Microsoft had routinely tested their compiler with App Verifier then this use-after-free bug would never have shipped.

( see Increased Reliability Through More Crashes for App Verifier details )

Shipping bug free code is not a practical goal for most developers, but using the available bug finding tools can ensure that you don’t ship easily avoidable bugs, thus saving you and your customers time and money. Making it easy for customers to report crashes – by saving minidumps, and automatically creating repro cases – is also recommended.