If your Windows computer is running slowly – if a program takes a long time to launch, if a game has a poor frame rate, or if an idle application uses too much CPU time – the best way to investigate is to record an Event Tracing for Windows (ETW) trace. An ETW trace records a wealth of information (CPU sampling, context switches, disk I/O, custom data, and much more) that allows most performance problems to be understood by a trained expert. If you’re not a trained expert then you can still record an ETW trace, and then share it with somebody who is.
If a particular program is being slow or inefficient then you may be able to record an ETW trace and share it with the authors of that program. Quite often they can figure out what is going wrong, whether it is a bug on their side or an overheating CPU on your side. Tell them I sent you. They may be grateful for receiving an actionable report instead of vague complaints about slowness which they cannot reproduce.
Not all developers are equipped to analyze ETW traces, for technical and practical reasons – ask first.
Recording and sharing ETW traces has never been easier. Here are the three steps.
Recording ETW traces
If you prefer to learn through watching instead of reading you can view this brief video that demonstrates installing UIforETW and recording a trace. For written instructions, read on:
- Get the latest release of UIforETW. This is an open source tool for recording and managing ETW traces. It makes recording traces easier while adding additional information such as input events to make analysis easier. Download etwpackage.zip, extract the contents, and run etwpackage\bin\UIforETW.exe. This will install the necessary versions of the Windows Performance Toolkit. Wait for the installations to finish.
- Now click Start Tracing. ETW tracing will begin. By default it goes to in-memory circular buffers and can be left running indefinitely, recording the last 10-60 seconds (actual duration varies) of activity. When you have reproduced the slowdown type Ctrl+Win+C from wherever you are (you don’t need to switch to UIforETW) to save the trace buffers to disk. You should enter a description of what happened in the Trace information field associated with your trace. Detailed descriptions are ideal, as they tell the analyst what the problem is and where in the trace it occurred.
- Right-click on the list of traces and select Browse folder to open the documents\etwtraces folder containing the traces. There will be a .etl file and a .txt file for each trace. Upload them to your favorite file-sharing service to share with someone who can analyze the traces.
Be aware, however, that ETW traces can contain personal information. ETW traces record information about all processes on your system. Typically this include the names of files being read and written, so an analyst may be able to tell what document you were editing, or what music you were listening to. However the traces will not include the contents of the files or the names of files that are on-disk but not referenced. The Input tracing information is very important for a successful analysis but it defaults to Private mode, where all letters are recorded as ‘A’ and all numbers are recorded as ‘1’, to avoid being a key-logger. Full mode input tracing can be useful, but enable it with caution, for obvious reasons. And, be thoughtful about who you share ETW traces with.
ETW traces also include full information about your hardware, and version numbers of any software that is running when the trace is recorded
That’s it. That’s all it takes.
Extra bonus steps
- If you install Intel’s Power Gadget (and launch UIforETW after the Power Gadget install completes) then additional information about CPU frequencies, power draw, and temperature will be recorded. Sometimes this is vital, and other times it doesn’t matter.
- If you are reporting problems in Chrome then, starting with version 46 (beta, as of September 2015), it is possible to get some of Chrome’s tracing events to show up in ETW traces. This may include additional information such as URLs from any of your tabs so be mindful about the privacy implications. To use this feature you have to enable it in Chrome by going to chrome://flags/, searching for “trace-export”, enabling “Enable exporting of tracing events to ETW”, and then relaunching Chrome. You can then select which Chrome tracing events are exported by selecting categories from UIforETW’s Settings dialog. This feature is best used in cooperation with a Chrome developer who is investigating your issue and can recommend categories. https://crbug.com is the best place to start these discussions. Note that in many cases a Chrome trace may be a better option than an ETW trace.
Recording great traces
Some traces are better than others. If your description of your problem is vague, or if your trace doesn’t capture the critical moment, then the analyst may not be able to identify your problem. Here is an example of a bad description:
your program sucks and its always slow why cant u make it work better bruce said send a trace lol
And here is a good description, from a trace that was well recorded:
I clicked the foo-bar widget and it took about ten seconds to update during which time WizzyFuzz was hung. I saved the trace about two seconds after WizzyFuzz started responding again. This hang happens about 10% of the time when I click the foo-bar widget. I’m using the default settings and fuzzing a two-hundred cubit Wizzy.
You don’t, however, have to describe your hardware. ETW traces already contain this information.
For the last four years I have been writing about how to record and analyze ETW traces. I may be obsessed.
My first attempt used a series of batch files to call xperf. It worked, and it helped lots of people record traces without having to learn the peculiar xperf syntax for recording traces. But batch files are horrible. This was not good.
Then Microsoft released wprui. This was a point-and-click UI for recording trace. Wprui was a huge improvement, but it was not easily extensible. I briefly promoted it but it is ultimately missing too many features that I need.
Then I decided to write my own UI. The first versions of UIforETW worked well and had the features that I needed. UIforETW also worked around a few xperf bugs, but the initial versions were too hard to use. Initially you had to build UIforETW from source, and you had to track down the Windows Performance Toolkit installers yourself. Who has time for that? The most recent UIforEW releases, from v1.11 and beyond, are turnkey. They include pre-built binaries and the WPT installers. So, if you just need to record or analyze a trace, grab the latest release and you are set.
Analyzing ETW traces
If you are given the job of analyzing one of these ETW traces look for blog posts in the xperf category. In particular, look at the ETW Training Videos I created to help new analysts get started – just ignore the information on recording traces.