Xperf Basics: Recording a Trace (the ultimate easy way)

This post can be found through https://tinyurl.com/etwtracing, and an alternate version of these instructions can be found here.

If your Windows computer is running slowly – if a program takes a long time to launch, if a game has a poor frame rate, or if an idle application uses too much CPU time – the best way to investigate is to record an Event Tracing for Windows (ETW) trace. An ETW trace records a wealth of information (CPU sampling, context switches, disk I/O, custom data, and much more) that allows most performance problems to be understood by a trained expert. If you want to analyze traces, go here. If you’re not a trained expert then you can still record an ETW trace, and then share it with somebody who is.

If a particular program is being slow or inefficient then you may be able to record an ETW trace and share it with the authors of that program. Quite often they can figure out what is going wrong, whether it is a bug on their side or an overheating CPU on your side. Tell them I sent you. They may be grateful for receiving an actionable report instead of vague complaints about slowness which they cannot reproduce.

Not all developers are equipped to analyze ETW traces, for technical and practical reasons – ask first.

Recording and sharing ETW traces has never been easier. Here are the three steps.

Recording ETW traces

If you prefer to learn through watching instead of reading you can view this brief video that demonstrates installing UIforETW and recording a trace. For written instructions, read on:

1. Get the latest release of UIforETW. This is an open source tool for recording and managing ETW traces. It makes recording traces easier while adding additional information such as input events to make analysis easier. Download etwpackage.zip, extract the contents, and run etwpackage\bin\UIforETW.exe. This will install the necessary versions of the Windows Performance Toolkit. Wait for the installations to finish.
2. Now click Start Tracing. ETW tracing will begin. By default it goes to in-memory circular buffers and can be left running indefinitely, recording the last 10-60 seconds (actual duration varies) of activity. When you have reproduced the slowdown type Ctrl+Win+R from wherever you are (you don’t need to switch to UIforETW) to save the trace buffers to disk. You should enter a description of what happened in the Trace information field associated with your trace. Detailed descriptions are ideal, as they tell the analyst what the problem is and where in the trace it occurred.
3. Right-click on the list of traces and select Browse folder to open the documents\etwtraces folder containing the traces. There will be a .etl file and a .txt file for each trace. Upload them to your favorite file-sharing service to share with someone who can analyze the traces.

Be aware, however, that ETW traces can contain personal information. ETW traces record information about all processes on your system. Typically this include the names of files being read and written, so an analyst may be able to tell what document you were editing, or what music you were listening to. However the traces will not include the contents of the files or the names of files that are on-disk but not referenced. The Input tracing information is very important for a successful analysis but it defaults to Private mode, where all letters are recorded as ‘A’ and all numbers are recorded as ‘1’, to avoid being a key-logger. Full mode input tracing can be useful, but enable it with caution, for obvious reasons. And, be thoughtful about who you share ETW traces with.

ETW traces also include full information about your hardware, and version numbers of any software that is running when the trace is recorded

That’s it. That’s all it takes.

Extra bonus steps

• If you install Intel’s Power Gadget (and launch UIforETW after the Power Gadget install completes) then additional information about CPU frequencies, power draw, and temperature will be recorded. Sometimes this is vital, other times it doesn’t matter, and sometimes it just causes crashes (thanks Intel).
• If you are reporting problems in Chrome or Edge then, starting with version 74, it is possible to get some browser tracing events to show up in ETW traces. This may include additional information such as URLs from any of your tabs so be mindful about the privacy implications. To use this feature you select which Chrome tracing events are exported by selecting categories from UIforETW’s Settings dialog – the “latency” category is a good first choice. This feature is best used in cooperation with a Chrome developer who is investigating your issue and can recommend categories. https://crbug.com is the best place to start these discussions. Note that in many cases a Chrome trace may be a better option than an ETW trace.
• Starting with version ~91 (when v8_enable_system_instrumentation=true was turned on by default) you can get function names for JavaScript code in Chrome and Edge. You need to run the browser with –js-flags=”–enable-system-instrumentation” and then check Trace JavaScript in the UIforETW Settings UI (not yet in a released version).

Recording great traces

Some traces are better than others. If your description of your problem is vague, or if your trace doesn’t capture the critical moment, then the analyst may not be able to identify your problem. Here is an example of a bad description:

your program sucks and its always slow why cant u make it work better bruce said send a trace lol

And here is a good description, from a trace that was well recorded:

I started tracing in UIforETW and then clicked the foo-bar widget and it took about ten seconds to update during which time WizzyFuzz was hung. I saved the trace about two seconds after WizzyFuzz started responding again. This hang happens about 10% of the time when I click the foo-bar widget. I’m using the default settings and fuzzing a two-hundred cubit Wizzy.

You don’t, however, have to describe your hardware. ETW traces already contain this information.

Analyzing ETW traces

If you are given the job of analyzing one of these ETW traces look for blog posts in the xperf category. In particular, look at the ETW Training Videos I created to help new analysts get started – just ignore the information on recording traces – or go to https://tinyurl.com/etwcentral.