ETW Central

Over the last few years I’ve written over forty blog posts that discuss ETW/xperf profiling. I’ve done this because it’s one of the best profilers I’ve ever used, and it’s been woefully undersold and under documented by Microsoft. My goal has been to let people know about this tool, make it easier for developers and users to record ETW traces, and to make it as easy as possible for developers to analyze ETW traces.

Some of those posts have aged poorly, and the rest are hidden amongst the 210+ posts (really? wow) I’ve written. The purpose of this page is to be a central hub that links to the ETW/xperf posts that are still relevant. Also, I’ve updated many of the older posts to reflect changes in the ETW toolset (technically known as the Windows Performance Toolkit). For convenience this page is accessible as

The most important post describes how to record ETW traces. This is important because ETW traces can be recorded on one machine, and analyzed on another. This means that your customers or relatives can record ETW traces and then you can analyze them. Remote diagnosis of issues is a wonderful superpower. The article describing how to record an xperf/ETW trace can be found here – share it with those who have performance problems:

Xperf Basics: Recording a Trace (the ultimate easy way)

Once you’ve got a trace you need to analyze it (or share it with someone who can). The most comprehensive resource I’ve created for learning how to analyze ETW traces is the series of three videos I created in 2014. More information and links to the videos can be found here:

ETW Training Videos Available Now

For more details, or if you don’t want to watch videos, there are many tutorial blog posts available, listed here in rough order of importance:

Doing precise analysis of an ETW trace requires knowing exactly what the many columns in the tables mean. Some of those table columns are documented in these blog posts, updated in 2016 for the latest version of WPA:

ETW investigation write-ups:

Some of my favorite blog posts are those that tell a tale of noticing some software that I use being slow, recording a trace, and figuring out the problem. In most cases this let me come up with a workaround, and in many cases the (ridiculously!) detailed bug reports or the attention the posts drew led to the problems being fixed.

Many of the articles linked to below have not yet been updated. It’s an ongoing process but I think it’s worth publishing this now without waiting for all of the updates to finish.

I’ve categorized the investigations by what product was investigated. And, as a reminder, with the exception of the fractal software investigation all of these are looking at problems in software that I don’t work on.

Visual Studio and VC++ code-gen:

Windows Performance Toolkit (profiling the profiler!):


Windows Live Photo Gallery:

Western Digital driver (initially thought to be Windows):


Other people’s ETW investigation write-ups:

Obsolete posts

Some of the blog posts are now completely obsolete and are listed here only for historical interest:

About brucedawson

I'm a programmer, working for Google, focusing on optimization and reliability. Nothing's more fun than making code run 10x as fast. Unless it's eliminating large numbers of bugs. I also unicycle. And play (ice) hockey. And sled hockey. And juggle. And worry about whether this blog should have been called randomutf-8. 2010s in review tells more:
This entry was posted in uiforetw, xperf and tagged , , , , . Bookmark the permalink.

67 Responses to ETW Central

  1. Pingback: Summarizing Xperf CPU Usage with Flame Graphs | Random ASCII

  2. Pingback: UIforETW – Windows Performance Made Easier | Random ASCII

  3. Pingback: Xperf Analysis Basics | Random ASCII

  4. Pingback: Exporting Arbitrary Data from xperf ETL files | Random ASCII

  5. Pingback: The New WPA Xperf Trace Viewer–New Bugs and Old | Random ASCII

  6. Pingback: WPA–Xperf Trace Analysis Reimagined | Random ASCII

  7. Pingback: Process Tree from an Xperf Trace | Random ASCII

  8. Pingback: Xperf Wait Analysis–Finding Idle Time | Random ASCII

  9. Pingback: Xperf and Visual Studio: The Case of the Breakpoint Hangs | Random ASCII

  10. Pingback: Visual C++ Debug Builds–”Fast Checks” Cause 5x Slowdowns | Random ASCII

  11. Pingback: Visual Studio Single Step Performance Fixes | Random ASCII

  12. Pingback: Make VC++ Compiles Fast Through Parallel Compilation | Random ASCII

  13. Pingback: You Got Your Web Browser in my Compiler! | Random ASCII

  14. Pingback: Self Inflicted Denial of Service in Visual Studio Search | Random ASCII

  15. Pingback: Xperf Symbol Loading Pitfalls | Random ASCII

  16. Pingback: Slow Symbol Loading in Microsoft’s Profiler, Take Two | Random ASCII

  17. Pingback: Windows Slowdown, Investigated and Identified | Random ASCII

  18. Pingback: Windows Slowdown, Investigated, Identified, and Now Fixed | Random ASCII

  19. Pingback: Making VirtualAlloc Arbitrarily Slower | Random ASCII

  20. Pingback: Hidden Costs of Memory Allocation | Random ASCII

  21. Pingback: Fixing another Photo Gallery performance bug | Random ASCII

  22. Pingback: Faster Fractals Through Better Scheduling | Random ASCII

  23. Pingback: PowerPoint Poor Performance Problem | Random ASCII

  24. Pingback: Defective Heat Sinks Causing Garbage Gaming | Random ASCII

  25. Pingback: Thread Naming in Windows: Time for Something Better | Random ASCII

    • brucedawson says:

      I don’t think logman is necessarily better or worse than xperf but I don’t think it can do the things that I do with xperf, so it is of little interest to me. Disclaimer: I’m not a logman expert, but I think this is true.

      TL;DR – UIforETW/xperf/WPA can work miracles and I explain how to do this in this blog. logman? Dunno.

      • Severin Pappadeux says:

        Not that it can do something different, but on my W10 installation it came with the system, C:\Windows\System32\logman.exe. Basically, if you install something on user system, looks like it is much better to run logman as an events watchman for your app

  26. Pingback: Power Wastage On An Idle Laptop | Random ASCII

  27. Hi Bruce, awesome work! Is there a place for RFEs? In particular, it would be great to be able to change the “trace to memory” window as well as specify the length of time before a trace is automatically saved to disk and/or stopped. Thanks!

    • brucedawson says:

      Feel free to create issue suggestions at

      Configuring the memory used and the auto-trace elapsed time has been suggested, but such changes must be made carefully to avoid giving users unnecessary foot-guns. “It just works” must continue to be as true as possible. That is, configuration is reasonable as long as it adds value without undue complexity or user risk.

      • Thanks. The complexity issue I understand but your point of view on risk seems overly cautious. As someone who has programmed in C/C++ for over 25 years, I have no lower extremities left to shoot! In my experience, a high amount of configurability is suitable for all but the most skittish, particularly when the *defaults* just work. In debugging performance and other issues, exploration is part of learning the ropes and I wouldn’t limit configurability unnecessarily in order to keep safe a small percentage of users that would try to do silly things on production systems. Yes I understand that having more configuration options multiplies the bug surface area and maintenance costs but I think that most users willing to experiment with advanced functionality would also be willing to help debug and improve those features, directly or indirectly. EOP (End of Pitch).

        • brucedawson says:

          I don’t disagree. Having it work out-of-the-box is most important. Having the configuration options as uncluttered as possible is also important, since every option that is added makes the remaining options harder to find. But I don’t think we’re really disagreeing in any meaningful way.

          > most users willing to experiment with advanced functionality would
          > also be willing to help debug and improve those features

          Well, the great thing about UIforETW being open source is that they can just grab a copy, change the constants, and party-on-Garth!

  28. Pingback: UIforETW is No Longer a CPU Hog | Random ASCII

  29. Pingback: ETW Flame Graphs Made Easy | Random ASCII

  30. Pingback: WPA Symbol Loading is Much Faster, but Broken for Chrome | Random ASCII

  31. Pingback: Thread Naming in Windows: Time for Something Better | Random ASCII

  32. I think another slowdown is on very large source files. If the function(s) being debugged can be migrated to smaller source files, its debugs faster. This was observed on a single source file that was 20K+ lines long.

  33. Pingback: [Перевод] Профилирование: оптимизация - Новини дня

  34. Pingback: Xperf Basics: Recording a Trace (the ultimate easy way) | Random ASCII

  35. Pingback: Making Windows Slower Part 1: File Access | Random ASCII

  36. Hi Bruce, adore your blog!
    Do you know, is it possible to collect PMInterrupt stacks somehow?
    I’ve found the event in,
    but didn’t manage to collect stacks for it:(

    • brucedawson says:

      I think that ETW handles PMC counters by recording their values during context switches. If so then recording PMC interrupt stacks is meaningless. It’s an idea that’s only applicable if the interrupts fire after a certain number of events. But, I may be misunderstanding. I’d recommend sharing what you’ve tried so others can learn from what did or did not work.

  37. Pingback: Study Notes Weekly No.3(Use odbcconf to load dll & Get-Exports & ETW USB Keylogger) | MottoIN

  38. Pingback: C++ Performance – Do the ski gloves fit?

  39. Pingback: Taskbar Latency and Kernel Calls | Random ASCII – tech blog of Bruce Dawson

  40. Pingback: Taskbar Latency and Kernel Calls (via – Windows Kernel News

  41. Pingback: [Перевод] Почему для открытия меню Windows читает один файл сто тысяч раз? | Терещенко. Просто. Профессионально

  42. Pingback: Почему для открытия меню Windows читает один файл сто тысяч раз? - Дорвеи и Сателлиты

  43. Pingback: 63 Cores Blocked by Seven Instructions | Random ASCII – tech blog of Bruce Dawson

  44. Pingback: [Перевод] 63 ядра заблокированы семью инструкциями – CHEPA website

  45. Pingback: [Перевод] 63 ядра заблокированы семью инструкциями —

  46. Pingback: [Перевод] 63 ядра заблокированы семью инструкциями | Терещенко. Просто. Профессионально

  47. Pingback: O(n^2), again, now in WMI | Random ASCII – tech blog of Bruce Dawson

  48. Pingback: Why does Windows read one file a hundred thousand times to open a menu? / Habr – Designers story

  49. Pingback: What Outranks Thread Priority? | Random ASCII – tech blog of Bruce Dawson

  50. Pingback: CH What Outranks Thread Priority? - CyberHero.TV

  51. Pingback: [Перевод] Расследование: что выше, чем приоритеты потоков в Windows? | Терещенко. Просто. Профессионально

  52. Pingback: Расследование: что выше, чем приоритеты потоков в Windows? / Хабр

  53. Pingback: Awesome C/C++ Learning Resources | Everything That You Need

  54. Pingback: Measuring and Optimizing CPU Performance – Nobl9

  55. Max says:


    Thanks for all your training material for WPA/WPR. I’m trying to learn more about Windows and Services in particular. I’m wondering how you would go about using WPA to find out what triggered a particular service to start? I know how to isolate a service into it’s own svchost.exe and I can zoom into where the service starts and I see that services.exe is the process that starts it, but I don’t know how to go about finding out who/what called upon the services.exe to actually start it up.

    Thanks again for all your material!

  56. Pingback: The Easy Ones – Three Bugs Hiding in the Open | Random ASCII – tech blog of Bruce Dawson

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.