Windows software trace preprocessor

The Windows software trace preprocessor (WPP; the preprocessor and related support tools are known as WPP Software Tracing) is a preprocessor that simplifies the use of WMI event tracing to implement efficient software tracing in drivers and applications that target Windows 2000 and later operating systems. WPP was created by Microsoft and is included in the Windows DDK. Although WPP is wide in its applicability, it is not included in the Windows SDK, and therefore is primarily used for drivers and driver support software produced by software vendors that purchase the Windows DDK.

Background

Software tracing is a specialized use of logging to record information about a program's execution. This information is commonly used for debugging. In contrast to event logging, the primary purpose of which is to produce records of events that can be audited by system administrators (see for example Event Viewer) or analyzed by management tools, software tracing is primarily a debugging aid for software developers. As such, many of the non-functional requirements of event logging, such as localizability or a standards-based output format, are explicitly non-goals for most applications of software tracing. On the other hand, software tracing has special requirements for performance that are not generally as important in event logging. For example, one common use of software tracing, in/out tracing, produces output at the entry point and return of functions or methods so that a developer can visually follow the execution path, often including parameters and return values, in a debugger or text-based log file (this can be seen as a run-time analog of a sequence diagram). This type of tracing, although useful for developers, can greatly hurt performance of a software product if it cannot be disabled (either at compile-time via conditional compilation, or at run-time via flags).

Additional considerations special to software tracing include the following:

• In proprietary software, tracing data may include sensitive information about the product's source code.
• If tracing is enabled or disabled at run-time, many methods of tracing require a significant amount of additional data be included in the binary, which can indirectly hurt performance even when tracing is disabled.
• If tracing is enabled or disabled at compile-time, getting trace data for an issue on a customer machine depends on the customer being willing and able to install a special, tracing enabled, version of your software.
• Certain types of software, such as drivers, need to meet strict performance requirements even with tracing enabled.

Due to the first two considerations, traditional methods of software tracing use conditional compilation to enable or disable tracing (and inclusion of tracing data) at compile-time. For example, using the C preprocessor, one might define a macro DebugOut as follows:

#ifdef _DEBUG
#define DebugOut(msg, ...) \
    DebugPrintf(__FUNCTION__ "(" __FILE__ ":" TO_STRING(__LINE__) ")\t" \
        msg, __VAR_ARGS__)
#else
#define DebugOut(msg, ...)
#endif

where TO_STRING is a macro that converts the line number (__LINE__) to a string and DebugPrintf is a printf-like function that might for example output text to the debugger.

Then, the following code:

DebugOut("Error %d occurred\n", error_code);

would produce output similar to the following on debug builds only:

SomeFunction(file.c:78)    Error 217 occurred

Another technique for certain types of tracing (particularly in/out tracing) is to use instrumentation. While this technique can address many of the major concerns, it is not always available (typically only in managed code).

WMI event tracing is an example of a technology that addresses in particular performance of tracing in performance-critical code such as drivers. It can also address the concern of controlling the distribution of sensitive trace information by letting a developer define the human-readable tracing data ("Error %d occurred\n" in the example above) separately from the code so that it is not built into the product (in the code, a specific message is referred to by its message number). However, there are some important limitations:

• WMI event tracing cannot, by itself, automatically generate the "SomeFunction(file.c:78)" part of the trace message. This is a limitation of all such technologies, not specific to WMI event tracing.
• Requiring the human-readable part of the tracing data to be separated from the code can decrease the readability of the code.
• Using this technique can introduce significant development overhead for "one-shot" tracing messages.

Operation of WPP

WPP is run prior to compilation (in other words, before even the C preprocessor), and generates a trace message header for each file that it processes (by default this header is filename.tmh, where filename is the name of the processed source file). This header must then be explicitly included into the source file, for example:

// File: file.cxx
// This file is an example of using WPP
#include "file.tmh"

WPP's understanding of C/C++ syntax is very limited. In particular, it does not expand macros (except in special circumstances where it is necessary), nor does it handle pragmas or perform any semantic analysis.

A developer specifies one or more tracing macros that WPP should handle, via a configuration file, special annotations in comments, command line parameters, or some combination of these methods. Each time WPP encounters one of the macros that it is supposed to handle, it generates a trace message macro. In other words, if for example DoTrace is a tracing macro, WPP will generate a separate macro for each occurrence of DoTrace. The generated trace message macros are disambiguated by file name and line number, and, using various preprocessor tricks, WPP in turn defines the original tracing macro so that it will expand the appropriate trace message macro at each occurrence.

How trace message macros are generated by WPP depends on a template file (the format of the file is undocumented). The default template files included with WPP specify that the string of a trace message should be included in an annotation (using the __annotation feature of the Microsoft Compiler). These strings are not included in the compiled code, but are included in the debugger symbol file in a format that tools included with WPP can understand. The trace message macros also include the logic for enabling or disabling tracing via flags and the calls to WMI event tracing APIs.

Limitations

• Because WPP doesn't expand macros, it won't recognize an instantiation of a tracing macro that is included in the definition of another macro. For example, if DoTrace is a tracing macro, and a macro CheckForErrors is defined as:

#define CheckForErrors(error_code) \
    if (IsError(error_code)) \
    { \
        DoTrace("Error %d occurred\n", err); \
        HandleError(error_code); \
    }

then WPP will not generate the trace message macros for DoTrace where CheckForErrors occurs. WPP provides an ad hoc workaround for this issue, but there still exists a small class of macros that cannot be expressed even using the workaround.

• The default template file generates code that will only work properly with the Microsoft compiler. Although this is not an inherent limitation of the preprocessor, the fact that the template file (which controls what code is generated in the trace message header) uses an undocumented format means that in practice WPP will only work properly with the Microsoft compiler.
• Earlier versions of WPP caused compiling errors when more than one trace macro header was included into a source file (for example, if a source file with tracing included a header that had tracing in inline functions). This is fixed in the most recent version. Note that this is also a limitation of the template file, not the WPP tool itself.
• Because trace message macros are disambiguated by file and line number, there can be only one tracing macro per line in the source code.

External links

• WPP Software Tracing Reference on MSDN