Security Briefs

Events in Windows Vista

Keith Brown

Contents

Events and XML
The Event Manifest
Templates and Events
Building and Registering Your Resource DLL
Generating Events at Run Time
Summary

In last month’s column, I took a closer look at the eventing system that’s been around since Windows NT®, showing how you should be using it via the Microsoft® .NET Framework 2.0. In this column, I’ll show you the new eventing system, Windows® Eventing 6.0. You’ll be happy to know that you can get started experimenting with it today, since Windows Vista™ has this feature!

Figure 1 shows the new Event Viewer in Windows Vista. One of the most obvious changes from the previous version is the number of logs. If you look under the Windows Logs folder, you’ll see the classic event logs: Application, Security, System, as well as some newer logs such as the Setup log, which is designed for application setup programs. You’ll also see a folder for logs created specifically for particular applications and services. It may not look like there are many logs here, but if you open up the Microsoft subfolder, then open up the Windows subfolder, you’ll find well over 50 logs that are specific to various components of the Windows Vista operating system. For example, the Backup service has its own log, the WinLogon service has its own log, and so on.

Figure 1** Windows Vista Event Viewer **(Click the image for a larger view)

With so many logs, I initially wondered how an administrator would be able to find anything. But after spending some time looking at the Overview and Summary page, I realized that this new Event Viewer allows you to gather events that aren’t necessarily all from the same log into a single view. By expanding the list of Error events, I see events that came from the Application, System, and other more specific logs, such as the one for Virtual Server.

The old Event Viewer allowed you to filter a log based on criteria such as event level (Information, Error, and so on) and event source (sometimes referred to as an event provider). The new Event Viewer allows for similar filtering, but also allows you to create saved views, each of which can aggregate events from as many logs as you want. Application developers can now ship their own custom views along with their applications to help administrators find relevant messages. For example, in the folder called Custom Views I’ve created a view of all Error events across all logs that have occurred in the last hour. I also created a second custom view that finds all administrative or operational events in any log that comes from the source in the sample code for this application.

Events and XML

When you create a filter for a custom view, you’re actually creating a series of XPath queries that the eventing system will use to select the events for the view. Below is the view definition for one of my custom views. I’ve added some white space and shortened some text so it’ll be readable here, but it should give you a flavor for what’s going on.

<QueryList>
  <Query Id=”0” Path=”Application”>
    <Select Path=”Application”>
      *[System[Provider[@Name=’...’]
      and (Level=1 or Level=2)]]
    </Select>
    <Select Path=”MyCompany-...”>
      *[System[Provider[@Name=’MyCompany-...’]
      and (Level=1 or Level=2)]]
    </Select>
  </Query>
</QueryList>

Notice that each Select element has a Path. This indicates which log the XPath query will run against. So right out of the gate, it’s pretty clear that a single view can host events from several logs. Next, look at the query itself. It selects all events that have a System element with a Provider child element that has a particular Name attribute and a Level attribute of 1 or 2. Level is the numeric representation of whether the event is Critical, Error, Information, or some other type.

Wait, you might be thinking, events are represented as XML now? I was pretty excited to see this, because as with many developers, I’ve made a pretty big investment in XML already, and if working with events is as easy as working with XML, I’m a happy camper. Of course one of the first concerns I had when I saw this was the size of the logs. XML isn’t the most compact representation for data. But as you’ll see when I drill a little deeper into the guts of Windows Eventing, the event data is actually stored in a binary format, and is only rendered as XML on demand. And if you read my last column, you learned about message files, which factor out even more redundant data: the strings that make events readable by humans. Those strings often make up the majority of the data for an event, and factoring them out into message files is still an important feature that is used in the new eventing system, both for conserving space in the logs and for localization.

Figure 2 shows what an event looks like in XML. This is a failed logon audit event; apparently I mistyped my password recently. Again, I’ve shortened things a bit for readability, but this will give you a feel for what a typical event looks like.

Figure 2 An Event

<Event xmlns=”https://schemas.microsoft.com...”>
 <System>
   <Provider Name=”...-Auditing” Guid=”...” /> 
   <EventID>4625</EventID> 
   <Version>0</Version> 
   <Level>0</Level> 
   <Task>12544</Task> 
   <Keywords>0x8010000000000000</Keywords> 
   <TimeCreated SystemTime=”2007-01-15...” /> 
   <EventRecordID>22</EventRecordID> 
   <Correlation /> 
   <Execution ProcessID=”588” ThreadID=”5736” /> 
   <Computer>mario</Computer> 
 </System>
 <EventData>
   <Data Name=”SubjectUserSid”>S-1-5-18</Data> 
   <Data Name=”SubjectUserName”>MARIO$</Data> 
   <Data Name=”SubjectDomainName”>WORKGROUP</Data> 
   <Data Name=”SubjectLogonId”>0x3e7</Data> 
   <Data Name=”TargetUserSid”>S-1-0-0</Data> 
   <Data Name=”TargetUserName”>Keith</Data> 
   <Data Name=”TargetDomainName”>MARIO</Data>
   ...
  </EventData>
</Event>

There are two main elements in this event. The first part, System, includes not only information about the event itself, but also the environment in which the event was generated. Note that the process and thread IDs are recorded, along with the computer name (in the Name attribute of the Provider element). The second part, EventData, is where the parameters for the event are located. This is actual data inserted by the process that generated the event—in this case, the Local Security Authority subsystem.

Remember how filters in the Event Viewer use XPath queries to determine which events to show in a view? The query I showed earlier only looked at the System element, but now you can see how a query could drill into the EventData element as well, allowing filters to be much more precise than they ever were before.

Once again, keep in mind that none of the friendly, localized text of the message that you normally associate with an event is actually stored in the event itself, as you can see by examining the event in Figure 2. That text is stored in the provider’s message file. And the events themselves aren’t stored in XML, but rather in binary records. So the XML you’re seeing and the message text for the event was all pulled together, or rendered, when I examined the event through the Event Viewer. You can do the same thing programmatically by calling the EvtRender API.

Now that you’ve gotten a glimpse of what the new eventing system looks like from an administrator’s perspective, let’s drill down and see what it takes to generate these newfangled events for Windows Vista and Windows Server® code-named "Longhorn."

The Event Manifest

A traditional message file adheres to a well-documented format and works well for localizing messages. However, the information that a message file contains is insufficient to describe events, especially with all the new attributes introduced in Windows Vista. Additionally, when reading a .mc file, it is impossible to know which strings correspond to events and which do not.

Fortunately, Windows Vista introduces a new format for defining messages and events. Not only does this new format make it possible to fully describe events and all of their attributes, but as you might expect, it’s based on XML, which means parsing one yourself will be a no-brainer. I can see using this to generate helper classes for eventing like the one I built by hand in my last column. That’s the kind of code you don’t want to be writing by hand anyway!

To define a set of Windows events, you need to create a manifest, which is an XML file that contains a bunch of metadata, including descriptions of the events you plan to generate. Figure 3 shows an example of a manifest that I’ve put together for the sample code that accompanies this column.

Figure 3 Sample Manifest File

<instrumentationManifest 
  xmlns=”https://schemas.microsoft.com/win/2004/08/events”>

  <instrumentation xmlns:xs=”https://www.w3.org/2001/XMLSchema” 
    xmlns:xsi=”https://www.w3.org/2001/XMLSchema-instance” 
    xmlns:win=”https://manifests.microsoft.com/win/2004/08/windows/events”>

    <events>

      <!-- Publisher Info --> 
      <provider name=”MyCompany-MyApp-MyComponent” 
        guid=”{559b3019-e149-441f-b59b-d5ac6b87c7c3}” 
        symbol=”MY_PUBLISHER” resourceFileName=”...\events.dll” 
        messageFileName=”...\events.dll”>

        <channels>

          <!--  import the Application log defined in winmeta.xml --> 
          <importChannel chid=”admin” name=”Application” /> 

          <!--  define my own channels --> 
          <channel chid=”op” 
            name=”MyCompany-MyApp-MyComponent/Operational” 
            type=”Operational” isolation=”Application” enabled=”true”/>
        </channels>

        <!-- Event Templates --> 
        <templates>
          <template tid=”dbConnFailed”>
            <data name=”DatabaseName” inType=”win:UnicodeString” /> 
            <data name=”ConnectionString” 
                  inType=”win:UnicodeString” /> 
            <UserData>
              <DatabaseConnectionFailure xmlns=”https://example.org”>
                <name>%1</name> 
                <connectionString>%2</connectionString> 
              </DatabaseConnectionFailure>
            </UserData>
          </template>
        </templates>

        <!-- All the Events that can be published by this Publisher --> 
        <events>
          <event value=”1001” level=”win:Informational” channel=”op” 
            symbol=”MYEVENT_APP_STARTED” 
            message=”$(string.msgAppStarted)” /> 
          <event value=”1002” level=”win:Informational” channel=”op”    
            symbol=”MYEVENT_APP_STOPPED” 
            message=”$(string.msgAppStopped)” /> 
          <event value=”1003” level=”win:Critical” channel=”admin” 
            symbol=”MYEVENT_DB_CONN_FAILED” template=”dbConnFailed” 
            message=”$(string.msgDatabaseConnectionFailed)” /> 
          <event value=”1004” level=”win:Critical” channel=”op” 
            symbol=”MYEVENT_DB_CONN_FAILED_DETAIL” 
            template=”dbConnFailedDetail” 
            message=”$(string.msgDatabaseConnectionFailed)” /> 
        </events>

      </provider>
    </events>
  </instrumentation>
  <localization>
    <resources culture=”en-US”>
      <stringTable>

      <!-- This is how event data can be used as part of 
           Message String --> 
        <string id=”msgAppStarted” value=”MyApp started.” /> 
        <string id=”msgAppStopped” value=”MyApp stopped.” /> 
        <string id=”msgDatabaseConnectionFailed” value=”Failed to connect 
         to the %1 database using the following connection string: %2. 
         Check to see that the database is available, or update web.config
         with a new connection string in the appSettings section.” /> 

      </stringTable>
    </resources>
  </localization>
</instrumentationManifest>

Let me walk you through the various sections of the manifest. The first section defines a provider, which must be assigned a unique GUID and a name that will be recognizable to an administrator. The standard format for publisher names is [CompanyName]-[ApplicationName]-[ComponentName]. As you can see, there are some attributes on the publisher where you need to supply the full path to the executable file that contains your message definitions. However, the Event Log API will expand environment variables, so you can use strings like "%ProgramFiles%\MyApp\events.dll". Because these paths might vary depending on where you the user chooses to install your program, you will need to modify the manifest in your setup program.

The next section defines a set of channels. Channels are where your application will send events, although at run time you won’t be thinking about channels at all since, in the manifest, each event predefines which channel it should be funneled through. You simply raise the event and the Windows Eventing plumbing will route the event to its designated channel. What’s on the other end of a channel? Most likely it’ll be an event log or a high-performance trace log, depending on the type of channel. As you learn more about Windows Eventing, you may read documentation that blurs the distinction between channels and logs, so don’t let that surprise you. The concept of an event channel is new to Windows, and it may become more architecturally significant in the future.

Note that I referred to the Application channel by importing it. This is the global Application event log that you can see in Figure 1. I then create a new channel, and this will ultimately create a new event log file. This is a lot like the custom event log that I created in my previous column, and it will be used to store operational events that would otherwise clutter up the global application log.

There are four types of channels. Administrative and Operational channels point to traditional event logs and may be serviced by subscriptions and event-forwarding services. The difference between an administrative and operational log is mainly how it’s used. Administrative events should be infrequent and actionable; they should fire when there is a problem that the administrator needs to attend to. Operational events are the more mundane My app started/My app stopped type of events.

The other two types of channels, Analytic and Debug, are high-performance channels that will not be dumped into an event log because it’s assumed that they will generate a boatload of events. These channels are used for a high-performance form of event tracing known as Event Tracing for Windows (ETW). You can start a trace by enabling a channel in the Event Viewer, which will also allow you to view the events in the viewer (you will need to select "Show Analytic and Debug channels" in the View menu). For more information, see "Improve Debugging And Performance Tuning With ETW" from the April 2007 issue of MSDN® Magazine.

Templates and Events

Next in the manifest is the template section. This is where you define the structure of the <EventData> section of your events. By factoring out event structure into a template, you can reuse that template with different events. Take a look at how I defined the dbConnFailed template (see Figure 3). The first thing you’ll see are two data elements. This is like the formal argument list for the event: it lists an ordered set of data that your application is expected to supply when generating the event.

After the formal argument list is an optional section called <UserData>. This is where the manifest ends and your creativity takes over, because whatever XML you put inside the UserData element is what is going to show up in the event’s EventData section. Because this particular template is for a database connection failure, I called my top-level event data tag DatabaseConnectionFail-ure. Note that I also put it into its own XML namespace, which is natural because it’s an extension, not a known part of the schema for manifest files. If you forget to do this in your own templates, the message compiler will stop with an error. Do you see the placeholders where the data goes? %1 and %2 indicate the DatabaseName and ConnectionString arguments, respectively.

Before leaving the templates section, I want to point out that the formal parameter list is strongly typed. I’ve provided a list in Figure 4 of types that are recognized out of the box. These types and a bunch of other metadata are defined in winmeta.xml, which is a file provided with the Windows SDK for Windows Vista and Windows Server "Longhorn." You can find this file in the Include directory of the SDK, and you’ll need either to copy it into the directory where you’re compiling your manifest or to use the –W flag to specify the full path to winmeta.xml, because the message compiler needs it to process a manifest. If you forget to do this, MC.EXE will complain and remind you.

Figure 4  Predefined Input Types

Speaking of the message compiler, you’ll need to make sure you’re using the version of MC.EXE that comes with the Windows Vista and Windows Server "Longhorn" SDK. This is easy when you’re experimenting; for now just be sure to run the message compiler from a Windows SDK command prompt as opposed to a Visual Studio command prompt.

Let’s move on to the events section. Here you can see that I’ve defined four events, each of which has a unique numerical value. This is the event ID, and it should be different for each event you define. Then there’s a level that determines how critical the event is. There are a set of predefined levels in winmeta.xml that should be familiar to anyone who has spent time working with the classic Windows event log. Next comes the channel, which refers to one of the channels defined earlier. This defines the initial direction that the event will travel. Features such as event forwarding and subscriptions, which are beyond the scope of this column, may ultimately determine where the event lands, but in most cases, the event will simply be recorded in the log specified in the channel definition, and that will be that (the event is written into the log no matter what; forwarding will use a copy of the event).

Each event also includes a human-readable string that describes it. Note that instead of hardcoding the text into the message itself, I refer to a string table, which you’ll find in the last section of my manifest. This allows the strings to be localized for each culture that your application supports. Also note how my database connection failure message includes placeholders just like the template did: this is how you plug those same parameters into your messages at run time.

Building and Registering Your Resource DLL

If you read my April 2007 column on traditional eventing, you’ll find that building a resource DLL containing your messages is a similar process on Windows Vista. Here’s my batch file that builds the resource DLL for the sample code:

REM buildEventDll.cmd
mc MyManifest.xml
rc /fo events.res MyManifest.rc
link /DLL /NOENTRY /SUBSYSTEM:WINDOWS /MACHINE:X86 events.res

As a side effect of running the message compiler, a C header file will be created with all of the symbols you specified in your manifest. You’ll see some of these symbols being used in Figure 5.

Figure 5 Generating Events

#include <windows.h>
#include <winevt.h>
#include <evntprov.h>
# pragma comment(lib, “wevtapi.lib”) 

// this header file generated by MC.EXE
#include “MyManifest.h”

void wmain() {
    REGHANDLE pub;
    EventRegister(&MY_PUBLISHER, 0, 0, &pub);
    EventWrite(pub, &MYEVENT_APP_STARTED, 0, 0);
    EventWrite(pub, &MYEVENT_APP_STOPPED, 0, 0);
    EventUnregister(pub);
}

Once the DLL is built, make sure your manifest points to it properly (recall the publisher attributes that have full paths pointing to the resource DLL). Then you can run a new tool called wevtutil.exe to register your application with the new eventing system. Do this from a Windows SDK command prompt:

wevtutil im MyManifest.xml

You can also unregister your application:

wevtutil um MyManifest.xml

One of the things this tool will do when you install a manifest is write some registry keys that indicate the path to your resource DLL. As of this writing, if those paths are not correct, wevtutil.exe won’t complain, but you won’t get the behavior you desire. Side effects of incorrect registration can include error events in the log or correct events but no messages.

Once you’ve registered your manifest, all of that metadata becomes available to an administrator, even before you generate your first event. From what I’ve seen on Windows Vista, custom event logs that you create by defining new channels don’t show up until you actually generate an event that targets that particular channel.

Generating Events at Run Time

Currently there are no wrappers in the .NET Framework for the new Windows Eventing APIs, so I’ll be showing some C++ code here. Bear with me; the folks at Microsoft are busy building some sample wrappers that you can use, and the next release of the .NET Framework (code-named "Orcas") will expose this functionality directly!

There are only a few APIs you need to know to get started generating events, and the short code snippet in Figure 5 shows a simple example of them in action. The sample code available for this column on the MSDN Magazine site shows how to go one step further and pass arguments for your events, as well as check for errors returned by these APIs.

The first line of code wires you up to the eventing system and the metadata that you registered earlier using wevtutil.exe. Before your application shuts down, be sure to unregister it. This code is really easy to write; you’ll find the majority of your time will be spent crafting the manifest, which is as it should be!

Summary

What about applications that use the old techniques for eventing? Will they work on Windows Vista and Windows Server "Longhorn?" Yes, they will. They won’t have nearly as much control over the structure of their events, but all those events you were writing to the Application event log will show up just fine in the new global Application event log, and all the traditional eventing APIs and .NET classes will still work on those platforms.

If you’re writing new code that’s targeting Windows Server "Longhorn," you should seriously consider using the new APIs and getting the full benefit of Windows Eventing 6.0!

Send your questions or comments for Keith at  briefs@microsoft.com.

Keith Brownis a cofounder of Pluralsight, a premier Microsoft .NET training provider. Keith is the author of Pluralsight’s Applied .NET Security course, as well as several books, including the .NET Developer’s Guide to Windows Security, which is available in print and on the Web.