This article may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. To maintain the flow of the article, we've left these URLs in the text, but disabled the links.

Tester, Take Two-TESTREC.EXE Updates Previous Version of the Tester Utility John Robbins Code for this article: Bugslayer0600.exe (256KB)

Recently, someone asked me to talk about a time during the development process when I learn the most. Being a debugger kind of guy, I'm sure they expected me to talk about the times I've worked on the nastiest bugs. While you do learn a bit slogging through the debugger, the real eye-opening learning experiences have occurred when I have had to do a subsequent version of a project.

      When you create the new release, the first thing you have to do, which can be quite instructive, is deal with all the bugs that you left in the product or that surfaced in the field. And sometimes you have to face the fact that what you thought was a wonderful design that would scale infinitely into the future just plain stinks! To me, that epiphany is a true Zen moment. You have to bravely face the mortality of something you created and, most importantly, you have to make it right. What makes the situation even more painful is that you have to admit to your teammates and to your manager that you weren't as smart as you thought you were. While your ego takes a beating, that's the price you pay for enlightenment.
      On the other hand, in creating a subsequent version you can sometimes reach true nirvana and still learn a great deal. Some of my proudest moments have been when I learned that my design was sufficiently flexible and strong enough to extend in ways that I had never originally thought of. While that's wonderful when you are doing the subsequent version work yourself, the ultimate high is when others can pick up your work and easily take it in new directions. That's the moment where the "guru" term starts getting flung around!
      To get the good and the bad, you need to live with your designs for a while. However, that doesn't mean you should live with the mistakes forever. If a design or implementation will not meet your needs for the future, you should consider scheduling time to rewrite the problem area. Whenever I am scheduling a project, I always expect to spend 15-25 percent of the time as reinvestment in the code base. Sometimes the problem areas are obvious, but other times they are not. To find the less obvious areas, I spelunk through the bug tracking system looking for the modules or subsystems that had the highest number of problems.
      So far I have been talking mostly about implementation problems. But a much more difficult determination to make is whether you got your feature set wrong. If you are working on very vertical applications, sometimes you are the only game in town, so your customers don't know any better. Most of us are in competitive situations and can tell by looking at our company's bottom line if we hit the mark. While you might think you have the coolest product since sliced bread, your customers vote with their dollars. If it's too hard for them to get their job done, they might vote for your competitor's product. Please note that I am talking about missing features and functionality, not plain buggy products. I'm assuming you have been reading the ol' Bugslayer column all along and don't suffer from those afflictions any more.
      Let's take a look at an app that's missing features or functionality. Back in the April 1999 issue of Microsoft® Systems Journal, I showed you a cool utility called Tester. Tester allows you to send keystrokes to another application so you can easily develop regression tests that take care of testing for you. Since it is a COM-based system, you can use any programming language you want to write the tests. Unfortunately, it's far too difficult to write those tests. Even I found it harder to use than it should have been.
      In the original column, I mentioned that I needed to break down Tester into two parts: the part that plays back keystrokes and the part that records keystrokes. I meant to write the second half earlier, but other column topics hijacked my attention and sent me off on interesting tangents. After a couple of e-mails telling me that I didn't do the full jobâ€"meaning I was missing featuresâ€"I thought I had better take care of business and make a more feature-complete utility. I had my Zen moment.
      This month, I will finish the second part of Tester with TESTREC.EXE, the recorder utility. With this utility, Tester as a whole is much easier to use and will allow you to whip up some automated regression tests that help speed up your testing considerably. I'll call the new version Tester Two.
      First, I will quickly reintroduce Tester so everyone has an understanding of the first version's capabilities. Then I'll move into the requirements for TESTREC.EXE. Finally, I will discuss some of the implementation highlights. As you'll see, the implementation was much more than I bargained for.

Tester Recap

      Tester is a COM object that offers a couple of interfaces that allow you to get information about and control other windows on a computer. The following were my requirements when building the first version:

• Given an input string of keystrokes formatted the way they would be if you were going to pass them to the Visual Basic® SendKeys function, Tester can play the keystrokes to the active window.
• Tester can find any top-level or child window by title or class.
• Given any arbitrary HWND, Tester can get all the properties of the window.
• Tester must notify the user's script of specific window creation or destruction so it can handle potential error conditions or do advanced window handling.

      I personally like to use Windows® Script Host (WSH) as my scripting environment. Figure 1 shows an example script, which starts Notepad and says a few things. While playing the keys and fiddling with windows are concepts that are easy to grasp, handling arbitrary window creation and destruction is a little more elusive. It's a very nice feature, so I want to make sure that everyone understands it.
      One of the most difficult issues you face when automating an application is when the unexpected happens. If you are doing what you should be doing, you will be using Tester scripts on your debug builds. Since you are also using liberal assertions, you never know when one can pop up. Obviously, you don't want to have to write your script so that you have to check for those assertion message boxes each time you do a particular operation in your script. By having the notification handlers, you can set a handler in the background that is triggered by Tester when a window is created or destroyed and matches certain criteria. That way you have an extra level of safety in your scripts to make sure you are automating the correct windows properly and not accidentally telling Windows Explorer to delete your application's directory.

Tester Two Requirements

      The main point of Tester is to send keystrokes over to another window. Tester does that quite well, but the problem is the format of the TInput PlayKeys method. I used the Visual Basic SendKeys format, which is not intuitive at all. Here are the requirements that I laid out for Tester Two:

• The ability to record keystrokes and place them into a string compatible with PlayKeys and SendKeys.
• When generating a Tester script, the script is self-contained so the saved script is ready to run.
• The user can edit the automatically generated script before saving it.
• Tester can properly set focus to a specific window, including any child control, to ensure that playback goes to the correct window.

      In general, the requirements seem straightforward. However, you are probably wondering about recording and playing back mouse movements and clicks. Originally, I had planned to include them, but I ran into so many problems getting the keyboard recording straight that I simply ran out of time. While I could have simply done something with hardcoded pixels, I desperately wanted to find a way to avoid scripts that were hardcoded to the resolution on which you recorded them. To me, a regression testing script with a hardcoded screen resolution is not very useful. Consequently, mouse handling remains a goal for a future version of Tester.

Using Tester Two

      Recording your testing scripts is a piece of cake. Start TESTREC.EXE, press CTRL+R or click on the record button in the toolbar, press ALT+TAB to shift to your application, and type away. When you start recording, TESTREC.EXE minimizes itself to the task bar and changes its title to RECORDING! so you know what it's doing.
      After recording the keystrokes you want, there are several ways to stop recording. The simplest way is to set the focus to the TESTREC.EXE application by clicking on it or using ALT+TAB to set the focus. Alternatively, pressing CTRL+BREAK or CTRL+ALT+DEL will stop recording as well. When you are recording, the edit control in TESTREC.EXE will contain a VBScript-compatible script of your actions. You can edit the script or save it to a file.
      After you have recorded and saved a script, just run it under WSH to play back the keystrokes. The Tester COM object in TESTER.DLL uses TINPUTHLP.DLL and TNOTIFYHLP.DLL, so make sure they are in your path.
      Before I move on to the implementation, I want to give you a few tips on how to maximize your Tester usage. First, like any development task, you should have a plan for what you intend to accomplish before you record in one session someone typing all of Shakespeare's sonnets into your application.
      Your goal should be to make your scripts as generalized as possible so you can reuse them. For example, you should only have one script to open files. This means you should only record small operations. You can chain a bunch of small scripts together with .WSF files for WSH 2.0 in Windows 2000, or with batch files if you have to support Windows NT® 4.0 or Windows 98.
      After you record the keystrokes to your application, you might want to edit the output so that it appears more readable. TESTREC.EXE just records the keystrokes as a stream, with predefined breaks for outputting the information, so you should tighten up the calls to PlayKeys. You should also edit the recorded keys to remove any mistakes (such as BACKSPACE) so you are only playing back known, good input.
      TESTREC.EXE knows how to ensure that the proper window gets focus. In Tester Two, this is done through the ALT+TAB handling, so you might want to ALT+TAB to your application every once in a while without actually shifting focus to a new application. For example, if you are testing a data entry application, enter the data and tab to the next field. Press ALT+TAB to generate the appropriate code to set the focus to the new child window. Part of setting the specific focus involves verifying that the child window exists. The extra ALT+TAB keystrokes will ensure that the appropriate child window exists and is ready for input.

Implementing TESTREC.EXE

      The TESTREC.EXE implementation turned out to be much more involved than I ever expected. The issue wasn't having to write tons of code, but trying to get the proper states set when building up the output keystrokes. Making the output understandable to humans was a challenge as well.
      The first thing I needed to figure out was how to record keystrokes. In Windows, there is only one way: a journal record hook. There's nothing very exciting about journal hooks except handling the WM_CANCELJOURNAL message properly. When the user presses CTRL+ALT+DEL, the operating system executes any active journal recording hooks. This makes sense because it would be a pretty serious security breach to allow an application to record the keystrokes that make up the user's password. To handle WM_CANCELJOURNAL in a manner that keeps the implementation details hidden, I used a message filter to monitor for it coming through. You can see all the hook details in HOOKCODE.CPP in this month's code distribution at the link at the top of this article.
      Of course, since a journal record hook is a global hook, debugging gets to be a little interesting. I played around to see if I could debug everything on one machine; on Windows NT 4.0, I proceeded to lock up the entire user interface. The only way to debug TESTREC.EXE is by remote debugging with the Visual C++® debugger. For more information about remote debugging, see my Bugslayer column in the August 1999 issue of MSJ.
      I did learn something new about the Visual C++ debugger while working on TESTREC.EXE. When you have remote debugging set up, the Build Execute program name menu option will start the program on the remote machine. I thought that was pretty cool!
      Before I jump into the really nasty guts of keystroke generation, you might want to look for SendKeys on MSDNâ„¢ Online. The discussion that follows assumes you are familiar with the basic gyrations of the keystroke format.
      After I was merrily recording with a shell of a hook, I needed to think a bit about the best approach for implementing the recording processing. I decided to keep the actual recording and output parts generic so I could change them at will as well as test them in isolation. Since journal record hooks can reside inside the EXE and any hook processing should be as lightweight as possible, I simply used a C++ class, CRecordingEngine, to handle the processing. The CRecordingEngine class constructor takes a pointer to a CRecordingUIOutput class. This class, which I defined in RECORDINGENGINE.H along with CRecordingEngine, is an abstract base class that CRecordingEngine calls for output.
       Figure 2 shows RECORDINGENGINE.CPP, the part where I spent way too much time. I had to do horrendous amounts of testing to make sure I had everything output properly. It all revolved around handling the CTRL, SHIFT, and ALT keys, especially in nested states. I'm sure you are reading this wondering how hard could it be to record keystrokes and plop them into a string? Believe meâ€"I originally thought it would be simple, too!
      I started out by doing a finite state diagram of key actions so I could get a handle on the processing. I was originally going to include that state diagram for the keyboard handling, but the diagram is so complicated that it probably would have taken the entire MSDN Magazine art department three or four months to make it understandable enough to read. The other problem is that it would take two full pages to show. Yes, keyboard state is that confusing!
      The first problem I encountered was what I will call the nested modifier states. As you know, the CTRL, ALT, and SHIFT keys are special modifier keys. CTRL+A, SHIFT+A, and ALT+A have three different meanings. If Windows allowed you to press only a single modifier at a time, my life would have been easy. In SendKeys format, a nested modifier looks like +(AB^(CD)). The + indicates a SHIFT, and the ^ indicates a CTRL. The following keystroke sequences produces that SendKeys string:

While there's some work involved with the previous example, try keeping the state on something like the following:

From the June 2000 issue of MSDN Magazine.