Share via

Implementing Custom Test Types

  • Article

Visual Studio 2008 Team System integrates complex testing functionality into the Visual Studio IDE. Integrating testing functionality into Visual Studio allows developers easier access to load and unit testing without forcing a context switch to another tool. Furthermore, integrating testing into Visual Studio allows for metrics about testing to be easily marshaled against the Team Foundation Server, which helps improve communication. It is also possible for organizations and Visual Studio Integration Partners to extend the types of tests available by leveraging the Test Type Extensibility Framework to add their own fully integrated test types.

Test Framework

Test Types in Visual Studio Team System are implemented, at the lowest level, as VsPackages. This lets test types integrate seamlessly with the Visual Studio IDE. If you have custom test types and want to interface with core testing components within the IDE, such as the Test Manager, Test View, and Test Results tool windows, you have to implement these core classes:

  1. TestType Class: Provides a GUID that can be used to permanently define the test type among the various components used to create the custom test type.

  2. TestElement Class: The core test functionality provided by the custom Test Type is implemented as a Test Element instance. Because this type may be marshaled to other machines in the process of testing, this class must be serializable.

  3. TestResult Class: Handles results for each test instance of the custom type, and stores them for interaction with reporting interfaces.

Because of the complexity of extending test types, and the amount of code that you would be forced to copy and paste, this walkthrough will guide you through an existing example rather than walk you through the process of creating your own custom test type. By focusing on an existing example, you will be able to better get a feel for the type of extensibility that is possible.

To find and view the custom test type sample

  1. Download and extract MyTest2008.zip.

  2. Open the MyTestSample.sln solution. After the MyTest project is loaded, expand the References node in Solution Explorer.

    References to core assemblies required for Visual Studio Extensibility, such as the Microsoft.VisualStudio.Shell, and related assemblies, are listed, as well as references to core testing libraries. All these libraries are required to successfully implement a VsPackage that can be used as a custom Test Type.

  3. In Solution Explorer, double-click MyTestPackage.cs to open it in the editor.

    Within the Test Type Extensibility Framework, VsPackages interact with Testing Components found in Visual Studio Team System through the use of special registration attributes. In this case, MyTestPackage exposes the RegisterTestTypeNoEditorAttribute to indicate to the testing framework that this package exposes a simple Test Type. This simple Test Type does not provide an editor to allow testers to specify criteria and settings. The MyTestPackage VsPackage also exposes a Tool window, which is used to visually display test result information after instances of the MyTest test class are run. This information is provided to the Visual Studio IDE by the ProvideToolWindowAttribute.

  4. Note too that MyTestPackage.cs proffers a specialized Service to the Visual Studio IDE via the ProvideServiceForTestTypeAttribute. Code to proffer the service is handled in the private OnCreateService() method, which gets mapped during class creation. In the OnCreateService() method, code simply ensures that required input is in the correct order, then creates and returns a new instance of the MyTestTuip class. The new class acts as a wrapper that is designed to house and administer various Test Type related functionality and interfaces.

  5. In Solution Explorer, double-click MyTestTuip.cs to open it. Near the bottom of the file is a public interface used to identify the SMyTestService. This interface, in turn, is implemented by the MyTestTuip class. This class is near the top of the file. Together, these objects inform the Visual Studio IDE of the presence of the MyTest Test Type during run time. In other words, these objects merely serve to facilitate communication between the custom Test Type and the IDE. They are not the implementation of the Test Type itself.

    With the necessary framework now in place to expose testing functionality via the proper registration attributes, and to manage interaction with the IDE, we can implement the custom functionality for a new Test Type. Providing custom test functionality in Visual Studio Team System requires the implementation of four key classes:

    • MyTest

    • MyTestResult

    • MyTestAdapter

    • MyTestTip

    These classes provide base functionality for a new Test. Additional functionality, such as custom editors, custom viewers, or the ability to extend the run configuration settings, require the implementation of other classes.

  6. In Solution Explorer, double- click MyTest.cs to open it in the editor. MyTest implements the TestElement class supplied by the Test Type Extensibility Framework, describes the properties of a test during run time. Because this class might be marshaled between machines during a distributed test, this class must be serializable. Therefore, the SerializableAttribute above the class's definition is required.

  7. Expand the TestType instance code region. Near the top of the MyTest class, note that the class exposes a static TestType class with a hard-coded GUID. This GUID, or TestType marker, will be used to uniquely identify the custom type and all its constituent elements during run time. It will not identify instances of the type.

  8. Expand the Constructors code region. Next, scroll to the constructors for MyTest. The first constructor is the default construct required for serialization. The comments for the second instructor indicate that this constructor is usually called by the Test Tip (o TIP).

    In the Testing Framework, tests are processed at run time, but users frequently wish to save test results for future use to detect trends, avoid regression, or analyze performance changes, and so forth. The purpose of the TestTip class is to provide tips, or instructions, to the custom Test Type about how to serialize and load results. In other words, the TIP is where Test Type designers can specify the serialization mechanism for their Test Types. Because they are responsible for the implementation details, they are free to choose a serialization format that makes sense to their particular test domain. A simple storage mechanism would be to simply persist test data to disk in a flat file, or maybe as xml. However, because the implementation details are left to the designer, it might make sense to store the data in a SQL Server database, or to expose a custom object that could hydrate and dehydrate data through the use of a Web Service. The core concept is flexibility, which is allowing Visual Studio customers and Partners the ability to store and load test data for their own test extension types in whatever manner makes sense.

  9. In Solution Explorer, double click MyTestTip.cs to open it in the editor. Note first that it exposes a single, mandatory, property: TestType.It returns the TestType that is used to uniquely identify the custom type. Look at the implementation details for the Load() and Save() methods. These methods just persist data to a simple StreamWriter. However, they use collections of ITestElement objects as a handle for the serialization and deserialization process. It is here that the MyTest, which subclasses the TestElement base class, instances are created from disk, or saved to disk.

  10. Close MyTestTip.cs, and return to MyTest.cs.

    With a serialization and deserialization mechanism now in place, coupled with descriptive information about the nature of the test itself (and a framework to inform Visual Studio and the Test Type Extensibility Framework that the test type is available), there now just remains one final detail: how the test actually interfaces with Visual Studio at run time to actually perform custom test logic.

  11. Expand the TestElement overrides code region, and locate the Adapter property. Other properties in this class describe the type of functionality exposed by this custom Test Type, such as whether or not the test is read only, or can be invoked from the command line, and so forth. The Adapter property returns an object that points to the MyTestAdapter class.

    The function of the Adapter is to allow custom test logic to be tightly integrated into the Test Framework provided by Visual Studio Team System. The VsPackage and supporting objects are used to register a Test Type with Visual Studio. When a Test Type is registered, new instances of the Test Type can be loaded by end users, or testersbut the Adapter implementation actually handles the custom logic and testing functionality at run time. Its purpose is to bridge the gap between your testing objects for serialization and reporting and the existing framework, which actually processes tests.

  12. In Solution Explorer, double- click MyTestAdapter.cs to open it in the editor. Expand the IBaseAdapter methods code region and locate the Run() method. In this method, an ITestElement is passed in, along with an instance of a testing context object for processing. The ITestElement is an instance of MyTest. Results from each run are stored in a new instance of the MyTestResult class which is created just above the try/catch block. Within the method itself, some simple code is wrapped in a timer block, and outcome as well as duration are then recorded by setting properties on the result instance. You set properties by registering the result instance with the test context near the end of the method.

  13. If you wish to compile and run the sample, make sure to follow the instructions found in the readme.txt file that accompanies the solution.