How to: Specify a Test Run Configuration

  • Article

You can control the way test runs proceed by specifying settings that are stored in test run configurations. Test run configuration settings are saved to disk in files that have the .testrunconfig extension. You can create and save multiple run configurations, and then make a specific run configuration active, which is the test run configuration that is used when tests are run. For information about how to make a run configuration active, see How to: Apply a Test Run Configuration.

The procedures in this topic describe various aspects of working with run configurations. The first group of procedures describes how to create and delete run configurations.

The second group of procedures describes how to set the values within run configurations. To set these values, you use the test run configuration editor, a dialog box that has the following pages:

  • General. On this page, specify a name for the test run configuration and an optional textual description. You can also specify a scheme for creating the names of test runs, which are used to identify test runs in the Test Results window.

  • Controller and Agent. Select whether to run tests on the local computer, or remotely, by using a test controller. Running tests locally lets you debug your tests and your production code. You cannot use integrated debugging when you run tests remotely.

    If you choose to run tests remotely, you identify the controller to use by specifying its network name or its IP address. For more information, see Working with Controllers, Agents, and Rigs.

    Note

    When the test controller is the local computer, identify the computer by specifying localhost.

    You can also specify names and values for properties of an agent. If an agent being used has those properties, the test or tests you are running will use that agent. If no agent has the properties that were requested, the test run will fail.

  • Code Coverage. On the code coverage page, you specify what to instrument: which DLLs or .exe files, and, for ASP.NET projects, which directories. You can also specify whether to instrument in place and how to re-sign the files. This process starts by making a copy of the DLL. Then, the DLL is changed, that is, instrumented, so that code coverage statistics can be gathered from it during test runs. For more information, see How to: Obtain Code Coverage Data and Instrumenting and Re-Signing Assemblies.

  • Deployment. Before tests are run, they are always copied to a deployment directory to ensure isolation and produce more reliable test results. Use the Deployment page to specify any additional files or directories you want copied to the deployment directory. For more information, see Test Deployment Overview and How to: Configure Test Deployment.

  • Hosts. Select ASP.NET if you want unit tests to run in the same process as an ASP.NET site. For other test types, including Web tests, choose Default. If you select ASP.NET, you can make additional choices to configure the way ASP.NET tests are run. For more information, see Overview of ASP.NET Unit Tests.

  • Setup and Cleanup Scripts. On this page, you can select scripts to be run before tests are run, and after tests are run, respectively. To identify a script, type its path in the box under Setup Script or Cleanup Script. Alternatively, click the ellipsis (…). This opens the Open dialog box, which you can use to select a script file.

    Setup scripts and cleanup scripts run before and after test runs, regardless of the types of tests that are contained in those test runs. For more information about scripts that are run together with test runs, see Test Deployment Overview.

  • Test Timeouts. You can use this page to can specify time-out values for test run and for individual tests. For more information, see How to: Set Time Limits for Running Tests.

  • Web Test. Set values to be used when a Web test is run. Note that the Simulate Think Times setting is ignored when the Web test is aggregated into a load test. For more information, see How to: Set Think Times in a Web Test.

    Note

    You cannot configure all aspects of test runs by using the test run configuration editor. For some aspects, you need to directly edit the XML file that contains test run configuration settings. For more information, see How to: Edit a Test Run Configuration File.

    Note

    The Controller and Agent page and the Web Test page are available only if you have Team Edition for Testers installed.

Creating and Removing Test Run Configurations

To add an initial test run configuration to your solution

  • Add a test project to your solution.

    A test run configuration is added automatically to the solution under the Solution Items folder.

    Note

    Each run configuration you create is listed as a choice for the Select Active Test Run Configuration and Edit Test Run Configurations options on the Test menu.

To add a subsequent test run configuration to your solution

  1. In Solution Explorer, right-click Solution Items and then point to Add, and then click New Item.

    The Add New Item dialog box appears.

  2. In the Categories pane, click Test Run Configuration.

  3. (Optional) In the Name box, change the name of the test run configuration file.

  4. Click Add.

    The new run configuration file appears in Solution Explorer, under the Solution Items folder.

    Note

    The list of test run configurations that Visual Studio displays — for example through the Select Active Test Run Configuration option on the Test menu — is derived from the list of test run configuration files in the Solution Items folder. This means that if you move a run configuration file to another location in your solution hierarchy, it can no longer be used as a test run configuration from within the Visual Studio IDE.

  5. The <run config file name>.testrunconfig dialog box appears.

    You can now set and save run configuration values. For more information, see the procedures in Setting Test Run Configuration Values.

    Note

    Each run configuration you create is listed as a choice for the Select Active Test Run Configuration and Edit Test Run Configurations options on the Test menu.

To remove a test run configuration from your solution

  • Under the Solution Items folder in Solution Explorer, right-click the test run configuration you want to remove, and then click Remove.

    The run configuration file is removed from your solution. This change is reflected in the list of choices for the Select Active Test Run Configuration and Edit Test Run Configurations options on the Test menu.

Setting Test Run Configuration Values

To edit a run configuration if you have a solution open

  1. Under the Solution Items folder in Solution Explorer, double-click the test run configuration file that you want to open.

    - or -

  2. Point to Edit Test Run Configurations on the Test menu and then click a run configuration on the submenu.

    The run configuration opens for editing.

  3. Make changes on the individual pages for each aspect of the run configuration. For more information, see the descriptions of each page earlier in this topic.

  4. Click Apply, and then click Close.

See Also

Tasks

How to: Apply a Test Run Configuration
How to: Edit a Test Run Configuration File
How to: Configure Test Deployment
How to: Obtain Code Coverage Data
How to: Set Think Times in a Web Test

Concepts

Test Deployment Overview
Overview of ASP.NET Unit Tests