Using a Summary Results File with a Generic Test

  • Article

 

Through the use of a summary results file, a generic test can generate specific, detailed, test results. A summary results file is an XML file that conforms to a specific XML schema. It provides a mechanism that lets your test report detailed results for portions of the generic test, such as per-test-case results if the generic test is used to run multiple third-party test cases.

Requirements

  • Visual Studio Enterprise

Note

When you run a generic test remotely and it communicates results through a summary results file, you cannot view the results immediately. All the tests in the test run must first complete before you can view the results of the generic test, even if the generic test itself has already completed.

Creating and Using a Summary Results File

Using a summary results file consists of the following main steps.

Note

It is assumed that you already have a running program or test that you want to use as the basis for your generic test, and that you can edit its source code.

First, run xsd.exe, the XML Schema Definition Tool. This is a command-line utility that is accessible from a Visual Studio Command Prompt. Your goal is to make the xsd utility output a class, in the language of your choice, that contains specific parameters and properties. When you run xsd, one of the options you specify is the XML schema file SummaryResult.xsd, which you can find in the folder <drive>Program Files (x86)\Microsoft Visual Studio 12.0\Xml\Schemas. For information about running xsd.exe, see XML Schema Definition Tool (Xsd.exe).

Next, edit the program you will wrap as your generic test. Using the class generated by xsd, add parameters and properties to the classes of your generic test. In this step, you change your code to make it output data while the test runs. The data that is output is in a format that can be displayed in the generic test [Results] page in Visual Studio.

Finally, create the generic test itself to wrap your existing program. This process is described in How to Create a Generic Test. In your generic test, under Results Settings, choose Summary Results File and specify a path to a folder where you want the summary results file to be placed.

When you run the generic test, the generic test writes data to the summary results file, according to the changes you made in the source code. This is in addition to the results it reports in the .trx file of the test run. After the test runs and the summary results file exists, Visual Studio can parse it and display the in-depth results you coded your test to generate.

Example Summary Result File

<?xml version="1.0" encoding="utf-8" ?>  
<SummaryResult>  
    <TestName>ParentTest</TestName>  
    <TestResult>Passed</TestResult>  
    <InnerTests>  
        <InnerTest>  
            <TestName>InnerTest1</TestName>  
            <TestResult>Passed</TestResult>  
            <ErrorMessage>Everything is fine.</ErrorMessage>  
<DetailedResultsFile>D:\Documents and Settings\Results.txt</DetailedResultsFile>  
        </InnerTest>  
        <InnerTest>  
            <TestName>InnerTest2</TestName>  
            <TestResult>Failed</TestResult>  
            <ErrorMessage>Something went wrong.</ErrorMessage>  
            <DetailedResultsFile>D:\Documents and Settings\Results.txt</DetailedResultsFile>  
        </InnerTest>  
    </InnerTests>  
</SummaryResult>

See Also

How to Create a Generic Test
Walkthrough: Creating and Running a Generic Test