Visual Studio 2005 Team System: Building Robust and Reliable Software

  • Article

 

Kamran Iqbal
Microsoft Corporation

May 2004

Applies to:

   Microsoft Visual Studio® 2005 Team System

Summary: Describes the code analysis and performance tools available in Visual Studio 2005 Team System. (11 printed pages)

Note   This document was developed prior to the product's release to manufacturing, and as such, you may find inconsistencies with the details included here and those found in the shipping product. The information is based on the product at the time this document was created and should be used for planning purposes only. Information is subject to change at any time without prior notice. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Contents

Introduction
Analysis Tools Support
Our Solution
Tool Integration within Visual Studio
Conclusion

Introduction

To develop robust and reliable software, developers need an integrated set of analysis tools that help them in detecting code defects and performance problems earlier in the development cycle.

Analysis Tools Support

In the past, lack of support for code analysis in Visual Studio has led developers to buy third party tools, build custom analysis tools, or ship without analyzing code.

  • Buy third-party tools at an additional cost.

    Choosing a tool from a wide range of analysis tools that meet software analysis requirements and are within budgetary constraints is not always an easy task. In addition, these tools may not integrate within the Visual Studio Integrated Development Environment (IDE) and usually require additional time to learn to successfully use them.

  • Build a custom analysis tool.

    Building a custom analysis tool requires resources, skills, and experience. This is not practical for most development teams within a company and is an option that few companies can afford.

  • Ship code without analysis.

    Shipping code without performing analysis may seem like the right approach under tight deadlines and budgetary constraints, but it may turn out to be a costly mistake down the road when the software fails after it is deployed.

Our Solution

In Visual Studio 2005 Team System, developers will see a new class of analysis tools that are fully integrated within the IDE. This tight integration of the analysis tools within the development environment helps developers detect and fix code defects and performance problems early in the product development cycle. This also helps teams manage the Software Development Life Cycle (SDLC) effectively and efficiently.

The two categories of analysis tools available in Visual Studio 2005 Team System for building robust and reliable software are:

  • Code Analysis Tools
  • Performance Tools

Integration of analysis tools within the development environment helps developers in detecting coding, performance, and security related issues. In addition, code analysis tools can be used as a part of the check-in policy for a nightly build process, enabling development teams to correct defects before code is checked into the source tree. By correcting problems earlier rather than later in the development cycle, teams reduce the overall cost of fixing code defects.

In addition to the full integration within the IDE, developers can use these tools from the command-line.

Tool Integration within Visual Studio

In Visual Studio 2005 Team System, developers can conduct static analysis and dynamic analysis of their code by using Code Analysis and Performance Tools.

Code Analysis Tools

The goal of the Code Analysis Tools is to make it possible for developers to use static analysis on their projects to detect and correct code defects. To achieve this goal, Visual Studio 2005 Team System includes two tools:

  • PREfast
  • FxCop

PREfast

Prefast is a static analysis tool that provides information to developers about possible defects in their C/C++ source code. Common coding errors reported by Prefast include buffer overrun, un-initialized memory, null pointer dereference, memory and resource leaks.

IDE Integration

To make it natural for developers to use analysis tools, Prefast is fully integrated within the IDE. Developers can easily enable PREfast by selecting Yes (/prefast) on the project's Property Pages, as shown in Figure 1.

Aa302177.vsts-dev-fig01(en-us,MSDN.10).gif

Figure 1. Enable PREfast

During the build process, any PREfast warnings generated for the source code appear in the Error List. These warnings include defect path information, if available; double-clicking the warning highlights the defect path that led to the warning in the code editor, as shown in Figure 2.

Aa302177.vsts-dev-fig02(en-us,MSDN.10).gif

Figure 2. PREfast Warnings

#pragma support

Developers can use the #pragma directive to treat warnings as errors, or disable warnings, as shown below.

#pragma warning (error: 6260)  //treat warning 6260 as an error
#pragma warning (disable: 6011)  //disable warning 6011

Annotation support

PREfast also supports annotations to improve the accuracy of the code analysis. Annotations provide additional information about pre- and post- conditions on function parameters and return types.

#include </prefast/SourceAnnotations.h>
[SA_Post( MustCheck=SA_Yes )] double* CalcSquareRoot
( 
   [SA_Pre( Null=SA_No )] double* source, 
   unsigned int size
)

In the above example:

  • [SA_Post ( MustCheck=SA_Yes)] requires caller to check the return value of CalcSquareRoot
  • [SA_Pre ( Null=SA_No)] requires caller to pass non-null parameter "source" to CalcSquareRoot

Command-line support

In addition to the full integration within the development environment, developers can also use PREfast from the command-line, as shown below.

C:\>cl /prefast Sample.cpp

FxCop

FxCop is a static analysis tool that analyzes managed code assemblies and reports information about the assemblies, such as violations of the programming and design rules set forth in the Microsoft .NET Framework Design Guidelines. FxCop represents the checks it performs during an analysis as rules. A rule is managed code that analyzes an assembly and returns a message about its findings. Rule messages identify any relevant programming and design issues and, when possible, supply information on how to fix the target.

IDE Integration

To make it natural for developers to use analysis tools, developers can select Run FxCop on the project's Property Pages, as shown in Figure 3.

Aa302177.vsts-dev-fig03(en-us,MSDN.10).gif

Figure 3. Enabe FxCop

Additional options for including or excluding rules and treating rules as warnings or errors can also be accessed from Property Pages, as shown in Figure 4.

Aa302177.vsts-dev-fig04(en-us,MSDN.10).gif

Figure 4. FxCop Rules

Once FxCop is enabled, during the build process, FxCop reports rule violations in the Error List, as shown in Figure 5.

Aa302177.vsts-dev-fig05(en-us,MSDN.10).gif

Figure 5. FxCop Rule Violation

The Error List provides information about the rule violation and suggestions to correct the problem. You can easily navigate to the offending line of code by simply double-clicking a rule violation in the Error List.

MSBuild Integration

The next release of Visual Studio radically improves the build process by introducing a new build engine called MSBuild. FxCop can be invoked through the MSBuild system. This allows developers to run FxCop over managed projects from the command-line.

Performance Tools

In Visual Studio 2005 Team System, Performance Tools allow developers to measure, evaluate, and target performance-related issues in their code. These tools are fully integrated into IDE to provide a seamless and approachable user experience.

The Performance Tools support two methods of profiling:

  • Sampling
  • Instrumentation

During sampling, the data collection infrastructure periodically interrupts the application as it executes to determine what function is being executed and increments that function's sample count. It stores information about the call stack leading up to the function call.

After an application exits, all of the data gathered is generated into a report file and can be easily viewed using the reporting features integrated in the IDE.

Sampling provides the advantage of low overhead, since the application is only interrupted periodically. This allows the application to behave closer to how it would in the real-world. The drawback to this approach is that it can only get relative performance data for the functions that were sampled. It is possible that a function you wanted to sample did not get sampled and therefore, no information is available about it.

Instrumentation provides the advantage of gathering exact performance data for specific portions of the application. During instrumentation, "enter" and "exit" probes are inserted into the application's functions. These probes report back to the data collection infrastructure and allow users to capture exact amounts of time (and other metrics) that a function took to execute.

Profiling Applications

The preferred usage pattern for profiling an application is to begin with sampling, and then instrument specific aspects of their application based on the results produced by sampling.

The process of profiling an application is straight forward. You begin by creating a new performance session. In Visual Studio 2005 Team System, you can use Performance Session Wizard to create a new performance session.

Performance Session Wizard

Performance Session Wizard sets up the environment necessary for profiling an application. In Visual Studio 2005 Team System, the wizard provides built-in support for EXE, DLL, and ASP.NET applications.

You can create a New Performance Session using the wizard, as shown in Figure 6.

Aa302177.vsts-dev-fig06(en-us,MSDN.10).gif

Figure 6. Launch Performance Wizard

Note   To manually create a new performance session use the New Performance Session command on the menu. You can use this technique to manually profile other type of applications such as Windows Services.

Performance Explorer

A Performance Session is created as a result of running the session wizard or manually creating a session. It is visually represented in Performance Explorer, as shown in Figure 7.

Aa302177.vsts-dev-fig07(en-us,MSDN.10).gif

Figure 7. Performance Explorer

The Performance Explorer presents hierarchical structure to the user. The root node of the hierarchy represents the Performance Session. The properties of this node are the properties the user sets when the Performance Session was created. If the user used Performance Session Wizard to create the session, then these properties reflect the values the user chose as they walked through the wizard. If the user created the Performance Session manually, then these properties contain their default values.

The root node has two children, the Targets node and the Reports node. The Targets node contains one or more targets, which can be an .EXE, .DLL, or ASP.NET application.

The Reports node contains all the reports that are relevant to a particular Performance Session.

Performance Session Reports

Once the application finishes executing, a performance session report is automatically added to the Reports node. These reports can be viewed with the following views:

Summary View

The Summary view gives developers a starting point in their investigation. It shows the most expensive functions during the execution of the application, as shown in Figure 8. From each data point in this view, users can navigate to more detailed views.

Aa302177.vsts-dev-fig08(en-us,MSDN.10).gif

Figure 8. Summary View

Functions View

The Functions view shows all of the functions that were called during the execution of the application across all of the modules that were referenced by this application, as shown in Figure 9. The information shown in this view depends on the method (sampling vs. instrumentation) used for profiling.

Aa302177.vsts-dev-fig09(en-us,MSDN.10).gif

Figure 9. Functions View

Caller/Callee View

The Caller/Callee view provides the details for the functions listed in the Functions view, as shown in Figure 10.

Aa302177.vsts-dev-fig10(en-us,MSDN.10).gif

Figure 10. Caller/Callee View

Callstack View

The Callstack view allows users to drill into specific call traces and analyze which traces have the greatest performance impact.

Aa302177.vsts-dev-fig11(en-us,MSDN.10).gif

Figure 11. Callstack View

Type View

The Type view provides information about the number of instances and total bytes of a particular type, as shown in Figure 12.

Aa302177.vsts-dev-fig12(en-us,MSDN.10).gif

Figure 12. Type View

Command-line support

Performance Tools feature are available through command-line tools. This allows the users the flexibility of running these tools from command-line or using them to automate tasks using scripts.

Conclusion

Visual Studio 2005 Team System includes a new class of analysis tools which are fully integrated within the development environment and are also available from the command-line. These powerful yet easy to use tools help developers build robust and reliable software by providing them with code defect and performance bottleneck detection.

For more information on the other members of Visual Studio 2005 Team System, see: