Operations Guidance for Team Foundation Server

Microsoft Corporation

September 2007

Summary: You can manage your deployment of Microsoft Visual Studio 2005 Team Foundation Server more effectively if you create your own operations plan. As you create your plan, you should understand key elements of the architecture of Team Foundation Server and how your deployment topology affects operations. This white paper explains those elements in detail, so that you can avoid common problems with your deployment.

Applies to:

   Microsoft Visual Studio 2005 Team Foundation Server, including:

   - Team Explorer

   - Team Foundation Build

   - Team Foundation Version Control

Contents

Overview

Understanding Your Team Foundation Server Topology

Managing Service Accounts

Managing Groups, Users, and Permissions

Managing Backup and Recovery

Managing Configuration Information

Monitoring Team Foundation Server

Creating an Operations Plan

Conclusion

Author Bio

Additional Information

Overview

You can use this guide to help you understand your deployment of Team Foundation Server and keep it running throughout the software-development life cycle. Team Foundation Server depends on an alignment of permissions, configuration information, and service accounts across the components to operate correctly. These components include not only Team Foundation Server itself, but also the technologies on which it depends, such as Microsoft Windows SharePoint Services and Microsoft SQL Server Reporting Services. If you understand how these components interoperate, you can more effectively perform routine tasks, such as managing user permissions, backing up your deployment, and monitoring its overall health. In addition, you can avoid causing problems when you alter your deployment to meet the changing needs of your organization.

This guide is not designed to help you plan or troubleshoot a deployment. However, you will be better able to troubleshoot an existing deployment or plan a new one if you understand how Team Foundation Server operates. You can find more information about Team Foundation Server planning, installation, troubleshooting, and operations online in the Visual Studio 2005 Team Foundation Installation Guide and the Visual Studio 2005 Team Foundation Administrators Guide on the Microsoft Web site.

Understanding Your Team Foundation Server Topology

The design of your specific deployment affects how you perform tasks with Team Foundation Server, from providing users with basic access to managing functionality. Moreover, you can better manage configuration changes if you understand how the specific configuration choices of a deployment work within Team Foundation Server. Specifically, you must know the following information to manage Team Foundation Server effectively:

  • Which physical computer or computers host the application and data tiers for Team Foundation Server
  • Which physical computer or computers act as build computers
  • How the physical computers communicate with each other
  • How client computers communicate with Team Foundation Server

Figure 1 illustrates the underlying physical architecture of Team Foundation Server and the communications among the components, and the table below it identifies where on each Team Foundation Server logical tier you can find configuration information.

Figure 1. Team Foundation Server Operations Architecture

Team Foundation Server Operations Architecture

Logical Tier Configuration Information

Client

Windows Registry

Client Cache (Visual Studio)

Application

Windows Registry

IIS (IIS Manager)

Team Foundation Server Config files

SQL Server Reporting Service data sources (TFSReports)

Data

Windows Registry

Team Foundation Server Integration Database

Database roles in SQL Server

Your deployment of Team Foundation Server might be as simple as a single-server installation alongside the client tools that will access it. Conversely, it might be as sophisticated as any other scalable, geographically distributed, high-availability server product that contains intellectual property that is critical to the future of a company. Configuration information that is critical to the operation of Team Foundation Server will be present on all of those computers and will be exposed through the configuration points that are outlined in the diagram. Managing Team Foundation Server becomes much easier if you understand the specifics of your deployment and how it was configured. You should determine when to perform specific operational tasks, based somewhat on your deployment choices, but mostly on your business needs. How often you need to perform various operational tasks, such as managing accounts, performing backups, and monitoring usage, is up to you.

How Communications Affects Operations

Every project in Team Foundation Server depends on the communication among the physical components to provide functionality for source control, bugs, tasks, documents, reports, and so on. To deploy Team Foundation Server and keep it running, you must understand its communication paths. Specifically, you must know the ports and protocols on which Team Foundation Server depends. Because Team Foundation Server is a distributed client and server application, communication-related concerns are an inherent part of routine operations. For more detailed information about these communication paths, see Team Foundation Server Security Architecture on the Microsoft Web site.

Managing Service Accounts

Team Foundation Server requires multiple service accounts for basic operation. Your operations plan should include managing service accounts when you perform certain tasks, such as:

  • Installing Team Foundation Server.
  • Updating the passwords for service accounts.
  • Changing the accounts that are used as the service accounts.
  • Moving Team Foundation Server to a different environment.

For example, if account passwords expire every 90 days, you will need to build password updates into your operations plan.

The following table lists all service accounts for Team Foundation Server and their purposes:

Service Account Purpose

Team Foundation Server service account (TFSSERVICE)

  • Used as the application-pool identity by the application pools for Team Foundation Server (TFS AppPool) and Windows SharePoint Services (TFWSS and TFSWSSADMIN).
  • Used by the SharePoint Timer Service and Windows services in Team Foundation Server. The Windows services include Code Coverage Analysis Service and TFSServerScheduler.

Team Foundation Server Reporting service account (TFSREPORTS)

  • Used by data sources for SQL Server Reporting Services.

Team Foundation Server Proxy service account (TFSPROXY)

  • Used by Team Foundation Server Proxy, if the proxy is installed and configured.

For more information about how to update password information or change accounts for both TFSSERVICE and TFSREPORTS, see How to: Assign a New Account to a Team Foundation Server Service on the Microsoft Web site. For more information about the system accounts and service accounts for Team Foundation Server, see Managing and Resetting Service Accounts and Passwords on the Microsoft Web site.

Managing Groups, Users, and Permissions

Team Foundation Server, Windows SharePoint Services, and SQL Server Reporting Services all maintain their own information about groups, users, and permissions. Your operations plan should include managing this information when you perform certain tasks, such as:

  • Creating a team project in Team Foundation Server.
  • Changing the membership of a team project.
  • Creating a custom user group.
  • Adding or removing a user or a group of users.
  • Changing permissions for a user or a group of users.
  • Moving Team Foundation Server to a different environment.

For example, every time you create a team project in Team Foundation Server, you must ensure that the users for that project have been added to the appropriate groups and that they have the permissions that they require to do their jobs. You will need to build the checklist for this into your operations plan.

To make managing users and permissions easier, you can add users to Active Directory groups that are specific to their organizational roles. You can then add those groups to Team Foundation Server, Windows SharePoint Services, and SQL Server Reporting Services, and assign appropriate permissions to those groups. By taking this approach, you can reduce the amount of configuration and management that is required for individual users. However, you still must carefully consider how you want to manage users and permissions, not only for team projects in Team Foundation Server, but also across Team Foundation Server, Windows SharePoint Services, SQL Server Reporting Services, and the Windows operating system itself.

Managing Users and Groups Across Projects

When you manage Team Foundation Server and the projects on each server, you should ask some basic questions, such as those that follow. Your answers to these questions can help you determine what groups and permissions are appropriate for your deployment needs.

  • Do team projects require unique permissions, or should all contributors have access to all team projects? This question is a major factor in determining how many groups you need to manage your users.
    • If you can grant contributors the same permissions for all projects, the simplest approach is to use a single Active Directory group to manage those permissions.
    • If you must grant specific permissions for each team project, you might need to manage a complex matrix of groups. In this situation, you must determine whether you can manage users and groups more easily in Active Directory or in Team Foundation Server, Windows SharePoint Services, and SQL Server Reporting Services. If you can create and manage Active Directory groups, you can create these groups more easily than managing users for each team project across the three programs. However, if you have limited control or no control over the creation and management of groups in Active Directory, you will almost certainly have to manage users and groups in Team Foundation Server, Windows SharePoint Services, and SQL Server Reporting Services. You can still create custom groups to help manage the specific permissions for those users.
  • What groups will your organization require? You should review the standard groups in Team Foundation Server and determine whether they meet your needs. As mentioned previously, if your organization requires additional groups to perform specific business roles, you can create them by creating custom groups in Team Foundation Server. To define custom groups, you should determine what activities members of that group perform, and what minimum permissions they require to perform that work. As you add more groups, management becomes more complex and more costly. A good design balances the needs of security against the additional overhead. To help strike this balance, you can determine the smallest number of users for which you will create and maintain a custom group, and then you can consolidate groups that are smaller than the size that you chose. If necessary, you can grant specific permissions to individual users, instead of to groups. However, this strategy also increases the administrative overhead. Common sense is your best guide, but you should never grant users and groups more permissions than what they require to perform their work.
  • Will all users exist in the same domain or workgroup? Depending on your configuration, Team Foundation Server can support users and groups from more than one domain, or users from both a workgroup and a domain. For more information, see Managing Team Foundation Server in a Workgroup, Managing Team Foundation Server in an Active Directory Domain, and Trusts and Forests Considerations for Team Foundation Server on the Microsoft Web site.
  • Will all contributors have the same access to the entire version-control tree, or will permissions vary for each contributor? You manage permissions for Version Control similarly to how you manage permissions for file systems. You can create specific permissions for a folder or allow it to inherit permissions from its parent. As an alternative, you can explicitly deny permissions to one or more folders. By combining these approaches, you maximize flexibility in how you can define permissions. If you must restrict specific parts of the version-control tree, you should design the folder structure to make managing permissions easier. Because you can take advantage of inheritance, you should design the folder structure so that specific permissions are applied to as few folders as possible. For more information about how to control access to source code, see Team Foundation Source Control Permissions on the Microsoft Web site.
  • Do you need to manage permissions uniquely for builds on the build server? If you must manage permissions on the build-server shares, you must maintain Active Directory groups to accomplish this task. Ideally, you can use the same groups for Team Foundation Server permissions as for build shares. Otherwise, you must add and maintain additional Active Directory groups for this purpose.

Project Roles and Associated Default-Group Memberships

When you create a project in Team Foundation Server, the project has three default groups to which you can assign users and groups of users: Project Administrators, Contributors, and Readers. In addition, the Team Foundation Administrators group appears as a group in every project. Your operations plan should include managing the membership of these default groups when you perform certain tasks, such as:

  • Creating a team project in Team Foundation Server.
  • Changing the membership of a team project.
  • Adding or removing a user or group of users to or from a default group.

For example, if a team of developers joins a team project, and you manage users in the default groups, you will need to add those developers to the appropriate groups. For each business role in Team Foundation Server, you must create groups and assign permissions to those groups or users in Team Foundation Server, Windows SharePoint Services, and SQL Reporting Services.

Every business has required functions that translate into roles, or, to put it more simply, into jobs that have specific responsibilities and duties. You must determine what group memberships and permissions are appropriate for each person's role in Team Foundation Server. Also, you must determine what group memberships and permissions match that role most closely in Windows SharePoint Services and SQL Server. If the default groups in Team Foundation Server are not specific enough to meet your business needs, you can create custom groups with specific permissions to better map to business roles. As the following table outlines briefly, you must assign each user to a group in each program, based on what role the user plays:

Business Role Program Required Group Memberships

Project Lead

Team Foundation Server

Project Administrators

Windows SharePoint Services

Site Administrators

SQL Server Reporting Services

Content Manager

Project Contributor

Team Foundation Server

Contributor

Windows SharePoint Services

Contributor

SQL Server Reporting Services

Browser

Project Reader

Team Foundation Server

Reader

Windows SharePoint Services

Reader

SQL Server Reporting Services

Browser

Team Foundation Server Administrator

Team Foundation Server

Team Foundation Administrators

Windows SharePoint Services

SharePoint Administration group in SharePoint Central Administration

SQL Server Reporting Services

Content Manager

System Administrators

Project Leads

A project lead is generally in charge of a team project and can maintain a work item database, a team project portal, and the permissions and security for that project. Project leads cannot create team projects.

Project Contributors

A project contributor is the hands-on manager, designer, developer, or tester who works on a project. Project contributors know about Team Foundation Server, but they use it only as a tool to deliver their commitments.

Project Readers

A project reader is a high-level project manager, business analyst, or a manager on a related project. Project readers must be able to see the status of a particular project, but they do not contribute specific deliverables to that project. A project reader has no work items that are associated with the project.

Team Foundation Server Administrators

An administrator of Team Foundation Server is an IT administrator who manages the server itself. Members of this role have more permissions than any other user of Team Foundation Server. Administrators maintain Team Foundation Server, and they administer permissions and security for users and groups. For most organizations, members of this role create projects and manage the overall health and operation of the server, but they are not necessarily involved in any particular project. Members of this role might customize the process guidance for their organization, if the process guidance does not vary between projects.

For more information about default groups in Team Foundation Server, see Default Groups on the Microsoft Web site. For more information about how to set group membership for each of these roles, see Team Foundation Server Administrator Permissions, Team Foundation Server Project Lead Permissions, and Team Foundation Server Contributor Permissions on the Microsoft Web site. For more information about how to customize process guidance, see Customizing Team Foundation Server for Your Business on the Microsoft Web site.

Customizing Groups and Permissions in Team Foundation Server

In addition to the default groups, you can create custom groups to reflect roles that are described in your chosen process guidance or specific business needs. Your operations plan should include managing custom groups when you perform certain tasks, such as:

  • Creating a project in Team Foundation Server.
  • Changing the membership of a project.
  • Reviewing membership and permissions for each custom group.
  • Adding or removing a user or a group of users to a custom group.
  • Changing the permissions for a custom group.

For example, if a team of developers joins a project, and you manage users in custom groups, you will need to add those developers to the appropriate groups.

You must assign permissions to custom groups when you create them. These groups will not exist outside Team Foundation Server. Therefore, you must be sure that the users and groups that belong to these custom groups belong also to the appropriate groups in Windows SharePoint Services and SQL Server Reporting Services.

You can also customize the permissions that are granted to any group or user in Team Foundation Server, with the sole exception of the Team Foundation Server Administrator group. As a best practice, you should assign users to groups and then customize the permissions for those groups in Team Foundation Server, instead of to individual users.

For more information, see Custom Groups, Team Foundation Server Permissions, and Team Foundation Server Default Groups, Permissions, and Roles on the Microsoft Web site.

Tools for Managing Users and Groups

As mentioned earlier, Active Directory provides a powerful way to manage users and groups across Team Foundation Server and the programs on which it depends. If you cannot manage groups by using Active Directory, you can define and manage groups inside Team Foundation Server itself. Because security groups for Team Foundation Server are contained in each installation, you must manage security groups on each server. However, members of the Team Foundation Server Administrator group will have complete control over those groups. Nevertheless, you still must add the users in these groups to the appropriate groups in Windows SharePoint Services and SQL Server Reporting Services.

After you have determined the best way to manage users and groups across your deployment, you can directly add, modify, deactivate, reactivate, view, and remove users to and from Team Foundation Server, Windows SharePoint Services, and SQL Reporting Services. Admittedly, this process is cumbersome and subject to error, especially over time. A few external tools can help you manage this process. The following tools are not part of Team Foundation Server itself and are not supported as part of the product. However, they might help you in your specific deployment:

Team Foundation Server Administration Tool - By using this tool, you can add users to Team Foundation Server, Windows SharePoint Server, and SQL Server Reporting Services quickly through one common interface. Also, you can change permissions on one or more of those programs, identify errors, and view all users and their permission sets across Team Foundation Server, Windows SharePoint Server, and SQL Server Reporting Services. This tool is published on CodePlex, Microsoft's open source project hosting Web site. For more information, see Team Foundation Server Administration Tool on the Microsoft Web site.

Team Foundation Server Permission Manager - By using this tool, you can add or remove members from groups in Team Foundation Server, roles in SQL Server Reporting Services, and roles in Windows SharePoint Services. Also, you can set permissions in Team Foundation Server at the server and project levels, set AreaPath and version-control permissions, and create users in Team Foundation Server. In addition, you can assign permissions to a group that are identical to those of an existing user or group that you specify, and save user permissions as a template for later use. For more information, see Team Foundation Server Permission Manager on the Microsoft Web site.

Managing Backup and Recovery

As the diagram near the start of this document shows, data for Team Foundation Server is stored on the data tier, in Microsoft SQL Server 2005 databases. Your operations plan should include managing backup and recovery when you perform certain tasks, such as:

  • Backing up Team Foundation Server databases.
  • Creating disk images.
  • Performing trial restoration of Team Foundation Server databases.
  • Testing the validity of backup data.
  • Maintaining data-backup sets.
  • Reviewing data-backup storage and retrieval.

For example, if your backup strategy includes maintaining daily data sets for a month, you will need to include the testing and storing of those data sets in your operations plan.

Backing up these databases is a key aspect of protecting your deployment against loss. One advantage of the architecture for Team Foundation Server is that all of the data that you need for restoring a deployment is stored in these databases.

If you are familiar with backing up SQL Server databases, you will find backing up and restoring Team Foundation Server databases familiar. As with all deployments, you should consider keeping a record of the service accounts that you use in a specific deployment, the computer names, and the setup options for later reference. This record can be useful if you must restore the servers completely, although the database backup itself is almost always sufficient. You should always keep copies of all recovery materials, documents, and database and transaction-log backups at an offsite location. Also, you should perform trial data restorations periodically to verify whether your files are backed up properly. A trial restoration can reveal hardware problems that do not appear with software verifications.

Managing Backup Media

When you back up and restore a database, you must back up the data onto media (for example, tapes and disks). Your operations plan should include provisions for managing media, such as:

  • A tracking and management plan for storing and recycling backup sets.
  • A schedule for overwriting backup media.
  • A decision to use either centralized or distributed backups when you have deployed a multi-server environment.
  • A means of tracking the useful life of media.
  • A procedure to minimize the effects of the loss of a backup set or backup media.
  • A decision to store backup sets onsite or offsite, and an analysis of how this will affect recovery time.

To safeguard against a natural disaster, such as a fire or an earthquake, you should consider keeping duplicates of your server backups in a different location. This policy will help protect you against the loss of critical data. As a best practice, you should keep three copies of the backup media, and store at least one copy offsite in a properly controlled environment.

Larger enterprise clients might need to recover data in a specific time frame. These customers might benefit from SQL Server database mirroring or clustering technology. If you use these technologies, the databases for Team Foundation Server (which include the necessary data for Windows SharePoint Server and SQL Server Reporting Services) are mirrored to a second database server that is kept at a geographically distant site. You must configure SQL Server clustering when you install Team Foundation Server, but you can configure SQL Server mirroring at any time after installation. For more information about SQL Server mirroring for Team Foundation Server, see Mirroring the Team Foundation Data-Tier Server on the Microsoft Web site. For more information about clustering the data tier for Team Foundation Server, see Clustering the Data-Tier Server on the Microsoft Web site.

Backing Up Computers in Your Deployment

Because data for Team Foundation Server is stored in SQL Server databases, you do not have to back up the entire computer on which Team Foundation Server is installed. You can simply back up the databases and reporting services keys, reinstall or reimage the computers, and then restore the data. If media fails, or if a disaster that involves those computers occurs, you can reinstall Team Foundation Server, Windows SharePoint Services, or SQL Server Reporting Services to provide a cleaner and more reliable alternative to restoring from a backup image. However, you can use disk imaging to back up your computers.

Disk Imaging

To help reduce recovery time if a system fails or a disaster occurs, you can use disk imaging to back up the computers in your deployment of Team Foundation Server:

  • You should shut down your deployment of Team Foundation Server before you create any disk images.
  • For best results, you should always restore each disk image to the same hardware from which it was taken. You can cause problems if you restore an image to a hardware configuration that is different from the original system.

Use the following guidelines if you use disk imaging to back up your deployment.

Computers Running Team Foundation Server, SQL Server Reporting Services, SQL Server Analysis Services, or Windows SharePoint Services

After you create a disk image of the computer, you should update the image each time that you change the system, such as when you apply a service pack. You can then restore this image after a failure.

Computers Running SQL Server 2005

You can use disk imaging to back up computers that are running SQL Server 2005. However, you can back up only the SQL Server installation itself in this manner. You should not try to back up data for SQL Server by using disk imaging. You cannot apply log-file backups and roll forward to the point of failure after you recover data from a disk-image backup. To back up your SQL Server data, you must use database and transaction-log backups for SQL Server 2005.

Considerations for SQL Clusters in SQL Server 2005

If you are using the functionality in SQL Server 2005 for backing up and restoring data, you should consider selecting a shared drive that is separate from the Data and Logs drives. If you take this approach, you can access the resulting backup and transaction log files more easily if the primary node fails over to the secondary node.

Disk Images Shared Among Computers

You should always create a unique disk image for each computer in your deployment. If you restore the same disk image to multiple computers, you could cause problems with services that run on those computers.

Backing Up Databases

Data for Windows SharePoint Services and SQL Server Reporting Services is stored as part of the database set for Team Foundation Server. Because this data is stored in seven related databases, you might want to use marked transactions in the transaction logs of each database. Transaction-log marks help guarantee absolute consistency in the distributed data set. A transaction-log backup across all seven of the databases for Team Foundation Server will complete quickly enough to provide enough protection in most environments. However, you might prefer to use third-party hardware or software data-management solutions if the size of data or other factors in the environment require near-zero latency in the backup set.

Because SQL Server 2005 supports the Volume Shadow Copy Service, you can create snapshots of a database set by using a backup utility that leverages that technology. By taking such an approach, you would reduce the risk of minor inconsistencies when you restore the data. However, you generally do not need this level of precision. Databases for Team Foundation Server are loosely coupled; any latency issues generally are limited to relationships between things such as work items and change sets that are created during the backup window.

For more information about how to back up Team Foundation Server, see How to: Back Up a Team Foundation Server and How to: Back Up the Reporting Services Encryption Key on the Microsoft Web site.

Recovering Databases

To recover data for Team Foundation Server, you must restore its databases. All of the data for Windows SharePoint Services and SQL Server Reporting Services is stored in those databases; therefore, you do not have to perform any specific backup or restoration of those programs to recover project data. After any system failure or disaster, you start with the basic recovery steps, and then you recover specific parts of your deployment.

Before You Start a Recovery

Before you recover data for your deployment, you must perform the following tasks:

  • Make sure that your deployment of Team Foundation Server is offline.
  • Reinstall Microsoft Windows and any required service packs and software updates, if necessary.
  • Verify appropriate domain and network functionality.
  • If you are replacing a complete server, name it the same as the original server.

After you finish replacing hardware and installing the operating system, you should recover specific parts of your deployment by following the installation guide for Team Foundation Server.

Recovering the Data Tier

If you can restore data to the same computer as the original data tier, you avoid having to rename the data tier in the configuration information for Team Foundation Server. For more information, see Restoring the Team Foundation Server from Backups and How to: Restore Team Foundation Server Data on the Microsoft Web site.

If you must restore data to a computer that is different from the original data tier, you must update also each of the five locations in which configuration information for Team Foundation Server is stored. You follow the same process as if you were moving a Team Foundation Server installation from one set of hardware to another. You are, in fact, "restoring" the data to a different server. For more information, see How to: Move Your Team Foundation Server from One Hardware Configuration to Another on the Microsoft Web site.

Recovering Data to the Point of Failure

To recover your databases to the point of failure, you must be able to back up the transaction log that is currently active. If media failure or other problems prevent this strategy, you must recover the databases to an earlier point in time.

To recover to the point of failure
  • Back up the transaction log that is currently active.
  • Restore your backups of the Master and MSDB databases.
  • Restore the most recent full database backup.
  • Restore any differential database backups that were created since the most recent full backup.
  • Restore transaction-log backups up to and including the last backup log that was written before the point of failure.
  • Recover the database.

Recovering Data to an Earlier Point in Time

If you cannot back up the currently active transaction log after a system failure or disaster, you must recover the database to an earlier point in time. The procedures for this strategy vary according to whether you are recovering a deployment that comprises one database or multiple databases.

Recovering the Multiple Databases

If you want to recover your deployment to a point before the point of failure, you must use named transactions to recover data to a specific named mark.

You must restore all Team Foundation Server databases in your deployment to the same marked transaction. These databases include the database for SQL Server Reporting Services, the configuration database for Windows SharePoint Services, and the content database for Windows SharePoint Services. These databases are considered part of the seven databases for Team Foundation Server.

To recover data to a specific mark in a named transaction
  • Restore your backups of the Master and MSDB databases.
  • Restore the most recent full database backup.
  • Restore any differential database backups that were created since the most recent full backup.
  • Recover the database at the specified marked transaction in a transaction-log backup. You must use the Transact-SQL RESTORE LOG statement and the WITH STOPATMARK='mark_name' clause to restore to a named mark.
  • Repeat these steps for each database in your deployment, recovering each to the same named mark.

For more information about how to recover a database to a specific mark in a named transaction, see Recovering to a Named Transaction on the Microsoft Web site.

Recovering the Application Tier

No data for Team Foundation Server is stored on the server that is running the application tier. Therefore, you do not need recover that server in the traditional sense, although you will need to restore the keys for SQL Server Reporting Services. However, you must have a functioning application tier to restore your deployment to full functionality. If you configured a fail-over server for the application tier, you can fail over to that server without having to reinstall any software. For more information, see How to: Activate a Fail-Over Application-Tier Server on the Microsoft Web site.

You can reinstall the application tier on the same computer as before. In that case, you do not need to change the configuration information for the application tier, because the information will be the same. However, you must restore the SQL Server components on the application tier to the same service-pack and software-update levels as they were before the failure. These components also must match the version level on the data tier. If these components do not match, Team Foundation Server will not start.

If you must reinstall the application tier on a different computer, you must not only ensure that the service-pack and software-update levels are the same as before, but also you must update the configuration information before your deployment will operate correctly. For more information, see How to: Connect a Different Application-Tier Server to the Data Tier on the Microsoft Web site.

Managing Configuration Information

Team Foundation Server depends on SQL Server, SQL Server Reporting Services, Microsoft Internet Information Services (IIS), the Windows operating system, and Windows SharePoint Services. Most of the configuration information for Team Foundation Server is stored in five locations:

  • IIS configuration information - Team Foundation Server application tier
  • Team Foundation Server configuration files (web.config, proxy.config) - Team Foundation Server application tier
  • SQL Server Reporting Services data sources (for example, TFSREPORTS data) - Team Foundation Server application tier
  • Team Foundation Server integration database - Team Foundation Server data tier
  • Windows Registry - Team Foundation Server application, data, and client tiers

Before you change the configuration of Team Foundation Server in any way, you must take into account all of the sources of configuration information. Your operations plan should include managing configuration information when you perform certain tasks, such as:

  • Renaming a server that is part of a Team Foundation Server deployment.
  • Changing Team Foundation Server hardware.
  • Changing ports or protocols that are used by Team Foundation Server.
  • Changing the domain or workgroup of Team Foundation Server.

For example, if you decide to change your Team Foundation Server deployment to use HTTPS and Secure Sockets Layer (SSL), you will need to change the configuration information.

For more information about how to change the configuration information for Team Foundation Server, see Managing Team Foundation Server Configuration Settings on the Microsoft Web site. For more information about how to change registry settings for Team Foundation Server, see Registry Settings in Team Foundation Server Components on the Microsoft Web site. For more information about SQL Server databases and SQL Server Reporting Services, see Managing SQL Server Services at the Microsoft Web site. For more information about databases for Team Foundation Server, see Team Foundation Server Security Architecture on the Microsoft Web site. For a specific example of changing Team Foundation Server configuration to use different communication protocols, Team Foundation Server, HTTPS, and SSL, see Securing Team Foundation Server with HTTPS and Secure Sockets Layer (SSL) on the Microsoft Web site.

Monitoring Team Foundation Server

As with every other server technology, you should monitor the health of Team Foundation Server proactively and continuously. Team Foundation Server depends on SQL Server, IIS, ASP.NET, Windows SharePoint Services, and Microsoft Windows Server 2003. You must monitor each of these technologies to maintain the overall health of your deployment. The extent of the complete system leads to a challenge for effective monitoring. Therefore, your operations plan should include what tools are best suited for which monitoring task, what problems to monitor, and how frequently you should monitor your system.

Team Foundation Server itself is essentially a special ASP.NET SQL Server application. If you are familiar with monitoring this kind of application, you can use the same skills as you monitor Team Foundation Server.

To follow best practices for monitoring, you must answer three critical questions:

  • What should I monitor?
  • What monitoring tools can I use?
  • How do I interpret and act on the data that I get?

What to Monitor

As mentioned previously, the scope of the infrastructure for Team Foundation Server is quite large. You should decide proactively what you want to monitor and how to decide whether some value that you notice requires action. For example, you might decide to act if a CPU peaks at more than 80 percent for longer than 10 minutes. You can document this decision somewhere, so that others have a defined threshold and avoid individual personal judgments. Of course, there are hundreds of other examples. If you collect all of these variables and states at one location, you have a documented information model about the health state of your environment for Team Foundation Server. This strategy is referred to also as the health model or health information.

The health model is a set of observable conditions that define the states of the system. The previous example defines the threshold for monitoring the CPU usage. As this example shows, the health model is more about convention and consensus than science and math.

As an administrator of Team Foundation Server, you must decide what to monitor and how to assess whether a state has changed by using the thresholds as reference points. Without a health model, you have no reference points against which you can measure the health of your deployment.

What Monitoring Tools Are Available

You can use Windows Event Viewer to monitor server-state changes. Entries in this log indicate critical errors, warnings, and information. The challenge is to understand the impact of a state change and the action that is required to bring the system back to the normal state. For more information (including the list of event IDs), see Understanding Monitoring Tools for Team Foundation Server and Monitoring Event Logs on the Microsoft Web site.

If you need very detailed and fine-grained logging and diagnostic information, you can use tracing. You can activate .NET tracing to produce very detailed diagnostic information. Typically, you need this type of monitoring for troubleshooting immediate problems. However, you must have a working knowledge of Team Foundation Server and its architecture to be able to interpret this information. For more information about tracing, see How to: Use Web Services to Enable and Configure Trace for Team Foundation Server Components, How to: Change the Trace Output Directory for Team Foundation Server Components, and Trace Settings for Team Foundation Server on the Microsoft Web site.

Performance monitoring differs substantially from monitoring logs. To monitor performance, you must observe a specific set of performance counters over time. For example, you might want to monitor performance to address concerns about response time. It is difficult to respond to user complaints about the response time for downloading the source tree for a specific project, if you do not have any data about usual download times. For more information about performance monitoring, see Evaluating Team Foundation Server Performance and How to: View Team Foundation Server Performance Counters on the Microsoft Web site.

The third category of monitoring is service availability. Availability means both whether a Team Foundation Server service is functioning (up and running) and the responsiveness of that service (process time). To achieve service availability without interruptions, you have to monitor the exposed services proactively. Of course, it is a daunting job to monitor this aspect of any product constantly. If you have a monitoring system installed, it usually also covers the services monitoring. For more information about viewing Team Foundation Server services, see How to: View Team Foundation Server Services on the Microsoft Web site.

How to Interpret and Act upon the Monitoring Data

Logging data, tracing data, performance-monitoring data, and service-monitoring data all require a different approach to understand and interpret. You first must understand and verify that something has happened, and then determine an action to return the system to a healthier state, if necessary. Each deployment has its own process for acquiring this understanding and determining courses of action. However, any process will require a concentrated effort over time. You can develop your own customized response information more easily, if you keep a record of the data that you gather as you monitor your deployment and the actions that you take to respond to unfavorable changes. You might invest in a commercial software suite to help you automate the collection and retention of this data.

Monitoring Version Control

You must address many variables when you monitor version control and a team build environment. If you have a good understanding of development cycles, you can predict more accurately what to monitor closely in Version Control and Team Foundation Build. Additionally, if you understand their limits, you can address any issues more proactively. Version Control and Team Foundation Build effectively are limited by the project limits that are found with work items. For more information, see Team Foundation Server Project Limits on the Microsoft Web site.

Team Foundation Server includes many performance counters for monitoring Version Control. Depending on your focus, the counters in the following table might interest you. For a complete list of counters, see Monitoring Performance on the Microsoft Web site.

Counter Name Description

Current File Uploads

The number of files that are being uploaded at a particular moment to the Team Foundation Server Version Control service

Current File Uploads/Sec

The rate at which files are being uploaded to the Version Control service for Team Foundation Server

Current Server Requests

The number of requests that the Version Control service for Team Foundation Server is processing

Current File Downloads

The number of files that are being downloaded from the Version Control service for Team Foundation Server

Current File Downloads/Sec

The rate at which files are being downloaded from the Team Foundation Server Version Control service

Current Requests/Sec

The rate at which the Version Control service for Team Foundation Server is processing requests

Average Response Time

The average amount of time that the Version Control service for Team Foundation Server took to process a single request

Monitoring Team Foundation Build

As with any toolset, the usage that is defined by your deployment will vary dramatically. For example, a project with a single build environment with a single build script will have very different usage from a team project with multiple build environments and many build scripts. To monitor Team Foundation Build effectively, you must determine monitoring criteria that suit your deployment needs. The following list includes some items that you might want to monitor in Team Foundation Build:

  • Average time to execute a build
  • Number of times that a build occurs
    For example, a daily build should occur only once per day.
  • Number of failed builds that occurred within a specific time frame
  • Number of builds that occurred outside of business hours
  • Standard performance criteria from the server for Team Foundation Build
    For example, you can monitor percentages of CPU utilization.
  • Notification of successful builds

The following tools and procedures might help you determine some of the important factors in your Team Foundation Build environment:

Creating an Operations Plan

You can create your own list of daily, weekly, monthly, and as-needed tasks to best suit the needs of your business. In your list, you should consider linking each task to the specific how-to documentation that is available for that task online in the MSDN Library; or you might create your own how-to documents, with the specifics of your deployment of Team Foundation Server included in the steps.

The following table lists operational tasks that you might include as part of your operations plan:

Daily Weekly Monthly As-Needed Troubleshooting (Proactive) Troubleshooting (Reactive)

Back up data

Review Team Project Web sites

Run TFSSecurity to review users, groups, and permissions

Manage users and groups

Review services to ensure that all required services are running

Troubleshoot Team Foundation Server

Run builds

Review server performance data

Back up reporting services key

Control access to source control

Run trace on Team Foundation Server components

Troubleshoot Source Control

Back up transaction logs

Perform full backup

Change passwords for service accounts

Revise permissions

Review Event Log

Troubleshoot Team Foundation Build

Test restoring data/verify data backup

Overwrite backup media from previous month

Change file-type associations in Source Control

Evaluate server performance

Troubleshoot the data warehouse

Check average response time for version control

Manage service accounts

Troubleshoot Team Foundation Server Proxy

Process the data warehouse

Troubleshoot Team Foundation Reporting

Move Team Foundation Server to new hardware

Troubleshoot Team Project Creation

Test restoring Team Foundation Server data

Ideally, each item in this table would link to documentation that provides specific details about how to perform each required task in your plan.

Conclusion

Team Foundation Server is a distributed system with dependencies on Windows SharePoint Services, SQL Server, IIS, and the Windows operating system. By understanding the configuration and communication requirements of Team Foundation Server within your deployment topology and by creating your own operations plan, you can manage your deployment of Team Foundation Server more effectively.

Additional Information

You can find more information in the following topics on the Microsoft Web site:

Builds

Overview of Team Foundation Build

How to: Run a Build on a Build Type

Data

Backing Up the Team Foundation Server

Creating Transaction Log Backups

Working with Transaction Log Backups

How to: Back Up a Team Foundation Server

How to: Back Up the Reporting Services Encryption Key

Restoring the Team Foundation Server from Backups

Scheduling Synchronization with the Data Warehouse

How to: Set the Processing Interval for the Data Warehouse

Monitoring

How to: Use Web Services to Enable and Configure Trace for Team Foundation Server Components

How to: Change the Trace Output Directory for Team Foundation Server Components

Monitoring Event Logs

How to: View Team Foundation Server Performance Counters

Trace Settings for Team Foundation Server

How to: View Team Foundation Server Services

Moving

Team Foundation Server Move Types

How to: Move Your Team Foundation Server from One Hardware Configuration to Another

How to: Move Your Team Foundation Server from One Environment to Another

How to: Move from a Single-Server to a Dual-Server Deployment

patterns & practices

patterns & practices: Visual Studio Team System Guidance

Permissions

Team Foundation Server Permissions

Team Foundation Server Default Groups, Permissions, and Roles

How to: View Permissions

Team Foundation Server Administrator Permissions

Team Foundation Server Project Lead Permissions

Team Foundation Server Contributor Permissions

Service Accounts

Managing and Resetting Service Accounts and Passwords

How to: Assign a New Account to a Team Foundation Server Service

Team Project Portals

Using the Team Project Portal

How to: Upload or Save Documents to the Project Portal

Troubleshooting

Strategies for Troubleshooting Team Foundation Server

Troubleshooting Team Foundation Server Permissions and Security

Troubleshooting Team Foundation Server Performance Counters

Troubleshooting Team Foundation Server Command-Line Tools

Troubleshooting Team Explorer

Troubleshooting Microsoft Excel and Microsoft Project Work Item Integration

Troubleshooting Source Control Issues

Troubleshooting Team Foundation Build

Troubleshooting the Data Warehouse

Troubleshooting Team Foundation Server Proxy

Troubleshooting Team Foundation Reporting

Troubleshooting the New Team Project Wizard

Users and Groups

Managing Users and Groups

Default Groups

Custom Groups

Utilities

TFSSecurity Command-Line Utility Commands

Permission Command

Version Control

File Types

How to: Control Access to Team Foundation Source Control

Author Bio

Mark Brown is a Team System Ranger and a Senior Consultant with Microsoft Consulting Services in the United States.

David Caufield is a Team System Ranger and a Lead Architect with Microsoft Consulting Services in the United States.

Mario Contreras is a Team System Ranger and a Senior Consultant with Microsoft Canada's MCS group.

Bill Essary is a software architect for Team Foundation.

Susan Ferrell is a technical writer and the content lead for Team Foundation Server on the user-education team for Visual Studio Team System.

Sudhir Hasbe is the program manager for Team Foundation Server Administration and Operations.

Bijan Javidi is the Team System Lead Ranger and a solutions architect for Visual Studio Team System.

John Steer is a Senior Security Consultant with the Microsoft ACE Services team.

Jeremiah Talkar is a Team System Ranger and a Solutions Architect with the Microsoft Mission Critical Program.