Visual Studio 2005 Team System: Designing Distributed Systems for Deployment

Article
03/22/2011

Bill Gibson and Alex Torone
Microsoft Corporation

May 2004

Applies to:
Microsoft Visual Studio® 2005 Team System

Summary: This article examines software design tools available in Visual Studio 2005 Team System for building distributed applications. (12 printed pages)

Note This document was developed prior to the product's release to manufacturing, and as such, you may find inconsistencies with the details included here and those found in the shipping product. The information is based on the product at the time this document was created and should be used for planning purposes only. Information is subject to change at any time without prior notice. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Introduction
Challenges of Designing Distributed Systems
Improving Distributed System Design and Deployment
Overview of the Distributed System Designers
Extensibility
Integrating with Visual Studio 2005 Team System
Conclusion

Introduction

The Distributed System Designers are tools that support the graphical design and validation of distributed systems. This toolset includes tools for application architects, designers, developers and operations architects. The Distributed System Designers are an early deliverable from the Dynamic Systems Initiative (DSI), aimed at improving the development, deployment and management of enterprise-class distributed systems. For more information about DSI, see Microsoft Dynamic Systems Initiative. These tools are part of Visual Studio 2005 Team System.

Service-oriented architectures serve as the basis for the next generation of distributed applications. The Microsoft "Indigo" platform will provide the industry-leading implementation of service-oriented systems. "Indigo" will build upon the SOAP and Web services support provided in the Windows platform today and add support for a comprehensive range of transports and system topologies, enabling secure, reliable, and durable message-based communications between services. While there are important advances inherent in the Indigo platform, enterprises can get started now developing service-oriented systems using SOAP, XML messaging, and ASP.NET Web services. For more information, see Indigo.

Challenges of Designing Distributed Systems

Designing and deploying distributed systems is a fairly complicated process. This section explains some of the problems that can arise.

Visualizing the Design of Distributed Systems

Visualizing the system structure as an integral whole becomes increasingly difficult as the system becomes more and more fragmented in a service-oriented architecture. In addition, enterprises typically have many different systems that have accumulated over time as departments purchase, develop, or acquire a variety of applications. Because individual systems might use any number of programming technologies, it is frequently difficult to share functionality and data between them. To enable interoperability, developers and architects are increasingly required to design message-based interfaces; designing new messages and ensuring conformance with existing message schemas is crucial. Interoperation through messages is at the heart of service-oriented architectures.

Keeping the Design and Code Synchronized

Keeping system design documentation current requires close communication between architects and developers. But even with the best of intentions, system design documentation frequently becomes out-dated and inaccurate as soon as coding begins. Synchronizing design documents with fast-changing code quickly becomes an overwhelming task that is soon abandoned.

Designing for Deployment

Software and hardware vendors often presume developers are aware of every nuance of platform configuration (SQL, IIS, BizTalk, and so on), and that operations management has a thorough understanding of frameworks and message protocols used by application developers. While operations should be part of the overall software development life cycle, this group is organizationally and functionally disconnected from development. Operations and development rarely collaborate proactively, and often come together reactively and late in the development cycle to diagnose problems that could have been prevented earlier.

Consider the simple case of developing and deploying a Web service. While the developer's primary concern is the implementation of the service, they invariably have to concern themselves with security and authentication models, what other supporting services are required on the target environment, and what configuration settings are required to cause the Web service to operate as desired. Operations needs to understand what protocols and services are required by the new service, and whether corporate IT policy has been followed. The disconnect between development and operations leads to deployment issues often resulting in mismatched configuration, or worse, a design that is incompatible with the datacenter resulting in a large amount of the IT budget spent correcting deployment problems.

While many organizations attempt to address the communication issue with documentation, design reviews, and elaborate diagrams, they simply lack tools and a common language to effectively enforce and communicate their policies. Furthermore, these processes are so far removed from the actual tools that developers and operations use everyday that the process itself contributes to the problem.

Configuring Applications for Security

Securing a distributed application is a time-consuming and complicated process because numerous technologies and settings that can impact the design of an application are involved. Currently, there is no integrated way to represent the security configuration of the application or the security requirements of the datacenter while designing the application. As a result, it is hard to ensure that security is implemented correctly.

Improving Distributed System Design and Deployment

The Distributed System Designers provide an integrated design experience with the goal of enabling the visual design and validation of distributed systems. Designers use the System Definition Model (SDM) as the underlying meta-model that describes connectivity, configuration and relationships not only for the application services, but also for the run-time environments. SDM is based on a multi-layered model that includes applications, application hosting environments, network topology, operating systems, and physical devices. This model allows the Distributed System Designers to not only describe designs at each layer, but also to express constraints and policies at each layer that can cut across all layers of a distributed system.

The Distributed System Designers support an integrated model of two domains—development and operations. This allows the designers to address customer problems in the following ways:

Provides a common language (based on SDM) for describing the design and configuration of a distributed system.
Allows developers to express what the application requires of the run-time environment.
Allows operations to express application runtime, security, and connectivity requirements that are the policies of the target deployment environment.
Uses abstractions that allow developers and operations to communicate on common ground.
Integrates with the existing Visual Studio project system and .NET technologies.
Provides full synchronization between visual design elements and code.
Includes an extensibility framework to allow new kinds of application and hosting systems to be modeled.

The following section elaborates on the capabilities of the individual designers and editors in the Distributed System Designers toolset.

Overview of the Distributed System Designers

The Distributed System Designers include the following designers:

Application Connection Designer
Logical Datacenter Designer
System Designer
Deployment Designer

Application Connection Designer

The Application Connection Designer (ACD) helps a developer or architect to define and configure applications that will be composed into systems for deployment. A set of pre-defined application prototypes is provided in the Toolbox, which will include Web services, Web applications, Windows applications, External Database, External Web services, and External BizTalk services. A generic application prototype is also included for other application types to be documented in the design.

The application connection diagram shows the applications defined within a single Visual Studio solution. The diagram can be created from scratch by adding each application from the Toolbox, or it can be reverse engineered from an existing solution or from existing projects.

An example application connection diagram is shown Figure 1.

Figure 1. Application Connection Designer

The ACD shows both "internal" and "external" applications. Internal applications are individually deployable applications being defined within the solution from which one or more deployable systems might be composed. Each internal application typically results in a single project that defines a host-able executable (for example, a Web application or Windows application). Resources can be included in this project or located in other referenced or shared projects, including class library projects. External applications allow developers to visualize references to other systems that will need to be resolved during deployment.

Each application is depicted as a box, with smaller shapes representing connection points, either to or from the application. These are referred to as endpoints. Services offered by the application are represented by provider endpoints (solid shapes); connection points to services offered by other applications are represented by consumer endpoints (clear shapes). Applications are connected to each other through their endpoints.

While the SDM model focuses on the type, configuration and connection of endpoints, individual applications can extend this model to represent the definition of the behavior provided by a service. Web service endpoints can be defined, for example, in one of two ways: clicking on a Web service provider endpoint allows the definition of operations for that Web service in the Endpoint Details window; or, the behavior definition can be imported from an existing Web service description language (.wsdl) file. By extending the model in this way, the ACD supports a complete design experience, allowing the complete specification of the behavior and configuration of Web service applications, for example.

The connections shown on the ACD depict the current configuration of the applications within the development environment. If the application is debugged, it will follow the connection pathways shown.

The ACD supports a deferred implementation approach, allowing a user to create and validate a design before committing the design to code. Adding an application definition to the ACD from the Toolbox does not create the corresponding projects, code and configuration files immediately. First-time generation of code is referred to as "implementation." Application definitions can be implemented incrementally or all at once. An implemented application is distinguished by a drop-shadow on the diagram surface. Once an individual application is implemented, the generated code files, configuration files, and the application design displayed on the diagram surface are kept continuously synchronized as long as the diagram and code files are open. If the diagram is closed for some reason, re-opening it will synchronize it with the code, and update it with any changes made to the code while the diagram was closed. In this way, the design model in the ACD can easily be kept current with any changes made to the code, a common problem with other tools. If the application is created by reverse engineering an existing project or solution, it is already considered to be implemented and is automatically synchronized thereafter.

The ability to defer implementation allows architects and application designers to concentrate on the functional design and validation of the system and defer some of the implementation decisions that might best be made later by the development team. These decisions might include choosing the programming language and template, or the server location used for Web projects.

Because the diagram is stored as a file (.dsdgm file), it can be included in source code control and incorporated into the normal workflow of a team. Information about implemented applications is stored in .sdm files within each project. This allows projects to be reused in multiple solutions and work to be divided among the developers on a team.

The ACD provides access to a complete model of application configuration settings and the ability to define hosting constraints. Hosting constraints allow the application designer to specify requirements on the hosting environment. By defining hosting constraints on application definitions in the Settings and Constraints Editor, application developers can request a required set of features be available in the target deployment environment, thus communicating to operations important application requirements. Operations can then decide whether they want to make changes to the logical datacenter to accommodate those requests. The application settings and hosting constraints are validated against logical datacenter settings and constraints as part of deployment validation (see Deployment Designer).

Logical Datacenter Designer

The Logical Datacenter Designer (LDD) is used to create diagrams of interconnected logical servers that represent the logical structure of a datacenter. These logical datacenter diagrams communicate important information to the developer about the target deployment environment. Using this designer, operations architects can specify and configure the types of servers in the datacenter, the types of communications allowed, specific communication pathways, and types of services enabled.

Logical datacenter diagrams will typically be created and owned by operations analysts, but will be used by developers. After creating a logical datacenter diagram, the operations analyst can lock and version the diagram throughout the application development life cycle to keep it current with datacenter design changes. Like the other Distributed System Designers, the Logical Datacenter Designer is fully integrated with Visual Studio. However, logical datacenter diagrams are created independently of the application development process. These diagrams are stored as .lsad files and can be reused by other distributed systems that will be deployed in the same target environment.

An example of a simple logical datacenter diagram in the process of being built is shown in the following illustration.

Figure 2. Logical Datacenter Designer

In the LDD, the Toolbox contains pre-defined logical server prototypes that can be added to the diagram surface. Logical servers represent application hosts in the datacenter. Native types include a Windows Client server, Web server (IIS server), SQL Server, and a Generic server. With the exception of the Generic server, each prototype defines a group of settings and applicable constraints that are used to configure the server. The Generic server is included for documentation purposes only and allows the existence of other types of servers within the datacenter to be modeled. Each logical server has two endpoints specifying a specific communication protocol. Endpoints can be added to logical servers to communicate with other logical servers through that endpoint.

By editing the provided settings, the configuration of logical servers within the datacenter can be modeled. Instead of editing the provided settings, these can also be imported from real servers. Once the datacenter is described, policy constraints regarding the types and configuration of applications that can be hosted in this representation of the datacenter can also be specified using the Settings and Constraints Editor. These constraints are authored against settings in the application layer because they are used to constrain the types of applications the servers can host and to specify the configuration required for the applications. For example, ASP.NET security settings can be constrained for applications hosted on a Web server or the version of the .NET Framework required for applications hosted on a Windows Client Server. Marking certain settings as fixed specifies that these settings cannot be overridden by development.

The Settings and Constraints Editor is shown in Figure 3.

Figure 3. Settings and Constraints Editor

The Toolbox also contains zones and endpoints. Zones represent communication boundaries in the datacenter. Zone endpoints represent connection points between zones and servers. Communication pathways between zones and servers are controlled through zone endpoints, which can be configured to determine the types of communication protocols that are allowed into and out of the zone. Server endpoints are used to specify communication pathways and protocols allowed between servers and those used to send or receive traffic across zones.

The configuration settings and application constraints defined in the logical datacenter diagram will be validated against the settings and constraints defined on applications as a part of deployment validation (see Deployment Designer). In this way, datacenter policies and deployment requirements can be communicated from operations to development teams.

System Designer

The System Designer is used to compose and configure systems from the applications that are defined in the ACD. In the Distributed System Designers, a system is the unit of deployment, and is a configuration of one or more applications and other sub-systems. Because systems can be composed from other systems, they allow large-scale distributed system scenarios to be defined. An application architect can design a complex multi-tiered system as a hierarchy of "nested" systems.

Creating a system definition allows a deployment configuration to be defined independently of the development configuration shown on the ACD. Where allowed, application settings defined on an application definition in the ACD can be overridden on the application use in a system diagram. Multiple system definitions can be created, each with distinct configurations of the applications defined in the solution. This allows different configurations to be defined for different planned deployments, and perhaps for different datacenter configurations, different geographic deployments or for different customers.

An example system diagram containing a nested system is shown in Figure 4.

Figure 4. System Designer

Deployment Designer

The Deployment Designer is used to define the deployment of a specific system into a target logical datacenter. The Deployment Designer will typically be used by developers and architects. In the Deployment Designer, applications in the system are bound to logical servers in the target datacenter. The functionality offered by this designer highlights the communication benefits inherent in the SDM model that underlies the Distributed System Designers. Once a deployment is defined, it can be validated on demand. Validation confirms that all applications in the system are bound to logical servers and then checks that the applications meet the application constraints specified in the logical datacenter diagram. Validation also confirms that logical servers meet the hosting constraints specified on the application connection diagram and the system diagram.

Validation also ensures that required communication pathways exist and determines whether the correct communication protocols exist and are compatible between applications and host servers.

An example deployment diagram is shown in Figure 5.

Figure 5. Deployment Designer

All validation errors are displayed in the Visual Studio Error List, which provides a simple navigation mechanism for correcting and reconciling errors. When an error in the Error List is double clicked, the Deployment Designer opens the appropriate diagram, selects the appropriate application or logical server, and navigates to the appropriate setting, allowing it to be corrected. This provides the ability to correct configuration errors before deployment, even before a complete implementation of the system. From the Deployment Designer, a report of all required application and datacenter configuration settings can be generated and used to create a script for custom deployment tools.

Extensibility

The Distributed System Designers will include an extensibility model that allows partners to extend the platform and add support for other protocols, and additional kinds of applications and logical servers.

Integrating with Visual Studio 2005 Team System

The Distributed System Designers provide support for design and deployment of distributed applications. The designers integrate with other Visual Studio 2005 Team System tools in several ways:

Distributed system development projects designed, developed, and tested for deployment using the Distributed System Designers, are managed using Visual Studio 2005 Portfolio Management Tools.
Applications are unit tested using Visual Studio 2005 Team Test.
Source code control and integrated work item tracking using Visual Studio 2005 Team Foundation Server and Client.

Conclusion

The Distributed System Designers are a collection of designers that simplify the process of designing and validating distributed systems based on service-oriented architectures.

Service-Oriented Architectures: Service-oriented architectures are expected to serve as the basis for the next generation of distributed applications, but it is difficult to visualize and design their necessarily fragmented structure. The Application Connection Designer and System Designer will help application architects, designers, and developers visualize and design service-oriented systems as a whole.
Design for Operations: The Distributed System Designers create diagrams to represent the application layer and the application hosting layer of a system. These diagrams help those who design and deploy applications to validate an application configuration against operational requirements when the application is first deployed and throughout its lifetime, as the services that comprise it evolve.
Executable Design: Through the unified model in the Distributed System Designers, application and infrastructure architects capture rich metadata during the design process. This metadata can be used both to generate code and to validate application design against the datacenter definition. This model lets application architects create a more formal and rigorous design. The design stays synchronized with the code, remaining relevant throughout the application's lifetime, which helps application architects maintain and extend the application as needed.

The Distributed System Designers help enterprises successfully construct Web services-based service-oriented applications and efficiently prepare them for successful deployment in the datacenter.

For more information on the other members of the Visual Studio 2005 Team System, see: