Enabling the Service-Oriented Enterprise

by William Oellermann

Summary: Building Web services is pretty easy today. Building lots of Web services is a little more difficult, and managing those Web services is really difficult. As the number of services and service consumers grow, the fundamental benefits of service orientation diminish if certain IT capabilities are not provided. Let's look at a model that can facilitate the identification and prioritization of these capabilities for the service-enabled enterprise.

Contents

A Capability-Driven Maturity Model
Road Map Perspectives
ESOMM Capabilities
Repeatable Layer Capabilities
Supportable Layer Capabilities
Extensible Layer Capabilities
A Word on Maturity Models
Applying the Model
About the Author
Resources

When talking with customers and colleagues about Web services, you'll find that much of the excitement and interest is focused on the specifications that have helped this technology quickly become a de facto standard: SOAP, WSDL, XML, WS-Security, and so forth. While this standardization is critical, it often distracts us from the next level of discussions about how to really take full advantage of this common stack. These protocols give us greater reach and reuse than we have ever previously achieved with software, but their use alone will not fulfill that potential.

All the work invested in standards and consortiums allows us to develop services that can interoperate with other applications and systems, but it doesn't necessarily buy us reuse. Even more importantly, it doesn't buy us manageability. If you happen to work in or with an organization that has attained some success with Web services, you have no doubt encountered the need to address, if not the pain of not addressing, service testing, deployment, versioning, and management. These areas become very problematic when they are not identified prior to building a portfolio of services.

This condition has already made some organizations victims of their own success. Approaches that work with a handful of services often falter as the number of services and their usage increase. If you don't have the principles and practices in place to promote reuse and provide manageability as your use of services grow, you will hit the "scalability ceiling." (See the sidebar, "A Word on Maturity Models.")

Discussing this issue is very difficult, even among the savviest and most experienced of architects, much less other less-technical participants that play a role in the services ecosystem. Some of this difficulty stems from the use of overloaded terms such as service, but much of it comes from the sheer complexity involved with a massively distributed architecture that touches many areas of an organization. If something is difficult to discuss, just think of how difficult it can be to define a strategy or plan for it.

These challenges led us to look at the use of a model to help facilitate the discussion of service enablement and the planning of a service-enabled enterprise. With the contributions and feedback of many organizations and thoughtful contributors over the last two years, we have developed the Enterprise Service Orientation Maturity Model (ESOMM).

A Capability-Driven Maturity Model

All 4 layers, 3 perspectives, and 27 capabilities defined in the ESOMM are designed as a road map to support services—not any specific service with any specific use, implementation, or application, but any service, or more specifically, any set of services. If you intend to develop only one service, the value this model provides will be limited. The objective of ESOMM is to help you overcome the problem of scalability at a group, department, division, or enterprise level. This objective means it can be applied as part of a grassroots initiative or an enterprise strategy. Your ability to provide certain capabilities might be limited by the extent of your overall organizational alignment, but that factor should not impede your ability to start building and implementing a plan leveraging ESOMM.

Developing a service-enablement strategy is not a trivial undertaking and should not be treated as a short-term or one-time effort. At its apex, service orientation is intended to enable a higher level of agility for an organization, so it can provide an expedient response to the needs of the business and its customers. Successfully achieving this agility means providing a plan that is both extensible and durable. By focusing on the service, and not the message, implementation, or usage, ESOMM can help you to build such a plan that can be applied broadly by any organization. As the preferred service implementations, technology choices, and usage patterns change, the model should provide a durable road map that can endure those changes.

ESOMM consists of four layers, each containing a set of capabilities that as a whole establish a solid level of accomplishment and value to the organization for that layer (see Figure 1). This value should be fully realized if all the attached capabilities are provided. It is possible, and certainly likely, that an organization may address parts of multiple layers; the objective is not to satisfy a layer so much as it is to identify the level of value an organization desires.

The first layer of maturity in ESOMM, called usable, consists of the appropriate use of standards and protocols to design and develop services that are usable across an organization's platforms and technologies. Building services correctly is absolutely critical before you can successfully establish many of the higher-order capabilities up the SOA stack. The true test of how well your services are built is how much commonality and consistency is present across all of your services, regardless of the tools and technologies used.

The repeatable layer addresses many of the capabilities that make services efficient to develop, deploy, and maintain across your organization. Without addressing key capabilities in this area, it will be very difficult to successfully scale your use of Web services.

The supportable layer encapsulates many of the capabilities necessary to reliably provide services for mission-critical applications that are supportable. These capabilities include the operational support of services, but just as important, the self-service capabilities provided to consumers. Self service is an important tool to enable your services to scale by keeping your developers and architects from spending an inordinate amount of time in correspondence with potential consumers of their services. This layer is very difficult to approach successfully without some success in making services repeatable in their deployment and management.

The extensible layer represents the pinnacle of realizing the business agility promised by services and the reuse of services as the building blocks of other services, or service aggregation. This notion usually involves providing capabilities direct to customers and/or partners through services that generate new revenue channels, if not new business models. There are several capabilities required to make this realization possible, and they make up the majority of the focus at this extensible layer.

Road Map Perspectives

Through engagements and interaction with customers over the last couple of years, we recognized a need to address different perspectives through this model. As we discussed the capabilities in each layer of the model, it became apparent that there were varying levels of appreciation or consideration for them, depending on the audience's role and challenges. For instance, one organization had a hard time appreciating the need for contract-driven interface design. They were quite content using tools to dynamically generate service contracts and felt they were very successful with that approach.

When talking with the users of the group's services, however, we got an entirely different perspective by learning that the use of specific data types and element patterns caused a lot of difficulty during implementation. By recognizing that not all capabilities are driven by the service owners or providers, stakeholders are able to have greater appreciation for some of the capabilities and make more informed decisions about approaching and prioritizing them.

Figure 1. The four maturity layers of ESOMM (Click on the picture for a larger image)

The capabilities in each layer of the ESOMM are categorized into one of the three perspectives: implementation, consumption, and administration (see Figure 2). Implementation capabilities target the development and deployment of Web services from the provider's perspective. This perspective is the one most organizations appreciate by default and requires the least amount of clarification. Consumption capabilities are those that cater to the consumers of your services, making them easier to implement and therefore more widely and successfully adopted. We believe this is the most neglected perspective, yet it can be the most critical in determining your success with service orientation. If it is difficult to leverage your services, your success will be limited, no matter what other areas you have optimized. Administration capabilities use the operational and governance aspects of Web services across the organization. This perspective is becoming more recognized and appreciated, but there is still more that needs to be understood about its value and impact.

The additional value of perspectives is the aid they provide in prioritizing the implementation of specific capabilities in the model. While there is an ability to ignore certain capabilities while progressing up the model, it is fairly risky to skip capabilities at a lower layer within the same perspective. Those prerequisites are intended to not only help you grow your system logically, but also to help you learn how services are utilized and supported across your organization because this information will impact the design of certain capabilities at higher layers.

It is also important to realize that an organization can choose to focus on one or two of the perspectives, but the overall maturity, and, hence, the overall value of your services, depends on the appropriate level of attention to all three. Neglecting any of the three will have consequences—some more readily apparent than others. Such consequences would be akin to ignoring one of the food groups; you can do it, but it would limit some of the meals you could enjoy and not be ideal for your overall health.

ESOMM Capabilities

There are 27 capabilities outlined in the ESOMM model, and each could easily take an article to explain in full detail (see Figure 3). Here, we'll look at a brief overview of them to help articulate the premise of each capability and its alignment to the associated perspective and layer.

The implementation perspective includes the development processes and design patterns capabilities in the usable layer (1):

Development processes. Like design patterns, development processes speak to a concept with which development teams are very familiar, but is focused specifically on the implementation of services. At a basic level, this concept would generally consist of the use of common standards supported across the most common platforms and tools, including SOAP, WSDL, WSA protocols, and the WS-I Profiles. At a more advanced level, the capability would include the definition of repeatable processes for the design of service interfaces, the deployment of services, and the publication of services. Again, the objective is to make these processes broadly available across platform boundaries.

Design patterns. Whenever you develop software, it is good to take advantage of proven design patterns that are appropriate for that implementation. This particular capability speaks to the design patterns that are appropriate and necessary, specifically for use with services. We have good information available in this space, starting with the four tenets of service orientation (see Resources). Other examples of relevant patterns include asynchronous execution, callback support, exception handling, and the use of custom SOAP header data. The key to fulfilling this capability is defining patterns that can be used by developers across all the appropriate tools and platforms.

From the consumption perspective of the usable level there are the explicit contracts and testability capabilities:

Figure 2. Three perspectives on the maturity layers (Click on the picture for a larger image)

Explicit contracts. The most basic information you should provide to consumers of your services is explicit contracts, which includes not only richly detailed schemas (leveraging the power of XSD), but rich policy data that shares appropriate context information. This information allows them to know not only the makeup of the body of the message payload, but the header elements that are supported and required. The difficulty of this capability is that there is very little support for some of the standards emerging around the sharing of policy, so it requires some thought and effort to address sufficiently.

Testability. This capability provides consumers with a mechanism to test the invocation of a service to support implementation and troubleshooting from their side. One of the most challenging aspects of implementing distributed systems is integration testing. With so many dependencies, it can take a lot of coordination to simply complete a call through the entire chain, even for a null operation or an invalid request. Web services are in a unique position to abstract applications from the back-end systems performing the processing; a testing environment, either physical or virtual, can be a strategic tool in improving implementation efficiency and reducing the need for manual support.

The administration perspective of the usable layer contains the basic monitoring and security model capabilities:

Basic monitoring. The support of Web services starts with support for the basic infrastructure hosting and providing those services. This support includes the basics of operational excellence, which includes the monitoring of key counters and logs that could indicate processing issues and provide alerts on the triggers indicating outages for critical services. This capability does not include service-level visibility or monitoring, as that is not a trivial undertaking and will come further up the model's stack.

Security model. Once the adoption of services takes hold, organizations will find them used in a multitude of applications—both external and internal applications. It is important to have a standard security model that can support each of the primary scenarios to meet business requirements. This model involves creating one or more credential stores that can be managed and supported as part of your services ecosystem. Examples of these credential stores include certificate authorities, an LDAP directory, Active Directory directory services, and databases. While credentials make up a critical component of your service-security model, fully addressing this capability also means supporting the requirements around access, authentication, authorization, and encryption. These approaches don't necessarily need to be implemented as features, if they are not required at the time, but a complete security model needs to be defined for when the time comes. This implementation will likely be the first step in defining your service policy.

Repeatable Layer Capabilities

The implementation perspective includes the service blocks and common schema capabilities in the repeatable layer (2):

Service blocks. A key to reducing the amount of "routine" logic pushed down to each and every service is the use of a concept called service blocks. This capability is an architecture model that provides common routines that should be leveraged broadly such as transformations, point-data lookups, and utility processes. One example of a service block would be a logging service, and in fact our logging application block could be service-enabled to meet this need. For this model to be successful, these service blocks should be available to any and all services across your organization.

Common schema. When providing services that work with common entities, it is very helpful to have a common definition for those entities to minimize the amount of effort necessary to provide and consume those services. Because of the multiple systems typically involved in providing the business logic and data for services, this definition is usually an ongoing effort that will not necessarily ever be considered complete. However, even establishing common definitions for a handful of core entities can reap tremendous benefits for service-enablement efforts.

The consumption perspective for the repeatable layer includes the service discoverability and self provisioning capabilities:

Service discoverability. Once you start to provide more than a handful of services, discoverability becomes a key capability to help efficiently guide consumers in the identification of the appropriate service to use. This guidance can be accomplished manually or programmatically with tools (for example, through UDDI). Unfortunately, just providing a list of services on a Web page doesn't prove sufficient at this level, so one of the keys to guiding the discovery process is intelligent categorization. One or more well thought out taxonomies can help potential consumers look intuitively in the correct places for services based on a domain or specific implementation.

Self provisioning. As you develop more services, it will become more challenging for consumers to request access to them. While you, as a provider, might be interested in automating the back end of the user-provisioning process, it is more important early on to allow consumers to quickly identify and leverage the tools available to request access. If this process is too difficult or problematic, your adoption rate will likely suffer, limiting your overall success. This capability can be achieved with a wide range of solutions ranging from very simple to very complex.

The administration perspective of the repeatable level includes the enterprise policies and deployment management capabilities:

Enterprise policies. Service policies are important to define, support, and enforce as you grow your organization's use of services. The starting point for this effort was developing a security model, but at this level it is necessary to broaden the policy to support a set of possible options available depending on the different criteria of a given scenario. For example, a service used within a department will likely have a very different set of policies from the same service used by partners. At this level, the definition of internal categorization(s) becomes important as it can help you to define baseline templates aligned with some of your most common scenarios to promote consistency and efficiency. Success in this area isn't driven as much by the choices made as it is by the establishment, use, and distribution of policies to those impacted.

Deployment management. The best way to promote and enforce a consistent governance model for your organization is to define a common deployment process for services that provides real value. For instance, if your deployment process automatically provides services with operational support, it will quickly become widely adopted. To further strengthen its role, it is also important that this process is flexible and easy to use, so that developers, architects, testers, and operators across the organization will buy into and adhere to the established model.

Supportable Layer Capabilities

The implementation perspective of the supportable level (3) contains the schema bank, executable policy, and versioning capabilities. Let's look at them in more detail:

Schema bank. Creating and supporting a schema bank is the next step in leveraging a common schema. Doing so allows developers to leverage and reuse existing entity definitions to build messages that are able to adapt automatically to schema extensions through includes and imports. This leverage is a very challenging area that very few customers are successfully tackling today, but will quickly become an invaluable tool as your services start to evolve.

Executable policy. While the request message initiates the invocation of a service, you will often want to leverage policy information to determine exactly what should be executed under certain conditions. Policies are often thought of as a tool for external use, but policies can also be a powerful tool for providing context internally. If you need to change the execution process or behavior based on who is making the request, what format the data takes, or what time of day it is, executable policy will play an important role in service enablement.

Versioning. The versioning of Web services is something that can and should be handled differently from the typical approaches for other software because of its complexity and extensibility. For instance, the versioning of a service needs to be treated separately from the versioning of the messages involved in the service. Fortunately, with the inherent flexibility of XML and XML schemas, there is an ability to support this complexity through both implicit and explicit versioning methods. However, this ability is a feature that the typical service development tools do not take advantage of out of the box and requires some planning and support to execute consistently.

Figure 3. Capabilities of the maturity layers (Click on the picture for a larger image)

The consumption perspective of the supportable layer is where we find the execution visibility, service portal, and explicit SLAs capabilities:

Execution visibility. Perhaps the single, biggest factor in your ability to scale your use of Web services is your ability to support increasing service consumption by providing rich self-service capabilities to potential and existing consumers. If every implementation of a service requires a series of phone calls or e-mails, some simple math will show you that your team's ability to develop services will quickly be consumed by support needs. A single portal that provides access to the right information can be a huge factor in reducing the demand on your development and support teams.

Service portal. This capability speaks to a specific resource that could be accessed through the self-service portal. To make your service consumers more self-sufficient, provide them ample visibility into their use of your services. More visibility builds on the ability to test services by providing a level of partitioned access that helps them to understand what is occurring during execution to use troubleshooting and also deduce whether something is failing to meet their expectations. This capability would also share historical activity to help consumers track how often the service is consumed by their application(s).

Explicit SLAs. To effectively set expectations for your services to consumers, you must provide explicit service-level agreements (SLAs). Providing SLAs is a very challenging capability because there is not a standard available to communicate this information for Web services. However, various forms of documentation or communication can be used to effectively share important information in this area including the expected lifespan, the anticipated processing time for requests, and any planned downtime. It is important to note that without a sufficient level of service visibility and management internally, it will be nearly impossible to support SLAs.

The administration perspective of the supportable level contains the provisioning model, monitoring, and auditing capabilities:

Provisioning model. The next step in making services self sufficient and supportable is a streamlined process for provisioning new users. This model would be used to support external and internal users and would include the handling of requests for access, creating new credentials, and providing alerts notifying consumers of changes to services or account status. This capability builds on the self-provisioning capability defined in the repeatable layer (2) to provide an end-to-end provisioning process for service consumers.

Monitoring. Monitoring and visibility into service execution is one of the most challenging aspects of supporting services on an enterprise scale because of the distributed nature of Web services. The only way to address this support effectively is to embrace your services as a system because support on a service-by-service basis is an inefficient and frustrating endeavor. While there are many ways to establish this system, we believe that a common execution architecture is the most effective means.

Auditing. The key to truly trusting the services you provide in your organization is a reliable audit model that can serve as a tracking mechanism for who did what and when. This mechanism not only helps to resolve any disputes that arise, but can also play a strategic rule in an enterprise-compliance program.

Extensible Layer Capabilities

Now, let's look at the implementation perspective's capabilities for the extensible layer (4), service orchestration and service collaboration:

Service orchestration. To leverage Web services in well-defined business processes, you must have the ability to orchestrate. This ability involves workflows that may involve manual as well as automated processes, but it allows you to define a fairly consistent and repeatable usage of services that can be changed over time.

Service collaboration. Once you've established services as autonomous functions that are maintainable and supportable, you can start to aggregate them to provide more sophisticated and complex services. We refer to this aggregation as service collaboration. Through collaboration, you can define synchronous and asynchronous execution patterns that allow you to support entire processes consisting of existing services. As an example, you could take a service that sends an e-mail and a service that retrieves all schedule changes and expose a new service that sends an e-mail to customers notifying them of schedule changes.

The consumption perspective of the extensible layer includes the external policy and service SDKs capabilities:

External policy. To establish a secure infrastructure for providing services to external consumers and partners, you must establish external policies, which include not only externally supportable security policies, but the policies you have defined for how your services can be bundled with other third-party services. Such policies include branding, payment, and usage terms that are important for protecting your business interests.

Service SDKs. Building a business model or channel through Web services requires taking the level of support for service consumers to the next level. This advance to the next level could include developing and providing prepackaged software development kits (SDKs) that allow developers to quickly provision and implement your services. These SDKs potentially include UI components for direct use in the developer's application(s) and would be tailored to target development environments, as there is no standard for cross-platform development tools today.

Finally, the administration perspective of the extensible level includes the automated policy management and business analytics capabilities:

Automated policy management: Once you start exposing services to a very broad audience and using policies to define your interoperability support, you will start to get requests for changes in policy—for example, changes to SLAs or security requirements. These requests should be processed in an automated fashion. That way they don't create significant overhead in scaling your interaction with a broad customer base.

Business analytics. The pinnacle of visibility across a system of enterprise services is what we refer to as service analytics. This visibility combines the technical data from service metadata, activity logs, and events and allows you to cross reference it with policy information, business data, and user data to provide rich views into every dimension of your services. This information can be used to determine what services are making money or what user tendencies are present between internal and external consumers of your services.

A Word on Maturity Models

Like other maturity models, ESOMM was derived from Carnegie Mellon's Capability Maturity Model (CMM, see Resources). However, we borrowed the notion of a capability-driven maturity model and stopped there. Instead of applying service orientation to the CMM, we took those principles and applied them to the service-orientation paradigm, essentially building the road map from scratch. This model does not focus on processes because the focus is on IT capabilities, not organizational readiness or adoption. While you will recognize some conceptual similarities with CMM, you can also see that ESOMM is a decidedly different application of the maturity model concept.

Although there can be value realized through some of the other service-related maturity models out there, be aware of how much of the CMM implementation it subsumes. CMM was designed to address an organization's processes related to software engineering, which is a decidedly different problem from defining an IT plan for service orientation. If you adopt a model that blends these two, it could be overwhelming and counterproductive for the organization. Even worse, if you adopt a model that simply renamed some of the CMM levels and capabilities, you could be going in the wrong direction. That would be like taking a map of England with the name London replaced by Paris and trying to go to the Eiffel Tower: It simply isn't going to work!

Buying into the capability-driven maturity model is one decision. Choosing the appropriate road map is another. Make sure you are picking a map that applies to your objectives. No maturity model applies to everyone in all circumstances; make sure you choose one that takes you to your destination.

Applying the Model

Now that we've had a chance to briefly review the components of ESOMM, you are hopefully already thinking about how it can be applied to your organization. To be clear, applying this model should not be looked at as a one-time activity or short-term process. Instead, the model is best leveraged as a working plan that can be modified over time as the usage of services and your experience grows.

Unfortunately, the downside of using the term maturity with a model is that people will immediately want to know what layer their organization is at to get a sense of their status or identity. As it happens, there is no appropriate way to answer the question, "what layer is my organization?" Instead of giving an overall grade based on one of the layers, we take the approach of giving each layer its own level of maturity, ranging from one to five, based on half-point increments.

Table 1. A sample ESOMM rating

A maturity level of one means you haven't had the opportunity to address the capabilities at that layer, while five means you have mastered all of the capabilities defined at that layer. A typical result is provided in Table 1. This rating for each layer is determined by the strength of the corresponding capabilities, ranging from one to five, with a strength rated as a five and a weakness of one. The average of these qualities will give you an overall grading for the layer, rounded to the nearest half point. For example, looking at layer 1, if you have three of the six capabilities gauged as a strength, two gauged as adequate, and one gauged as a weakness, the overall maturity of the level would be 3.5.

What shouldn't be lost in this rating discussion is the fact that ESOMM is intended to be leveraged as a road map, more so than as an assessment tool. While it is important to know where you are, getting an exact bearing is less important than identifying the capabilities you need to address to continue advancing the value of service enablement in your organization. As long as you are willing to ask yourself some hard questions in an objective manner across all the relevant groups, you should be able to get a good understanding for your current challenges. If you apply the strategy and objectives of your organization, you should be able to identify which capabilities you will need to address in the near, short, and long term.

It is terribly difficult to have a concise, constructive conversation about the service-enabled enterprise. The goal here is to empower you as a software architect with some tools and some of the information necessary to address the right questions in helping your organization achieve real value through this very powerful enablement technology.

Hopefully, this brief overview of ESOMM has provided you with enough information to start applying some of the concepts in your service enablement efforts. Each capability could easily fill an entire article, so your feedback of any specific area is welcome.

Many people have contributed to ESOMM and the concepts presented here, but I would like to thank especially Wolf Gilbert, Scott Beaudreau, Michel Burger, and Byron Howard for their contributions.

About the Author

William Oellermann is a solutions architect with Microsoft Consulting Services, who is focused on driving and supporting strategic service enablement opportunities in the communications sector. William came to Microsoft in February 2004 and has spent over 10 years as a software architect, spanning consulting and corporate roles at Avanade, Ericsson, and Micrografx. He has been a thought leader in the Web services space for over six years as a frequent speaker at industry conferences, author of multiple articles in trade magazines, and author of Architecting Web Services (APress, 2001). When not keeping up with the latest service enablement trends and developments, William enjoys helping his wife keep up with their two very active daughters.

Resources

Carnegie Mellon Software Engineering Institute (SW-CMM)

Microsoft Developer Network (MSDN)
"Architecting Industry Standards for Service Orientation," Josh Lee (Microsoft Corporation, 2005)

This article was published in the Architecture Journal, a print and online publication produced by Microsoft. For more articles from this publication, please visit the Architecture Journal website.

© Microsoft Corporation. All rights reserved.