Frederick Chong, Gianpaolo Carraro, Eugenio Pace (all from Microsoft Corporation) & Matt Baldwin (Hostway Corporation)

November 2007

Summary: Service delivery is a broad discipline that amalgamates knowledge and best practices from technical areas, such as networking, security, instrumentations and monitoring, operating systems, business processes, and system automation. Despite the complexity of this subject matter, this topic is not brand new. Practitioners in the telecommunications industry have given so much thought to this discipline that they are able to distill the essence of their industry management framework into letters in the FCAPS acronym (Fault, Configuration, Accounting, Performance and Security Management).

Contents

The Impetus for Specialized SaaS Hosting
Optimizing for ISV Needs
Cost and Operational Compliance
The SaaS Hosting Continuum
Realizing SaaS Hosting
    The SaaS Delivery Platform
    The SDP SDK
Conclusions
References

The Impetus for Specialized SaaS Hosting

The case for specialized Software as a Service (SaaS) hosting can be easily rationalized by examining an incompatible pair of anecdotal fact and reality:

• Fact: When we explore the expertise of software vendors today, we find that most of them will not claim operating a hosting environment as a core competency. Actually, many would also cite operational excellence as a key barrier of entry into the SaaS market. This observation should not come as a surprise to the reader.
• Reality: However, when we take a look at the prevailing market situation, delivering SaaS applications often depends on hosting solutions that are “home-brewed” by the software vendors themselves. This is to say that in addition to implementing the application logic, many SaaS software vendors themselves will need to design, develop, and integrate the operational modules for their service offerings. As illustrated in Figure 1, the service delivery components, such as billing, metering, and logging, are built by the application vendor and integrated into the application stack.

The key observation from the datum above is that many ISVs are re-implementing operational components by necessity, not because those infrastructure additions provide significant value to differentiating the core features of the applications. Moreover, building these components add to the time and cost of delivering the application. Therefore, for the software vendors, most of such redundant work and expenditure can be eliminated if the software vendors can obtain the same operational functions from third parties that specialize in SaaS hosting.

Figure 1. A simple, non-optimized hosting platform (Click on the picture for a larger image)

There are many existing examples that can attest to the benefits of specialization. In all of the following examples, specialization leads to better implementation of the resulting platforms that are also available at lower costs to the platform users:

• Before operating systems became commodity products, programmers had to write their own virtual memory managers and multi-tasking process schedulers. Such low-level system features are implemented and fine-tuned in modern operating systems; this frees application developers to focus more time on the application logic instead.
• Before relational databases like Microsoft SQL Server came along, advanced application developers had to code and implement data management and retrieval modules for their line-of-business applications. Today, many people rely on off-the-shelf database products to manage their application data. Only those who have very unique data input and output patterns will attempt to code their own data storage and query engines.
• Web development used to require deep expertise with the HTTP and HTML. In the past, Web developers would build code modules to abstract the internals of the protocol and markup language so that less experienced programmers can collaborate in the development project. When the Microsoft ASP.NET Framework shipped, it relieved Web developers of the repetitive tasks of implementing standard Internet components.

Now we are seeing similar trend with SaaS hosting infrastructures. Frequently, software vendors have to implement hosting modules themselves in order to interact with the operational environment. Figure 2 shows how the infrastructure services components could be generalized and consolidated into collections of business and application services within a SaaS Delivery Platform (SDP). Applications will interact with the infrastructure services through interfaces exposed by the SDP. When SDPs become as commercially available as operating systems and database engines, software vendors will be able to concentrate on the core application development tasks, thus reducing the time and cost to market.

Figure 2. A full-featured SaaS delivery platform (Click on the picture for a larger image)

The features of an SDP are not limited to the infrastructure services; hosters can (and should) also offer management solutions that software vendors can use to manage, monitor, and configure their applications. Furthermore, through business management control panels or digital dashboards that act on data collected by the SDP services, software vendors can also monitor higher-value business information such as service usage patterns and uptake, outstanding payments, and billing transactions. Such information enables the software vendors to derive business intelligence that assists them in formulating their market strategy.

It is important to understand that the SDP is not a replacement of a hoster’s existing offerings (such as co-location or dedicated hosting). On the contrary, the hoster’s goal is to augment the total revenue stream by attracting additional customers with SaaS hosting needs. However, because of space or total power consumption constraints (which are usually the limiting resources of a hoster), the hoster may decide that the most effective use of their resources is to focus on the SDP business instead of their traditional offerings. A hoster that decides on such a business shift may reallocate development resources that are working on dedicated automation and services to building the shared SDP services. Because the SDP services are reusable by many more customers (as opposed to one customer benefiting from each dedicated hosting development), this business shift could yield greater monetary return.

Considering the rapid growing interests and adoption of the SaaS delivery model, the primary goal of this paper, therefore, is to help hosting companies and software providers understand the software services hosting opportunities and issues through guidance in relation to the questions below:

• What is the impetus for specialized SaaS hosting?
• How can SaaS hosters attract ISVs to use their services?
• Should the SaaS providers build or outsource their hosting needs?
• What technical capabilities should SaaS hosters build and offer?
• How are the design and implementation of software services impacted by SaaS hosting?
• What assistance can SaaS hosters offer to the software vendors to address the gap between software development and service delivery?

Optimizing for ISV Needs

As a specialized SaaS hoster grows to accommodate increasing numbers of applications, it can be advantageous for the hoster to develop a menu of modules within the hosting environments that are pre-tailored for specific kinds of applications: one module might feature network and disks optimized for the application with high data I/O requirements, and another module might feature a lot of memory and processing power for crunching complex computations, and so forth. The menu of choices should also allow for different combinations of operating system platforms, application frameworks, and database choices.

Furthermore, by paying attention to application development trends on the market, such as popular choices of operating system platforms, application frameworks, and databases, SaaS hosters can identify the kinds of hosting modules it needs to develop to reduce the amount of effort software vendors need to spend on integrating their applications into the service delivery infrastructure.

Figure 3. Providing billing modules for WCF-based software services (Click on the picture for a larger image)

Figure 3 illustrates an example.  Let’s suppose that Northwind Hosting, as shown in Figure 3, is aware that the Microsoft Windows Communication Foundation (WCF) framework is a popular technology for implementing Web services. With this knowledge, Northwind Hosting can then provide management and run-time components that leverages the WCF platform architecture. Specifically, Northwind Hosting can develop a control panel that enables the software vendor to configure the WCF Web service operations as billable business actions, and then insert a billing run-time component in the WCF processing pipeline to intercept and pre-process incoming Web service requests. When an incoming Web service request maps to a billable business action, the billing interceptor triggers a billing event that is captured by the billing system and the transaction is recorded in the metering database.

More generally, a SaaS hoster can examine the end-to-end solutions architecture of a large number of hosted applications to identify common application architecture models, such as the one shown in Figure 4. With such an insight into the components and partitioning of distributed applications, the SaaS hoster can construct specific hosting environment that are fine-tuned for those application models. In addition, the SaaS hoster can also provide operation modules (like the WCF billing interceptor in the previous example), templates and prescriptive guidance to software vendors so that the applications can be architected to integrate and operate effectively in the pre-optimized hosting environment.

Figure 4. A common application model for which hosters can optimize (Click on the picture for a larger image)

In a specific example of an application that is implemented using the Microsoft .NET Framework, the SaaS hoster may provide the following suggested list of implementation guidelines to the software vendor to ensure smooth integration with the hosting infrastructure:

• Applications are made up of sets of Web sites, Web services, and databases
• Web sites must be implemented in ASP.NET 2.0 and must interact with WCF-exposed Web services.
• WCF services interact with SQL Server databases.
• WCF services that interact with a user repository using directory services.
• All .NET Framework assemblies are private to the application (no components should be installed in the global assembly cache).
• All configuration information should be stored in .NET Framework standard files (Web.config files).
• Exception logging and handling should be performed with Microsoft Enterprise Library.
• Data Access should be performed with Enterprise Library Data Access Application Block.
• Only standard extensions are allowed to Enterprise Library (meaning that extensions should have been performed through Microsoft published extensibility points).
• Enterprise Library binaries will be supplied by the SaaS hoster.

By complying with the guidance, software vendors can minimize the number of custom management modules they have to develop themselves. Instead, they can expect the SaaS hosters to provide them with the necessary platform tools to integrate with the hosting environment and reap the full benefits of being able to deliver the application cheaper, faster, and in many cases, with a higher operational service quality bar. Keep in mind that this does not eliminate the need for software vendors to design and implement their application with the running stage of the life cycle in mind. We call this development mentality “design for hosting” (DfH). We will describe DfH in more details later in this paper, and more specifically about how SDP tools should help the software vendor with the DfH process.

Cost and Operational Compliance

There are many benefits to using the SaaS hoster, but it may not always be practical for the software vendor to completely comply with the guidance it provides, and sometimes the hosting environment has to be configured in a non-standard way to meet the application requirements. For example, the typical users of a music e-commerce application frequently previews up to the first 20 seconds of music tracks and expects fast load time for each track that is being previewed. This application will require extremely fast disk and network I/O support in the hosting environment, which may not be available from the standard environment provided by SaaS hosters. Consider a second example when an application requires MQSeries networking infrastructure to integrate with an in-house mainframe system. In either case, the software vendor can expect to pay the hoster a higher premium to get the specialized equipment installed.

The key takeaway here is there is a cost to be paid from deviating from the standard SaaS hosting offerings. The greater the deviation, the more it costs the hoster (and consequently the software vendor) to offer the same quality of service. With the pre-configured sets of standard environments, the hoster is aware of the associated costs and labor to deliver the levels of services it expects to offer. Therefore, the hosting cost is usually the lowest for the software vendor when the application’s operational requirements can be satisfied by the standard environments provided by the hoster. In other situations when the application requires non-standard equipment and infrastructure to operate, it will most definitely cost the hoster (and hence the software vendor) more to deliver the same quality of service. Figure 5 summarizes this idea. At “compliance level 3”, a software vendor does not have any special requirements and obtains the offered services at the lowest price. For the same level of service, a software vendor coming in with atypical requisites that meets compliance level 1 can only get the hosting solution at a higher price.

Figure 5. Graph of compliances (Click on the picture for a larger image)

Deviation from standard hosting packages is not only limited to platform and infrastructure configurations. Differences in business practices can also lead to higher hosting cost. For example, for hosters that do billing on behalf of software vendors, the hoster may have standardized the invoicing systems to issue bills at certain designated times of the month, thereby making it possible for the hoster to lower postage costs by taking advantage of the presorted bulk mailing rate. If the software vendor’s billing cycle varies from the hoster’s established system, the software vendor can expect to pay the hoster additional service fee to cover the increased rate and labor.

The SaaS Hosting Continuum

The discussion thus far has explained the benefits of software vendor outsourcing SaaS hosting responsibilities to a third party SaaS hoster. However, in reality, a software vendor’s arrangement to host and operate their software services may not be clear-cut as a two way split, but it may be better portrayed as a continuum of decisions, as illustrated in Figure 6. Numerous factors can influence an ISV’s decision to keep the SaaS hosting in-house, to outsource to an external hosting solution, or to adopt a hybrid strategy of partial self and third-party host.

The previous section includes examples of unique application requirements that cannot be satisfied through common SaaS hosting environments. In the most extreme of those cases, the software vendor may not find a hoster who is willing to offer the extra services even at a premium price; therefore, the software vendor has no choice but to host and operate the special components themselves.

In addition to application platform requirements, unique regulatory and business compliance requirements can also lead the software vendor to lean toward a self-hosting decision. For example, if the application is used by an industry segment that has rigorous security requirements, such as mandating the use of IPSec between machine communications in the data center and using 4096-bit crypto keys (longer than most common corporate use) for RSA encryption in the database. The software vendor may not be able to find a SaaS hoster that already has a security and network infrastructure that is set up to deal with such specifications.

Alternatively, if the application’s operational dependencies consist of standard components and common practices employed by others in the industry, the software vendor can benefit from the economy of scale and professional services offered through third-party SaaS hosters. Creating new billing systems, monitoring systems, and provisioning systems can become huge undertakings that strain available research and development funds, especially when the software vendor is small and financially constrained. In addition, software vendors must also consider the time and cost it takes to staff and build up the necessary processes and experiences for operating the hosting environment. By choosing to integrate with third-party SaaS hosters’ infrastructures, software vendors can deliver their products to market cheaper and faster.

Figure 6. The SaaS hosting continuum (Click on the picture for a larger image)

As shown in Figure 6, SaaS hosting options can be represented as a continuum of choices, with wholly in-house solutions at one extreme end, wholly hosted solutions at the other, and a number of hybrid options in between.

For example, consider order fulfillment and billing. An ISV may decide to manage the order fulfillment process for the customers they acquired through their own channels but relies on the SaaS hoster to fulfill the orders that came through the hoster’s channel. The reasoning behind this decision could be due to the SaaS hoster’s business policy to retain a share of revenue for every customer that is processed by their order fulfillment service, regardless of the channel that the customer is acquired through. Alternatively, if the SaaS hoster charges a bulk rate for using its billing system, the ISV may decide to use the hoster’s billing system for invoicing all customers because there is financial incentive in consolidating all their billing activities through a single billing solution. Furthermore, the fact that they do not already have a billing solution that complies with the hosting industry processes and regulations make it compelling to contract the SaaS hoster’s billing services. In this scenario, the effect on revenue and the fact that there is a readily available (billing) solution are considerations that influenced the software vendor’s decision to partially develop and host a capability for bringing its tenants onboard while outsourcing the billing capability to a third party who can do it cheaper and better.

As a recap of key points before delving into the architecture sections, the motivations and principles for the SaaS hosting industry shift are the following:

• Considering the intense competition in the hosting industry, hosters can differentiate and streamline themselves by consolidating and leveraging the multitude of components in their existing hosting environment into a shared software service delivery platform.
• Software service providers can subscribe to a more cost effective service delivery infrastructure by adopting a shared SDP, especially when they are able to comply with the service delivery guidance given by their SDP hoster(s) and operate within the parameters of the shared infrastructure.
• The software vendor’s SaaS hosting decision is not always binary and can be hybrid, realized through a continuum of choices in between relying completely on a shared SDP to wholly self-hosting.

Realizing SaaS Hosting

At this juncture, we will switch gears to dive deeper into the architecture topic of the service delivery platform at three (interrelated) levels illustrated in Figure 7.

Figure 7. An application designed for hosting interfaces with the SDP using the information and tools from the SDK as an intermediary (Click on the picture for a larger image)

• SaaS hosting capabilities delivered through the Service Delivery Platform (SDP), which consists of infrastructure, operational (OSS) and business services (BSS), to support activities that occur during the run phase of the SaaS application life cycle.
• The SDP SDK abstraction layer with established interfaces to facilitate communications between the SaaS application and the SDP runtime.
• The SaaS application that provides application states and events information to the SDP for enforcing operation and business policies; collectively, the tasks of enabling an application to be operationally-ready is known as “Design for Hosting” (DfH).

The SaaS Delivery Platform

• This section delves into the following:
• SDP capabilities
• Identity and access management
• Ordering and provisioning
• Metering and billing
• Monitoring
• Other SDP services

The next sections describe each of these.

SDP Capabilities

The primary goal of the SDP is to provide a core set of common capabilities that software vendors need to manage and operate the application services, the user identities, and the business activities related to the use of the software services. At a high level, SDP capabilities can be conveniently grouped into categories consisting of identity and access management, order management and provisioning, application and systems monitoring, customer support and metering and billing. Each SDP capability provides the technology for accomplishing a well-defined set of tasks in its functional category.

Figure 8 illustrates the high-level architecture of SDP capabilities.

Figure 8. High level architecture view of SDP capabilities (Click on the picture for a larger image)

It is important to note that the SDP capabilities are not merely infrastructure software components and servers deployed on a broad scale; they are really specialized platform service modules that have been fine-tuned to comply with industry and operational processes that are essential to successful software service delivery.

Furthermore, although each SDP capability is designed to perform a well-defined set of functions, in reality, the activities and processes involved in delivering software services are all interlinked. Hence, the arrows in Figure 8 are to show the dependencies between the SDP capabilities. As an example, the provisioning module is triggered by an event generated by the order management module that in turn initiates a series of setup and configuration actions performed by the identity management, CRM and SLA monitoring modules. The design of SDP capabilities must take all these integration points into account in order to create a flexible and composable end-to-end service delivery solution.

One recurring architecture consideration when designing an SDP capability is the multiple levels of multi-tenancy that has to be supported in the management data model. For example, at one level, the identity management module within an SDP should allow the SaaS hosting provider to manage the identities of the software vendors they are hosting. Because a SaaS hosting provider is hosting multiple software vendor tenants, this first level of multi-tenancy allows the SaaS hosting provider to define different identities and access policies for each separate software vendor tenant. In turn, each of the hosted software vendors will need to manage their own set of application tenants that have signed up to use the software services. This is the second level of multi-tenancy that needs to be built into the identity management data model. Figure 9 provides a simple illustration of the multi-level multi-tenancy concept. In this example, Northwind is a SaaS hoster with ISV A and ISV B signed up as the software services providers. In turn, ISV A has Application Tenant A and Application Tenant B signed up as subscribers of the software service ISV A provides. Delegated administration is enabled for the hosted tenants at two levels. Specifically, both the software vendors and application tenants that are hosted by Northwind have the ability to administer their respective users and application functions.

Figure 9. Multi-level multi-tenant data model (Click on the picture for a larger image)

In the next few sections, we will describe the SDP capabilities shown earlier in Figure 8 in more detail.

Identity and Access Management

As mentioned earlier, a multi-level multi-tenant identity data model enables the identity management capability to delegate the administration of identities and access policies to respective administrators at separate levels: administrators in each software vendor tenant can grant to business and operation staff permissions for performing privileged tasks such as generating billing reports and starting database backups; similarly, application administrators from the subscribing application tenants can assign rights to their users for configuring and accessing application features.

Another function of the identity and access management capability is to facilitate single sign-on (SSO). There are a few key SSO scenarios to consider. The first scenario is to enable SSO between the applications that are hosted at the SDP. Frequently, SaaS hosters will bundle and resell multiple hosted applications from different software providers in a packaged offering. SSO between the bundled applications can reduce the unnecessary intrusions on the user experience and reduce the number of credentials that the user and application has to manage.

To enable SSO between multiple hosted applications, an approach is for the SaaS hosting provider to have the responsibility of being the identity provider. As the identity provider, the SaaS hoster provides the necessary infrastructure for registering, managing, and validating the identities of the application users. The next paragraph describes how SSO is accomplished.

It is good—and a common security practice—to authenticate and validate the user identity that is accessing a hosted application. In the approach we are describing, a user will first have to authenticate his or her identity with the identity provider. If the user’s credentials are valid, the identity provider issues a security token to the user as proof that the user is a valid tenant. The hosted application will check and verify for the presence and the authenticity of the security token before granting access to the user. Subsequently, if the user needs to access a different application, there is no need to re-input the user credential to re-authenticate the user. The same security token that is issued by the identity provider can be used as the proof of identity to gain access to another application, hence fulfilling the SSO requirement. The scheme that we have just described is, in fact, a common Web SSO pattern used in many Web-based solutions on the market.

The preceding scheme assumes that the hosted applications are capable of processing the security tokens that are issued by the hosting provider’s identity system. In reality, there will be situations when the hosted application has been designed and implemented to process a different type of security access token. For example, some applications will expect to receive a user name and password, and other applications may use proprietary HTTP cookies as security tokens.

The Security Token Exchange pattern is an architecture approach that can be used to work around circumstances when the applications cannot be modified to process the identity provider’s security token. The implementation of the pattern requires a security token exchange proxy component to be installed to pre-process the standard token issued by the identity provider. After the token is validated, the proxy component can map the represented identity to another set of user credentials that is expected by the hosted application. Figure 10 illustrates an example of security proxy components implementing the security token exchange for ISV A’s and ISV B’s application. The standard token STH is validated and swapped out for STA and STB, respectively.

Figure 10. Single sign-on scenarios (Click on the picture for a larger image)

The third SSO scenario is to enable users in enterprises to use their existing user corporate accounts for signing on to the hosted applications. Although multiple approaches have been used to accomplish such goals, some methodology are less scalable and more error prone than others. For example, directory synchronization, while the concept is straight forward, is not very scalable and efficient when synchronization needs to take place between the SaaS hoster’s identity system and many enterprises’ directories. Newer generations of standard-based federated identity solutions (such as Active Directory Federation Services) that build on trust relationships and policies allow identity systems to be loosely coupled and therefore more scalable. Through the use of identity federation solutions, businesses and software vendors can enable enterprise SSO by establishing trust relationships with SaaS hosters that are playing the role of a third-party identity provider. The ADFS articles referenced at the end of this paper provide more details about the mechanics of using federated identity solutions to enable SSO.

Ordering and Provisioning

Ordering and provisioning are capabilities that are closely related. Together, these two capabilities enable and facilitate the onboarding processes of two primary kinds of tenants: the software vendors and the application tenants.

On-boarding the Software Vendor

The process of provisioning the software vendor can be complex. The process involves evaluating and understanding the application design; making sure that the all aspects of the application platform requirements can be satisfied by the hosting infrastructure; and getting the software components of the application installed, deployed, and configured in the hosting environment. To ensure that all these tasks are completed within the expected go-to-market timeframe, the software vendor and SaaS hoster usually begin by working closely together in a qualification process. Ideally, the qualification process starts early in the application development cycle, so the hoster can work with the ISV to adopt “design-for-hosting” best practices. The qualification process should also identify any unique technical requirements the application will have. For example, if a financial application requires a connection to the SWIFT network used in international banking, the hoster needs to be able to accommodate this requirement.

A goal of the qualification process is to reveal crucial information, including technical and business dependencies and assumptions, so the provisioning process can be executed with minimal disruptive surprises. To meet this goal, the hoster should initiate a series of technical discovery exercises to gain the needed information. Potential questions to answer include the following:

• What is the nature and function of the application?
• What operating systems and application platforms does the application run on?
• What does the application architecture look like? Does the ISV have architectural design documents that it can share?
• What is the entities and data model in the applications? What are its database requirements?
• What are the networking requirements?
• Is the application using or dependent on any protocols that can raise firewall issues?
• Has an application prototype been deployed? If so, has the ISV generated any deployment documentation or other material that will be helpful during deployment?
• What are the application control points that can facilitate management and troubleshooting?
• Are there any legal or regulatory compliance issues to consider?
• What are the expected performance metrics and load profiles?

Eventually, information gathered through the qualification process is used in two parallel provisioning workflows to bring the software vendor onboard, as shown in Figure 11.

Figure 11. The software vendor on-boarding process (Click on the picture for a larger image)

In one workflow, the software images, along with descriptions and configurations of the hosting infrastructure and SDP capabilities, are used by the infrastructure and platform provisioning workflow to install and configure the application and its platform components. SDP capabilities that provide application and operational support (such as identity management and monitoring, respectively) are also provisioned according to the requirements.

The preceding workflow handles application deployment, but another workflow handles exposing and activating the software service so potential subscribers can review, subscribe, and access the software services through the product catalog and order management services.

On-boarding Application Tenants

The process of provisioning application tenants begins when the tenant (to-be) submits the orders for software services that are selected from a product catalog. Depending on the application, the order details should capture information such as the services that are being procured, customer organization, address, and contact and billing information. For more complex applications, the ordering process may not be self-serviced. Instead, the order may be taken by a customer service representative from the software vendor and batch-submitted on behalf of the service subscribers.

When a new order is entered through the order management application, a new order management workflow instance is started to coordinate a series of provisioning activities that will take place in the SDP. Because some of these fulfillment tasks may not be completed immediately, the order management workflow will normally return a pending status and an order tracking identification number to the user submitting the order.

Figure 12 illustrates the order management and order fulfillment process for application tenants.

Figure 12. Order management and fulfillment solution flow for onboarding application tenants (Click on the picture for a larger image)

The order details are sent to separate provisioning pipelines. In one pipeline, business-related information, such as the tenant’s organization, billing information, and service entitlement, is entered into the customer and billing systems. In another pipeline, an application provisioning request is put into a queue. When the application provisioning system picks up the request, it figures out the list of applications components that need to be configured and updates the software components according to the provisioning logic the software vendors provide. In some situations, especially when the application provisioning logic is fairly complex, the software vendor implements the automation scripts and code for making the changes and provides an interface to the SDP’s provisioning system to call into. A hybrid scenario is also possible. For example, the SaaS hoster may implement the scripts for installing new virtual directories on the Web servers and for creating new LDAP directory and user account entries in the identity system. At the same time, the software vendor provides a provisioning interface to call into database scripts for creating new tenant entries and application metadata that describes the default settings for tenant-configurable user interfaces, branding, color schemes, workflows, business rules, and entity data models.

Considering that processes for provisioning the tenant usually involve many moving parts in the SDP, it is essential to consider exception cases when tasks in the provisioning workflow cannot be completed successfully. An example of this is a failure to create an organization unit in the LDAP directory due to duplicate naming conflict. The solution to these exception cases often is to undo and roll back all the changes and log the offending errors so the support staff can troubleshoot, fix, and retry the provisioning steps again from a stable state. This means that the implementation of provisioning systems should understand transactional units of work and requires participating modules to provide interfaces for performing reverse logic to undo changes.

An order completion handler in the provisioning system coordinates statuses from different branches of the provisioning logic and logs a final provisioning status after all outstanding tasks are complete. The final log also contains provisioning requests that cannot be concluded successfully. For such cases, the errors are logged to assist further troubleshooting. For orders that are fulfilled, the handler sends a notification to the provisioned tenants with information about how to begin accessing and using the software services.

Metering and Billing

Like order management and provisioning, the metering and billing capabilities are interconnected. The metering capability is responsible for recording events and tracking details related to service usages which the billing capability uses—in conjunction with pre-defined sets of charging policies to generate the customer invoices.

Metering

Considering that software vendors may adopt different monetization schemes, it is therefore critical for the SDP platform to be sufficiently flexible in recording and billing service usages. In the simplest case of user license-based revenue model, for example, $65 per user per month, the metering logic is quite simple and only needs to track the number of user accounts that has been created within the billed month. To determine the final monthly amount of a tenant’s invoice, the billing service can simply multiple the total number of users by the monthly fee of $65 and deduct any prorated discounts to account for licenses that are not activated at the beginning of the month. With a more complicated scenario where the bill is computed based on other types of resource usage, the metering module will need to record usage events and track the billable “quantities.” Billing units can vary from the number of times a service is used, to the amount of network or CPU used, to more business-oriented metrics such as dollar amount of widgets purchased through the software application. Although the variations in metering units may seem broad, most units of measurement can be viewed as belonging to the two following categories:

• Resource usage. This category includes metering by CPU time, network bandwidth consumption, disk storage, and database size. User-based licenses, while not strictly resource-centric, can be correlated to the resource usage model with the assumption that more users imply more resource usage.
• Derived business value. This category includes charging tenants according to the measurable business value that is derived from using the software services. The previous example of dollar amount of purchased widgets fits into this classification. In another example, an application that processes loan applications—often a complicated, multi-stage process involving different institutions with their own fee structures—might charge at certain stages in the process to ensure that all creditors are paid promptly.

Clearly, some of these measurements, such as the number of bytes transmitted in an HTTP request, can be derived from the application platform stack, but only some of them can be computed and obtained from the application logic. The following figure shows two scenarios: one where the application will provide the metering information and another where a metering agent runs alongside with the application to provide metering data for the application. The second approach is often used when the application has already been implemented and when there is sufficient data that the metering agent can glean from the platform environment or the application request context to compute the billing information. Later on in the paper we will discuss design for hosting and metering is one of the areas that will require upfront consideration so that the application design and implementation can provide accurate usage information. In some cases when usage has gone beyond the scope of the agreed licensing terms (for instance, exceeded the number of concurrent user licenses), the metering agent or the application will also have to enforce the licensing contract. The license enforcement logic does not necessary have to take the harsh mean of denying access, but instead, should construct this as a sales opportunity by redirecting the user to a Website where the user can purchase the “currency” to use the service.

Figure 13. High level metering solution flow (Click on the picture for a larger image)

Figure 13 shows two approaches for capturing and recording the metering information: direct invocation or event-base. With the direct invocation mechanism, the application and metering agent calls the interface of a metering service that will record the call parameters (containing metering data) into a database. When an event-based mechanism is used, known metering event types are defined and registered by the metered applications. Every time a billable action is performed by an application, the application or its metering agent publishes an instance of a registered event. The published event contains service usage information. A metering event listener that has subscribed to the list of registered events types will receive a notification to process the published event instance. When processing the published event, the metering listener logs the metering data contained in the published event into a database which is periodically queried by the billing service to generate customer invoices.

Billing

Because the billing capability involves handling of money, hosters, ISVs, and possibly even government regulators and law enforcement are particularly sensitive to the implementation details of billing processes. Therefore, in preparation to offer a bill-on-behalf service, the SaaS hoster must have clear and specific answers to certain key business questions about the billing process.

For example, if an ISV uses a billing service module provided by the hoster to bill and collect money from tenants, will the hoster actually hold any collected monies in a custodial account for any amount of time, or will the billing service simply act as a direct conduit into the ISV’s bank or credit card account? Another area concerns the risks and mitigation assessment of the hoster’s access to customers’ unencrypted bank account or credit card details. For undertaking liability, which party will be responsible for bad debt collection?

In some cases, the answers to these questions can affect security and regulatory compliance requirements of the billing solution, so any potential hoster should devote some attention to resolving these business issues early in the solution development process.

Beyond the business issues, developing a billing system involves considerations as practical as deciding which payment methods the system will support, such as credit and debit cards, checks, and auto-bank payment. Such decisions will affect the kinds of payment gateways, solutions, and financial institutions that the SaaS hoster needs to work with.

Bill consolidation is another important consideration: customers who buy a number of different products and services should be billed with a single invoice, for simplicity’s sake. This is especially relevant for aggregators that offer comprehensive solutions including SaaS applications from different ISVs. The SDP may need to consolidate outstanding charges from multiple vendors with differing billing rules into a single bill for the customer.

As SaaS empowers software vendors to adopt many types of non-perpetual licensing schemes, the billing system must now also support numerous types of metering units and pricing policies adopted by different software vendors.

Because of these kinds of considerations, the billing systems used by SaaS hosters can quickly approach the same scale of complexity as those used by telecommunication providers. As a background, the billing systems used in many telecommunication companies today are frequently the most complex solutions supporting the telecommunication service business. A large part of the complexity stems from the multiplicity of business rules that are required to support a fairly dynamic industry with pricing structures and financial policies that can change quickly and at the same time, vary across regions, customer segments, and sizes of the business deals.

Figure 14 illustrates a typical solution flow for a billing-on-behalf system.

Figure 14. A high-level billing solution flow (Click on the picture for a larger image)

At the core of the billing solution illustrated in Figure 14 is a business rules–driven workflow engine that coordinates the end-to-end execution of a series of billing-related tasks. Customer billing is initiated by a timer process that checks the customer database at preset intervals to determine which customers are due for a billing action. New charges for the current billing period is computed by querying new service usage data from a database of billable transactions recorded by the metering system described earlier. To calculate the billed amount, the billing system applies a pricing structure template supplied by the software vendor as the formula for tallying the current total. The pricing template specifies the formula for what and how their applications should be billed for. For example, for one software vendor, the pricing formula is to charge 30 dollars for the first 50 widgets purchased through the software service and 25 cents for every subsequent widgets thereafter. If an application tenant has bought 60 widgets, the total bill for the month is 30 + (0.25 * 10) = $32.50. Another software vendor’s billing policy is 40 dollars per user license per month and if an application tenant is metered at 5 users the total bill would be 5*40 = $200. The key point of this discussion is to highlight the fact that pricing formula is frequently dynamic and the billing solution must account for this important differentiation. In some cases, in addition to the standard pricing policies, there may be application tenant specific pricing policies, such as a one-time volume discount that needs to be applied to adjust the final bill.

In addition to activities that occurred during the current billing cycle, the billing solution must also consider outstanding bills that are brought forward from the previous billing cycles. Business policies for handling overdue charges can depend on variables such as the length and amount of the owed payment. As you can see in Figure 14, business rules in the workflow control the actions required to account for the non-payment scenarios. In the more extreme cases when a payment has been overdue for a long period, the action can trigger the deprovisioning system to suspend an account or instigate a legal case for collection agencies.

Payment collection is frequently automated by using existing payment gateway solutions. These payment gateway products enable customers who have pre-approved auto-payment transactions to have their credit cards and bank accounts be automatically credited. However, in some cases, credit card and bank account information expire and the tenants will have to be notified to have their financial data updated. Updating customer account can typically be done by the tenants themselves through self-service Web sites. When the customer billing details are updated, the payment collection workflow will restart to resume and retry the transaction again.

Finally, printed invoices may be generated and sent with information to advice tenants on the remaining payment actions that they will need to take.

The billing capability is a mission-critical system in the SDP. Any unplanned downtime in the system can cause considerable financial losses to the SaaS hoster and its software tenants. Therefore, building redundancy into the billing solution is crucial to help with mitigating unforeseen failures or sudden surge in billing activities in the system. For example, a redundant message queue can enable various components in the billing solution to continue communicating requests and responses even when a peer queue is out of commission. The redundant queue can also be used to offload processing from a busy queue when there is a sudden rise in the number of payment to be processed.

Monitoring

Delivering reliable software services requires quality software and a quality operational environment. While good software is designed to minimize and handle failure gracefully, it needs the support of an operational environment that is equipped with the right tools and processes to monitor and correct errors. Therefore, the goal of the monitoring capability is to provide the infrastructure for detecting, diagnosing, and correcting operational issues.

Generally speaking, monitoring requirements are derived from the service-level agreement (SLA) between the software service providers and the SaaS hoster and SLA between the software service providers and their application tenants. Typically, the scope of the monitoring occurs in three categories: performance, availability, and security. Performance measure concerns metrics such as throughput and rate of response; availability tools and procedures manage the service accessibility and up time; and security monitoring helps mitigate the risks of unauthorized access to data and services.

Operational data coming from different sources contribute information to help detect abnormal operation behavior. Some examples of the management information sources are the software applications, the application platform runtime (such as the .NET Framework runtime), the operating system, or an application platform service (such as SQL Server). It is important to recognize that no one single data source can provide all the information needed to troubleshoot and debug all operational problems. Software issues can arise in any distributed component of the execution path, so it is critical for the operational process to be able to instrument and monitor the SDP at multiple technology levels.

To communicate and relay management information, the monitoring infrastructure can use a push or pull model for gathering operation data. Both models involve two actors: a producer of management data and a consumer of the management data produced. Frequently, the producer is the application or the system component that is being monitored. In some cases, a third management agent actor is installed and co-located with the managed subject and will collect and produce the management data on behalf of the managed subject. The management data consumer is usually a monitoring application that has been configured to use the collected data to detect anomalies in the operational environment. For example, a management application can be configured to collect and analyze CPU usage performance counters from server farms. In addition, the management application has a rule configured to send an operation notification when it detects a situation where the average CPU load has been stabilizing at over 80 percent utilization during the past 10 minutes. In turn, the notification generated can be used to enable a more dynamic operation environment where additional servers are automatically provisioned to handle extra load when necessary.

Figure 15 illustrates various types of monitoring. With the push model, the application itself or the management agent (proxy) pushes information to the monitoring infrastructure. This approach is commonly used to monitor for events that are triggered by application state changes or operational situations that are not anticipated to occur at regular intervals. For example, when the application detects that it has lost its connection to the tenant database, it raises an application error event that gets written to an application event log. Security monitoring is another type of management application that can rely on the application to log and push the security failure events through notification mechanisms.

Figure 15. Health, performance, and availability monitoring (Click on the picture for a larger image)

With the pull model, the monitoring application periodically collects information from the applications and management agents. For example, to monitor an application’s availability, the monitoring station will invoke a management interface of the application and monitor responses from the application. Depending on the number and rate of responses received, the management rules and algorithms at the monitoring station can help compute the application’s level of availability. This approach is suitable for retrieving timed management data that measure the current level of service qualities such as rate of performance and availability.

Among all the SDP capabilities, monitoring may be the most familiar to existing hosters because of its similarity in approaches for monitoring current generations of hosting infrastructures. This topic is also very well explored in existing management literature and there are many available products for implementing the monitoring solutions. Additional resources on this topic are referenced at the end of this paper.

Other SDP Services

In addition to the main capabilities previously discussed, the SaaS hoster can offer additional value-added services that will help the software vendors and their tenants meet industry regulations and obtain a higher level of assurance on their data protection. These services include the following:

Auditing. Many businesses and organizations are required to collect and preserve information that can be used by auditors to assess compliance with regulatory or industry standards. SaaS hosters may be required to help tenants collect information about the inner workings of the hosting platform or processes that they are unable to obtain by themselves. Trying to anticipate every possible reporting requirement that a tenant may face can be impractical, but hosters should consider designing their platforms to produce audit trails that satisfy common compliance regimens, such as SAS 70 and PCI DSS for credit card payment processors. 

Backup and restore. SaaS hosters can provide such services to software vendors to help safeguard their customers’ data against loss due to accidents and criminal activities. The additional challenges that SaaS hosters could face in providing data backup and restore services are the more granular tasks of restoring data within a multi-tenant environment. Database restore operations are typically “all or nothing”: the most recent backup set is retrieved and the database is restored to the state it was in at the time of the backup. Though this may be appropriate in some cases, some SaaS applications are designed to store data from multiple tenants in a common set of database tables, with a tenant ID number being used to associate certain records with a specific tenant. With a shared database configuration, an all-or-nothing restore will roll back the data of other tenants, even if they don’t need to be restored. The operation staff needs to be able to restore the data of a single tenant if requested, regardless of how it is stored.

For more information about working with data in a SaaS environment, see “Multi-Tenant Data Architecture,” at http://msdn2.microsoft.com/en-us/library/Aa479086.

The SDP SDK

Up to this point, we have described the capabilities of the SDP, which are often implemented as software services that the software vendors’ applications can interact with through a set of well defined SDP APIs. To assist the ISV in designing applications that can utilize the SDP capabilities to the fullest extent, the hoster should provide a comprehensive SDK that includes all the resources necessary to understand and interface with the SDP’s services and processes. As with any SDK, this can include API and schema documentation, code samples, and a variety of development tools tailored for the SDP.

Unlike many other SDKs that application developers are familiar with, the SDP SDK must take into consideration the fact that the production SDP services are usually not accessible to software services that have not been provisioned in the SaaS hoster’s SDP environment. The restriction on the services’ accessibility is to ensure that the security and performance of the production SDP services are not adversely affected by third-party software services that hasn’t been fully tested and certified.

As a substitute, the SaaS hoster can provide dummy stub implementations to help the software vendors design, test, and verify the application’s communications with the SDP services.

For example, a simulated billing environment might include a stub BillTenant() method that accepts a tenant ID number and monetary amount. The stub method does not perform any real billing function. Instead, the code inside the method only queries the tenant database to verify that the ID number is valid and returns a status code indicating success or failure. The application will have no real tenants during development, of course, so the SDK can also include a number of “sandbox” tenant accounts for testing. Aside from providing a stub implementation, the SaaS hoster can also make a set of SDP services available in a separate pre-production SDP infrastructure to provide a more realistic environment for testing end-to-end scenarios. The SDP services in such environment will use the same code image as those in production, and mock-up business data to simulate the test cases.

In general, the tools and resources offered through the SDK need to assist the ISV in four broad areas:

Building. This includes project templates that automatically generate default code and configuration; reference libraries, run-time modules, and components that contain code implementation of the SDP interfaces; and “design for hosting” components that extend the application development environment, such as Microsoft Visual Studio Extensions.

Testing. As previously described, the SDK includes stubs for helping developers ensure that they are making calls into the SDP correctly and that any callback functions work as expected.

Packaging. The “Metadata-Driven Service Delivery” section later in this paper describes an approach that uses domain-specific language (DSL) for describing the application platform, deployment, and operational requirements of the software services. Packaging tools include comprehensive documentation of the DSL, designer, and code libraries to produce manifests that enable the SaaS hoster to provision new applications with minimal human intervention.

Planning. A more advanced SDK can include business planning tools. For example, financial planning tools that can simulate different monetization models and show the impact of moving from a subscription to a transaction-based model; another example is a growth forecasting tool that can compute the effects of different growth alternatives—thus enabling the software vendor to weigh increasing hosting cost against additional revenue. In addition, these tools may either operate with simulated data or use the history of data that the hoster has accumulated for the software vendor while hosting the application(s).

Using the SDP Services

A well-designed SDP not only defines a clear separation between the roles and responsibilities of the hoster and those of the software vendor; it also facilitates effective two-way communications between the hoster and software vendor components through interfaces implemented by both parties. Figure 16 illustrates an SDP interface.

Figure 16. A simplified SDP interface diagram (Click on the picture for a larger image)

To facilitate bi-directional communications, the SDK provides a client-side service agent proxy for the application to interact with the infrastructure services hosted in the SDP. These interfaces provide the mechanisms for applications to perform tasks such as triggering a metering event or authenticating and retrieving the user context of an application tenant.

In many situations, SDP infrastructure services will also want to instruct the application to perform certain management tasks, such as providing application management state or performance counter data to health monitoring services. To support routine maintenance operations, management control and command need to send signals to the applications to initiate a graceful shutdown. Such service to application communications is typically accomplished through pre-defined callback interfaces. By implementing the callback interfaces, applications expose well-known control entry points that infrastructure services can use to invoke the applications.

Dependency Injection

To further address the separation of concern between application development and operational tasks, the SDK should also implement and facilitate the use of the Dependency Injection pattern. Dependency injection enables application development to be designed with hosting in mind, but to also loosely couple the implementation details and issues across development and operation functions. We illustrate with the following pseudo code example to show how an implementation of a Customer class would look like with and without dependency injection:

// Pseudo code: Without dependency injection:
class Customer
{
   IBilling b = new BillingSystem();
 
…
}

// Pseudo code: With dependency injection:
class Customer
{
    [Dependency()]
    IBilling b; 
}
In configuration file:
<config>
<BillingProvider interface=”IBilling” type=”NorthWindBilling”/>

</config>

Without dependency injection, the implementation of the Customer class directly references the BillingSystem class and has a tight dependency on the specific type of billing system that is implemented by the BillingSystem class. A code implementation of BillingSystem class (it can be a stub) is needed to run and test the Customer class.

With dependency injection, the billing object in the Customer class is instantiated based on configuration metadata. In this case, the billing object has a dependency on any object class that implements the IBilling interface. The SDP SDK provides software modules that serve as object factory and uses metadata in the application configuration file to instantiate the corresponding class type supporting the IBilling interface on behalf of the Customer class.

A framework such as the Policy Injection Application Block (http://msdn2.microsoft.com/en-us/library/bb410104.aspx ) can be used to provide such functionalities.

Bolting on SDP Services

The preceding assumed a hoster-software vendor working model where it is feasible for the software vendor to make modifications to the application to comply with management tasks and events that occur during operation.

Under certain situations, such as when it is deemed too complex to modify the software or when the software vendor is delivering the application through multiple hosting providers, the software vendors may be reluctant to embed hosting provider–specific SDK parameters in their application code. Under these circumstances, the hosting provider should provide the alternative of using less or non-intrusive mechanisms to facilitate operational communications.

More specifically, many Web applications run on application platforms that allow pre-processing and post-processing modules to be injected in the application processing pipeline. For example, for ASP.NET applications, a custom ASP.NET httpmodule can be implemented and configured to execute just before and after the application code to handle the user’s requests.

When the application code cannot be modified to signal to the SDP that it is processing a billable request, an httpmodule provided by the hosting provider can be configured to intercept the application request and to send a billing event to the SDP’s metering service.

Figure 17 illustrates ASP.NET httpmodule communication.

Figure 17. Enabling operational communications non-intrusively with ASP.NET httpmodule (Click on the picture for a larger image)

Metadata-Driven Service Delivery

“Architecture Strategies for Catching the Long Tail” (http://msdn2.microsoft.com/en-us/library/Aa479069.aspx), the first article in this series, described the use of metadata for customizing multi-tenant applications. More specifically, metadata-based architecture makes it possible to configure and spawn identical application instances for an arbitrary number of tenants with the behavior and appearance of each application tailored to the tenant’s needs, without requiring custom code.

Likewise, by using metadata to describe the application’s operation requirements and configurations, communications between the software vendor and the SaaS hoster can be streamlined. Furthermore, in conjunction with provisioning systems that can understand and make use of the semantic meaning of the metadata, much of the software vendor’s provisioning process can be automated and expedited.

To communicate hosting and operation information to one another, the SaaS hoster and application provider must agree on a lingua franca that serves as the domain-specific language (DSL; that is, languages designed for and intended to be useful for specific kinds of tasks) for SaaS hosting.

Figure 18. Using designers to generate metadata (Click on the picture for a larger image)

To simplify the task of generating metadata that describes application and operational requirements, hosting providers can provide designer tools to software vendors. Figure 18 illustrates this key concept of capturing requirements through designer tools. The designer tools include templates and wizards that guide the software vendor through the process of capturing operation-related parameters, such as the application and health models, performance KPIs, and metering and billing policies. After the required information is entered, the designers are capable of generating hosting manifests—that is, data files with hosting requirements expressed as metadata. The structured metadata in the manifest further enables the SaaS hosters to automate their provisioning systems, because hosting requirements can now be unambiguously parsed and processed by the provisioning logic in place to support the application and software vendor provisioning process.

Conclusions

Service delivery is a broad discipline that amalgamates knowledge and best practices from technical areas, such as networking, security, instrumentations and monitoring, operating systems, business processes, and system automation. Despite the complexity of this subject matter, this topic is not brand new. Practitioners in the telecommunications industry have given so much thought to this discipline that they are able to distill the essence of their industry management framework into letters in the FCAPS acronym (Fault, Configuration, Accounting, Performance and Security Management).

In many ways, FCAPS also appropriately describe the mantric objectives of the software service delivery function. However, the current situation is different with respect to the fact that most of the software application vendors do not have the financial clout and execution capacity of telecommunication companies to take on, let alone excel, in operational management. In addition, compared to the number of telecommunication companies in the world, there are many more software service providers that are faced with the service delivery quagmire.

As we have indicated, the topical surface areas of SaaS hosting is wide and deep. Although not discussed in this paper, operational efficiency and maximizing the tenant “density” of data centers are subjects of burgeoning relevance. Industry interests in technologies like virtualization will also grow in locked steps as SaaS hosters increase their volume of business amidst the rising tide of higher power consumption and costs.

Essentially, it is the preceding confluence of technical and business needs in growing numbers that leads to the current opportunistic demand for specialized SaaS hosters. However, the partaking of the SaaS hosting opportunity does not come for free. In order to differentiate and maximize the value proposition to the software vendors, SaaS hosters should optimize their offerings by doing the following:

They should focus on the SaaS hosting service offerings for application platforms that yield the highest return on investment. By specializing, SaaS hosters can often offer a higher quality solution at lower price, especially when they can also leverage economy of scale in their operations.

They should develop knowledge and expertise in the application platforms that the software vendors use to build their software services. Oftentimes, with such insight, SaaS hosters can develop operational software modules that plug into and extend the functionalities of the application platform. When infrastructure services can be bolted onto existing software applications without requiring changing the application code, it becomes much more convenient and attractive for the software vendor to use the SaaS hosting offering.

They should offer “design for hosting” guidance to the software vendor. In addition to traditional design discussion forums, the guidance can also be delivered through documentation consisting of operation design patterns, APIs in software development kit, designers, wizards, and application development tools that simplify the “design for hosting” processes.

They should adopt a metadata-centric approach for communicating operational requirements between the software vendors and SaaS hosters. Such an approach helps increase the fidelity and validity of information that is communicated. When operational parameters can be determinately described through a domain-specific language, it becomes possible to streamline and systematize the process of capturing the operational requirements from the software vendors and automate many facets of the service delivery infrastructure.

References

“Overview of Active Directory Services (ADFS) in Windows Server 2003 R2” on the Microsoft Web site:
http://www.microsoft.com/WindowsServer2003/R2/Identity_Management/ADFSwhitepaper.mspx

“Introduction to ADFs” on Microsoft TechNet:
http://technet2.microsoft.com/windowsserver/en/library/c67c9b41-1017-420d-a50e-092696f40c171033.mspx?mfr=true

“Simplify Single Sign-on Using ADFS” on Microsoft TechNet:
http://www.microsoft.com/technet/technetmag/issues/2006/07/Simplify/

“System Center Operations Manager 2007 SDK” from the Microsoft Developer Network on MSDN:
http://msdn2.microsoft.com/en-us/library/bb437575.aspx

“Monitoring in .NET Distributed Application Design” from the Microsoft patterns & practices Developer Center on MSDN:
http://msdn2.microsoft.com/en-us/library/ms954612.aspx