What is BPEL4WS?

Build Better Business Processes with Web Services in BizTalk Server 2004

Jon Fancey

Code download available at:BPEL4WS.exe(158 KB)

Contents

From Object-Orientation to Service-Orientation
The New Lingua Franca
Boundaries
Conclusion

The potential of the service-oriented enterprise is huge. Service-oriented architecture (SOA) will simplify the building, managing, and maintaining of distributed systems, in part because the technologies you will use to build these new systems are standards-based, thanks to the collaborative efforts of companies like IBM BEA, and Microsoft. Working together has enabled them to establish many Web services specifications, including WS-Security, WS-Addressing, and WS-Policy.

In this article I will focus on one Web services specification that is critical, yet has been largely overlooked: the Business Process Execution Language for Web Services (BPEL4WS, or BPEL).

I will describe why BPEL is important and what it offers you if you are implementing Web services today or planning to in the future. I'll also present concrete examples using BizTalk® Server 2004. I'll discuss the various advantages of BPEL throughout the article, but for a quick look at some of the most important ones, see the sidebar "Benefits of BPEL." For background on all the great benefits of SOA, take a look at Aaron Skonnard's Service Station column in the February issue of MSDN®Magazine.

From Object-Orientation to Service-Orientation

With the ability to easily integrate with and invoke Web services comes the problem of the coordination and management of messages as they are routed from service to service. As time passes, it will become increasingly unlikely that any interesting business processes will execute in a single message exchange. Traditionally, implementing business logic meant writing code (C++ or C#, for example). With BizTalk Server, however, many common requirements can be realized without coding a single line. It seems reasonable to code manually when dealing with a few services, but this becomes more problematic as the number and complexity of services increases. Enter BPEL—the first step towards abstracting the collaboration and sequencing logic out of platform-dependent code and into a formal process definition based on XML, WSDL, and XML Schema.

Successful SOA implementations require reusable logic and service autonomy. In order to achieve them, you need to start thinking of applications as the collaboration of message exchanges. Of course, when you begin exposing application functionality as services, you relinquish a certain amount of control because you no longer know how your services are being employed or even who is using them. I'll take a look at how you can come to grips with that fact later on in this article.

If I were to describe business orchestration in musical terms, the orchestra would be made up of Web services, BPEL would be the sheet music, and the composer would be the coordinator or business process manager (BPM). Designing an application architecture that fits this description using ubiquitous standards in such a way that the specifications themselves do not intrude on the process is the great challenge in Web services today. BizTalk Server is the solution to that problem. Let's look at how BizTalk Server uses Web services, BPEL, and Web Services Enhancements (WSE) as parts of a BPM solution.

The New Lingua Franca

As I mentioned, BPEL is one of a number of technologies that provides a rich Web services infrastructure. Each of the specifications (some still waiting for vendor implementations and consequently widespread adoption) addresses a different area of concern. BPEL was submitted to OASIS last year, where it is currently undergoing finalization (see www.oasis-open.org). However, you can take advantage of a growing number of major specifications right now in your orchestrations and Web services by using WSE and BizTalk Server in tandem.

BPEL gives Web services architects the kind of flexibility that the open database connectivity standard (ODBC) gave database architects. That standard enabled uniform data access regardless of the actual database used. BPEL also enables a process to be defined in one environment and executed in another. Another advantage is that it clarifies business processes and enables the designers of the business processes to define the orchestrations themselves (more on this shortly).

There are a couple of different methods by which the BPEL specification can be implemented. For example, you could run the BPEL through an interpreter the way a scripting language is executed, or you could convert or compile it into a more efficient format prior to execution.

BizTalk Server takes the second approach and realizes a number of benefits from doing so. BizTalk Server enables the import of BPEL 1.1-compliant processes and converts them to XLANG/s, the Microsoft process definition language. XLANG/s has been supported in all previous versions of BizTalk Server, thus it provides important backward compatibility. XLANG/s also provides many of the common facilities required to implement processes such as mapping, conversion, and transformation functions. Currently the BPEL standard only allows such functionality to be exposed as additional services. This can incur performance overhead without runtime optimization.

BizTalk Server goes even further than merely converting to XLANG/s by actually compiling the converted XLANG/s into .NET assemblies prior to execution, resulting in a five-fold performance improvement over the previous version of BizTalk Server.

Perhaps the main reason BPEL has received less attention than other specifications is that most vendors (including Microsoft) do not expect it to be created the same way you create WSDL or XSD—using an XML editor. Instead, tooling in BizTalk Server provides a rich graphical process editor to create orchestrations—the Orchestration Designer.

Support for interoperability in BizTalk Server enables BPEL documents to be imported and turned into orchestration (.odx) files. An orchestration can then be viewed and edited graphically in the designer and even exported back to BPEL if required. This allows companies or business units to share a business process quickly and easily, even if they run on completely different hardware and operating systems. This can make enterprises much more agile, particularly where changing a process could result in taking advantage of a brand new market opportunity.

By focusing on interoperability, BizTalk Server embraces BPEL without being constrained by it. Consider, for example, that the major processing requirement in messaging systems is transformation—the conversion from one format or structure to another. BPEL can't do this in the standard, but BizTalk Server can.

To give you an idea of what's going on behind the scenes, take a look at the code in Figure 1. It's important to bear in mind that BPEL is a substrate that binds pre-existing Web services together and is not intended to be used to implement the services themselves. This is one of the reasons that the feature set in BPEL is constrained to simple expressions and constructs. BPEL allows you to form new Web services through recombination and enables the assembly of complex business processes.

Figure 1 Sample BPEL Process—Adding Two Numbers

<?xml version="1.0"?> <process xmlns:wsdl="https://schemas.xmlsoap.org/wsdl/" xmlns:q2="https://tempuri.org/Arithmetic" xmlns:q1="https://tempuri.org" xmlns:xsd="https://www.w3.org/2001/XMLSchema" name="Example1.Add" targetNamespace="https://tempuri.org" xmlns="https://schemas.xmlsoap.org/ws/2003/03/business-process/"> <partnerLinks> <partnerLink name="Arithmetic" partnerLinkType="q1:ArithmeticType" myRole="portRole" /> <partnerLink name="InternalArithmeticService" partnerLinkType="q1:InternalArithmeticServiceType" partnerRole="portRole" /> </partnerLinks> <variables> <variable name="AddRequest" messageType= "q1:__messagetype_Example1_ArithmeticSchema_AddMessageIn"/> <variable name="AddResponse" messageType= "q1:__messagetype_Example1_ArithmeticSchema_AddMessageOut"/> <variable name="InternalAddRequest" messageType="q2:AddSoapIn" /> <variable name="InternalAddResponse" messageType="q2:AddSoapOut" /> </variables> <sequence> <receive partnerLink="Arithmetic" portType="q1:ArithmeticPT" operation="AddTwoNumbers" variable="AddRequest" createInstance="yes" /> <assign> <copy> <from variable="AddRequest" part="part" query="/*[local-name()='AddMessageIn' and namespace-uri()='https://Example1 .Arithmetic']/*[local-name()='X' and namespace-uri()='']" /> <to variable="InternalAddRequest" part="A" query="/*[local-name()='int' and namespace-uri()='https://tempuri.org']" /> </copy> <copy> <from variable="AddRequest" part="part" query="/*[local-name()='AddMessageIn' and namespace- uri()='https://Example1 .Arithmetic']/* [local-name()='Y' and namespace-uri()='']" /> <to variable="InternalAddRequest" part="B" query="/*[local-name()='int' and namespace- uri()='https://tempuri.org']" /> </copy> </assign> <invoke partnerLink="InternalArithmeticService" portType="q2:AddSoap" operation="Add" inputVariable="InternalAddRequest" outputVariable="InternalAddResponse" /> <assign> <copy> <from variable="InternalAddResponse" part="AddResult" /> <to variable="AddResponse" part="part" query="/*[local- name()='AddMessageOut' and namespace- uri()='https://Example1 .Arithmetic']/*[local- name()='Z' and namespace-uri()='']" /> </copy> </assign> <reply partnerLink="Arithmetic" portType="q1:ArithmeticPT" operation="AddTwoNumbers" variable="AddResponse" /> </sequence> </process>

Figure 2** Relationship Between BPEL, WSDL, and XML Schema **

A BPEL definition generally requires two other document types, WSDL and XML schema, and BizTalk Server orchestrations are no exception to this. Figure 2 shows the relationship between them. BPEL extends WSDL to both provide and consume Web services in an abstract way. This enables one of the great strengths of WSDL, namely the ability to separate the abstract message and port information required at composition time from the physical binding and address details required at invocation time. The WSDL and XSD for this example are given in Figure 3 and Figure 4.

Figure 4 XSD for Adding Two Numbers

<?xml version="1.0" encoding="utf-16"?> <xs:schema xmlns="https://Example1.Arithmetic" xmlns:b="https://schemas.microsoft.com/BizTalk/2003" targetNamespace="https://Example1.Arithmetic" xmlns:xs="https://www.w3.org/2001/XMLSchema"> <xs:element name="AddMessageIn"> <xs:annotation> <xs:appinfo> <b:properties xmlns:b="https://schemas.microsoft.com/BizTalk/ 2003"> <b:property distinguished="true" xpath="/*[local- name()='AddMessageIn' and namespace- uri()='https://Example1.Arithmetic']/*[local- name()='X' and namespace-uri()='']" /> <b:property distinguished="true" xpath="/*[local- name()='AddMessageIn' and namespace- uri()='https://Example1.Arithmetic']/*[local- name()='Y' and namespace-uri()='']" /> </b:properties> </xs:appinfo> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="X" type="xs:int" /> <xs:element name="Y" type="xs:int" /> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="AddMessageOut"> <xs:annotation> <xs:appinfo> <b:properties> <b:property distinguished="true" xpath="/*[local- name()='AddMessageOut' and namespace- uri()='https://Example1.Arithmetic']/*[local- name()='Z' and namespace-uri()='']" /> </b:properties> </xs:appinfo> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="Z" type="xs:integer" /> </xs:sequence> </xs:complexType>

Figure 3 WSDL for Adding Two Numbers

<?xml version="1.0"?> <definitions xmlns:tns="https://tempuri.org" xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:q2="https://Example1.Arithmetic" xmlns:q1="https://tempuri.org/WebService2/Service1" xmlns:bpel="https://schemas.xmlsoap.org/ws/2003/03/business-process/" xmlns:plnk="https://schemas.xmlsoap.org/ws/2003/05/partner-link/" targetNamespace="https://tempuri.org" name="Example1" xmlns="https://schemas.xmlsoap.org/wsdl/"> <import namespace="https://Example1.Arithmetic" location="Arithmetic.xsd" /> <types> <xsd:schema xmlns:xs="https://www.w3.org/2001/XMLSchema" xmlns="https://Example1.Arithmetic" xmlns:b="https://schemas.microsoft.com/BizTalk/2003" targetNamespace="https://Example1.Arithmetic"> <xs:element name="AddMessageIn"> <xs:annotation> <xs:appinfo> <b:properties xmlns:b= "https://schemas.microsoft.com/BizTalk/2003"> <b:property distinguished="true" xpath= "/*[local-name()='AddMessageIn' and namespace- uri()='https://Example1. Arithmetic']/*[local-name()='X' and namespace- uri()='']" /> <b:property distinguished="true" xpath="/*[local- name()='AddMessageIn' and namespace- uri()='https://Example1. Arithmetic']/*[local-name()='Y' and namespace-uri()='']" /> </b:properties> </xs:appinfo> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="X" type="xs:int" /> <xs:element name="Y" type="xs:int" /> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="AddMessageOut"> <xs:annotation> <xs:appinfo> <b:properties> <b:property distinguished="true" xpath="/*[local- name()='AddMessageOut' and namespace- uri()='https://Example1. Arithmetic']/*[local-name()='Z' and namespace-uri()='']" /> </b:properties> </xs:appinfo> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="Z" type="xs:integer" /> </xs:sequence> </xs:complexType> </xs:element> </xsd:schema> </types> <message name="__messagetype_Example1_ArithmeticSchema_AddMessageIn"> <part name="part" element="q2:AddMessageIn" /> </message> <message name="__messagetype_Example1_ArithmeticSchema_AddMessageOut"> <part name="part" element="q2:AddMessageOut" /> </message> <portType name="ArithmeticPT"> <operation name="AddTwoNumbers"> <input message= "tns:__messagetype_Example1_ArithmeticSchema_AddMessageIn" /> <output message= "tns:__messagetype_Example1_ArithmeticSchema_AddMessageOut" /> </operation> </portType> <plnk:partnerLinkType name="ArithmeticType"> <plnk:role name="portRole"> <plnk:portType name="tns:ArithmeticPT" /> </plnk:role> </plnk:partnerLinkType> <plnk:partnerLinkType name="InternalArithmeticServiceType"> <plnk:role name="portRole"> <plnk:portType name="q1:Service1Soap" /> </plnk:role> </plnk:partnerLinkType> </definitions>

A cursory examination of the BPEL vocabulary in Figure 1 reveals many familiar concepts such as scoped variables and type-compatible assignments. BPEL also provides logical constructs such as conditionals, loops, and parallel processing. The BPEL in Figure 1 needs little explanation. The basic constructs are termed activities in BPEL, the full list for which is provided in Figure 5. Because of the flexibility an XML-based representation provides, activities can be combined and nested in interesting ways. This process is particularly suited to a graphical design environment.

Figure 5 BPEL Activities

Many BPEL features relate to timing and synchronization. The loosely coupled, highly latent world of Web services requires a much more casual attitude to sequence and time than do regular applications. To see what I mean, think about how an orchestration runs. Waiting for a message is a passive activity and something in the orchestration will almost certainly have to wait for the appropriate message to appear. BizTalk Server deals with messages by implementing subscriptions. An orchestration can subscribe to a particular message type, and all messages received go into a persistent store called the message box, implemented using SQL Server™. The message box is examined and the subscribed orchestrations are executed by the runtime. This results in orchestration instances being created, each running with its own state. This is important for reasons I'll discuss shortly.Benefits of BPEL

• Cross-vendor support
• Enables process interoperability
• XML-based language
• Powerful control flows allowing arbitrary nesting
• Currently (BPEL 1.1) based on WSDL 1.1, XML Schema 1.0, XPath 1.0 and WS-Addressing
• Sync and async messaging
• Uses callbacks and correlation sets to compose services
• Transactions
• Atomic and long-running support using correlation sets
• Exception handling
• Compensation and fault processing

The next activity I want to discuss is the invoke activity. It enables an orchestration to call a Web service to help fulfill the implementation of the process. This functionality is present in BizTalk Server with the Send and Receive shapes. In the example in Figure 1, I am simply invoking a Web service that will add two numbers and return the sum. The expected WSDL definitions are referenced to provide the necessary binding and context. This case is a little contrived, but it does illustrate the ability to decouple externally exposed services from changes to internally defined services.

For example, this internal service could be replaced with another one or the schema changed to alter the message structure without affecting the caller in any way. Such encapsulation is important in order to achieve reuse of existing services as components of a larger orchestration. A public Web service for a bank that enables the transfer of funds between accounts will consist of internal invocation and collaboration of a number of different systems internally. Exposing these internal systems APIs to the outside world often leads to brittleness and can be highly insecure.

Instead, an orchestration can be used to provide a facade to bridge the outside world and your internal systems in a loosely coupled way. This is a common requirement in enterprise application integration (EAI) scenarios and typically employs content-based routing. Bear in mind that orchestrations in BizTalk Server do not have to be exposed as Web services. They can be run by a variety of means using different configurations. An orchestration's trigger may be an XML document dropped via FTP into a watched file system folder, for example.

It should be apparent by now that a messaging system has a large number of requirements that are not directly related to the execution of any one particular service. Because these requirements cut across all services, they are driving the creation of BPM software infrastructure. BizTalk Server is both a BPM and integration product. It consists of around 1.8 million lines of managed code. This gives a good indication of the size of a BPM application—you don't want to start writing one yourself.

If you're going to build business processes using existing services, you need to have those services in place first. But that's not the reality for most companies today, so it's critical to make your services generic and reusable, rather than services targeted to a particular consumer. Adopting an interface-first design approach makes this much easier to achieve.

The visibility of the actual running business processes represents a huge step forward and with BizTalk Server you can create a new process by linking together available "shapes" in the graphical editor. These shapes map naturally to BPEL activities already described and allow a process to be modeled by dragging and dropping these activities onto the designer surface. Developers can do this using Visual Studio® .NET 2003, and business analysts can use Visio® to design orchestrations. As you can see in Figure 6, the receive shape (labeled ReceiveMessage) is synonymous with the BPEL Receive activity. Similarly, the send shape (labeled ReplyMessage) maps to the Reply activity in BPEL. The goal is that each type of user has the best tools for their own familiar environment yet they share the same artifacts.

Figure 6** Orchestration in BizTalk Server 2004 **

Now let's consider a more complex scenario. One of the assumptions made explicit with distributed service-based systems is that they will occasionally fail. By their nature, timeouts can occur due to network latency or the endpoint being unavailable, for example. Messages may not be delivered, may be delivered multiple times, or may be delivered in an unexpected order. These characteristics make transactions problematic as the traditional approach of using atomicity to guarantee consistency and isolation through a two-phase commit scheme (such as that provided by the Microsoft Distributed Transaction Coordinator, or MS DTC) cannot be employed. With the WS-Transaction specification there will be a comparable solution, but even then there are many cases when transactions spanning organizational boundaries are not appropriate. A different approach taken in BPEL is based on the notion of compensation.

Compensation enables long-running transactions (LRTs) to be managed so that if failure occurs, a defined compensatory action will be invoked. One example is an airline reservation system. In booking a ticket, you may have to provide payment before your seat on the flight can be confirmed. This process could take several seconds. If the seat cannot be confirmed, the payment must be refunded so that it appears that the transaction never took place. LRTs are the mechanism by which Web services can achieve this state. Because of this, the term LRT is somewhat of a misnomer as it isn't really a transaction at all. Each action and compensatory action taken is distinct from the others. BizTalk Server implements this scheme and logically ties this unrelated activity coordination together to manage it and attempt to prevent inconsistencies. Note, though, that this is not a fail-safe mechanism because it does not preclude the possibility that inconsistencies could occur. It is also important to realize that this is not a failing of BizTalk Server itself; rather it is due to the pragmatic approach taken in dealing with failures within a loosely coupled distributed system.

Another problem Web services introduce is the lack of shared knowledge or state. This should be viewed as a strength because it reduces any dependency or unnecessary knowledge by one component of another. It does, however, require a different approach to solving a perennial problem. For example, an LRT may consist of a fire-and-forget style of asynchronous message being sent. It may be expected, though, that at some point in the future the service to which the message was directed will respond in a callback-style pattern or "solicit-response." When this occurs, the original orchestration must correlate the inbound message with the original outbound one. BPEL provides support for this with correlation sets. Correlation sets in BizTalk Server are based on the notion of specific process instances, as I mentioned earlier.

You must mark an orchestration in BizTalk Server as able to be activated in order to indicate that it can be the start of a new process instantiation. From that point on, if an activity is executed that causes the orchestration to suspend execution, it can be resumed later when the callback comes in. BizTalk Server supports this BPEL feature by allowing the receipt of a message and correlating it back to the original in-flight suspended orchestration instance. As the callback message is likely to differ from the original one that initiated the orchestration, BizTalk Server also provides the facility to alias correlation information. This allows unique data items to be identified across different message types in order for the BizTalk Server engine to associate them. This is achieved through property promotion, that is, the process of annotating the schema type being used with the relationships between different elements or complex types with the same meaning.

For example, a customer ID may be present in all messages but is unique to a particular customer. Because of this uniqueness property, it can be used to correlate different messages to a single orchestration instance even though in one message type it may be called CustID while in another message type, it may refer to the same entity called Customer.

BPEL contains other examples of synchronization activities all supported in BizTalk Server. One of the more interesting is the pick activity, which BizTalk Server provides via the Listen shape (see Figure 5). This activity allows the selection of a logic path based on the message type received. An orchestration that can receive one of several different messages can use this activity to wait until the first message is received and continue down a particular path based on its type. This is useful when you are dealing with a number of partners using different message structures but the semantics of the actual business process being created are the same for each.

Boundaries

As I've said, BPEL should be viewed purely as an orchestration language, and this is where the interoperability benefits lie. Because of this, BPEL imposes some restrictions on the expressions allowed in conditional activities and assignments. In order to discourage its use in implementing the actual business logic of a service the specification actually mandates that only simple expressions are allowed. This is because its stated purpose is to control the execution of the business logic components, namely the Web services.

Consequently, a BPEL orchestration cannot accomplish much by itself. A BPEL orchestration cannot actually implement a business process. Instead, it acts as a coordinator by passing messages from system to system, employing other services along the way. The orchestration only knows how to call, consume, and manage the calling of the other Web services and nothing more. BPEL is also just a specification, not a product.

For it to be useful, it has to exist as an implementation on a specific platform and that means implementation-specific code will be written to integrate with and extend it. This is likely to change as the specification process continues on to recommendation status, with the possibility that concessions will be made to provide for some of the common requirements in a standard, interoperable way. These current limitations make it possible to define orchestrations in BizTalk Server that cannot be expressed in BPEL. Although this is unavoidable, the extensibility BPEL allows will likely lead to greater improvements in this area over the next few years, and in particular, will lead to greater interoperability.

Conclusion

BPEL is first and foremost an orchestration language. It is enjoying widespread industry support and adoption by providing a full-feature set to define business process interactions. BPEL's major promise is that the creation of abstract and executable schemes can be defined as business process and run in any compliant engine. While today a single organization may not need to run BPM engines from multiple vendors, many organizations are moving towards implementing connected business processes across extended enterprises. In these situations especially, the ability to specify those aspects of a shared process in a technology-neutral manner is very advantageous.

The push is on to make Web services work both inside and outside of the enterprise. Many core specifications are now complete with implementations emerging in WSE technology today and the new set of .NET technologies for building and running connected systems (known as "Indigo") tomorrow. BPEL will almost certainly become central to complex business orchestration, but with a clear technology roadmap in place you can be sure that your Web services investment today will reap rewards for years to come. BizTalk Server unlocks the door to this interconnected world and enables you to do more with less.

Jon Fancey runs an IT consultancy in the UK that specializes in enterprise-class software design and implementation. You can reach him at msdn@jonfancey.com or read his blog at www.jonfancey.com.