Web Services Protocol Workshops Process Overview
Jorgen Thelin
Program Manager, Distributed Systems Standards, Microsoft Corporation
October 2004
Summary: The WS-* workshop process is a method to produce well-engineered, quality Web services specifications. Feedback and interoperability testing by multiple vendors produces specifications that are stable, known to work, and ready for consideration as an industry standard. The WS-* workshop process has many similarities with the process for software development, and is a deliberate attempt to apply the best practices from the software engineering community to the task of producing Web services specifications. WS-* protocol workshops allow the Web services community to be involved in the process of validating and refining Web services specifications. (12 printed pages)
Contents
What Are WS-* Protocol Workshops?
Why Do Microsoft and Industry Partners Conduct WS-* Workshops?
WS-* Workshop Flow: the Specification Development Process
WS-* Workshop Results
WS-* Workshops Forward Plan
Active Participation in the WS-* Workshop Process
Summary
Resources
What Are WS-* Protocol Workshops?
Web services protocol workshops are part of the standards-development process that Microsoft and industry partners use to develop well-engineered, quality Web services specifications. The WS-* specifications are the framework elements of an end-to-end architecture for distributed systems. Among other benefits, the WS-* architecture enables platform-neutral interoperability.
WS-* Workshops are structured, face-to-face meetings hosted by one of the co-authors of a Web Service specification.
WS-* Workshop meetings exist to gather feedback on the specification documents and experience implementing the specifications.
WS-* Workshops involve the Web services community (including end-user companies, academics, ISVs and vendors) in the process of refining and testing Web services specifications.
WS-* protocol workshops are an example of Microsoft's commitment to openness and industry cooperation around Web service specifications and multi-vendor interoperability.
Types of WS-* Workshops
There are two types of WS-* workshops:
- Feedback Workshops gather comments on a specification from implementers and interested parties.
- Interoperability Workshops test product implementations to validate the ability of vendors to develop interoperable applications scenarios using the corresponding specification protocol(s).
More details on the mechanics of WS-* workshops are detailed below after a review of the history and goals of the WS-* workshop process.
History of WS-* Workshops
WS-* Protocol Workshops have a solid history beginning in early 2003, as illustrated in the following diagrams and tables.
The SOAP and WSDL specifications proceeded through a prototype version of the workshop process during 2001 and 2002, and this experience led to the refined and formalized WS-* workshop process now in use.
.gif)
Figure 1. History of WS protocol interoperability workshops
The WS-* workshop process builds upon the successful community consultation around previous Web services specifications—including SOAP and WSDL. The WS-* workshop process copies many of the best aspects of the SOAPBuilders group—including soliciting broad feedback and hands-on interoperability testing. However, one improvement over previous consultation mechanisms is that the WS-* workshop process provides a clear commitment from co-authors and contributors to make licenses broadly available for intellectual property (IP) necessary to implement the specifications.
Table 1. WS Protocol Workshop History
| Workshop Event | Date | Companies |
|---|---|---|
| WS-Policy and WS-Trust Feedback | February 2003 | 15 |
| WS-Policy and WS-Trust Feedback | March 2003 | 14 |
| WS-Reliable Messaging Feedback | July 2003 | 11 |
| WS-Reliable Messaging Interop | October 2003 | 8 |
| WS-Secure Conversation / WS-Trust Interop | November 2003 | 5 |
| WS-Federation Feedback | November 2003 | 15 |
| WS-Eventing Feedback | February 2004 | 18 |
| WS-Transactions Feedback | March 2004 | 16 |
| WS-Federation Passive Requester Profile Interop | March 2004 | 7 |
| WS-Eventing Interop | April 2004 | 7 |
| WS-Reliable Messaging Interop | May 2004 | 5 |
| WS-Discovery Feedback | May 2004 | 12 |
| WS-Device Profile Feedback | August 2004 | 13 |
| WS-Secure Conversation / WS-Trust Interop | October 2004 | 6 |
| WS-Metadata Exchange Feedback | October 2004 | 15 |
As of October 2004, 66 different companies have attended workshop meetings.
Why Do Microsoft and Industry Partners Conduct WS-* Workshops?
The main reason for the WS-* workshop process is to produce well-engineered, quality specifications.
Secondary benefits of WS-* workshops include:
- Workshops provide proof of the interoperability of the Web services specifications.
- Workshops help to uncover any inconsistencies with other WS-* specifications.
- Interop workshops allow implementation experience to be gained much earlier in the development process, which helps shape the specifications and validate the overall design.
- Workshops foster community involvement among partners, vendors, academics, consultants, authors and other interested parties to shape and improve the development of the Web services specifications in a formal setting
- Workshops apply software testing disciplines to the way Web services specifications are developed. This provides additional review and test cycles in order to increase the effectiveness of finding and removing defects in the specifications.
- Workshops help determine whether the specification is appropriate and ready to move to an industry standards-setting organizations for consideration for approval and ratification
WS-* Workshop Process Flow: the Specification Development Process
The diagram below shows the steps taken in progressing a specification through to completion. The stages involved in the specification development process are: Authors drafts, Feedback Workshops, Interoperability Workshops, and Ratification as a Standard.
.gif)
Figure 2. WS-* specification development process
The WS-* workshop process has many similarities with the process for software development, and this is a deliberate attempt to utilize many of the best practices from the software engineering community. All software engineering processes involve a series of development stages, with quality control checkpoints between each to determine whether software is ready to proceed to the next stage in the process, and this approach is mirrored in the WS-* workshop process.
WS-* Specification Development, Phase 1: Author Drafts
The specification authors first work on an initial draft known as an Author Draft.
There may be many revisions until the authors arrive at an interim version of the document they are happy with (a Public Consultation draft).
At this stage, the specification is published on the authors' Web sites, such as the Microsoft Developer Network (MSDN) Web Services Developer Center.
WS-* Specification Development, Phase 2: Feedback Workshops
Once a Public Consultation draft has been published, the WS-* Workshops phases begins. The first step is a Feedback Workshop.
Feedback Workshop: Goal
A Feedback Workshop allows the authors to provide an introduction and education on the specification to the Web services industry. Workshop participants are encouraged to raise questions or provide detailed feedback on the contents of the specification.
A Feedback Workshop provides a quality control checkpoint early in the development process to provide confirmation that the fundamentals are correct and the specification is ready to move to the next stage.
Feedback Workshop: Audience
The audience for Feedback Workshops is anyone interested in the specifications, including vendors, customers, academics, and IT professionals.
Feedback Workshop Attendance
Attendance at WS-* workshops is open to all.
Anyone can attend a WS-* workshop regardless of whether they received an invitation directly from the specification author companies or not.
The only requirement for attendance is that participants sign a feedback agreement to allow the specification authors to utilize the feedback they receive in a royalty-free manner. Without such an agreement, there is a danger that any feedback the authors incorporated in the specification could introduce intellectual property and patent encumbrances that would prevent the authors granting licenses to these specifications on a royalty-free basis. The feedback agreements provide a well-defined legal framework around the WS-* workshop process that keeps the specifications clear of any IP taints.
Feedback Workshop: Format
The format of a Feedback Workshop is a one-day meeting with presentations from the specification authors for part of the day and round-table discussion for the remainder of the day.
The meeting minutes record all the feedback and issues brought up at the workshop.
A Feedback Workshop is very similar to the practice of code reviews in a typical software development process.
Workshop Invitations
Invitations are sent out to all participants of previous workshops, plus academics and interested parties and contacts in the industry.
The invitations are also posted to various industry mailing lists frequented by Web services infrastructure developers. An example of such a mailing list is the SOAPBuilders list on Yahoo Groups.
Recipients of the invitation are encouraged to forward the invitation to colleagues inside or outside their companies.
Anyone can ask to be added to the workshop invitation mailing list.
Discussion Lists
Each workshop has an associated mailing list for follow-on discussions after the face-to-face meetings.
The aim of the mailing lists is to permit community engagement in the ongoing development of the specifications and allow additional feedback to be collected over time.
This mailing list is also the place where discussion of on-going interoperability testing will occur.
See the Resources section below for the list of available mailing lists.
Feedback Workshop: Outputs
The output from a Feedback Workshop includes a list of questions, comments, clarifications, and issues for the specification authors. It also may produce some suggestions for future interoperability scenarios.
The authors then review each of those items and agree to a resolution.
Feedback Workshop: Next Steps
The usual outcome of a Feedback Workshop is that the authors create a new version of the specification that incorporates the feedback.
In most cases this new specification version is termed the Interoperability Draft (if the authors' opinion is that the specification is ready to go forward to multi-vendor interoperability testing), although there may be some occasions where the authors feel that the revisions are sufficiently major that another Feedback Workshop is required.
WS-* Specification Development, Phase 3: Interoperability Workshops
Once a stable Interoperability Draft is published, an Interoperability Workshop is arranged for multi-vendor testing of the specifications.
Interoperability Workshop: Goal
The goal of an Interoperability Workshop is to test the interoperability between different implementations of the specifications and gather any issues or problems in achieving that.
Interoperability Workshop: Audience
The audience for Interoperability Workshops is companies, organizations, and individuals who have implemented the specifications.
It is recognized that many companies will be bringing pre-release code to the Interoperability Workshops, so mechanisms are built in to the operating procedures and agreements for the workshops to respect the privacy of vendors and not allow the use of the interoperability results for competitive leverage or public relations advantage. This helps to create a constructive and co-operative environment in the workshop meetings.
Interoperability Workshop Attendance
Any company, organization, or individual that have implementations of the specifications being tested are able to attend an Interoperability Workshop.
As with Feedback Workshops, participants at Interoperability Workshops need to sign a feedback agreement to allow the specification authors to utilize any feedback they receive in a royalty-free manner.
Interoperability Workshop: Format
The format of an Interoperability Workshop is a 1-, 2-, or 3-day meeting where participants bring their own implementations for a hands-on round-table interoperability lab.
The specification authors define a set of interoperability scenarios to drive the testing efforts. Test cases and scenarios are developed and distributed in advance of the workshop meeting—typically at least 6 to 8 weeks in advance of the workshop to allow sufficient time to implement the scenarios.
The Interoperability Workshop meeting provides a focused opportunity for vendors of Web services software to test against other vendors' implementations in a structured and collaborative manner.
An Interoperability Workshop is very similar to the practice of system and integration testing in a typical software development process. Both involve executing the implementations in a close-to-real-world runtime environment, and testing that everything fits together and works as expected.
Interoperability Scenarios
The specification authors create a set of interoperability testing scenarios for use in each Interoperability Workshop, and circulate that document to interested parties and appropriate mailing lists for community comment on the proposed test cases.
Scenarios are test cases that are close to the anticipated real world usage patterns of the specification and are designed to validate that interoperability can be achieved outside a simple research lab setting.
Each participant in an Interoperability Workshop brings an implementation of the scenarios for that workshop. Participants build the scenario implementations with their own choice of products, tools, programming languages, and operating systems.
Almost all Interoperability Workshops will involve some scenarios that involve composition of multiple specifications, as that is the situation most likely to be found in any significant real world usage. For example, few enterprise-grade applications will run without some security features enabled, so some of the scenarios in the Interoperability Workshop for WS-ReliableMessaging cover composition with Web services security specifications (such as WS-Security and WS-SecureConversation).
Interoperability Workshop: Outputs
The outputs from an Interoperability Workshop are a measure of interoperability between implementations, a list of questions, comments, clarifications, and issues for the specification authors, and possibly some suggestions for additional interoperability scenarios from participants.
Interoperability Endpoints
After the Interoperability Workshop, the vendors participating are encouraged to make Interoperability endpoints available for ongoing testing among implementations.
These endpoints allow the reach of the Workshop to be extended in both time and location. This allows any vendors who were not able to attend the face-to-face meeting for any reason to test their implementations too.
Face-to-face testing still has many significant advantages compared to endpoint testing though. These include easier diagnosis of interoperability problems, faster turnaround of interoperability queries, providing a deadline to help all vendors get their implementations ready for testing and to allow testers and developers to talk directly to each other.
Interoperability Workshop: Next Steps
After an Interoperability Workshop, the specification authors review the outcome of the testing, and any issues raised during the interoperability testing. They make a decision whether the specification needs final amendments, and whether the testing has achieved sufficient coverage of the specification features to warrant final release to a standards-setting organization for consideration for approval and ratification.
Transitioning to Phase 4: WS-* Workshop Process Exit Criteria
Authors make the decision on whether a specification has demonstrated the required quality for release to a standards-setting organization based on many factors.
The most significant exit criteria involves assessing the technical closure on the specification, the interoperability achieved, and the overall fit with the rest of the WS-* architecture.
Technical closure consists of three main facets: composition, coverage, and proven interoperability.
- Composition—Does the specification cleanly fit together with the other specifications in the Web services architecture stack? Can these specifications work successfully and interoperate together?
- Coverage—Have all the main features of the specification been interoperability tested, so that all the major functions of the specification are exercised?
- Proven interoperability—Has the testing in the workshops achieved proven interoperability with multiple implementations of the specification?
If the authors decide that the test results were not sufficiently good or that significant problems or ambiguities were noticed in the specifications, or that there are additional features in the specification that have not yet been tested, then another Interoperability Workshop and/or publication of another specification revision will occur.
When a specification has achieved release quality it is ready to exit the workshop process and pass to the next stage of its development, namely consideration for approval by a standards-setting organization, and eventual ratification.
WS-* Specification Development, Phase 4: Ratification As a Standard
Once the co-authors and community have completed the design work on a specification, and once broad agreement on the contents, quality, and interoperability of the specification has been demonstrated through the workshop process, the specification is then ready to move to a standards-setting organization to formally recognize that agreement in the industry.
At this point, the specification is processed under the procedures operated by the standards-setting organization.
WS-* Workshop Results
A solid core of companies have supported Web Services Protocol Workshops from across a wide range of sectors, including enterprise and Web services specialists, industry vertical organizations, value-added networking specialists, devices vendors, and academia. As of October 2004, 66 different companies have attended workshop meetings.
There have been some very successful results from the testing at interoperability workshops. For example, an interoperability workshop held in March 2004 covered the WS-Federation Passive Requester Profile specification. This workshop had the highest turnout of any interoperability event to date. The testing was a great success, with all implementations successfully completing the baseline interoperability scenario by the end of the first day. Many of the companies moved on to test an additional scenario as a follow-on exercise—this was also successfully completed by the end of the afternoon.
Feedback from workshops has been used to refine and update various WS-* specifications. For example, feedback from previous WS-ReliableMessaging Feedback and Interoperability Workshops was included in the version of that specification republished in March 2004. Also the WS-Addressing, WS-Policy, WS-SecureConversation, and WS-Trust specifications were republished recently incorporating workshop feedback.
As of October 2004, two WS-* specifications (WS-Security and WS-Addressing) have moved on to a standards-setting organization after completing their journey through the workshop process.
Both SOAP and WSDL also benefited from an earlier version of the workshop process where multi-vendor interoperability testing validated the specifications before they moved to a standards-setting organization.
Several more specifications are close to the end of the workshop process, and they will be moving to standards-setting organizations throughout 2005.
WS-* Workshops Forward Plan
A series of workshops are planned for the remainder of calendar year 2004 and in 2005. Below is the current snapshot:
Security Workshops
WS-Federation Active Profile Interop
Reliability Workshops
WS-ReliableMessaging Interop
Transactions Workshops
WS-Transactions Interop
Infrastructure Workshops
WS-Policy Interop
WS-MetadataExchange Interop
Devices Workshops
WS-Discovery Interop
WS-Device Profile Interop
WS-Eventing Interop
Management Workshops
WS-Management Feedback
WS-Management Interop
Active Participation in the WS-* Workshop Process
Workshops are open to everyone in the Web services community.
Interested parties can become actively involved in the workshop process to track or help shape the evolution of the Web services specifications through the following actions:
- Get more information on workshops
- Join the workshop mailing list(s)
- Attend workshop meetings
Getting More Information on WS-* Workshops
Full details of the workshop process and history are available on the MSDN Workshops home page. This page also includes the details of all upcoming and previous workshops.
Joining the WS-* Workshop Mailing Lists
Interested parties can join the workshop mailing lists on Yahoo Groups to post queries on the specifications, or discuss the interoperability scenarios proposed for future Interoperability Workshops.
See the Resources section below for the list of available mailing lists.
Attending WS-* Workshop Meetings
Interested parties can come to future Feedback Workshops to find out about the details of the Web services specifications directly from the specification authors, provide technical input on the specifications, and discuss how their own usage scenarios can be satisfied by using the WS-* architecture.
Anyone with an implementation of the relevant specification can attend an Interoperability Workshop and bring that implementation for round-table testing with other Web services vendors.
Workshop participants are asked to a sign the feedback agreement included in the invitation pack. This agreement allows the specification authors to utilize any feedback they receive through the workshop process without encumbering the specifications with intellectual property and patent claims that would prevent the authors from granting licenses to these specifications on a royalty-free basis.
Summary
WS-* Workshops are a vital part of the standards-development process used by Microsoft and industry partners to create well-engineered, quality, interoperable Web services specifications.
WS-* Workshops are a structured mechanism for applying software engineering "best practices" to the development of Web services specifications.
WS-* Workshop meetings have already provided valuable community feedback to the specification authors, which has been incorporated in updated versions of several Web services specifications.
When a specification exits the WS-* workshop process, it is ready to move to an industry standards-setting organization for consideration for approval and ratification.
Resources
MSDN Web Service Specifications home page
MSDN Web Service Protocol Workshops home page
Workshop mailing lists: