Using Web Services Enhancements to Send SOAP Messages with Attachments

Article
06/30/2006

Jeannine Hall Gailey
Web Consultant

February 2003

Applies to:
   Microsoft&174; ASP.NET WebMethods
   Web Services Enhancements 1.0 for Microsoft .NET
   WS-Attachments specification
   DIME specification
   Microsoft SOAP Toolkit 3.0

Summary: See how Web Services Enhancements 1.0 for Microsoft .NET supports sending attachments using DIME and WS-Attachments specifications. (13 printed pages)

Introduction
Using DIME with Web Service Enhancements
Advanced DIME Support in WSE
Issues Using WSE for DIME
Conclusion

WS-Attachments is a proposed Web service standard that leverages the DIME message encapsulation protocol designed to facilitate sending attachments with SOAP messages. In this article, you will see how the new Web Services Enhancements (WSE) 1.0 for Microsoft .NET supports sending attachments using DIME and WS-Attachments specifications.

Introduction

WS-Attachments prescribes a method that uses Direct Internet Message Encapsulation (DIME) for sending and receiving SOAP messages with additional attachments, like binary files, XML fragments, and even other SOAP messages. Based on a specification submitted by Microsoft and IBM to the Internet Engineering Task Force (IETF), DIME is designed to encapsulate a SOAP message and its related attachments in a MIME-like way. As with SOAP, DIME messages can be sent using standard transport protocols, like HTTP, TCP, and UDP. DIME supports streaming data. DIME can even be used without SOAP, although there is limited ability for DIME to describe the contents of its messages in this instance.

Microsoft has chosen to support DIME and WS-Attachments in the first version of Web Services Enhancements (WSE), a solution for developing secure, robust and scalable Web services using Microsoft .NET. This article deals with implementing DIME rather than delving into the protocol itself. For an additional discussion of the DIME protocol, see my article on DIME in the December issue of MDSN Magazine.

While not strictly limited to use with SOAP, the impetus for DIME is to more efficiently transport attachments with SOAP messages, which will be particularly useful for Web services that need to include large binary files such as media files or binary data files. Of course, without DIME, you can still send data in a SOAP message. For example, if I wanted to send large media files in a SOAP message to a requesting client, I could certainly do this by encoding the binary attachments as Base64 XML and including them in the body of the SOAP message. However, the processing gets bloated when attachments are very large, and even trickier if they are digitally signed. Also, when I want to send other SOAP messages, XML documents, or XML fragments that have a different character encoding than the main SOAP message, things can get complicated. These are a few scenarios where being able to attach binary data to a SOAP message makes more sense.

DIME has been optimized for use with SOAP messages. By relying on the metadata that already exists in a SOAP message, a DIME parser is not burdened with having to read through as much metadata in the DIME message itself. This makes DIME messages faster and more efficient to parse. For more information on the benefits of DIME over MIME, see Matt Powell's article, Understanding DIME and WS-Attachments.

What Are Web Services Enhancements?

In order to drive Web service interoperability in the enterprise, the major players in XML Web services (including Microsoft, IBM, and Verisign) have proposed new specifications that will improve interoperability in areas that are crucial for Web services, such as security, reliable messaging, and sending attachments. To support these new proposed standards, Microsoft has released Web Services Enhancements (WSE) 1.0, which is made up of a set of classes that implement these new protocols, as well as a set of filters hosted by Microsoft ASP.NET that intercept incoming and outgoing SOAP messages and interpret or generate the SOAP headers to support the requested functionalities. WSE 1.0 provides support for the following specifications:

WS-Security
Web Services Security Addendum
DIME
WS-Attachments
WS-Routing
WS-Referral

For more information, see Web Services Enhancements for Microsoft .NET.

Using DIME with Web Service Enhancements

WSE v1.0 supports DIME with SOAP attachments using ASP.NET, and reading and writing DIME messages to and from an I/O stream. In this section, we will walk through the process of adding a binary attachment to a SOAP message in ASP.NET. Later, we will discuss the stream support for DIME in the WSE.

How the WSE Runtime Implements DIME

For SOAP messages with attachments, the WSE runtime implements a DIME-compliant message parser that is able to translate the records of an incoming DIME message and extract the primary SOAP message from the first DIME record and any encapsulated files from successive records as attachment objects. After being extracted from DIME, the primary SOAP message is then passed to the WSE message pipeline where the series input filters evaluate the SOAP message for any other WSE-supported headers. The following figure shows how an incoming DIME message is handled by the WSE runtime and ASP.NET.

Figure 1. WSE runtime and ASP.NET handling incoming DIME message

For outbound SOAP response messages with attachments, the DIME message parser in the WSE builds a new DIME message where the first DIME record is the outbound SOAP response and any specified attachments are encapsulated in subsequent DIME records. The following figure shows how a DIME message is assembled by the WSE runtime.

Figure 2. WSE runtime assembling a DIME message

A rich API enables programmatic control of the WSE runtime and its support for DIME. Objects in the Microsoft.Web.Services.Dime namespace provide support for DIME. For a SOAP-based Web service hosted by ASP.NET, the SoapContext class in the Microsoft.Web.Services namespace controls the behavior of the WSE runtime by specifying the use and properties of the advanced Web service protocols for a given SOAP message. In the WSE programming model, when a SOAP request message is received, the runtime generates an HttpSoapContext object for the incoming request. The HttpSoapContext.RequestContext property accesses the SoapContext object for the message and enumerates the information contained in the WSE-specific header elements of the message.

When incoming DIME messages conform to the WS-Attachments specification, the WSE runtime handles them as SOAP messages with attachments, and then extracts the primary SOAP message and all of its attachments. For each attachment, a DimeAttachment object is added to the DimeAttachmentCollection, and these attachments are accessed through the SoapContext.Attachments property. Similarly, attachments can be added to the DimeAttachmentCollection for the SoapContext of an outgoing message so that the WSE runtime will include them as DIME records in the outgoing message.

Now, let's walk through the process of creating a Web service response as a DIME message with attachments.

Configuring Web Services Enhancements for DIME

Even with WSE installed on the ASP.NET Web server, there are a few additional DIME-specific configurations that need to be made to the ASP.NET application in order to use attachments with DIME. After creating a new ASP.NET Web service project in Microsoft® Visual Studio® .NET, references to the Microsoft.Web.Services.dll assembly needs to be added to the project. You must also add a new type to the soapExtensionTypes element by adding a new add element in the Web.config file for the project as follows:

<configuration>
    <system.web>
      ...
        <webServices>
            <soapExtensionTypes>
                <add type=
                "Microsoft.Web.Services.WebServicesExtension,
                 Microsoft.Web.Services, 
                 Version=1.0.0.0,
                 Culture=neutral,
                 PublicKeyToken=31bf3856ad364e35" 
                priority="1" group="0" />
            </soapExtensionTypes>
        </webServices>
    </system.web>
</configuration>

In this example, the value of the type attribute must not include any breaks or extra spaces. If the webServices and soapExtensionTypes elements do not already exist, they must be added to the Web.config file as well. An easier way to configure your WSE-based project to use DIME is to install the WSE Settings tool, an unsupported add-in to Visual Studio that allows you to easily set up Web service projects that use WSE.

Adding Attachments to a SOAP Message Using WSE

Consider that we have a Web service class named ImageDimeService with a public method that returns a binary JPEG image file. In the code-behind page for the service's .aspx file, you should create aliases for these namespaces with the using directive as follows:

using System.Web.Services
using Microsoft.Web.Services
using Microsoft.Web.Services.Dime

In this example, the ImageDimeService class is defined as follows:

[WebService(Namespace="http://example.com/dime/",
  Description="Web service returns one or more JPEG files using DIME.")]
public class ImageService : System.Web.Services.WebService
{
  ...

The following GetImage method implements the WebMethod that returns the requested JPEG files:

[WebMethod]
public string[] GetImage(string[] imageNameCollection)
{
  // Get the SoapContext for the response message
  SoapContext myContext = HttpSoapContext.ResponseContext;

  // Create an array that returns URIs for the related DIME attachments.
  string[] retUri= new string[imageNameCollection.Length];

  int i = 0; // Iterrator

  // Create a DimeAttachment object for each file specified 
  // by the values of the imageID array. 
  Foreach (string imageName in imageNameCollection)
  {
    // String that represents the file name and path for the attachment.
    string filePath = "C:\\images\\" + imageName + ".jpg";
    
    // Create a new DIME attachment using the the file name and 
    // specifying the encoding of the attachment using the MIME media 
    // type of image\jpeg.
    DimeAttachment dimeImage = new DimeAttachment(
      "image/jpeg", TypeFormatEnum.MediaType,
      filePath);

    // Generate a GUID-based URI reference for the attachment object
    // and assign in to the ID property of the DIME record.

    dimeImage.Id = "uri:" + Guid.NewGuid().ToString();

    // Add the new DimeAttachment object to the SoapContext object.
    myContext.Attachments.Add(dimeImage);

    // Add the generated URI to array that is returned.
    retUri[i] = dimeImage.Id;

    i++;

  }

  // Return the array of URIs that match the ID vaules of 
  // the attachments.
  return retUri;
}

In this WebMethod, the SOAP request message from the client contains a collection of image names. For each name in the collection, the Web service retrieves an associated JPEG file from the C:\images folder and adds the image to the DimeAttachmentCollection for the ResponseContext. When the WebMethod returns, the WSE runtime makes the response message the primary DIME record and each DimeAttachment a subsequent record, in the order in which they were added to the DimeAttachmentCollection.

Extracting Attachments from a DIME Message

In order for a Web service consuming a client application to receive and handle DIME messages, WSE must also be installed on the client. In order to use the WSE DIME support with a Microsoft .NET client application in Visual Studio, you must add a reference for the Microsoft.Web.Services.dll assembly. Also, after adding a Web reference to the DIME-based Web service, you must modify the proxy class in the References.cs file so that it inherits from the Microsoft.Web.Services.WebServicesClientProtocol class in WSE. For example, the client Web service proxy generated for the ImageService Web service from the previous example needs to be modified as follows:

public class ImageService : Microsoft.Web.Services.WebServicesClientProtocol

If you have previously installed the WSE Settings tool on the client, Add Web Reference will automatically generate a separate proxy class ending in "Wse" that already inherits from the proper WSE class. In that case, you can just use the ImageServiceWse class as the service proxy.

The following example shows how you might consume the ImageService Web service to retrieve multiple JPEG files as DIME attachments.

try 
{
  // Call the Web method passing an array of image names, and
  // capture the URI values in the returned array in case we 
  // need to reference the attachments by ID.
  string[] retUri = myService.GetImage(imageNameCollection);
}
catch (Exception ex)
{
  // Handle any exception errors.;
}

// Check if response message contains any attachments.
if (myService.ResponseSoapContext.Attachments.Count > 0)
{
  // Display each attached JPEG image file.
  for (int i=0; i<myService.ResponseSoapContext.Attachments.Count;i++)
  {
    // Stream the attached image into a new Bitmap object to display.
    Bitmap myImage = new 
      Bitmap(myService.ResponseSoapContext.Attachments[i].Stream);
    
    // Do something with the image.

  }
  // Warn the user if no images were returned.
  MessageBox.Show("No images were returned");
}

In this example, each JPEG attachment is written from memory into a new Bitmap object as a data stream using the DimeAttachment.Stream property.

Advanced DIME Support in WSE

While WSE allows you to easily encapsulate attachments with outgoing SOAP messages, the WSE support for DIME provides some more advanced functionality that I will discuss in this section.

Chunking Large Attachments

As you may have seen in my previous article on DIME in MSDN Magazine, the DIME specification supports the ability to chunk large attachments into multiple records, which is useful when dealing with very large attachments. The theory behind chunking is that by breaking a large attachment into chunks, you do not have to buffer the entire attachment in order to write it to a single DIME record. WSE allows you to specify the chunking size of attachments in bytes by setting the DimeAttachment.ChunkSize property. However, since ASP.NET WebMethods do not support streaming, when using WSE with ASP.NET the entire DIME message gets buffered in memory, which limits the usefulness of chunking in this implementation. A better method for transporting large attachments using DIME is streaming DIME over an appropriate transport.

Streaming Data Using DimeReader and DimeWriter

While DIME is a message-based format, the fact that DIME records are serialized and can be chunked enables it to be used effectively over packet-level transport protocols such as TCP and UDP. WSE 1.0 supports the streaming of DIME messages to and from System.IO.Stream objects, such as NetworkStream, rather than being limited by a message-based protocol like HTTP. When streaming as DIME, the WSE runtime chunks the outgoing DIME records so that the record size of the object being streamed does not need to be defined. The last record chunk is written when the end of the stream is reached. The following sample takes an input stream from a file (such as a large binary image file), writes the stream to a series of chunked DIME records, and writes the DIME message to a Stream object, which might be a NetworkStream over TCP:

// Sample transforms a binary input stream to a DIME stream of specified 
// MIME media-type.
static void WriteToDime(Stream inputStr,Stream dimeStr,string mediaType)
{
  // create writer for DIME message out to the dimeStream
  DimeWriter myDW = new DimeWriter(dimeStr);

  // Create a GUID for the DIME record ID.
  Guid guid = Guid.NewGuid();
  string id = string.Format("uuid:{0}", guid.ToString());

  // Create a new DIME record where the MIME media type is specified
  // by mediaType and contentLength of -1 to specify chunking.
  DimeRecord myRecord = myDW.LastRecord(id, mediaType, 
    TypeFormatEnum.MediaType, -1);

  // Set a rather small size for each record chunk at 4KB.
  myRecord.ChunkSize = 4096;

  // Use BinaryWriter to write a stream to the DimeRecord.
  BinaryWriter myWriter = new BinaryWriter(myRecord.BodyStream);

  // Write bytes from incoming stream to BinaryWriter.
  int size = 4096;
  byte[] bytes = new byte[4096];
  int numBytes;
  while((numBytes = inputStr.Read(bytes, 0, size)) > 0)
  {
    myWriter.Write(bytes, 0, numBytes);
  }

  // Cleanup
  myDW.Close();
}

Notice that unlike a SOAP-based implementation of DIME where the entire DIME message gets read into memory, when streaming to DIME, you can use DimeRecord.ChunkSize to limit the amount of memory that you need to use.

The following method implements the opposite procedure, namely reading a DIME stream and extracting the record contents to a binary stream.

static void ReadFromDime(Stream dimeStr, Stream outStr)
{
  // Create a reader for the streamed DIME message.
  DimeReader myDR = new DimeReader(dimeStr);

  // Create a DimeRecord to read the current (and only) DIME record.
  DimeRecord myRecord = myDR.ReadRecord();

  // Create a reader to read the record contents into.
  BinaryReader myReader = new BinaryReader(myRecord.BodyStream);

  // Write bytes to the output stream.
  int size = 4096;
  byte[] bytes = new byte[4096];
  int numBytes;
  while((numBytes = myReader.Read(bytes, 0, size)) > 0)
  {
    outStr.Write(bytes, 0, numBytes);
  }
  
  // cleanup
  myReader.Close();
  myDR.Close();
}

Issues Using WSE for DIME

As with any version 1.0 product, there are some limitations in how the WSE supports DIME, WS-Attachment, and related specifications. These issues are as follows:

Securing Attachments

While WSE provides comprehensive support for securing SOAP messages according to the WS-Security specifications, this support does not extend to attachments to SOAP messages sent in a DIME message. When you tell WSE to sign and encrypt the primary SOAP message, the SOAP message passes through the WSE SecurityOutputFilter where it gets appropriately secured. Only after passing through the various filters does the SOAP message get encapsulated as a DIME message with the attachments, which means that the attachments themselves are not secured in any way since they can be read by any DIME parser.

If you need to secure the attachments in DIME messages, you should use one of the .NET Framework-supported encryption mechanisms provided by the System.Security.Cryptography namespace. Also, since the DIME specification currently defines no method for signing DIME message headers, WSE has no way to determine if DIME messages have been tampered with. In particular, you may use an href to indicate a particular attachment ID. There is no built-in support in the WSE to verify that some middleman did not modify the ID and point the reference at something completely different. Therefore, when using DIME to transport sensitive data, you should always secure the stream or use a secured transport.

Referencing Attachments

WSE supports the ID field for DIME records, which can be set using the DimeAttachment.Id property. This enables you to assign and access attachments using friendly ID string values rather than integer index values. For example, you could access an attachment with an ID of Tom simply by specifying ["Tom"] instead of the array index [0] as follows:

myService.ResponseSoapContext.Attachment["Tom"];

Technically, ID should be a URI, but this is not enforced by WSE.

The WS-Attachment specification prescribes that attachments be referenced from the primary SOAP message by the ID values of the DIME record in which they are contained. This is to improve performance of DIME parsers by enabling them to read the primary SOAP message and then quickly locate an attachment by parsing only record headers until the desired record is found. However, in this initial version of WSE, Microsoft has taken a more autonomous view of attachments, which means that the WSE runtime merely extracts each attachment in the order received and keeps each as an object in memory to be accessed as needed. In the previous example, I referenced the attachments by specifying a URI ID value for each attachment in the collection and then returned a collection of these URI values in the body of the message.

No WSDL Support for DIME

Although the WSDL Extension for SOAP in DIME specification details a method for supporting DIME in the Web Services Description Language (WSDL), WSE does not include any updates to the WSDL generation functionality in ASP.NET or to the Wsdl.exe tool. This means that in order to publish the fact that a Web service supports DIME, you will have to manually edit and publish the WSDL file according to this new specification. Although describing the DIME functionality in WSDL is the most correct way to go, full description is not really necessary since you can easily check the SoapContext object on incoming messages to see if any attachments were included.

SOAP Toolkit 3.0 Interoperability

Since both WSE and the Microsoft® SOAP Toolkit 3.0 support the WS-Attachments specification, DIME messages containing a SOAP message with attachments can be created and read between these two runtimes. In fact, the SOAP Toolkit offers a more complete implementation of the WS-Attachments specification with support for referencing attachments from the primary SOAP messages, and the SOAP Toolkit 3.0 includes an updated WSDL Generator tool (Wsdlgen3.exe) which produces WSDL files that conform to the proposed WSDL Extension for SOAP in DIME specification. WSE does not support such descriptions for DIME-based Web services.

One thing to watch out for in such an interoperability scenario is using the high-level API provided by the SOAP Toolkit to consume a WSE-based Web service with attachments. Since the high-level API consumes a WSDL file to generate the necessary connection objects to marshal communication between the client and Web service, you will need to manually modify and publish the WSDL file for the WSE-based Web service to add the missing <dime:message> child elements to the <wsdl:input> and <wsdl:output> elements as defined in the WSDL extension specification. However, SOAP Toolkit 3.0 only handles DIME messages that conform to the WS-Attachments specifications, which means that it cannot be used to stream data as WSE can.

Conclusion

Web Services Enhancements for Microsoft .NET offers the most complete solution for sending and receiving attachments with SOAP messages. In addition to DIME support, the WSE runtime also hosts the process filter pipeline that enables you to easily handle WSE-supported headers in the DIME message's primary SOAP message. The WSE supports a non-SOAP-based implementation of DIME as well that enables you to use DIME to stream data as a series of DIME records.