Printer Friendly Version      Send     
Click to Rate and Give Feedback
Related Articles
This article presents an overview of the motivation behind new techniques that decompose problems into independent pieces for optimal use of parallel programming.

By David Callahan (October 2008)
We take a look at planned support for parallel programming for both managed and native code in the next version of Visual Studio.

By Stephen Toub and Hazim Shafi (October 2008)
Here we describe some of the more common challenges to concurrent programming and present advice for coping with them in your software.

By Joe Duffy (October 2008)
Here is an ASP.NET AJAX data-driven Web application that takes the best features from server- and client-side programming to deliver an efficient, user-friendly experience.

By Bertrand Le Roy (October 2008)
More ...
Articles by this Author
If you're unfamiliar with Windows Presentation Foundation (WPF), building that first Silverlight custom control can be a daunting experience. This article walks through the process.

By Jeff Prosise (August 2008)
As we'll show, with just a few lines of JavaScript you can build a general-purpose framework for incorporating page turns into Silverlight 1.0 apps.

By Jeff Prosise (May 2008)
: Jeff Prosise presents great tips for Silverlight development, which while it's gaining wide adoption, still needs more documentation and best practices so developers can make the most of the dazzling new features.

By Jeff Prosise (Launch 2008)
Jeff Prosise shows how you can implement drag-and-drop functionality in your Web app with ASP.NET AJAX.

By Jeff Prosise (January 2008)
Jeff Prosise explains when it's better to use UpdatePanel and when it's better to use asynchronous calls to WebMethods or page methods instead.

By Jeff Prosise (June 2007)
Jeff Prosise describes performance problems in an ASMX Web service that relied on legacy COM and Visual Basic 6.0 to perform key processing tasks and the approach he took to find a fix.

By Jeff Prosise (October 2006)


By Jeff Prosise (July 2006)
Data-driven site navigation is among the niftiest and most useful features in ASP. NET 2. 0. To get it working, all you do is create an XML site map file (or a SQL site map if you're using the MSDN®Magazine SqlSiteMapProvider), add a SiteMapDataSource, and bind a TreeView or Menu to the SiteMapDataSource.

By Jeff Prosise (June 2006)
More ...
Popular Articles
See how to build a document-level Visual Studio Tools for Office customization and integrate it with a content type in SharePoint.

By Steve Fox (May 2008)
Kenny Kerr sings the praises of the new Visual C++ 2008 Feature Pack, which brings modern conveniences to Visual C++.

By Kenny Kerr (May 2008)
In this article, author John Torjo presents a guide to his C++ GUI library called eGUI++ and explains how it makes user interface programming easier.

By John Torjo (June 2008)
If you're unfamiliar with Windows Presentation Foundation (WPF), building that first Silverlight custom control can be a daunting experience. This article walks through the process.

By Jeff Prosise (August 2008)
More ...
Read the Blog
Multicore systems are becoming increasingly prevalent, but the majority of software today will not automatically take advantage of this additional processing ability. And multithreaded programming, for anything but the most trivial of systems, is incredibly difficult and error prone today. In the October 2008 issue of MSDN ...
Read more!
Concurrent programming is notoriously difficult, even for experts. You have all of the correctness and security challenges of sequential programs plus all of the difficulties of parallelism and concurrent access to shared resources. In the October 2008 issue of MSDN Magazine, David Callahan describes ...
Read more!
A major advantage of AJAX and Silverlight applications is that they can transparently and continuously interact with a back-end service. The problem is that they run over HTTP, which wasn't designed with security in mind. In the September 2008 issue of MSDN Magazine, Dino Esposito shows you ...
Read more!
Unhandled exception processing shouldn't be a mystery. It's actually quite useful since it gives a crashing application an opportunity to perform last-minute diagnostic logging about what went wrong. In the September 2008 issue of MSDN Magazine, Gaurav Khanna discusses how ...
Read more!
Silverlight 2 data-binding features are simple to implement and let your apps communicate via WCF services with line-of-business applications, databases, and other services in your organization. In the September 2008 issue of MSDN Magazine, John Papa demonstrate how to build a ...
Read more!
The Security Development Lifecycle (SDL) team recently released details of the SDL process at microsoft.com/sdl. What you won't find in the publicly available SDL documentation is guidance specific to securing Web applications or online services. In the September 2008 issue of MSDN ...
Read more!
More ...
Wicked Code
Scalable Apps with Asynchronous Programming in ASP.NET
Jeff Prosise

Code download available at: WickedCode2007_03.exe (202 KB)
Browse the Code Online
Do you want to know a secret? A deep, dark, dirty secret? One that, if revealed, would cause great angst in the ASP.NET community and prompt shouts of "Aha!" from the anti-Microsoft crowd?
Most Web sites I've seen built with ASP.NET aren't very scalable, not because of a flaw in ASP.NET, but because of how the technology is used. They suffer a self-imposed glass ceiling that limits the number of requests they can process per second. These sites scale just fine until traffic rises to the level of this invisible ceiling. Then throughput begins to degrade. Soon after, requests start to fail, usually returning "Server unavailable" errors.
The underlying reason has been discussed many times in MSDN®Magazine. ASP.NET uses threads from a common language runtime (CLR) thread pool to process requests. As long as there are threads available in the thread pool, ASP.NET has no trouble dispatching incoming requests. But once the thread pool becomes saturated-that is, all the threads inside it are busy processing requests and no free threads remain-new requests have to wait for threads to become free. If the logjam becomes severe enough and the queue fills to capacity, ASP.NET throws up its hands and responds with a "Heck no!" to new requests.
One solution is to increase the maximum size of the thread pool, allowing more threads to be created. That's the course developers often take when their customers report repeated "Server unavailable" errors. Another common course of action is to throw hardware at the problem, adding more servers to the Web farm. But increasing the thread count-or the server count-doesn't solve the issue. It just provides temporary relief to what is in reality a design problem-not in ASP.NET itself, but in the implementation of the actual site. The real problem for apps that don't scale isn't lack of threads. It's inefficient use of the threads that are already there.
A truly scalable ASP.NET Web site makes optimum use of the thread pool. That means making sure request-processing threads are executing code instead of waiting for I/O to complete. If the thread pool becomes saturated due to all the threads grinding away on the CPU, there's little you can do but add servers.
However, most Web apps talk to databases, Web services, or other external entities, and limit scalability by forcing thread-pool threads to wait for database queries, Web service calls, and other I/O operations to complete. A request targeting a data-driven Web page might spend a few thousandths of a second executing code and several seconds waiting for a database query to return. While the query is outstanding, the thread assigned to the request is unable to service other requests. That's the glass ceiling. And that's a situation you must avoid if you care about building highly scalable Web sites. Remember: when it comes to throughput, I/O is evil unless handled properly.
I/O isn't evil, though, if it doesn't gum up the thread pool. And ASP.NET supports three asynchronous programming models that act as anti-gumming agents. These models are largely unknown to the community, in part due to scant documentation. Yet knowing how-and when-to use them is absolutely essential to building cutting-edge Web sites.

Asynchronous Pages
The first, and generally most useful, of the three asynchronous programming models ASP.NET supports is the asynchronous page. Of the three models, this is the only one that's specific to ASP.NET 2.0. The others are supported all the way back to version 1.0.
I won't go into detail about asynchronous pages here because I did that in the October 2005 issue (msdn.microsoft.com/msdnmag/issues/05/10/WickedCode). The upshot is that if you have pages that perform relatively lengthy I/O operations, they're candidates to become asynchronous pages. If a page queries a database and the query takes, say, 5 seconds to return because it either returns a large amount of data or targets a remote database over a heavily loaded connection, that's 5 seconds that the thread assigned to the request can't be used for other requests. If every request behaved like this, the application would quickly get bogged down.
Figure 1 illustrates how an asynchronous page neatly solves this problem. When the request arrives, it's assigned a thread by ASP.NET. The request begins processing on that thread, but when the time comes to hit the database, the request launches an asynchronous ADO.NET query and returns the thread to the thread pool. When the query completes, ADO.NET calls back to ASP.NET, and ASP.NET grabs another thread from the thread pool and resumes processing the request.
Figure 1 Asynchronous Pages at Work (Click the image for a larger view)
While the query is outstanding, zero thread pool threads are consumed, leaving all of the threads free to service incoming requests. A request that's processed asynchronously doesn't execute any faster. But other requests execute faster because they don't have to wait for threads to become free. Requests incur less delay in entering the pipeline, and overall throughout goes up.
Figure 2 shows the codebehind class for an asynchronous page that performs data binding against a SQL Server™ database. The Page_Load method calls AddOnPreRenderCompleteAsync to register begin and end handlers. Later in the request's lifetime, ASP.NET calls the begin method, which launches an asynchronous ADO.NET query and returns immediately, whereupon the thread assigned to the request goes back into the thread pool. When ADO.NET signals that the query has completed, ASP.NET retrieves a thread from the thread pool (not necessarily the same one it used before) and calls the end method. The end method grabs the query results and the remainder of the request executes as normal on the thread that executed the end method.
Not shown in Figure 2 is the Async="true" attribute in the ASPX's Page directive. This is required for an asynchronous page: it signals ASP.NET to implement the IHttpAsyncHandler interface in the page (more on this in a moment). Also not shown in Figure 2 is the database connection string, which includes an Async="true" attribute of its own so that ADO.NET knows to perform an asynchronous query.
AddOnPreRenderCompleteAsync is one way to structure an asynchronous page. Another way is to call RegisterAsyncTask. This has a couple of advantages over AddOnPreRenderCompleteAsync-the most important being that it simplifies the task of performing multiple asynchronous I/O operations in one request. For details on this, see the October 2005 installment of Wicked Code.

Asynchronous HTTP Handlers
The second asynchronous programming model featured in ASP.NET is the asynchronous HTTP handler. An HTTP handler is an object that serves as an endpoint for requests. Requests for ASPX files, for example, are processed by an HTTP handler for ASPX files. Likewise, requests for ASMX files are handled by an HTTP handler that knows how to deal with ASMX services. In fact, ASP.NET comes with HTTP handlers for a variety of file types. You can see these file types-and the corresponding HTTP handlers-in the <httpHandlers> section of the master web.config file (in ASP.NET 1.x, it's in machine.config).
You can extend ASP.NET to support additional file types by writing custom HTTP handlers. But even more interesting is the fact that you can deploy custom HTTP handlers in ASHX files and use them as targets of HTTP requests. This is the proper way to build Web endpoints that generate images on the fly or retrieve images from databases. You simply include an <img> tag (or Image control) in the page and point it to an ASHX that creates or fetches the image. Targeting an ASHX file with requests is more efficient than targeting an ASPX file because an ASHX file incurs much less overhead at processing time.
By definition, HTTP handlers implement the IHttpHandler interface. Handlers that implement that interface do their processing synchronously. The ASHX file in Figure 3 contains one such HTTP handler. At run time, TerraServiceImageGrabber makes multiple calls out to the Microsoft® TerraServer Web service to convert a city and state into latitude and longitude, retrieve satellite images ("tiles"), and stitch the images together to form a composite image of the specified location.
The results are shown in Figure 4. The page shown contains an Image control whose ImageUrl property targets the ASHX file in Figure 3. Once the user selects a city and state and clicks a button, the HTTP handler converts the input into a satellite image.
The results are impressive. But there's a catch. TerraServiceImageGrabber is a perfect example of how not to write HTTP handlers. Think about it. TerraServiceImageGrabber requires several seconds (at least) to make all its Web service calls and process the results. The vast majority of that time is spent simply waiting for Web service calls to complete. Repeated requests for the ASHX file can deplete the ASP.NET thread pool in a fraction of a second, preventing other pages in the application from being served up (or at least forcing them to be queued until a thread becomes available). You can't build a scalable application in this way unless you scale out the hardware. But why spend tens of thousands of dollars on a Web farm when one server might handle the load with properly written software?
Figure 4 TerraServiceImageGrabber at Work (Click the image for a larger view)
HTTP handlers don't have to be synchronous. By implementing the IHttpAsyncHandler interface, which itself derives from IHttpHandler, an HTTP handler can be asynchronous. When used correctly, an asynchronous handler utilizes ASP.NET threads more efficiently. This is done in the same manner as an asynchronous page. In fact, asynchronous pages leverage the asynchronous handler support that predated asynchronous pages in ASP.NET.
Figure 5 contains an asynchronous version of the handler shown in Figure 3. Async-TerraServiceImageGrabber is slightly more complex, but it is also far more scalable.
Asynchronous processing begins when ASP.NET calls the handler's BeginProcessRequest method. BeginProcessRequest makes an async call to TerraService via the TerraService proxy's BeginConvertPlaceToLonLatPt method. The thread assigned to the request then goes back into the thread pool. When the async call completes, another thread is borrowed from the thread pool to execute the ConvertPlaceToLonLatCompleted method. That thread retrieves the results of the last call, makes an async call of its own, and then goes back into the thread pool. This pattern repeats until all the asynchronous calls have completed, at which time the handler's EndProcessRequest method is called and the resulting Bitmap is returned to the requestor.
To forestall EndProcessRequest until the final Web service call has completed, AsyncTerraServiceImageGrabber returns its own implementation of IAsyncResult from BeginProcessRequest. If it returned the IAsyncResult returned by BeginConvertPlaceToLonLatPt instead, EndProcessRequest would be called (and the request terminated) the moment the first Web service call completed.
The class that implements IAsyncResult, TerraServiceAsyncResult, features a public CompleteCall method that can be called at any time to finish the request. Normally, AsyncTerraServiceImageGrabber calls CompleteCall only after the final Web service call has completed. However, if an exception is thrown in one of the methods that executes between BeginProcessRequest and EndProcessRequest, the handler caches the exception in a private field (_ex), calls CompleteCall to terminate the request, and then rethrows the exception from EndProcessRequest. Otherwise, the exception would be lost and the request would never complete.
AsyncTerraServiceImageGrabber is more scalable than its synchronous counterpart since it consumes ASP.NET threads for only a fraction of the total time required to process a request. The vast majority of the time, it's simply waiting for an asynchronous Web service call to complete.
In theory, AsyncTerraServiceImageGrabber can also outperform TerraServiceImageGrabber because, rather than making repeated calls to TerraService's GetTile method in series, it makes the calls in parallel. In reality, however, only two outbound calls targeting a given IP address can be pending at a time unless you increase the runtime's default maxconnection setting:
<system.net>
  <connectionManagement>
    <add address="*" maxconnection="20" />
  </connectionManagement>
</system.net>
Other configuration settings can also affect concurrency. For more information, refer to the Knowledge Base article "Contention, Poor Performance, and Deadlocks when You Make Web Service Requests from ASP.NET Applications" (support.microsoft.com/kb/821268).
Even if only one callout executes at a time, AsyncTerraServiceImageGrabber should perform no worse than TerraServiceImageGrabber. And its design is far superior because it uses ASP.NET threads as efficiently as possible.

Asynchronous HTTP Modules
The third asynchronous programming model you can leverage in ASP.NET is the asynchronous HTTP module. An HTTP module is an object that sits in the ASP.NET pipeline, where it can see-and even modify-incoming requests and outgoing responses. Many of the key services in ASP.NET are implemented in the form of HTTP modules, including authentication, authorization, and output caching. You can extend ASP.NET by writing custom HTTP modules and plugging them into the pipeline. And when you do, you should carefully consider whether those HTTP modules should be asynchronous.
Figure 6 contains the source code for a simple, synchronous HTTP module called RequestLogModule, which logs incoming requests in a text file named RequestLog.txt. It creates the file in the site's App_Data directory so users can't browse to it. (Note that the security principal that ASP.NET runs as-ASPNET or Network Service, for example-must have write permission to App_Data for this to work.) The module implements the IHttpModule interface, which is the one and only requirement for an HTTP module. When the module loads, its Init method registers a handler for HttpApplication.PreRequestHandlerExecute events, which fire from the pipeline in each and every request. The event handler opens RequestLog.txt (or creates it if it doesn't already exist) and writes into it a line of text containing pertinent information about the current request, including the time and date that the request arrived, the requestor's user name if the request is authenticated (or the requestor's IP address if authentication is turned off), and the requested URL. The module is registered in the <httpModules> section of web.config, prompting ASP.NET to load it each time the application starts up.
The problem with RequestLogModule is twofold. First, it performs file I/O in each and every request. Second, it uses a request-processing thread to perform the I/O-a thread that could otherwise be used to service additional incoming requests. This module, simple as it is, imposes a penalty on throughput. While you could mitigate the delay by batching the file I/O operations, a better approach is to make the module asynchronous (or better yet, batch the file I/O and make the module asynchronous).
Figure 7 shows the asynchronous version of RequestLogModule. Called AsyncRequestLogModule, it performs exactly the same work, but returns the thread assigned to the request to the thread pool before writing to the file. When the write completes, a new thread is borrowed from the thread pool and used to finish the request.
What makes AsyncRequestLogModule asynchronous? Its Init method calls HttpApplication.AddOnPreRequestHandlerExecuteAsync in order to register begin and end methods for PreRequestHandlerExecute events. The HttpApplication class features other AddOn methods for other per-request events. For example, an HTTP module can call AddOnBeginRequestAsync to register async handlers for BeginRequest events. AsyncRequestLogModule's BeginPreRequestHandlerExecute method uses the framework's FileStream.BeginWrite method to begin an asynchronous write. The moment BeginPreRequestHandlerExecute returns, the thread goes back into the thread pool.
AsyncRequestLogModule contains some thread synchronization logic that merits special mention. It's possible-likely even-that multiple requests running on multiple threads will write to the log file at the same time. To ensure that concurrent writes don't overwrite each other, AsyncRequestLogModule stores the position in the file for the next write in a private field shared by all module instances (_position). Before each call to BeginWrite, the module reads the position from the field and updates the field to point to the first byte following what's about to be written to the file. Logic that reads and updates _position is wrapped in a lock statement so that no more than one thread can execute it at a time. That prevents one thread from reading the position before another gets the chance to update it.
Now the bad news. For BeginWrite not to use another thread from the pool, the isAsync parameter to FileStream's constructor must be set to true, as I've done in my example. However, a little-known consequence of using FileStream.BeginWrite to initiate an asynchronous write is that there's no guarantee the write will actually be asynchronous, even if you've requested asynchronous operation. Windows® reserves the right to perform asynchronous file I/O synchronously if it believes that synchronous I/O will be faster. For more information, read the Knowledge Base article at support.microsoft.com/kb/156932. The good news is that if Windows writes to the request log synchronously, the writes are, in theory, performed so quickly that they have minimal impact on the scalability of the host application.

Conclusion
Asynchronous programming is an excellent way to build more scalable applications by using the ASP.NET thread pool as efficiently as possible. In my travels, I rarely see ASP.NET developers utilize asynchronous programming models, in part because they simply don't know that these models exist. Don't let sparse documentation stand in your way; start thinking asynchronously today, and you'll be building better applications tomorrow.
Be aware that the downloadable sample code accompanying this article comes in both C# and Visual Basic® versions. I frequently get e-mail asking for Visual Basic versions of the samples. This time, you don't have to ask; they're already there!

Send your questions and comments for Jeff to  wicked@microsoft.com.


Jeff Prosise is a contributing editor to MSDN Magazine and the author of several books, including Programming Microsoft .NET (Microsoft Press, 2002). He's also a cofounder of Wintellect, a software consulting and education firm that specializes in .NET.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.
Page view tracker