At Your Service: Performance Considerations for Making Web Service Calls from ASPX Pages

Matt Powell
Microsoft Corporation

July 22, 2003

Summary: Matt Powell shows you how to eliminate performance problems and the devouring of thread-pool resources with an asynchronous approach to Web service calls that uses Microsoft ASP.NET. (12 printed pages)

Download the associated sample code for this column.

Some interesting dynamics around creation of content for the Web is happening here at Microsoft. As you might imagine, many people from different groups across Microsoft are needed to create the volumes of technical information that you see on Microsoft.com. Lately, the people creating the content are re-focusing their content coverage, such as those in product documentation groups, Microsoft Product Support Services (which generates most of the KnowledgeBase), MSDN, and various others. PSS originally identified this common problem—performance issues when making Web service calls from ASP.NET Web pages—and we decided that the problem required more coverage than a KnowledgeBase article, but still fell outside the scope of product documentation.

However, it is a great subject for an "At Your Service" column.

The Scenario: Performance Devastation When Calling a Web Service from an ASP.NET Page

When we discuss Web services in this column, we expect that Web services will be consumed in a wide variety of scenarios. A primary scenario is Web services being accessed from a middle-tier environment like ASP.NET Web pages. The folks that support users of the MapPoint .NET Web service have had a recurring problem with people using their Web service. A call to MapPoint .NET may result in a relatively lengthy Web service call. By itself this isn't a problem, but there are some other factors that can make this a much larger problem than it appears.

HTTP Two-Connection Limit

The HTTP specification indicates that an HTTP client should make a maximum of two simultaneous TCP connections to any single server. This keeps a single browser from overloading a server with connection requests when it browses to a page with, say, 120 embedded thumbnail images. Instead of creating 120 TCP connections and sending an HTTP request on each, the browser will only create 2 connections and then start sending the 120 HTTP requests for the thumbnail images on those two pipes. The problem with this approach on the middle tier is that your middle tier may have 50 simultaneous users. If you have to make a MapPoint .NET Web service call for each of those users, then you will have 48 users sitting around waiting in line for one of those two pipes to free up.

Thread Pool Limits

ASP.NET handles incoming requests by servicing them with a pool of threads called the process thread pool. In normal circumstances a request comes in and a thread from the pool that is idle will pop in to service the incoming request. The "problem" is that the process thread pool will not create an infinite number of threads to handle a large number of requests. Having a maximum number of threads is a good thing, since if we created threads infinitely we would use all the machine's resources just to manage the threads. By limiting the number of threads that will be created, we keep the overhead of thread management to a manageable level. When a request comes in and all the threads in the thread pool are being used, then the request is simply queued until one of the busy threads completes and the freed thread can then handle the new request. This approach actually is more efficient than switching to a new thread because no thread swapping is required between requests. The problem is that the queue of waiting requests can grow significantly if threads are not being used efficiently, particularly on a fairly busy Web server.

Consider the scenario where you are making a Web service call from your ASP.NET page. If you make a synchronous call, the thread that you are running on will be blocked until that Web service call is completed. During the call, the thread cannot do anything else. It cannot handle another request despite the fact that it is doing nothing but waiting. If you have the default number of worker threads on a single processor machine (20), then it only requires 20 simultaneous requests until your threads are all used and you must queue further requests.

The Problem Is Not Restricted to Web Services

Making lengthy blocking calls from a Web page is not a problem seen only by people calling Web services. You would encounter the same problem if you made any number of lengthy calls: SQL Server™ requests, lengthy file reads or writes, Web requests of any sort, or accessing a concurrent resource where locking can cause significant delays. In fact, there are many Web service scenarios in which calling the service is quick enough and this is not a problem. However, you can probably understand that if you are calling the MapPoint .NET Web service potentially through proxies, over connections with a certain amount of latency, against a service that may take a bit of time to handle the request, you can see where delays could happen and problems could arise if sites are busy.

Tuning Around the Problem

Some aspects of this problem can be improved simply by making certain configuration settings to the environment. Let's take a look at some of the configuration settings that are available to tune around this problem.

maxconnections

The default two-connection limit for connecting to a Web resource can be controlled via a configuration element called connectionManagement. The connectionManagement setting allows you to add the names of sites where you want a connection limit that is different than the default. The following can be added to a typical Web.config file to increase the default value for all servers you are connecting, to a connection limit of 40.

<configuration>
  <system.net>
    <connectionManagement>
      <add address="*" maxconnection="40" />
    </connectionManagement>
  </system.net>

  <system.web>
     ...

It should be noted that there is never a limit to the number of connections to your local machine, so if you are connecting to localhost, this setting has no effect.

maxWorkerThreads and minFreeThreads

If you receive HTTP 503 errors ("The service is temporarily overloaded"), then you are running out of threads in your thread pool and your request queue has been maxed out (the default setting for appRequestQueueLimit is 100). It is possible to simply increase the thread pool size on IIS 5.0 installations. For IIS 6.0 installations (that are not running in IIS 5.0 compatibility mode), these settings have no effect.

maxWorkerThreads and maxIoThreads control the number of worker threads and threads handling new request submissions for ASP.NET. These settings need to be made in your Machine.config and will affect all Web applications running on your machine. maxWorkerThreads is part of the processModel element in Machine.config and if you look, you will notice that the default value for this setting is 20 threads per processor.

The minFreeThreads setting can be made in Machine.config or in your application's Web.config file under the httpRuntime element. What this setting does is refrains from using a thread from the thread pool to handle an incoming HTTP request if the number of free threads is below the set limit. This can be useful if you need a process thread pool thread to finish a pending request. If all the threads are being used to handle incoming HTTP requests, and the requests are waiting for another thread to complete processing, then you can get into a deadlock situation. An example of this would be if you are making asynchronous Web service calls to a Web service from your ASP.NET application and are waiting for the callback function to be called to complete the request. The callback will have to be made on a free thread in the process thread pool. If you look at your Machine.config, you will notice that the minFreeThreads setting defaults to 8, which is probably satisfactory if your worker thread pool has a limit of 20, but may be too small if you increase your pool size to 100.

It should be noted that thread pool limitations are exacerbated if your ASP.NET application is making Web service calls to the local machine. For instance, the test application that I created for this column calls a Web service on the same machine as the ASPX pages. Therefore, for blocking calls, a thread is being used for the ASPX page as well as for the ASMX Web service request. This effectively doubles the number of simultaneous requests our Web server is handling. In the scenario where we are making two simultaneous Web service requests (using asynchronous Web service calls), we end up tripling the number of simultaneous requests. To avoid these types of problems when calling back to the local machine, you should consider architecting your application to simply execute the code in the Web method directly from your ASPX code.

Windows XP Limitations

We should be sure to note that if you are doing some testing on a Windows® XP machine, another restriction you are running under is the artificial limitation on the number of simultaneous connections that the XP Web server is allowed to have. Because Windows XP is not a server platform, the number of simultaneous connections is limited to 10. This is usually fine for testing in a development environment, but can be severely limiting if you attempt any sort of stress testing. Connections from the local machine do not count against this limit.

The Real Solution: Asynchronous Request Handling

It is one thing to tune configuration settings around a problem and another to actually design your Web application in such a way that the problem is no longer an issue. Threads waiting for blocking calls to complete will never scale well, so the solution is to avoid the whole blocking issue altogether. The proper solution is to handle the request asynchronously. This manifests itself in two areas: making asynchronous Web service calls, and handling requests asynchronously in your ASP.NET Web application.

The First Piece in the Asynchronous Solution: Asynchronous Web Service Calls

In a previous column, I wrote about calling Web services asynchronously. Being able to free threads from waiting for Web service calls to complete is a key part of creating an asynchronous page handling model that frees up threads so more requests can be handled. And calling Web services asynchronously is fairly easy.

Consider the following Visual Basic® .NET code for an ASPX page:

' Excruciatingly Bad Performance Page based off of Synchronous
' Web service calls!
Public Class SyncPage
    Inherits System.Web.UI.Page

    Protected WithEvents Label1 As System.Web.UI.WebControls.Label
    Protected WithEvents Label2 As System.Web.UI.WebControls.Label

    Private Sub Page_Load(ByVal sender As System.Object, _
            ByVal e As System.EventArgs) Handles MyBase.Load
        'Call the web service
        Dim proxy As New localhost.Service1
        Label1.Text = proxy.Method1(500)
        Label2.Text = proxy.Method1(200)
    End Sub

End Class

This code is pretty straightforward. When the page loads, an instance of a Web service proxy is created and is used to make two calls to a Web method called Method1. Method1 simply returns a string that includes the input parameter passed to the method. In order to add a level of latency to this system, Method1 also sleeps for 3 seconds before it returns the string. The returned strings from the calls to Method1 are placed in the text for two labels on our ASPX page. This page provides very poor performance and sucks up threads from the process thread pool like a sponge. Due to the 3-second delay in the Method1 Web method, a single call to this page takes at least 6 seconds to complete.

The following snippet shows the code for a similar Web page, except now the calls to the Web service are being made asynchronously.

Public Class AsyncPage
    Inherits System.Web.UI.Page

    Protected WithEvents Label1 As System.Web.UI.WebControls.Label
    Protected WithEvents Label2 As System.Web.UI.WebControls.Label

    Private Sub Page_Load(ByVal sender As System.Object, _
            ByVal e As System.EventArgs) Handles MyBase.Load
        'Call the web service
        Dim proxy As New localhost.Service1
        Dim res As IAsyncResult 
            = proxy.BeginMethod1(500, Nothing, Nothing)
        Dim res2 As IAsyncResult 
            = proxy.BeginMethod1(200, Nothing, Nothing)
        Label1.Text = proxy.EndMethod1(res)
        Label2.Text = proxy.EndMethod1(res2)
    End Sub

End Class

Again, this page creates a Web service proxy and then proceeds to make two calls to the Method1 Web method. The difference is that we are calling BeginMethod1 instead of just calling Method1 directly. The BeginMethod1 call returns immediately so that we can start the second call to the method. Instead of waiting for the first Web service call to complete like we did in the first example, we are now starting both calls simultaneously. The calls to EndMethod1 simply block until the specific call completes.

It is important to note that once we return from our ASPX page the response is sent to the client. Therefore we cannot return from our Page_Load method until we have the data we need. That is why we must block for the Web service calls to complete. The good news is that now both calls are executing at the same time so the previous 6-second delay is now down to approximately 3 seconds. This is better, but it still creates blocking threads. What we really need is the ability to free up the thread handling the HTTP request while the Web service calls are completing. The problem is that the processing model for ASPX pages does not have an asynchronous execution paradigm. However, ASP.NET does provide a solution to this problem.

The Second Piece of the Asynchronous Solution: Asynchronous PreRequestHandler Execution

ASP.NET has support for something called HttpHandlers. HttpHandlers are classes that implement the IHttpHandler interface and are used to service HTTP requests for files with certain extensions. For instance if you look at the Machine.config, you will notice that there are a number of HttpHandlers that service requests for files with extensions like .asmx, .aspx, .ashx, and even .config. ASP.NET looks in the configuration information for a request for a file with a certain extension and then calls the HttpHandler associated with it to service the request.

ASP.NET also has support for writing event handlers that can be notified at various times during the processing of an Http Request. One such event is the PreRequestHandlerExecute event that occurs just before the HttpHandler for a particular request is called. There is also asynchronous support for PreRequestHandlerExecute notifications that can be registered for using the AddOnPreRequestHandlerExecuteAsync method of the HttpApplication class. The HttpApplication class is the class that you derive from if you have event handlers that you created behind the Global.asax file. We are going to use the asynchronous PreRequestHandler option to provide an asynchronous execution mode for making our Web service calls.

The first you thing you have to do before calling AddOnPreRequestHandlerExecuteAsync is create a BeginEventHandler and an EndEventHandler function. When a request comes in, the BeginEventHandler function will be called. This is when we will start our asynchronous Web service calls. The BeginEventHandler must return an IAsyncResult interface. If you are making a single Web service call, you can simply return the IAsyncResult interface that is returned from your Web service begin function (in our example the BeginMethod1 method returns an IAsyncResult interface). In the example I created, I wanted to perform the same operations that we made in our previous Web page examples where I illustrated synchronous and asynchronous Web service calls. This means I had to create my own IAsyncResult interface. My BeginEventHandler code is shown below:

Public Function BeginPreRequestHandlerExecute(
        ByVal sender As Object, _
        ByVal e As EventArgs, _
        ByVal cb As AsyncCallback, _
        ByVal extraData As Object) As IAsyncResult
    If Request.Url.AbsolutePath _
        = "/WebApp/PreRequestHandlerPage.aspx" Then
        Dim proxy As MyProxy = New MyProxy
        proxy.Res = New MyAsyncResult
        proxy.Res.result1 
            = proxy.BeginMethod1( _
                500, _
                New AsyncCallback(AddressOf MyCallback), _
                proxy)
        proxy.Res.result2 
            = proxy.BeginMethod1( _
                300, _
                New AsyncCallback(AddressOf MyCallback), _
                proxy)
        proxy.Res.Callback = cb
        proxy.Res.State = extraData
        proxy.Res.Proxy = proxy
        Return proxy.Res
    End If
    Return New MyAsyncResult
End Function

There are a couple other interesting things to note about this code. First of all, it is being called for every HTTP request being handled by this virtual directory. Therefore, the first thing I do is check the actual path of the request and see if it is for the page that I am servicing or not.

My function is called with a couple interesting input parameters. The cb parameter is the callback function passed to me by ASP.NET. ASP.NET expects that when my asynchronous work has completed, the callback function it supplied me with will be called. That is ultimately how they will know when to call my EndEventHandler. Again, if I was only making one Web service call, I could simply pass the callback to the BeginMethod1 call and the Web service call would take care of calling the function, but in this case I am making two separate calls. Therefore, I created an intermediary callback function that I pass to the two BeginMethod1 calls, and within my callback code I check to see if both calls have completed. If not, I return; but if so, I invoke the original callback. The other interesting parameter is the extraData parameter that holds the state for ASP.NET when it calls me. I must return the state information when I invoke the callback function indicated by the cb parameter so I store it in the IAsyncResult class I created. My callback code is listed below.

Public Sub MyCallback(ByVal ar As IAsyncResult)
    Dim proxy As MyProxy = ar.AsyncState
    If proxy.Res.IsCompleted Then
        proxy.Res.Callback.Invoke(proxy.Res)
    End If
End Sub

I should also mention that the class I created that implements IAsyncResult (called MyAsyncResult) is written in such a way that it checks both pending Web service call completions when the IsCompleted property is queried.

In my EndEventHandler, I simply get the results from my Web service calls and store them in the current request context. The context is the same context that will be passed to the HttpHandler. In this case, it will be the handler for .aspx requests so it will be available in my normal code. My EndEventHandler code is shown below.

Public Sub EndPreRequestHandlerExecute(ByVal ar As IAsyncResult)
    If Request.Url.AbsolutePath _
        = "/WebApp/PreRequestHandlerPage.aspx" Then
        Dim res As MyAsyncResult = ar
        Dim proxy As MyProxy = res.Proxy
        Dim retString As String
        retString = proxy.EndMethod1(proxy.Res.result1)
        Context.Items.Add("WebServiceResult1", retString)
        retString = proxy.EndMethod1(proxy.Res.result2)
        Context.Items.Add("WebServiceResult2", retString)
    End If
End Sub

Because the data for my .aspx page has already been received, the processing for the actual page is quite trivial.

Public Class PreRequestHandlerPage
    Inherits System.Web.UI.Page

    Protected WithEvents Label1 As System.Web.UI.WebControls.Label
    Protected WithEvents Label2 As System.Web.UI.WebControls.Label

    Private Sub Page_Load(ByVal sender As System.Object, _
        ByVal e As System.EventArgs) Handles MyBase.Load

        Label1.Text = Context.Items("WebServiceResult1")
        Label2.Text = Context.Items("WebServiceResult2")
    End Sub
End Class

It Isn't Just Theory—It Actually Works!

It makes sense that I'm wasting fewer resources if I'm not just blocking all my threads, but do the actual results really show a difference? The answer is a resounding "YES!" I put together the three test scenarios described in this column: making 2 blocking calls from Web page code, making 2 asynchronous calls from Web page code, and making 2 asynchronous calls from PreRequestHandler code. I stressed the three scenarios using Microsoft Application Center Test that sent continuous requests from 100 virtual clients for a period of 60 seconds. The results shown in the graph below indicate the number of requests completed during that period.

Figure 1. Requests completed in 60 seconds with 100 simultaneous clients

The asynchronous PreRequestHandler approach handles almost 8 times as many requests as the next fastest approach! So this approach can allow you to handle more requests, but how long is it actually taking for the individual requests to complete? The average response times for the three approaches are shown below.

Figure 2. Average completed response times with 100 simultaneous clients

The average response time for requests using the PreRequestHandler approach was only 3.2 seconds. Given that there was a built-in delay of 3 seconds for each Web service call, this is a highly efficient approach to the problem.

I would be remiss if I didn't mention that these unscientific numbers were based on my unscientific machine running in my very unscientific office. Of course, it makes sense that performance would improve if you free up threads that were not doing any work and had them actually do some work. Hopefully these results show that the improvements can be significant.

The PreRequestHandler approach is necessary because there is no asynchronous request handling mechanism built into the handler for .aspx requests. This is not true for all ASP.NET HTTP handlers. The .asmx request handler for Web services has an asynchronous model that I described in a previous column. The PreRequestHandler approach will work for all types of ASP.NET requests, but it is a bit easier to program with the asynchronous support built into the .asmx handler than it is to program with the PreRequestHandler.

Conclusion

Whenever you are executing any sort of lengthy process where performance is an issue, it is always a good practice to look into an asynchronous execution model. In the case of calling a Web service from an .aspx page, we determined that we could combine asynchronous Web service calls with an asynchronous execution paradigm provided by ASP.NET. This solves the problem of the lack of asynchronous support in the handling of .aspx requests. Performance problems and the devouring of thread-pool resources were eliminated with this asynchronous approach.

At Your Service

Matt Powell is the lead Content Strategist for the MSDN XML Web Services Developer Center. Besides writing numerous articles for MSDN and various magazines, and co-authoring Running Microsoft Internet Information Server from Microsoft Press, Matt also helped develop the groundbreaking SOAP Toolkit 1.0. When he is not delving into the depths of Web services, Matt helps his wife chase their kids around the greater Seattle area.