Instantly Increase ASP.NET Scalability

Each time a request for a .NET resource (.aspx, page, .ascx control, etc.) comes in a thread is grabbed from the available worker thread pool in the asp.net worker process (aspnet_wp.exe on IIS 5.x, w3wp.exe on IIS 6/7) and is assigned to a request. That thread is not released back into the thread pool until the final page has been rendered to the client and the request is complete.


Inside the ASP.Net Worker Process there are two thread pools. The worker thread pool handles all incoming requests and the I/O Thread pool handles the I/O (accessing the file system, web services and databases, etc.). But how many threads are there in these thread pools? I had assumed that the number of threads would vary from machine to machine – that ASP.NET and IIS would carefully balance the number of available threads against available hardware, but that is simply not the case. ASP.Net installs with a fixed, default number of threads to play with: The CLR for the 1.x Framework sets these defaults at just 20 worker threads and 20 I/O thread per CPU. Now this can be increased by modifying the machine.config, but if you are not aware of this, then 20 threads is all you’re playing with. If you have multiple sites sharing the same worker process, then they are all sharing this same thread pool.

So long as the number of concurrent requests does not exceed the number of threads available in the pool, all is well. But when you are building enterprise level applications the thread pool can become depleted under heavy load, and remember by default heavy load is more than just 20 simultaneous requests. When this happens, new requests are entered into the request queue (and the users making the requests watch an hour glass spin). ASP.NET will allow the request queue to grow only so big before it starts to reject requests at which point it starts returning Error 503, Service Unavailable.

If you are not aware of this “Glass Ceiling of Scalability”, this is a perplexing error – one that never happened in testing and may not reproducible in your test environment, as it only happens under extreme load.
So the first thing you can do to improve scalability is to raise the values. The defaults for the ASP.NET 2.0 are 100 threads in each pool per CPU and the defaults for the ASP.NET 3.x CLR is 250 per CPU for worker threads and 1000 per CPU for I/O threads, however you can tune it further using the guidelines below. 32 bit windows can handle about 1400 concurrent threads, 64 bit windows can handle more, though I don’t have the figures.

You can tune ASP.NET thread pool using maxWorkerThreads, minWorkerThreads, maxIoThreads, minFreeThreads, minLocalRequestFreeThreads and maxconnection attributes in your Machine.config file. Here are the default settings.

<system.net>

  <connectionManagement>

     <add address="*" maxconnection="2" />

  </connectionManagement>

</system.net>

<system.web>

  <httpRuntime minFreeThreads="8" minLocalRequestFreeThreads="4"  />

  <processModel maxWorkerThreads="100" maxIoThreads="100"  />

</system.web>

Here is the formula to reduce contention. Apply the recommended changes that are described below, across the settings and not in isolation.

    Configuration setting Default value (.NET Framework 1.1) Recommended value
    maxconnection 2 12 * #CPUs
    maxIoThreads 20 100
    maxWorkerThreads 20 100
    minFreeThreads 8 88 * #CPUs
    minLocalRequestFreeThreads 4 76 * #CPUs

    Set maxconnection to 12 * # of CPUs. This setting controls the maximum number of outgoing HTTP connections that you can initiate from a client. In this case, ASP.NET is the client. Set maxconnection to 12 * # of CPUs.
    Set maxIoThreads to 100. This setting controls the maximum number of I/O threads in the .NET thread pool. This number is automatically multiplied by the number of available CPUs. Set maxloThreads to 100.
    Set maxWorkerThreads to 100. This setting controls the maximum number of worker threads in the thread pool. This number is then automatically multiplied by the number of available CPUs. * Set maxWorkerThreads to 100.
    Set minFreeThreads to 88 * # of CPUs. This setting is used by the worker process to queue all the incoming requests if the number of available threads in the thread pool falls below the value for this setting. This setting effectively limits the number of requests that can run concurrently to maxWorkerThreads – minFreeThreads. Set minFreeThreads to 88 * # of CPUs. This limits the number of concurrent requests to 12 (assuming maxWorkerThreads is 100).
    Set minLocalRequestFreeThreads to 76 * # of CPUs. This setting is used by the worker process to queue requests from localhost (where a Web application sends requests to a local Web service) if the number of available threads in the thread pool falls below this number. This setting is similar to minFreeThreads but it only applies to localhost requests from the local computer. Set minLocalRequestFreeThreads to 76 * # of CPUs.

Note The recommendations that are provided in this section are not rules. They are a starting point. Test to determine the appropriate settings for your scenario. If you move your application to a new computer, ensure that you recalculate and reconfigure the settings based on the number of CPUs in the new computer.

By raising these values, you raise the “glass ceiling of scalability”, and in many cases that may be all you need to do, but what happens when you start getting more than 250 simultaneous requests? To make optimum use of the thread pool all IO requests that you know could take a second or more to process should be made asynchronously. More information on Asynchronous programming in ASP.NET is coming soon.

I recommend testing your new machine.config locally or virtually somewhere first because if you make a mistake - for example you paste this in to the wrong area, or there is already a <system.web> element and you paste in a second one - ALL websites on the box will stop functioning as ASP.Net will not be able to parse the machine.config!!!