Studies have shown that the average internet user will abandon a page
that takes more than four seconds to load.
If you are using ASP.NET, and you aren’t using asynchronous programming,
then you may have a problem.
Success may literally kill your site.
I discovered this problem a couple of years ago with a web app I was load
Quick summary of what I found:
Two copies of same app were running on the same server. I was load testing one.
As I ramped up beyond about 300-350 concurrent users bad things happened.
The time it took to return a page skyrocketed up to 30 seconds and beyond!
CPU and Memory on the server were barely being used.
I logged in to the second app and guess what? It was responding as quickly as
As I researched this issue I came across an MSDN article from 2007 by Jeff Prosise
Scalable Apps with Asynchronous Programming in ASP.NET.
Jeff is one of those “Card-Carrying Genius” types. He has written probably more
books than I’ve read and was one of the founders of Wintellect.
I’ve known several folks who worked for Wintellect, and they have all been damned sharp.
Here’s the opening of his article:
Do you want to know a secret? A deep, dark, dirty secret? One that, if revealed, would cause great angst in the ASP.NET community and prompt shouts of “Aha!” from the anti-Microsoft crowd?
Most Web sites I’ve seen built with ASP.NET aren’t very scalable, not because of a flaw in ASP.NET, but because of how the technology is used. They suffer a self-imposed glass ceiling that limits the number of requests they can process per second. These sites scale just fine until traffic rises to the level of this invisible ceiling. Then throughput begins to degrade. Soon after, requests start to fail, usually returning “Server unavailable” errors.
– Jeff Prosise
In a word: Threads. In two words: Thread Pool. Three words? Thread Pool Saturation.
Each person who connects to your app gets a thread from the .Net CLR Thread Pool.
As long as you don’t have more people than threads, everything’s great!
Unfortunately once you get more people than threads, new users go into a queue to wait for the
next available thread.
Browsers waiting for connections lead to frustrated users leaving the page.
The real fun happens when they leave, and then a thread is assigned to them and waits for them to respond.
The thread doesn’t know they left, so it waits until it reaches its timeout period. That means even though
they left, they still ate up a thread.
Going back to my story of the two identical apps, the reason the one not being load tested kept working
was because it had its own thread pool. The thread pool is at the CLR instance level. In ASP terms that means
Application or App Pool level.
Bad. Really bad.
I’ve created a super simple MVC app to illustrate.
To keep it simple I just did ye olde File: New Project and accepted the default MVC app.
The Home controller has three actions:
These do nothing but return their view.
In the real world, they would probably look something up in the database, so I put in a random delay to
So if you hit one of the pages, the app waits between .5 and 1.5 second before responding.
Here’s a bit of the code:
Next I used the Load Testing built into Visual Studio Ultimate to see how it performed.
I started with 200 users and added 200 every 10 seconds until I reached 2,000.
I held at 2,000 and ran for a total of five minutes.
Here is a chart of the results:
|Blue||Avg. Page Time||0.41||40.4||20.7|
The average page load time was 20.7 seconds.
That’s FIVE TIMES longer than the average person will wait!
If you’ll remember from Jeff Prosise’s article, the answer comes down to asynchronous
If you read his whole article, you probably came away with the impression that Jeff
is very smart and asynchronous programming is very hard.
In 2007 that was true (the hard part is less true now, the “he’s smart’ part is still true).
Fortunately .Net 4.5 gave us
Don’t let anyone kid you, asynchronous coding will make your head hurt.
You really don’t know the order in which things will happen and have to
take great care with variables.
I’ve worked a bit with Node.js, and it starts with the idea that everything should
One of the big differences between Node and ASP is that Node makes you declare synchronous code
whereas .Net makes you declare asynchronous code.
Having said that I find the nesting of callbacks in Node to border on spaghettification at times.
Until .Net 4.5 using asynchronous code in .Net was even more cumbersome.
await you can write code that looks like synchronous code, but
behind the scenes compiles to an asynchronous construct.
Let’s take a look at what changes need to be made to make the controller code I showed before
asynchronous, then see the results of running the same load test.
As you can see I added the
async keyword on the method and the
inside the method. I also made the method’s return type a
Task of type
instead of an
Also, I obviously couldn’t tell the Thread to sleep, so I swapped out
The result is that behind the scenes a callback will be added.
Before I was explicitly saying “Wait here and let this thread spin for n-milliseconds, then
Now I am saying, “Set a callback here, release the thread back into the thread pool, and
when n-milliseconds have passed, give me a thread and continue.”
Here is a chart of the results of running the asynchronous version:
|Blue||Avg. Page Time||0.83||17.8||2.94|
I think you’ll agree that’s quite an improvement.
Yes we had a few that took too long, but the average time to load dropped by an
order of magnitude.
Our proverbial average user wouldn’t abandon our site just because of the time to load a
As you can see there are a couple of gaps in the chart.
Unfortunately my notebook tends to have I/O log jams and the test was a bit too much for it.
I could move the test up to Visual Studio Online, but it would exceed my monthly allotment
of testing “virtual user minutes.” Since these results demonstrate the point I’m making, I
don’t think I’ll buy extra minutes to perfect the test.
My first test was with 10,000 users and my notebook shut itself down
because the CPU was overheating!
Fortunately 2,000 users sufficiently demonstrates the point.
You knew there had to be a catch, right? It couldn’t be this easy.
No, it’s not quite this easy.
In the real world you will be getting stuff from databases, file systems,
and web APIs.
If any of those do something synchronously, you will lose the ability to relinquish
the thread back to the pool.
Entity Framework has been an offender in this area, but from EF6 onwards there are
asynchronous methods you can use instead of synchronous ones.
I haven’t worked with asynchronous EF yet, so I can’t say much about it.
If you are creating anything other than a departmental app, you should be coding
Sure you can throw more hardware at the problem, but what a waste!
You could spin up the biggest, baddest, fire-breathing monster of a server and
you would still be limited by the number of threads in the thread pool.
You can increase the number of connections in the thread pool (and you very well
might anyway), or spread the load across multiple web servers (again, you will probably do so),
but the biggest bang for the buck comes with asynchronous code.
Hopefully I’ve piqued your interest in asynchronous ASP. Hopefully you’ll starting
doing it ASAP.
To help you learn more check out these links:
Scalable Apps with Asynchronous Programming in ASP.NET. Prosise’s article. Defines the problem, but his solutions are circa 2007.
Asynchronous Programming with Async and Await Good
intro to asynchronous coding with .Net.
Nodejs.org It wouldn’t hurt to spend some time with Node to see how an
asynchronous or die philosophy works. If nothing else it will make you appreciate how much
less complicated .Net now makes asynchronous code.
Intro To Node With Microsoft Tools Presentation
A presentation I did about using Node.js with Microsoft tools. I talked about this issue as well.
Scaling a standard Azure website to 380k queries per minute of 163M records with loader.io Excellent post by Troy Hunt.
Tasks, IDisposables and async/await Great post from Mark Rendle about not botching your code by trying to pass along an already disposed task. He makes a great point about the incremental time added by async/await and when it is worth the wait. Or await.