When is the right time to start adding(or thinking about adding) servers to your Web Application? What are the difficulties involved in going from a single server(DB and Web) to multiple?
For example:
Most of the time you start with one sever that you use for both DB and Web then you split you DB/Web onto a different server then go to multiple web servers(which creates session issues) then possibly a NAS for the DB etc etc.
When is the right time to start adding(or thinking about adding) servers to your Web Application? ...(going from a single server ... to multiple)
Think about it: From the beginning.
Start adding: When you get your first "server too busy" error. Anything sooner is premature optimization.
(Unless your web app is mission critical, in which case you probably aren't starting from scratch and don't need to poll the serverfault.com community.)
But seriously, for modern consumer web applications, getting "server too busy" can actually be a good thing. It certainly never hurt Facebook, Twitter, or YouTube. The danger with adding servers too early is that your app will never be as popular as you are expecting and you've ended up wasting money that could have been spent on feature development.
If you're one of the lucky few who actually has a hit web property, then (a) congratulations, and (b) you'll be able to measure your average response times from your log files and can take a more metric-driven approach to adding servers.
Personally, I always start with a separate web server from the DB server.
(based on IIS6 and SQL Server)
However, we do ALL this on a vmware server (3 node cluster in fact), and the webserver and the db server only have 1 vCPU and 1GB RAM. And they support a fairly busy suite of websites (external and intranet) without too much sweat.
But I know if I get a surge of popularity (and it might happen one day!) I can quickly reconfigure the vm's with more RAM and CPU.
On a practical note: Monitor your performance over time. Track changes and be proactive.
When is the right time to start adding(or thinking about adding) servers to your Web Application?
When you are designing your application. I've seen too many applications not designed to use multiple servers, and reverse engineering that later can be horrendous.
Make sure you test on multiple servers, too. Again. I've seen many applications work fine in dev/test only to fail in production because they couldn't deal with load balancers, or firewalls, multicasting hadn't been considered etc, etc.
And the time to add another server is when your capacity management statistics suggest that you will run out of capacity in just over the time it will take you to add another server.
You are collecting cpacity statistics? No? Then another thing to talk about with your application developers and infrastructure management people.
I disagree that waiting until you actually get 'server too busy' and irritating your users is the correct thing to do. Adding a new server to a production environment can be a lengthy process, and waiting until you have errors before you start is not really wise.
As others have mentioned, the question is vague, but there's really a pretty simple answer to it:
You need to add another server when limitations on the current one start costing you money.
If you're offing a free service, then there's really no incentive to add additional servers.
If you're making money off your service, then you can do a cost/benefit analysis: how will additional servers improve performance, and how much more profit will result?
Without any further information on what your application is, it's hard to give any specific advice, but here's some slides from a 2005 presentation by Brad Fitzpatrick on LiveJournal's infrastructure (with a heavy emphasis on database load balancing):
One of the things you should do is load test your existing one-server application to see how many sessions and users it can support at once. Once you know that, you can monitor those same metrics over time. When your numbers begin to approach the known limits, it's time to start preparing new hardware for deployment. You will need to consider the lead time for additional servers so that you can deploy before the existing system reaches its limit. This is a very general overview, and the details are covered very well in the book "The Art of Capacity Planning" by John Allspaw.
Having said that, I would break the database server and web server into their own systems as a starting point, then load test each to determine the theoretical load. Usually the web server will become the bottleneck first, so your application should be written with scalability in mind. The question mentions sessions, and depending on the platform you're using there are several ways to share sessions between servers, store them in the central database, or use load balancing hardware that will direct the same user to the same server unless it becomes unavailable for some reason.
But to answer the core question "when do we know when to do it," that will be all about the metrics you're collecting and comparing them against known limits arrived at through load testing.
In general, going from one to multiple front-ends is trickier than separating the front-end from the (usually database) back-end. However, careful use of load-balancers and "sticky" sessions can make this transition easier.
As for "when", the key to finding that is to measure the performance of your web (and database) server(s). Ideally in a test bench with realistic traffic-load, then see where the performance dips into "not good enough". Once the load on the live site starts getting close to the "not good enough", you order a new front-end.
If you know roughly when you'll be needing a new server and roughly how traffic volume grows over time, you can aim at having the new server deployed just before it's needed. Unfortunately, I can't give you any hard figures, as these things are very dependent on the specifics of your set-up.
