1

I'm very confused on techniques for scaling servers

Say you had one high traffic server running on one computer with one 12 core CPU, and one server for a database.

For a while that would work, but what about if the number of concurrent users becomes very high? How would one scale that? There is only so much that you can buff up one server to be able to handle.

I've searched on the internet for a while but couldn't really find an answer that outlines how one would do this. For instance, how does facebook handle so many users? If anyone has any answers or can point me to any resources I'd greatly appreciate it

Jamal H
  • 119
  • 4
  • *More* servers? Facebook, Google, etc. run hundreds of *thousands* of servers. – ceejayoz Jul 14 '17 at 16:58
  • how do they route users/communicate within those servers? – Jamal H Jul 14 '17 at 17:00
  • Googling "facebook scaling" will give you thousands of articles. Replication, sharding, caching, etc. etc. etc. Typically, these are techniques you tackle *when* you reach that sort of scale - be wary of premature optimization. – ceejayoz Jul 14 '17 at 17:03
  • 1
    Possible duplicate of [Can you help me with my capacity planning?](https://serverfault.com/questions/384686/can-you-help-me-with-my-capacity-planning) – mfinni Jul 14 '17 at 17:49

1 Answers1

5

you are correct there is only so much Memory and only so many processors you can through at a server. Not to mention the fact that if you only have a single server running your application you have all your eggs in one basket. What if your single server fails?

So we scale out, we add multiple servers running the same application to add scalability and high availability as well as load balancing.

We can scale out both the application tier and the database tier.

Another technique you can use is off loading. For example you can introduce a Content Delivery Network (CDN) in to your design. With A CDN we can have content cached on edge node around the globe. That way the users of our application can access content locally instead of having to connect to a set of servers that might be hundreds of thousands of miles away to access you application. a consequence of this is that you need less servers because most requests for content will be serviced by the CDN meaning less load on the servers themselves.

another way to offload is to add a caching tire between your Application servers and the database server. Using REDIS or MEMCACHED in memory caches the commonly requested queries can be cached on the REDIS / MEMCACHED cluster instead of being retrieved from the database servers. this speeds up access to the database and again will have the knock on effect that we will need smaller and fewer database servers.

Google, Facebook and all of the large internet applications use techniques and more.

CDNs https://en.wikipedia.org/wiki/Content_delivery_network

In Memory Caching https://docs.jivesoftware.com/jive/6.0/community_admin/index.jsp?topic=/com.jivesoftware.help.sbs.online_6.0/admin/CachingOverview.html

Scaling gracefully http://philip.greenspun.com/seia/scaling

Hope this helps,

Mike.

Michael Brown
  • 3,204
  • 2
  • 9
  • 10