25

I'm a newbie of server administration and I'm looking for a powerful hosting service to host my new website. This website is basically a back-end of an mobile online game, and it will:

  • handle up to 10 million of HTTPS request and mySQL queries a day
  • store up to 2000 GB file on the hard disk
  • transfer probably 5000 GB data in and out per month
  • it runs on PHP and mySQL
  • have 10 million records in mySQL database, for each record there are 5-10 fields, around 100 bytes each

I really don't know what kind of a server I need to handle these requirements, my question is:

  1. What CPU/RAM do I need for a dedicated server or VPS?
  2. What hosting companies are able to offer this kind of dedicated server or VPS?
  3. What about cloud computing? I've researched Amazon EC2 but it seems complicated to me. And I've contacted Rackspace but strangely they said Cloudsites is not suitable for my requirements. I wonder if there is other cloud hosting company.
  4. Any other alternative method?
sebix
  • 4,175
  • 2
  • 25
  • 45
Calvin
  • 361
  • 1
  • 3
  • 4
  • we got around this with 2 linux servers with 8 gigs of ram, the mysql is a mysql cluster and the DB is stored in memory fast fast, cpu is never much really if you use a good distro, and disk will only need to be used to take hourly snapshots gives you redundancy in case of failure. also you may wish to have mysqltuner installed so you can keep an eye on indexes etc and make the best usage of everything and also add lots of indexes and keep a log on slow queries, as for web this can be really cheap just add a load balancer on the front to split traffic – minus4 Mar 19 '10 at 08:58
  • Why not use a cloud service? Azure, Amazon, RackSpace, GoGrid, Heroku? – bbqchickenrobot Feb 20 '13 at 16:19

2 Answers2

33

A cheap desktop?

Let's get into the math.

  • 10 million requests.
  • That breaks down to 416667 requests per hour.
  • That breaks down to 6944 requests per minute.
  • That breaks down to 116 requests per second.

Double that (peak load) and we talk of a load a cheap quad core desktop can handle IF the queries are simple enough, and you don't really say how complex they are.

  • 5000 GB per month is trivial - seriously, same math applies.
  • That breaks down to 208GB / day
  • That breaks down to 8GB / hour
  • That breaks down to 148MB / minute

  • That breaks down to 2,5MB / second, 25Mbit. Double for peak - 50Mbit, trivial for any hosting center. Will cost you, though.

  • Store 2000 GB on the hard disc. That is 2x2000 GB hard discs in a RAID? Unless: it is for the database, an has lots of complex IO, then it is anything between some dozen discs and a LOT of 73GB 15.000RPM SAS discs in a RAID 10 (around 60 discs) to get the I/O needed - this question is not answerable without a LOT more info on data access patterns.

  • Runs PHP and MySQL - My mobile phone can do that ;) The question is how complex the application is. MySQL MAY or MAY NOT be an acceptable solution here, BTW l. - that would require more testing. There is a reason some people still use other larger commercial databases.

  • What CPU/Ram do I need for a Dedicated Server or VPS?

One would say that depends on logic (how much calculations in the PHP part, smartness or lack of programmers and a lot of other questions.

Seriously, this is a non-trivial setup. Get some specialists look into it.

Basically you need to get down and get your homework done. A lot of the questions are not answerable in this form. Especially because you don't seem to care about your data...

  • Backups?
  • No contingency plan? I mean, servers die - so you are OK with the site being down for days while replacement is configured?
Glorfindel
  • 1,213
  • 3
  • 15
  • 22
TomTom
  • 50,857
  • 7
  • 52
  • 134
  • thanks for your reply. the php is simple, I guess the main burden is on mySQL, I tested some mySQL queries on my laptop (Core2 Duo) with WAMP on Windows. with 10 mllions records in mySQL, on average each query costs 0.1 seconds. how much stronger with Quad Core be at handling mySQL queries? – Calvin Mar 19 '10 at 07:35
  • 2
    Forget the quad core. Your laptop SUCKS in IO - and IO is where databases aren lmimited. You have ONE hard disc, that is SLOW and ROBUST (latop). Servers use MULTIPLE hard discs that are FAST (but not robust). I use a quad core SQL Server fron MS and can handle more than 500 batches per second on simple selects (one batch being one select) without maxing out the CPU - but I do get a LOT of disc activity on a disc subsystme that is possibly more than 30 times as fast as yours (and that is not impressive yet). Discs are the limit. Plus proper programming. – TomTom Mar 19 '10 at 08:08
  • 1
    Your ssl traffic will need to be encrypted/decrypted, you may want to offload that on a balancer, and do a reverse proxy to an normal http server. This should keep latency down. you can also do you encryption in hardware too.......http://en.wikipedia.org/wiki/SSL_acceleration if budget is no concern for your database use http://www.ramsan.com/success/ccpgames.htm – The Unix Janitor Mar 19 '10 at 08:58
9

To add some of my experience which may be helpful:

  • As TomTom mentioned it is difficult/impossible to give exact specifications as a lot of it depends on the design and implementation of your application. Hardware that gives me or someone else X requests/sec may not work well for you.
  • I have a low end dedicated MySQL server (Intel Core2 Duo E4600 2.40 GHz, 4 GB RAM) serving an average of 100 requests/sec (close to 10 million/day) with a CPU idle rate of 90%. Other than some basic tweaks to the configuration it is running well due to being read heavy (+95% reads) and the active record set is easily contained in memory. Consider the size of your active set when choosing the amount of server RAM as it can make a big difference. Make sure you understand the difference between your database size and active record set size. For example, my databases total ~7GB but the active set is likely only a few 100MB.
  • Similarly, I have an Apache server of similar specs serving ~1 million requests per day which has an average ~95% CPU idle rate. Requests are a mix of very simple map data AJAX queries and more complex MediaWiki pages.
  • Benchmarking your specific application is a good start in trying to determine exactly what you need. You don't want to under estimate but over estimating can be just as bad due to the potential waste of money and effort.
  • Consider not only the average request rate but the peak rate. You don't want a server that just barely can handle the average rate as request rates can vary significantly over the day, week, and month. For example, I can get 3-4x the traffic during peak hours on weekends as I do in the minimum hours during the week. How much it varies will depend on your application and user base.
  • Can you cache any of your database/HTTP requests? This can drastically increase your request rate with cheaper/less hardware depending on how much you can cache.
  • Consider your scaling options for future growth now instead of later. A good option may be to use horizontal scaling which would let you start with minimal hardware and easily grow as needed.
  • Proper design of your application layer can have a huge effect on its ultimate performance. A bad SQL query on a table with no indexes can be orders of magnitude slower than a properly designed one. Similarly, poorly configured Apache/MySQL servers can be many of times slower than when correctly setup.
uesp
  • 3,384
  • 1
  • 17
  • 16