How much free memory should I have on my webserver?

Question

I have a webserver that's currently hosting two Wordpress sites and some java-based collaboration software. The server has 2G of memory and is currently using about 1.8G of the available memory. Right now what's on here is pretty much a pilot project that's getting negligible traffic so I think it's pretty clear that I'll be needing more memory.

I was wondering, if I was to release it, how I might anticipate my memory needs based on the traffic it gets. I've poked around on Google and what I've found has been a bit tenuous. Is there a good heuristic that one should use when calculating memory demands as a function of the base (no traffic) load on the server?

For reference, the output of free -m can be seen below:

             total       used       free     shared    buffers     cached
Mem:          2048       1832        215          0          0          0
-/+ buffers/cache:       1832        215
Swap:            0          0          0

To me this looks like actual memory used and isn't an illusion due to caching or anything else.

I figure the demands of my collaboration software will have to be experimentally tested so here's free -m without that software running:

             total       used       free     shared    buffers     cached                                     
Mem:          2048       1109        938          0          0          0                                     
-/+ buffers/cache:       1109        938                                                                      
Swap:            0          0          0   

My plan B to figure this out is to add a bunch of swap space to the server, give it some traffic and adjust according the the amount that swap gets used. I was just wondering if anyone had a good rule of thumb to estimate how much memory I should plan on in advance...or if what I'm thinking is nuts.

Many thanks in advance (I'm really quite new to this).

Answer 1

The answer is basically: It depends. In your specific case, the quality & efficiency of the sites you have installed can come into question.

For example, this week I was working on a server that is relatively moderate traffic, yet eating up 4GB of RAM. After doing a code review—the whole site is using an off-the-shelf CMS similar to WordPress but with customizations—we discovered a major bottleneck. Once we recoded the issue away, the server now uses about 2GB of RAM on an average day. A 50% drop in RAM usage!

I would not recommend doing something like split the server to use Ngnix as well as Apache. Just use Apache. The headaches of balancing the configs of two different servers is not worth the marginal benefit. Apache is fine. But that said, you should consider your WordPress PHP sites versus Java app versus MySQL in on the same box. In my experience, you should do the following:

1. Properly configure Apache: Apache is a good piece of software, but right out of the box it’s a memory hog. For example, I believe the default is to allow 255 connections per second? I can assure you most simple sites barely get 40 connections per second on a good day. So adjusting Apache to be realistic to your traffic will help. Also, there is a KeepAlive setting in Apache that works great! But out of the box, I believe it’s set to a MaxKeepAliveRequests of 100 which is fairly nuts. I usually set this to about 30 connections with a small KeepAliveTimeout of 2 to 3 seconds. The key is to have the KeepAliveTimeout to match speed it takes for an average page to download with a little bit of room for overhead/slowness. So if a page loads in 1 second, do a KeepAliveTimeout of 2 seconds.
2. Review the code for your WordPress sites for potential bottlenecks: Concentrate on the PHP core of it & clear up what you can. Look out for excessive MySQL calls & file system calls. This is where you will be able to make the app fly! Also, check the memory_limit in your php.ini and make sure it’s not higher than necessary. The default is 64M, but in many cases that can be lowered to 32M.
3. MySQL tuning or moving it onto it’s own server: After writing about MySQL above I realized you might be hosting your MySQL instance on the same box. Look into optimizing MySQL performance by running a script like MySQL tuning primer. Without tuning, MySQL will eat up all resources & big the system down. With tuning, MySQL will run better/faster & resources can be freed for other purposes. Also, consider moving your MySQL DB to a standalone server. You might have to learn how to properly network & firewall the server to allow your servers access but protect against hackers, but the performance benefit will be great.
4. Consider moving the Java app to another server: My rule of thumb is one major Java application per server. In general, Java apps can be memory hogs when compared to PHP setups like WordPress. By giving the Java app it’s own server to use, the WordPress sites will be much happier.

Regarding the MySQL tuning, that is something that can take a few weeks to nail down at the beginning. The reason being tuning scripts are based on real traffic MySQL sees. So you basically make your site live to the world, wait 2 days (at least), run the tuning scripts & then wait a few more days to tune some more. After a week or so you should be able to tune MySQL to work as well as it can with your setup.