understand correctly pm.max_children tuning

Question

I have done some investigations and I have found this to calculate and adjust pm.max_children value

https://myshell.co.uk/blog/2012/07/adjusting-child-processes-for-php-fpm-nginx/

but for example :

• I have 8Gb on my server
• I'm hosting 30 websites
• php-fpm average process size around 40mb
• php-fpm max process size around 80mb
• I want to allocate 5Gb max of my memory to php-fpm processes

If I apply this :

pm.max_children = Total RAM dedicated to the web server / Max child process size

So in my case :

pm.max_children = 5120 / 80 = 64

But If I add pm.max_children = 64 on each php-fpm website conf files, this means every website can use 64 children process X size of 1 process (ex 40mb) = 2560Mb

And If we imagine, at the same time, all the 30 website have reach the pm.max_children value, we will have : 2560Mb (max per website) x 30 websites = 76 800 Mb

Am I correct?

Si yes, this means when many websites are hosted on the same server, we have to divide the result of the calcul pm.max_children = 5120 / 80 = 64 by the number of websites hosted (here 30).

So 64 / 30 = 2,1 and pm.max_children = 2 per website

Is it correct or not?

Thanks

Answer 1

Your calculation is correct from what I gather.

Having many webs on the same server works only as long as not all webs use all available resources at the same time. This is what people usually call overprovisioning.

However, I suggest not to simply calculate pm.max_children around the available RAM, but around how many workers are actually necessary for the webs to function properly. Start with something lower and monitor the the php-fpm.log. If the max_children setting is reached, you will find it in the log and you can increase it.

Also, make sure that the PHP workers only live as long as necessary. For example, the following configuration will let a pool use up to 32 PHP workers if there is a burst of requests, but each worker will exit after 3 seconds of inactivity and free valuable RAM:

pm = ondemand
pm.max_children = 32
pm.process_idle_timeout = 3s

Use the ondemand process manager if you are low on RAM. It is a bit slower than the dynamic pm, but doesn't waste RAM for inactive websites.

If you want to control the total number of PHP processes, there is a setting called process.max in php-fpm.conf. I have never used it, but it seems to me you could use it to ensure that there are never more than a certain number of workers, regardless of how the pools are configured.

By the way, it is a very good idea to use separate pools for separate webs belonging to separate users. That way you won't have any problems with user permissions or with data cached from other webs.

Answer 2

I can only give you an advise based on our experiences.

We have only one PHP-FPM pool running to share resources (CPU and RAM).

Multiple pools allow different user accounts (e.g. www-data1, www-data2 ...) and may help to restrict access. Additionally you can assign different values for CPU and RAM consumption if required.

However following example uses only one pool:

; www.conf
;
; set pool management to have a fixed number of php workers
pm = static
; number of php processes (6 processes per CPU core)
pm.max_children = 48

I recommend using static pool management. This means there are always a fixed number of PHP workers.

; www.conf
;
; redirect worker stdout and stderr into main error log
catch_workers_output = yes

This might be helpful if your application runs into errors.

; php-fpm.conf
;
emergency_restart_threshold = 10
emergency_restart_interval = 1m
process_control_timeout = 10s

This is some basic life monitoring of your running PHP workers.

Do not forget to restart your PHP-FPM service after these changes.