Server being overloaded by flood attack

Question

I have a strong server, with 64GB Ram and amazing network and SSD for fast connections, but someone, I don't know why, is attacking my website, it seems that it is a script requesting my index.php page very often causing my MySQL unstable, even my server using a good hardware configuration. Check my logs file:

[Fri Nov 06 13:02:47.169346 2020] [proxy_fcgi:error] [pid 30232:tid 140367357691648] [client 164.68.100.114:35336] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.169560 2020] [proxy_fcgi:error] [pid 29914:tid 140366971922176] [client 164.68.100.114:33102] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.170482 2020] [proxy_fcgi:error] [pid 29692:tid 140364547610368] [client 164.68.100.114:2658] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.168929 2020] [proxy_fcgi:error] [pid 30229:tid 140366980314880] [client 164.68.100.114:40178] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.169336 2020] [proxy_fcgi:error] [pid 30229:tid 140366946744064] [client 164.68.100.114:2841] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.186088 2020] [proxy_fcgi:error] [pid 30229:tid 140367483582208] [client 164.68.100.114:39838] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.186853 2020] [proxy_fcgi:error] [pid 30229:tid 140367089354496] [client 164.68.100.114:2591] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.187609 2020] [proxy_fcgi:error] [pid 30229:tid 140367097747200] [client 164.68.100.114:2609] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.188884 2020] [proxy_fcgi:error] [pid 30229:tid 140366837704448] [client 164.68.100.114:3360] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190061 2020] [proxy_fcgi:error] [pid 30229:tid 140366946744064] [client 164.68.100.114:2841] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190768 2020] [proxy_fcgi:error] [pid 30230:tid 140367475189504] [client 164.68.100.114:3624] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190043 2020] [proxy_fcgi:error] [pid 30230:tid 140366611199744] [client 164.68.100.114:40394] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190882 2020] [proxy_fcgi:error] [pid 30230:tid 140366468523776] [client 164.68.100.114:40390] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190880 2020] [proxy_fcgi:error] [pid 30229:tid 140367114532608] [client 164.68.100.114:3085] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190894 2020] [proxy_fcgi:error] [pid 29692:tid 140364556003072] [client 164.68.100.114:33944] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190051 2020] [proxy_fcgi:error] [pid 30230:tid 140366544058112] [client 164.68.100.114:40392] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190914 2020] [proxy_fcgi:error] [pid 30229:tid 140365092873984] [client 164.68.100.114:33844] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.188394 2020] [proxy_fcgi:error] [pid 30230:tid 140366493701888] [client 164.68.100.114:40396] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.191808 2020] [proxy_fcgi:error] [pid 30229:tid 140365076088576] [client 164.68.100.114:40402] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190599 2020] [proxy_fcgi:error] [pid 30230:tid 140366560843520] [client 164.68.100.114:40386] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.190959 2020] [proxy_fcgi:error] [pid 30230:tid 140366577628928] [client 164.68.100.114:40388] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.192573 2020] [proxy_fcgi:error] [pid 30229:tid 140367466796800] [client 164.68.100.114:60224] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'
[Fri Nov 06 13:02:47.193573 2020] [proxy_fcgi:error] [pid 30229:tid 140366846097152] [client 164.68.100.114:60192] AH01071: Got error 'PHP message: PHP Warning:  mysqli::__construct(): (08004/1040): Too many connections in /var/www/master/database.php on line 19\n'

How I supposed to block this? Analyzing the logs deeply, all requests are made on HEAD method. If I block HEAD request for my website, I will have any problem or issue with Google bots, for example? I dont know if this is a solution, seems to be a trick. Also, I am using Cloudflare and this attacks persists. What should I do?

Thank you.

There are low-level tools for warding off such a "DOS attack". Search for such. — Rick James, Nov 06 '20 at 19:08
As far I know, apache by default contains "low-level" security to avoid this, but this is not working. — Kaique Vortex, Nov 06 '20 at 19:10
There isn't such "low-level security" to avoid this, as this is generally a valid use case that requests arrive in rapid fashion. — Tero Kilkanen, Nov 06 '20 at 22:56
Since these attacks last few minutes, my current `max_connections` on MySQL is 200, what about I update it to 30k? I have 64GB Ram DDR4 and Intel® Core™ i7-6700. Do you have any idea or experience if my server will handle it? — Kaique Vortex, Nov 07 '20 at 06:16
Take a look at mod_evasive for Apache: https://github.com/jzdziarski/mod_evasive - and be sure to triple check its configuration in your case. — Marc Stürmer, Nov 09 '20 at 09:56

score 1 · Accepted Answer · answered Nov 06 '20 at 21:20

1

Fail2ban might be a good option to watch the apache access LOG for unusual amounts of HEAD requests in a given period of time and create a temporary block in iptables firewall.

Jay Ta'ala has an article on his website that provides information on how to configure fail2ban for watching apache log files.

answered Nov 06 '20 at 21:20

Roderick Bant

149
3

Since these attacks last few minutes, my current `max_connections` on MySQL is 200, what about I update it to 30k? I have 64GB Ram DDR4 and Intel® Core™ i7-6700. Do you have any idea or experience if my server will handle it? – Kaique Vortex Nov 07 '20 at 06:16
Hard to say what the effect of raising the `max_connections` for MySQL wil be on the database server. But personally I would think twice about letting these attacks penetrate your infrastructure further than the webserver. Ideally I would try to block the attacks at the outer rings of your defenses as much as possible – Roderick Bant Nov 07 '20 at 09:50
I dont know if will be easy block these attacks, I am using Cloudflare already, and this requests are simple requests, if I try to block HEAD method, it will impact on my platform performance – Kaique Vortex Nov 07 '20 at 20:18
The idea is that you would use fail2ban not for simply blocking all HEAD requests, but only to block them when they exceed a certain amount of requests per minute or second for the IP making the high amount of requests – Roderick Bant Nov 08 '20 at 07:25
Are you sure that fail2ban control certain requests per minute? I take a look on the article you sent and searched on Google, and I could not find anything related to this – Kaique Vortex Nov 08 '20 at 07:50
Yes, you can set a config value for `findtime` and `maxretry` in your `jail.conf` file. Those will specify the period of time (findtime) in which to look for the maxretry amount of request before activating the block rule in iptables for the IP making the requests. After doing a bit more research I found [another article](https://unihost.com/help/how-to-protect-a-server-with-fail2ban/) which may be a bit more to the point for your situation. Do however make sure you replace the `GET|POST` bits with `HEAD` – Roderick Bant Nov 08 '20 at 08:20

Server being overloaded by flood attack

1 Answers1