Apache on Debian : server flooded by a lot of 400 , how to protect from it?

Question

My HTTPS server has been experiencing slowness for a few days, so I consulted the log file (the access.log, I use apache2). And I found out that my server is flooded by a lots of 400 :

If I change the apache config for stop listening the port 443, the flood of queries stop (but my website becomes inaccessible x) )

I tried to add some iptables rules. I try theses rules by example :

/sbin/iptables -N SYN_FLOOD
/sbin/iptables -A INPUT -p tcp --syn -j SYN_FLOOD
/sbin/iptables -A SYN_FLOOD -m limit --limit 10/s --limit-burst 10 -j RETURN
/sbin/iptables -A SYN_FLOOD -j DROP

(source)

When I add the last rule (-A SYN_FLOOD -j DROP), the flood stopped (great !) but my website become again inaccessible :/

My knowledges about server administration and the iptable command are very limited... why my website become inaccessible after this rules ? what should I modify ? maybe the use of iptable its not a good approach for my issue ?

Thank for any help/explanation :)

EDIT (after petitradisgris answer):

I installed and config fail2ban this night. I add theses jails :

> cat /etc/fail2ban/jail.local

# detect password authentication failures
[apache]
enabled  = true
port     = http,https
filter   = apache-auth
logpath  = /var/log/apache*/*error.log
maxretry = 3
findtime = 600
bantime = 86400

# detect potential search for exploits and php vulnerabilities
[apache-noscript]
enabled  = true
port     = http,https
filter   = apache-noscript
logpath  = /var/log/apache*/*error.log
maxretry = 3
findtime = 600
bantime = 86400

# detect Apache overflow attempts
[apache-overflows]
enabled  = true
port     = http,https
filter   = apache-overflows
logpath  = /var/log/apache*/*error.log
maxretry = 2
findtime = 600
bantime = 86400

# detect failures to find a home directory on a server
[apache-nohome]
enabled  = true
port     = http,https
filter   = apache-nohome
logpath  = /var/log/apache*/*error.log
maxretry = 2
findtime = 600
bantime = 86400

[apache-fakegooglebot]
enabled  = true
port     = http,https
logpath  = %(apache_access_log)s
maxretry = 1
ignorecommand = %(ignorecommands_dir)s/apache-fakegooglebot <ip>

[apache-badbots]
# Ban hosts which agent identifies spammer robots crawling the web
# for email addresses. The mail outputs are buffered.
enabled  = true
port     = http,https
logpath  = %(apache_access_log)s
bantime  = 48h
maxretry = 1

##To stop DOS attack from remote host.
[http-get-dos] 
enabled = true 
port = http,https 
filter = http-get-dos 
logpath = /var/log/apache*/access.log 
maxretry = 400 
findtime = 400 
bantime = 200 
ignoreip = 127.0.0.1
action = iptables[name=HTTP, port=http, protocol=tcp]

[iptables-dropped]

enabled = true
filter = iptables-dropped
banaction = iptables-allports
port = all
logpath = /var/log/messages
bantime = 1800
maxretry = 3

The config of my jails seems ok :

> sudo fail2ban-client status
Status
|- Number of jail:  9
`- Jail list:   apache, apache-badbots, apache-fakegooglebot, apache-nohome, apache-noscript, apache-overflows, http-get-dos, iptables-dropped, sshd

This morning, fail2ban ban IPs but only in sshd jail (strange...) :

> sudo cat /var/log/fail2ban.log* | grep Ban

2022-06-13 07:07:59,185 fail2ban.actions        [529]: NOTICE  [sshd] Ban 23.94.194.115
2022-06-13 07:08:32,529 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.156.122.114
2022-06-13 07:09:10,601 fail2ban.actions        [529]: NOTICE  [sshd] Ban 182.156.209.222
2022-06-13 07:09:18,831 fail2ban.actions        [529]: NOTICE  [sshd] Ban 186.10.125.209
2022-06-13 07:09:30,867 fail2ban.actions        [529]: NOTICE  [sshd] Ban 139.59.21.115
2022-06-13 07:11:15,736 fail2ban.actions        [529]: NOTICE  [sshd] Ban 213.136.90.174
2022-06-13 07:11:45,799 fail2ban.actions        [529]: NOTICE  [sshd] Ban 154.194.12.69
2022-06-13 07:14:50,174 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.156.124.5
2022-06-13 07:18:09,188 fail2ban.actions        [529]: NOTICE  [sshd] Ban 118.27.106.123
2022-06-13 07:18:55,509 fail2ban.actions        [529]: NOTICE  [sshd] Ban 104.248.89.194
2022-06-13 07:19:05,742 fail2ban.actions        [529]: NOTICE  [sshd] Ban 23.94.194.115
2022-06-13 07:19:11,811 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.154.104.24
2022-06-13 07:19:45,166 fail2ban.actions        [529]: NOTICE  [sshd] Ban 178.35.169.154
2022-06-13 07:20:36,453 fail2ban.actions        [529]: NOTICE  [sshd] Ban 139.59.21.115
2022-06-13 07:20:55,694 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.156.122.114
2022-06-13 07:21:33,806 fail2ban.actions        [529]: NOTICE  [sshd] Ban 186.10.125.209
2022-06-13 07:22:10,109 fail2ban.actions        [529]: NOTICE  [sshd] Ban 27.74.254.115
2022-06-13 07:22:56,385 fail2ban.actions        [529]: NOTICE  [sshd] Ban 213.136.90.174
2022-06-13 07:24:10,698 fail2ban.actions        [529]: NOTICE  [sshd] Ban 154.194.12.69
2022-06-13 07:25:34,062 fail2ban.actions        [529]: NOTICE  [sshd] Ban 104.248.62.102
2022-06-13 07:26:21,350 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.154.171.84
2022-06-13 07:29:57,842 fail2ban.actions        [529]: NOTICE  [sshd] Ban 118.27.106.123
2022-06-13 07:31:46,106 fail2ban.actions        [529]: NOTICE  [sshd] Ban 139.59.21.115
2022-06-13 07:33:49,576 fail2ban.actions        [529]: NOTICE  [sshd] Ban 27.74.254.115
2022-06-13 07:33:50,797 fail2ban.actions        [529]: NOTICE  [sshd] Ban 186.10.125.209
2022-06-13 07:34:22,899 fail2ban.actions        [529]: NOTICE  [sshd] Ban 213.136.90.174
2022-06-13 07:37:13,410 fail2ban.actions        [529]: NOTICE  [sshd] Ban 104.248.62.102
2022-06-13 07:38:07,498 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.154.171.84
2022-06-13 07:38:16,138 fail2ban.actions        [529]: NOTICE  [sshd] Ban 177.229.215.234
2022-06-13 07:40:44,582 fail2ban.actions        [529]: NOTICE  [sshd] Ban 104.248.89.194
2022-06-13 07:45:30,321 fail2ban.actions        [529]: NOTICE  [sshd] Ban 27.74.254.115
2022-06-13 07:46:31,019 fail2ban.actions        [529]: NOTICE  [sshd] Ban 186.10.125.209
2022-06-13 07:48:59,939 fail2ban.actions        [529]: NOTICE  [sshd] Ban 104.248.62.102
2022-06-13 07:49:51,227 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.154.171.84
2022-06-13 07:50:01,263 fail2ban.actions        [529]: NOTICE  [sshd] Ban 177.229.215.234
2022-06-13 07:50:59,390 fail2ban.actions        [529]: NOTICE  [sshd] Ban 46.19.137.50
2022-06-13 07:57:09,964 fail2ban.actions        [529]: NOTICE  [sshd] Ban 27.74.254.115
2022-06-13 08:01:50,508 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.154.171.84
2022-06-13 08:02:04,542 fail2ban.actions        [529]: NOTICE  [sshd] Ban 177.229.215.234
2022-06-13 08:02:39,206 fail2ban.actions        [529]: NOTICE  [sshd] Ban 104.248.89.194
2022-06-13 08:14:18,245 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.129.209.91
2022-06-13 08:16:07,008 fail2ban.actions        [529]: NOTICE  [sshd] Ban 167.99.158.168
2022-06-13 08:16:50,289 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.128.101.73
2022-06-13 08:24:37,536 fail2ban.actions        [529]: NOTICE  [sshd] Ban 104.248.89.194
2022-06-13 08:25:21,615 fail2ban.actions        [529]: NOTICE  [sshd] Ban 43.129.209.91

And the flood is always here...

sudo tail -f /var/log/apache2/site1_access.log /var/log/apache2/site2_access.log /var/log/apache2/site2_access.log

...
180.190.87.231 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
111.71.212.176 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
39.112.83.149 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
115.186.169.59 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
61.15.198.157 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
183.222.197.241 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
114.45.171.90 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
61.62.148.146 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
1.173.221.202 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
27.82.146.136 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
27.109.247.56 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
180.177.24.124 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
116.49.174.155 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
61.231.235.55 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
14.192.212.91 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
219.91.104.20 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
60.49.40.31 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
60.49.40.31 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
171.97.223.126 - - [13/Jun/2022:08:30:15 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
61.38.43.211 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
120.231.123.126 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
221.184.60.12 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
113.43.210.22 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
180.190.87.231 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
115.43.157.151 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
122.100.135.240 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
117.183.115.211 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
223.86.195.24 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
122.100.145.152 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
112.120.167.195 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
36.229.143.45 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
110.26.97.247 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
121.171.109.192 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
113.254.111.51 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
112.104.89.188 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
14.192.212.91 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
36.238.159.112 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
219.91.104.20 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
198.16.63.120 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
115.87.13.52 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
61.231.235.55 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
121.6.78.165 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
121.109.135.202 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
183.227.201.149 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
61.38.43.211 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
112.120.167.195 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
180.94.189.179 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
59.149.254.6 - - [13/Jun/2022:08:30:16 +0200] "GET / HTTP/1.0" 400 0 "-" "-"
...

I discover 'fail2ban', maybe there is a missing or mistake in ma config of jails ?

petitradisgris · Answer 1 · 2022-06-13T12:56:00.487

1

Try installing the 'fail2ban' software in your linux system: Tutorial in this link

The default confiiguration should help.

If not, you may easily write some specific rules that smartly block those traffic (banning IPs doing to much specific requests like that).

EDIT: here is an example of a jail that may fit your problem:

[ban-400]
logpath = /var/log/f2b-400.log
filter =
port = 80,443
failregex = ^<ADDR> \S+ \S+(?: \[\])? "[^"]*" 400\s
enabled = true

However:

the actual requests that gives a 400 error appears valid to me, so in my understanding, they should return a 200.
Thoses IP may comes from web scrapper (web browsers) and others things like that...So you don't want to ban an IP aiming at referencing you on a web browser...right?

So you should consider resolving this error 400 issue, where your apache2 should report a 200 status code. Did you disabled http1.0 in your config?

EDIT: regarding your iptable SYN_FLOOD.

The problem with it, is that a SYN request is part of a legitimate communication.

Moreover, as you are in TCP, a SYN request (and probably SYN/ACK) is sent many times to server until having response.

So, the server will count many SYN request per host (per second) and the iptables rules will block them...

edited Jun 13 '22 at 12:56

answered Jun 12 '22 at 23:19

petitradisgris

126
5

Hi and thanks you for you help ! I edit my initial post :) – spacecodeur Jun 13 '22 at 06:47
Updated my post too. – petitradisgris Jun 13 '22 at 08:24
http1.0 is not disable on ma apache config , may I disable it ? because when I go to my websites, the version used by HTTP is 1.1 (I see that in my access.log file). But if I disable all http v1.0 request, maybe I'll refuse legitimates queries ? (facebook bot, google bog, ...) – spacecodeur Jun 13 '22 at 08:51
1

When a client make a request in http1.0, your server will answer in http1.1 but in compatibility mode(so everything is fine). If you disable http1 0 for clients, this would lead to the error 400 answers totally similar of what you have. Don't disable it ;) – petitradisgris Jun 13 '22 at 12:21

Apache on Debian : server flooded by a lot of 400 , how to protect from it?

1 Answers1