2

My website constantly has several users online all the time. The server uses Apache/PHP, database and Memcached. On normal use, the application works well and fast.

However, the server appears to sometimes have "SYN flooding" attacks. I really believe/suspect these are intentional and not caused by our legit users. (as they appear to happen when there are some individuals that register new accounts and try to cause trouble)

Jun 27 22:12:21 xxxx kernel: [xxxx.xxxx] possible SYN flooding on port 443. Sending cookies. Jun 27 22:13:22 xxxx kernel: [xxxx.xxxx] possible SYN flooding on port 443. Sending cookies. Jun 27 22:14:25 xxxx kernel: [xxxx.xxxx] possible SYN flooding on port 443. Sending cookies.

Unfortunately, when that happens, my whole traffic is affected:

[Mon Jun 27 22:15:28.842067 2016] [mpm_event:error] [pid 12022:tid 132875292207712] AH00485: scoreboard is full, not at MaxRequestWorkers

enter image description here

My MaxRequestWorkers is 600. I have increased it a few times in the past.
Recently I also have increased net.ipv4.tcp_max_syn_backlog and ListenBackLog to 5000.
My server has 16GB RAM and 1 Gbps bandwidth.

I am not really happy that there appears to be someone that can easily control whether my website is alive or not.
What can one do to stop this?

Also, netstat seems to give me the IPs connected to the server right now.
Is it possible to get the top IPs at a given time in the past?

Nuno
  • 461
  • 1
  • 5
  • 23

1 Answers1

2
  1. Syn flood is kind of attack that is near to impossible to protect on single host.

Check SynCookies

cat /proc/sys/net/ipv4/tcp_syncookies

and enable (set to 1) it if disabled. It help legit users keep working.

Also You can try set lover /proc/sys/net/ipv4/tcp_synack_retries

https://stackoverflow.com/a/26674591/205355

  1. About IP

Synflood usually use spoofed random source IPs, so it can't be filtered based on source IP.

  1. As long as Your service is public, anyone can easily check it liveliness

  2. You can buy professional DDOS protection service. It use huge regional desributed cluster, geolocation aware DNS, corelation, ISP cooperation and more. It redirect clean user traffic to Your service on IP unknown to public (and hackers). But it can cost a lot, and can depends on power needed to survive and mitigate attack.

Community
  • 1
mmv-ru
  • 682
  • 6
  • 17
  • Just idea, make log of Syn in firewall and parse that log by awk or by script in preferred language, to count syn from same IP. Because of reason 3, more interesting to analyze web server log and try to detect IP of who check liviness of your site. It must be repeating one simple query. But it can be distributed on many IPs. If You found that IPs (its real, not a fake) You can try imitate attack success for that IPs. But this is challenge of shield and sword. Not a ultimate solution. – mmv-ru Jun 29 '16 at 10:16
  • From other side, You can try guess who benefits from Your site down? Possible it unfair competition? – mmv-ru Jun 29 '16 at 10:26