0

I have a dedicated server with 32GB RAM. It works all fine till some punters came to ddos my server. I had really high uplink bandwidth usage (over 150Mbps). I looked at netstat with the following command:

netstat -an | grep tcp | awk '{print $5}' | cut -f 1 -d : | sort | uniq -c | sort -n

It shows thousands of connections for multiple IPs: I used following iptable rules to limit connections per IP:

iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 15 --connlimit-mask 32 -j REJECT --reject-with tcp-reset 

iptables -A INPUT -m state --state RELATED,ESTABLISHED -m limit --limit 150/second --limit-burst 160 -j ACCEPT

I saved these rules and these rules are top in the list in iptables. I restarted the system to close all the connects and restored the iptable rules. Put the server online. Server was almost dead and there were thousands of connections again. I have UFW installed and only couple of ports are open.

What can I do to stop it? How can kill established connections per ip? BTW. all the ips are masked and fake ones.

Edit:

netstat -ant | awk '{print $6}' | sort | uniq -c | sort -n
      1 CLOSING
      1 established)
      1 Foreign
      2 SYN_SENT
      7 FIN_WAIT2
     16 LISTEN
     21 CLOSE_WAIT
     48 LAST_ACK
    209 SYN_RECV
    284 FIN_WAIT1
    772 ESTABLISHED
  35426 TIME_WAIT

output of iptraf -d eth0

IPTraf
l Statistics for eth0 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x                                                                                                                                                                      x
x               Total      Total    Incoming   Incoming    Outgoing   Outgoing                                                                                         x
x             Packets      Bytes     Packets      Bytes     Packets      Bytes                                                                                         x
x Total:       332826    128344K      192313   14538688      140513    113805K                                                                                         x
x IP:          332826    123518K      192313   11679988      140513    111838K                                                                                         x
x TCP:         332692    123507K      192311   11679924      140381    111827K                                                                                         x
x UDP:              0          0           0          0           0          0                                                                                         x
x ICMP:           134      11448           2         64         132      11384                                                                                         x
x Other IP:         0          0           0          0           0          0                                                                                         x
x Non-IP:           0          0           0          0           0          0                                                                                         x
x                                                                                                                                                                      x
x                                                                                                                                                                      x
x Total rates:      31150.2 kbits/sec        Broadcast packets:            0                                                                                           x
x                   25144.8 packets/sec      Broadcast bytes:              0                                                                                           x
x                                                                                                                                                                      x
x Incoming rates:    8907.0 kbits/sec                                                                                                                                  x
x                   14266.2 packets/sec                                                                                                                                x
x                                            IP checksum errors:           0                                                                                           x
x Outgoing rates:   22243.2 kbits/sec                                                                                                                                  x
x                   10878.6 packets/sec
user3404047
  • 1
  • 1
  • 2

1 Answers1

1

Actually, you are asking how to protect from DDOS attack. It's really depends on amount of traffic/packets to your system. I believe it's TCP syn flood in your case.

You need to check amount of pps and traffic with iptraf(iptraf -d eth0 for example).

Also you should look at sysctl: 

net.ipv4.tcp_syncookies
net.ipv4.tcp_synack_retries

And enable SYNPROXY in your iptables for attack duration: http://rhelblog.redhat.com/tag/synproxy/

However positive result depends on how strong DDOS attack is.

You could try to use some anti-ddos attack services, however i have no experince using them.

Navern
  • 1,569
  • 1
  • 9
  • 14
  • Thanks! I am going to try all your suggestions and would update you in 10 minutes. This attack is going on. My hosting is ddos protected but this attack is by-passing their mitigation system. – user3404047 Aug 15 '15 at 21:34
  • Look at net.ipv4.tcp_fin_timeout if you have too much connections in WAIT_TIMEOUT state. – Navern Aug 15 '15 at 21:38
  • I have turned on syncookies but it didn't help. Where can I see ".tcp_fin_timeout" and "net.ipv4.tcp_synack_retries"? Why the above iptable rules did not work to limit connections per ip? – user3404047 Aug 15 '15 at 21:45
  • Let me know please result of iptraf -d eth0 (or whatever interface you are using). You can use sysctl -a | grep, to check current value of this values. – Navern Aug 15 '15 at 21:51
  • Sorry for a lame question. This command open a bluescreen with some values. Do you want me to send you the print-screen? or what exact values you want me to copy you? Thanks! – user3404047 Aug 15 '15 at 21:53
  • There is total packets per second and traffic per second. It shows how strong attack is. Also you can for example change IP of your service and block all connections to old IP address. – Navern Aug 15 '15 at 22:08
  • I will copy you the values as the attack comes back again. Could you please (meanwhile) point out the problem in iptable rules? Why they are not limiting connections per IP? – user3404047 Aug 15 '15 at 22:13
  • It's hard to tell based on current output. You can run iptables -n -v -L --list-numbers and check how much of packets are going under your rule. Maybe limit is too high, maybe there is too much different IP addresses. Actually mitigating ddos attack with iptables only is pretty hard thing to do and you should use tcpdump and your brains:) – Navern Aug 15 '15 at 22:26
  • I update the question with the iptraf -d eth0 output. Please have a look. some values are increasing so fast. – user3404047 Aug 15 '15 at 22:37
  • 1
    If your getting DOS'ed i would highly recommend looking at Cloudflare. Its been a godsend for me. Not a perfect solution, but definitely moved things in the right direction. – Thomas Vincent Aug 15 '15 at 22:52