10

I've been reading about this everywhere all day, and from what I've gathered, TIME_WAIT is a relatively harmless state. It's supposed to be harmless even when there's too many.

But if they're jumping to the numbers I've been seeing for the past 24 hours, something is really wrong!

[root@1 ~]# netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n
  1 established)
  1 Foreign
 12 CLOSE_WAIT
 15 LISTEN
 64 LAST_ACK
201 FIN_WAIT2
334 CLOSING
605 ESTABLISHED
816 SYN_RECV
981 FIN_WAIT1
26830 TIME_WAIT

That number fluctuates from 20,000 to 30,000+ (so far, the maximum I've seen it go is 32,000). What worries me is that they're all different IP addresses from all sorts of random locations.

Now this is supposed to be (or was supposed to be) a DDoS attack. I know this for a fact, but I won't go into the boring details. It started out as a DDoS and it did impact my server's performance for a couple minutes. After that, everything was back to normal. My server load is normal. My internet traffic is normal. No server resource is being abused. My sites load fine.

I also have IPTABLES disabled. There's an odd issue with that too. Every time I enable the firewall/iptables, my server starts experiencing packet loss. Lots of it. About 50%-60% packets are lost. It happens within an hour or within a few hours of enabling the firewall. As soon as I disable it, ping responses from all locations I test them from start clearing up and get stable again. Very strange.

The TIME_WAIT state connections have been fluctuating at those numbers since yesterday. For 24 hours now, I've had that, and although it hasn't impacted performance in any way, it's disturbing enough.

My current tcp_fin_timeout value is 30 seconds, from the default 60 seconds. However, that seems to not help, at all.

Any ideas, suggestions? Anything at all would be appreciated, really!

Hamza
  • 101
  • 1
  • 1
  • 3

2 Answers2

5

A good discussion about time-wait is given by How to forcibly close a socket in time-wait.

According to this reference, the number of time-wait connections should correspond to the traffic in the last 4 minutes. Do these numbers approximately match?

Community
  • 1
David Harris
  • 239
  • 1
  • 4
2

I've had this problem with connections in a close-wait state when the other end isn't hanging up properly. This may be because of a malicious reasons, or because there is a problem in your networking stack.

The fact that you appear to be having problems with iptables points to problems with your network stack too.

It is worth switching your connection to another network port if you have one to see if the same problem occurs on that. The problem could also be with an upstream firewall, router or bad guy.

There are also some reports of issues with the RedHat (+Centos?) settings of tcp_tw_recycle. You may want to look into that.

rorycl
  • 848
  • 1
  • 6
  • 10