0

I would like to make my system more resilient against certain failure. When the system makes rapid outbound TCP connections fail, the nf_conntrack table fills up with TIME_WAIT entries. This causes other operations on the system to fail as there are no more nf_conntrack entries available.

I know that it is possible to modify certain parameters such as tcp_tw_reuse, & tcp_fin_timeout, but am reluctant to make drastic changes there (based on warnings like this)

What I was hoping to do prior to any such changes above and application changes is to harden my system against complete depletion of nf_conntrack entries by restricting outgoing connections by each susbsystem (by using the port numbers, ip addresses etc). I thought I would be able to add rules as follows: 

-A OUTPUT -p tcp --syn --dport 9702 -m connlimit --connlimit-above 3 -j REJECT --reject-with tcp-reset

But this seems to only impact active connections (which is understandable, as these are effectively gone). Is there a way to be able to limit outgoing connection per port/app in a way that it will take TIME_WAIT sockets into account?

Thanks

Community
  • 1
nhed
  • 520
  • 1
  • 6
  • 13
  • 1
    If you need more conntrack entries, why don't you just raise the number? – Michael Hampton Mar 04 '14 at 03:04
  • @MichaelHampton Not saying that I won't, but this would just buy me more time, but I'm trying to ensure first is that if one application goes wild that it does not impact others ... In my system it is easy to distinguish between said apps by socket parameters. – nhed Mar 04 '14 at 03:50
  • This is generally why keep your systems specialized, so that the system can be tuned to the primary app. If you do not trust your system+apps, barring the scenario of where the untrusted app is the one the system itself is designed around, you've already lost the battle. – Andrew B Mar 04 '14 at 04:05
  • @AndrewB I am with you there, but this is _not_ a webserver + mail server + DB + monitoring system. It just has the need to connect to more than one backend systems, as well as service customers at the front. While we intend to do our fine tuning, I claim that that if this feature existed, that it would have be considered, if nothing else, a good practice (like ulimit can be used to restrict resources for certain users/contexts) – nhed Mar 04 '14 at 04:29
  • It sounds like you're talking about a rate limit, rather than a connection limit. – Slartibartfast Mar 06 '14 at 04:17
  • @Slartibartfast The result may be a ratelimit. I'm looking to protect one subsystem of the system from failures in another subsystem. In this particular question I am looking to have a `connlimit` that applies to conntrack entries in any state, including `TIME_WAIT`. – nhed Mar 06 '14 at 14:23

1 Answers1

1

If you don't actually need NetFilter connection tracking, I'd suggest you either disable nf_conntrack altogether (by unloading it), or add a NOTRACK rule to the top of your iptables ruleset.

ch2500
  • 796
  • 5
  • 9
  • Yes, I already created NOTRACK rules for some of the traffic few days back, but this question is looking to implement a quota. While adding NOTRACK and some of the other suggestions would improve my situation - I think they don't give me enough control (and I'm a control freak). +1 for effort and potential help for future visitors, but won't mark accept as does not answer – nhed Mar 10 '14 at 01:51