-2

I have a streaming server with 10gbps card on it. I'm using ~2.5gbps.

My server is running perfect for 8-9 hours / day, and after that is going crazy, and everyone is freezing. At exactly the same time i reboot the system , and everything is back to normal.

I have monitor connections , bandwidth output everything. From 2.5 gbps, after 8-9 hourrs it goes to 1gbps thats why everyone is freezing, but after a reboot everything back to normal.

Anyone has any explanation about this?

This is my sysctl config

net.core.somaxconn = 65535
net.ipv4.route.flush=1
net.ipv4.tcp_no_metrics_save=1
net.ipv4.tcp_moderate_rcvbuf = 1
fs.file-max = 6815744
fs.aio-max-nr = 6815744
fs.nr_open = 6815744
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_rmem = 10000000 10000000 10000000
net.ipv4.tcp_wmem = 10000000 10000000 10000000
net.ipv4.tcp_mem = 10000000 10000000 10000000
net.core.rmem_max = 524287
net.core.wmem_max = 524287
net.core.rmem_default = 524287
net.core.wmem_default = 524287
net.core.optmem_max = 524287
net.core.netdev_max_backlog = 300000
net.ipv4.tcp_max_syn_backlog = 300000
vm.swappiness=1
net.netfilter.nf_conntrack_max=196608
net.ipv4.tcp_window_scaling = 1
vm.max_map_count = 655300
net.ipv4.tcp_max_tw_buckets = 1440000

The streaming is being done using PHP-FPM & nginx.

I'm on ubuntu 14.04 64bit with stock kernel. I tried the kernel 4.8 same results.

user3393046
  • 121
  • 2
  • 10

2 Answers2

0

throwing a bunch of config options and some random data about your system at us isn't at all useful.

You need to apply scientific method.

  • Install monitoring
  • Gather data
  • Use the data to help understand the system
  • Make changes based on what you have learned above
  • Monitor changes
  • Repeat analysis and make changes as required
  • Document changes

It's not rocket engineering or even alternate facts. Simple applied system management principles is all you need.

Further reading

user9517
  • 114,104
  • 20
  • 206
  • 289
0

Your post is somewhat confusing.

after that is going crazy

Implies increased activity, but you say...

everyone is freezing

...but then you contradict the assertion that "everyone" is affected by saying

From 2.5 gbps, after 8-9 hourrs it goes to 1gbps

If you are still sending out 1gbps, then clearly some clients are not affected. You also state:

The streaming is being done using PHP-FPM & nginx

...in which case it is not streaming. HTTP and fastCGI are not streaming protocols.

Based on the information you have provided, we cannot tell what ios wrong with your server, but a likely cause is that the nfconntrack table is full.

net.netfilter.nf_conntrack_max=196608

How many buckets are in use?

Its also possible your server is behind a misbehaving router/not receiving congestion notifications.

symcbean
  • 19,931
  • 1
  • 29
  • 49