Insanely high latency when full bandwidth is in use

0

When I use the full bandwidth my ISP is providing, I get insanely high latency in every website, more than 1000ms.

I did a test on 3 of my friends' internet connection, including mine. I took a reading of latencies of google.com before starting a download and took another reading during a download. And all 3 of their latencies were very close. It didn't fluctuate that much. But mine instantly went above 1000ms and mostly stuck around 1800-3000ms. I did a traceroute and found that most of the latency gets generated in the Default Getaway of my ISP. This is a recent problem with my ISP. It was fine before. And they are simply not doing anything regarding this.

What I want to know is, what is my ISP doing wrong? Anything in their configuration of routers or something else? Please help me out so that I can at least point them out their problem.

Thank you.

Abdullah Sabbir

Posted 2019-08-01T19:46:58.323

Reputation: 1

2latency will rise as you max out the bandwidth, its not an isp issue. – Moab – 2019-08-01T19:49:23.577

1@moab 100% agreed its not an ISP "issue" but it could be due to ISP changing their QoS settings, and most likely introducing bufferbloat. – davidgo – 2019-08-01T20:01:24.530

1If you max out your bandwidth, latency will suffer. – Moab – 2019-08-01T20:02:42.490

The way to fix this problem is to not saturate your connection. Add packet shaping/bandwidth limiting on your own router/computer so your ISP connection does not need to queue packets because they are coming in faster then the line can handle them. Also note that smaller pipes tend to have higher latencies then larger ones - if your connrction is small (lrss then a megabit or so) you might need to look at mtu as well, but dont do this unless you know you need to as it opens a can of worms. – davidgo – 2019-08-01T20:08:35.930

@davidgo I think you are right. I did a quick search on bufferbloat on google and did a test. I think my ISP's router does have this problem. Thank you. I'll let them know about this. – Abdullah Sabbir – 2019-08-01T20:09:07.657

@davidgo I understand about limiting the speed of my connection, I already use limit in my PC. But there are other family members in my home but they are not that tech-savvy. So I can't do it for all of them. And my connection speed is 9 Mbps. And I couldn't find a bandwidth limiting setting in my router's admin panel. And even if I could there are complications. My ISP has cache servers for some websites (like Netflix, YouTube, etc.). So I don't think it'll be easy to limit the speed from my side. – Abdullah Sabbir – 2019-08-01T20:13:42.753

@AbdullahSabbir - Ok, but if, when I was running an ISP and I would have had a (non premium) customer that complained about this I would not consider it my problem and would not do anything to fix it. You are generating too much traffic - this is your problem, not the ISPs – davidgo – 2019-08-01T20:14:54.590

None if that is the ISP's problem - and indeed them caching content is beneficial for you. get a router which supports dd-wrt and integrate it into your network to do connection-wide bandwidth shaping. – davidgo – 2019-08-01T20:17:03.567

@moab You really need to read up on bufferbloat. 99.9% of the time people complain about latency on a fully-loaded connection, it turns out to be bufferbloat, not the tiny increase in latency that you'd have if your queues were properly managed to control latency while maximizing link utilization (SQM). – Spiff – 2019-08-01T21:58:19.533

@davidgo TCP is intentionally designed to keep the bottleneck link saturated, so that your download (or whatever) goes as fast as possible, so that you get the full benefit of the bandwidth you're paying for. The solution is not to leave some link capacity unused. The solution is to prevent a huge harmful queue of packets from building up on the middleboxes on either side of the bottleneck link. SQM (the solution to bufferbloat) keeps the link saturated while still keeping latency low, by managing queues better. – Spiff – 2019-08-01T22:07:44.703

@Spiff We are actually in agreement. (SQM is an example of connection-wide bandwidth shaping) – davidgo – 2019-08-01T22:14:31.353

@davidgo We are not in agreement if you are still telling people not to saturate their links. Everyone should saturate their links as needed! The enemy is oversized poorly managed queues in networking equipment, not saturated links. – Spiff – 2019-08-01T22:34:22.790

@spiff but the SQM solution you posit reduces the connections speed to prevent the link from being saturated. https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details specifically states (in different ways multiple times actually) "you need take control of the bottleneck link away from the ISP and move it into the router so it can be fixed. You do this by entering link speeds that are a few percent below the actual speeds. "

– davidgo – 2019-08-01T22:45:53.180

Let us continue this discussion in chat.

– Spiff – 2019-08-02T01:50:25.637

Answers

1

High latency when the link is saturated is the telltale sign of a widespread router/modem design flaw known as bufferbloat.

Run http://dslreports.com/speedtest (if it supports your country) from a desktop/laptop web browser to check your bufferbloat grade.

The solution to bufferbloat is Smart Queue Management, using algorithms like Cake and FQ-CoDel. SQM is supported by many aftermarket Linux-based open source router firmware distributions such as OpenWrt, and some vendors have started to build SQM into their factory firmware as well.

One turn-key solution is the IQrouter from evenroute.com, which is a decent TP-Link model preloaded with OpenWrt, with SQM set up ahead of time, and with some custom software to keep SQM tuned at all times.

There are two widespread false myths that well-meaning but under-informed geeks often spread when this subject comes up, so let me do some myth-busting:

  1. Saturating a link does NOT automatically mean high latency. With proper queue management, you can keep your link saturated and still have decently low latency. Networking protocols such as TCP are intentionally designed to keep links saturated so that your network operations (file downloads or whatever) complete as fast as possible, so you get all the bandwidth you're paying for.
  2. Using QoS to prioritize one class of traffic (e.g. games) over another (e.g. video streaming) is not a solution to bufferbloat. It just shovels the problem into someone else's lap. SQM is the solution to bufferbloat.

Spiff

Posted 2019-08-01T19:46:58.323

Reputation: 84 656

Asked: 2019-08-01T19:46:58.323

Viewed: 51 times

Active: 2019-08-01T22:31:26.790