Getting corrupted data from the Internet - shouldn't TCP checksums prevent this?

1

The last 3 days, I've been getting corrupt data when surfing and downloading from a 3G internet connection. This is easiest tested by downloading a zip file from anywhere (e.g. https://get.videolan.org/vlc/2.1.3/win32/vlc-2.1.3-win32.7z) and testing the archive for checksum errors. Tried the same transfer with a different ISP, and the zip archive was OK.

How is this possible, given that TCP segments have a 16 bit checksum? Are the TCP segments not transferred unaltered from the server I connect to (e.g. get.videolan.org) all the way to my computer?

Some background info:

  • I connect via a Samsung GT-I8730 phone, which acts as a 3G modem+WiFi hotspot.
  • I get the same problem on 2 different laptops connected to the phone.
  • I've tried USB tethering instead of WiFi, but the result is the same.
  • When I use a VPN service (IPsec IKEv2 native client), downloaded files are not corrupt. Still the same ISP and the same phone.
  • I've tried a few large WiFi file transfers between the 2 laptops without any issues.

John Reynolds

Posted 2014-05-02T12:55:12.593

Reputation: 129

Its entirely possible the ISP is doing something to the traffic. It works with a VPN because the ISP isn't able to detect what your downloading. I would ask your ISP about this behavior. – Ramhound – 2014-05-02T13:19:44.430

Answers

2

Does your downloaded filesize exactly match what's on the server?

You might get checksum errors upon unzipping a file if the download did not complete.

Your ISP may be operating a transparent HTTP proxy that could also be interfering with the download. This is likely if a VPN makes it go away.

Some further thoughts based on your comments:

  • WIFI does not involve your phone, so that eliminates the laptop as an issue. (scratch that just realized you're using it as a hotspot. Eliminates your phone's Wifi hardware as a culprit, though.)

  • Try downloading the .7z file directly to your phone if you have enough storage. Then transfer it to your computer and verify. If you get it OK without using a VPN here, it takes your ISP out of the loop.

  • If possible, apply any baseband updates to your phone.

  • Your phone is acting like a router in this situation. Routers reissue TCP packets when they forward traffic. It may be correctly receiving the data from the remote server, but not passing it on correctly to your system, either due to a hardware defect or a bug. It's possible unencrypted traffic may be getting caught up in some sort of compression that would not be happening with encrypted VPN traffic. Consider updating the OS on your phone if you can or the application you are using to route 3G to Wifi users.

LawrenceC

Posted 2014-05-02T12:55:12.593

Reputation: 63 487

Yes, the corrupt files are the same length as the correctly downloaded one. – John Reynolds – 2014-05-02T17:41:55.610

Just for the heck of it, if you are under Windows, run the corrupt file against TrID http://mark0.net/soft-trid-e.html and verify the file type, or use the file command against it if you on Linux. Does this only happen when you download zip files?

– LawrenceC – 2014-05-02T19:06:58.920

TrID correctly recognizes the file as a 7z. Everything I download gets corrupt, but 7z files are easy to check with 7zip's test function. If I download an exe file (e.g. Firefox Portable) twice and compare the 2 downloads, the lengths are equal but comp.exe finds about a dozen differing bytes. It seems that the 3rd bit (counting from MSB) differs in those bytes, judging from the files I've compared so far. Sometimes it's 1 when it should be 0, sometimes it's the other way around. – John Reynolds – 2014-05-02T19:55:23.110

Tried downloading to the phone and then USB -> laptop, and checked the file which is still corrupt. The phone is runs Android, so there will be no OS updates ever. It seems possible that the ISP messes up the TCP segments, and that using the VPN tunnel protects against this. – John Reynolds – 2014-05-03T15:24:00.143

Update: as of Monday morning, the 3G speed has increased from <1Mbps to >10 Mpbs and downloaded files are no longer corrupt. – John Reynolds – 2014-05-05T15:14:20.037

Asked: 2014-05-02T12:55:12.593

Viewed: 825 times

Active: 2014-05-02T20:43:20.627