2

I've got multiple websites (all of which being reverse-proxied via the same server) failing to properly load on one of my computers.

This particular case can be reproduced every single time, where a single JS file will stay stuck at "pending" in the chrome developer tool, despite Fiddler showing that the request has been completed with the correct headers and response.

I tried the following:

  • incognito tab
  • Firefox, edge, internet explorer
  • Removed all vlan's from my computer
  • completely removing and reinstalling the network drivers
  • Try a fresh windows user

When I try a different computer it works perfectly fine, despite both computers having the same browser extensions and same antivirus, as well as both running an up-to-date installation of Windows 10.

If I run Chrome via HTTP Toolkit (software tool for intercepting HTTP requests) everything works fine.

These are the headers the server responds with (as shown in fiddler):

HTTP/1.1 200 OK
Server: nginx
Date: Sun, 03 May 2020 13:22:52 GMT
Content-Type: application/javascript
Content-Length: 577367
Last-Modified: Wed, 15 Apr 2020 21:18:39 GMT
Connection: keep-alive
ETag: "5e977a2f-8cf57"
Accept-Ranges: bytes

What other steps can I try to debug and pinpoint this issue?

This file is a wireshark capture of my PC communicating with my webserver, the file failing to load is /js/main.bundle.js?v=2.2.3

This file is the chrome net export

xorinzor
  • 269
  • 2
  • 4
  • 19
  • Can you try with another user on this computer ? What happens after the "pending" state, the dev tools shows that the request is cancelled ? – Swisstone May 06 '20 at 18:42
  • @Swisstone I let the page load for 10 minutes, then stopped the request. It went from `(pending)` to `(cancelled)` at that point. Also created a new windows user, and tried the same, but unfortunately with the same result. – xorinzor May 06 '20 at 20:17
  • Can you create a client-side network trace with Wireshark? – Bernhard May 07 '20 at 11:41
  • @Bernhard I added a link to the file to my post – xorinzor May 07 '20 at 14:43
  • So, the Wireshark trace has ca. 0,5MB of `main.bundle.js` that is transferred within roughly 0.1 sec. Towards the end, there is a bit packet loss and retransmissions, but that's not looking too severe. Can you keep the trace running for a bit longer to see what it is doing? The transfer looks rather complete... – Bernhard May 07 '20 at 14:49
  • @Bernhard I updated the wireshark trace with one that I left running for roughly 4 minutes. But I agree, and according to the fiddler trace the server does send a response that I don't see anything out of the ordinary with. The fact that all browsers are behaving the same however leads me to believe this is probably something within windows not working properly. – xorinzor May 07 '20 at 14:58
  • Alright, so the transfer is complete, the TCP session is kept open for roughly 1 minute (as expected by HTTP keepalive) and is then torn down. At latest after this 1 minute, the browser should be aware that the loading is finished (or at least terminated). You mention fiddler in your last response. Are there any other proxies or intercepting software involved? I definitely agree at this point that it is a client side problem. – Bernhard May 07 '20 at 15:08
  • @Bernhard no proxies or whatsoever. It's all via LAN too, when I connect my laptop via the same cable it works just fine. Both computers run the same antivirus (sophos home) so I doubt that'd be the cause either. – xorinzor May 07 '20 at 15:16
  • Alright, then let's focus on the client side: Can you test with Chrome or Chromium please. If the problem shows up there as well, open up `chrome://net-export/`and start a trace, do a page reload (and wait for >60 secs), stop the trace and make the file available? – Bernhard May 07 '20 at 15:28
  • @Bernhard interesting, didn't know that tool existed. Though the output makes no sense to me. I added the file to the question (ran it for roughly 75 - 80 seconds before stopping the page loading, and ending the export). – xorinzor May 07 '20 at 16:17
  • Extracting the relevant part from the net-export and quoting it here: `t= 69 [st= 7] HTTP_TRANSACTION_SEND_REQUEST_HEADERS --> GET /js/main.bundle.js?v=2.2.3 HTTP/1.1 t= 69 [st= 7] -HTTP_TRANSACTION_SEND_REQUEST t= 69 [st= 7] +HTTP_TRANSACTION_READ_HEADERS [dt=80452] t= 69 [st= 7] +HTTP_STREAM_PARSER_READ_HEADERS [dt=80452] t=80521 [st=80459] CANCELLED` It does not finish to parse the headers although the whole thing is sent according to Wireshark. – Bernhard May 07 '20 at 18:46
  • @Bernhard but the exact same happens in IE, Edge and Firefox too. So I'd be suprised if this is something related to the browser. Definitely not an easy problem this one, haha. – xorinzor May 07 '20 at 19:00
  • So, to summarize the analysis so far: Some retransmissions on the network layer, but ultimately it is transmitted successfully on the network. The browser does not see the transmission finish, in fact it never finishes even parsing the headers. I'm taking the guess here that there's some intercepting software in the way. You are mentioning Sophos Home in a comment, I'm going ahead and blame it for intercepting and breaking the traffic. – Bernhard May 07 '20 at 19:00
  • I'm going to disagree on Sophos being the cause. I run Sophos Home on my laptop (where it works) too, and the software never gave any notifications about blocking or intercepting something. Nothing to be found in the event history either. I guess I'll continue looking. – xorinzor May 07 '20 at 19:03
  • OK, challenge accepted ;) Do you know [Sysinternals Procmon](https://docs.microsoft.com/en-us/sysinternals/downloads/procmon)? Have it record the browser while doing the request and save the output to a file. – Bernhard May 07 '20 at 19:15
  • @Bernhard glad you're not giving up haha :) Because I'm not sure if it contains sensitive information I've emailed you the download link rather then putting a public link in the post for this capture file. – xorinzor May 08 '20 at 09:48

1 Answers1

1

Summarizing the results from the discussion as answer here:

The Wireshark trace shows some retransmissions on the network layer, but ultimately it is transmitted successfully on the network. This is not a network connectivity problem.

Digging through the Chrome browsers' chrome://net-export/ logs, one can seee that the browser does not even finish reading the headers from the network:

t= 69 [st= 7] HTTP_TRANSACTION_SEND_REQUEST_HEADERS --> GET /js/main.bundle.js?v=2.2.3 HTTP/1.1
t= 69 [st= 7] -HTTP_TRANSACTION_SEND_REQUEST
t= 69 [st= 7] +HTTP_TRANSACTION_READ_HEADERS [dt=80452]
t= 69 [st= 7] +HTTP_STREAM_PARSER_READ_HEADERS [dt=80452]
t=80521 [st=80459] CANCELLED

This means that the source of the problem likely sits in an intercepting (proxy) software on the client system, as network traffic reaches the system, nut not the application. As the OP mentions Sophos AV, I assumed this is the source of the problem.

While inspecting process communication with procmon.exe, Sophos could be identified as the culprit by the OP.

Bernhard
  • 911
  • 5
  • 13
  • Thank you so much for the help! I've submitted a ticket with Sophos to diagnose this further with them. – xorinzor May 08 '20 at 15:12