2

A simple WordPress static site run entirely on https.

Nginx conf here: http://pastebin.com/BrP0LThT

Only difference before / after is:

listen       443 ssl;
listen       443 ssl spdy;

Nginx 1.8.0 with SSL, no SPDY:

Response from transitions.js file:

HTTP/1.1 200 OK
Server: nginx/1.8.0
Date: Sun, 28 Jun 2015 18:13:30 GMT
Content-Type: application/javascript
Last-Modified: Wed, 03 Dec 2014 14:19:08 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
ETag: W/"547f1bdc-5267"
Expires: Sun, 12 Jul 2015 18:13:30 GMT
Cache-Control: max-age=1209600
Content-Encoding: gzip

Same server setup, with SPDY:

Response from same js file:

HTTP/1.1 200
cache-control: max-age=1209600
content-encoding: gzip
content-type: application/javascript
date: Sun, 28 Jun 2015 18:24:49 GMT
etag: W/"547f1bdc-5267"
expires: Sun, 12 Jul 2015 18:24:49 GMT
last-modified: Wed, 03 Dec 2014 14:19:08 GMT
server: nginx/1.8.0

Note how the server response for these files differ a lot once SPDY has been enabled.

Total time to load everything is pretty much exactly the same.

The fact it goes from a rather diagonal waterfall to a straight vertical drop is expected, multiplexing in full force.

But all TTFB green bars on those js, css and image assets get increased to about 280ms and the blue bars for Content Download time go from next to nothing, to over 1s each.

Detailed look here:

Note the uniform TTFB delays of about 280ms

It's all way too uniform to be a coincident.

iptables doesn't suggest any throttling. Nothing changed in the nginx conf either, other than enabling SPDY. As it's nginx 1.8.0, it's not the tcp_nodelay bug either. I have no special limiting configurations in my conf files or firewall. keepalive_timeout is 75 and the other keepalive options default.

Where should I look? What can I try? What might be the problem?

As bandwidth may be an issue now as many as 28 assets are downloading simultaneously, here is the graph of bandwidth utilisation. The busy JS downloading happens between 0.7s and 2s. Bar a weird nose-dive, bandwidth does max out (1.5Mbps) so perhaps the hosting environment has some influence here too.

Bandwidth Stats

JayMcTee
  • 3,763
  • 12
  • 20
  • First thing that jumps out is that 'Transfer-Encoding: Chunked' is missing in the SPDY enabled response. Please share your config (before and after) to help us find the cause. – Luc van Donkersgoed Jun 28 '15 at 22:45
  • Thanks Luc, pastebin of conf added above. The Transfer-Encoding: chunked, Connection: keep-alive, Vary: Accept-Encoding disappear, but reading up on those, that seems to be by design. I wonder whether it's either something related to GZip (double compression) or some buffer filling before transferring. FWIW this is a CentOS VPS. – JayMcTee Jun 29 '15 at 09:25
  • What is the time to `DOMContentLoaded` and `load` for both situations? – Luc van Donkersgoed Jun 29 '15 at 09:41
  • Before SPDY: DOMContentLoaded 2.478s - 2.534s (0.056s) Load 5.967s - 5.979s (0.012s) (Note 3 runs) After SPDY: DOMContentLoaded 2.741s - 2.795s (0.054s) Load 6.458s - 6.470s (0.012s) With SPDY on both counts marginally slower. With SPDY there is no longer a delay for the requests to start, but big delays in TTFB and Content Download. – JayMcTee Jun 29 '15 at 10:06
  • Or could the browser itself be throttling the bandwidth? As it now requests and receives as many as 25 files, might it by itself spread the available bandwidth so thinly across each concurrent download? – JayMcTee Jun 29 '15 at 12:18

1 Answers1

2

I believe what you are experiencing is consistent with how SPDY works.

In 'old' HTTPS, the browser will send requests to the server in a serial manner, which is what you're seeing in your first screenshot.

With SPDY, however, all requests are sent simultaneously, after which the server responds with the files in the order it deems optimal. This is what you're seeing in the second screenshot - notice that the start of all requests is at the same point in time.

The order in which the server delivers the requested files depends on server configuration, but more importantly, on resource prioritisation. The idea is to send JS and CSS files early, so the website can be painted. After that, SPDY should send the images and other resources.

Because you indicate that the time to DOMContentLoaded and load is not significantly different between SPDY and HTTPS, I think your server is behaving properly. If you want to achieve faster paint times, look into prioritisation.

Sources, and very interesting further reading:

As stated in the comments below, JayMcTee found out that his specific situation was caused by bandwidth - as SPDY executes all requests simultaneously, the bandwidth will be filled more easily, resulting in slower individual requests then when executed serially.

Luc van Donkersgoed
  • 619
  • 4
  • 15
  • Thanks for your answer. I concur that in large parts, what we're seeing here is simply the effect of SPDY, changing the shape of the waterfall. But I'm still concerned about the slower Content Download times. Take the same request, before and after SPDY: Request Start: 0.945 s, Time to First Byte: 396 ms Content Download: 2 ms, Bytes In (downloaded): 3.5 KB. ----- Versus: Request Start: 0.475 s (-470ms, expected), Time to First Byte: 287 ms (-109ms, somewhat expected), Content Download: 759 ms (+757ms or x360 slower), Bytes In (downloaded): 3.5 KB. This file is 360 times slower to DL! – JayMcTee Jun 29 '15 at 14:12
  • Ok, so the content download is a lot slower. But have other resources been downloaded in that time? Because that would still be expected behaviour. The long `Content Download` time would in fact be mislabeled and should be 'waiting for other downloads'. But the browser can't make this distinction, because all downloads are pushed over the same stream. – Luc van Donkersgoed Jun 29 '15 at 14:24
  • Yes, more is downloaded simultaneously. As much as 28 assets at the same time. Bandwidth in those tests does max out (1.5Mbps) so perhaps the wire is saturated? It's a cheap-ish VPS so perhaps they throttle bandwidth? I did look in to TCP throughput optimisation as well, like initcwnd and tcp_slow_start_after_idle etc. but no difference. I may have to ask the hosting provider. I have added a screenshot of the bandwidth usage profile. Thanks again for your continued insights. – JayMcTee Jun 29 '15 at 14:31
  • OK, a bit silly this, but those WebPageTests were run on DSL. Switch to Cable to bump 1.5Mbps client bandwidth to 5Mbps and everything looks way more healthy. Halving those blue Content Download bars. --- DomContentloaded 2.433s - 2.548s (0.115s) LoadEvent: 3.982s - 4.008s (0.026s). So the conclusion is that SPDY gives the most gains if your bandwidth allows all those simultaneous downloads. SPDY makes it a mass marathon. Non-SPDY is individual sprints. Thanks for helping me in the right direction. Perhaps if you mention bandwidth in your answer, I can then 100% accept it. Now it's 85% there. – JayMcTee Jun 29 '15 at 14:57
  • I've updated my answer to reflect your situation more precisely. – Luc van Donkersgoed Jun 29 '15 at 15:05