0
I'm looking for help on trying to troubleshoot a buffering issue using Amazon's CloudFront service. I've set up an S3 bucket and have it associated with a CloudFront service. I use the service to stream videos to a public channel I developed on the Roku box. I've been doing this since June.
Recently, I've noticed that some of the videos buffer constantly. I can't pinpoint a reason why some do and some don't. I've posted on the AWS and Roku forums, but, no response so far. If anyone knows of a method to troubleshoot this performance issue, I'd be all ears. I did try using the AWS bucket logging, but, wasn't sure what I should be looking for.
EDIT: From Capture logs...first is from a user, second from me.
2017-11-02 23:25:57 JAX1 84982 xx.xxx.32.124 GET xxx.cloudfront.net /regattas/club/2016_Master_Nationals_Riverfront_Club_A4__Heat_2_HD.mp4 206 - Roku/DVP-7.70%2520(087.70E04142A) - - Hit kWs3_MhCRPjjbgg1PNn-4dTWa7KgCWtczsZs6O8nflyQLxFvuPK2iQ== xxx.cloudfront.net http 205 0.001 - - - Hit HTTP/1.1
2017-11-02 23:25:56 JAX1 1520400 xx.xxx.32.124 GET xxx.cloudfront.net /regattas/club/2016_Master_Nationals_Riverfront_Club_A4__Heat_2_HD.mp4 200 - Roku/DVP-7.70%2520(087.70E04142A) - - Error UVOgLi06vlRmFT3Pj9Bw5l5rp1ZDJIuBl72IJYDNeLSbGib1siwdpw== xxx.cloudfront.net http 180 2.035 - - - Hit HTTP/1.1
2017-11-02 23:25:56 JAX1 85628 xx.xxx.32.124 GET xxx.cloudfront.net /regattas/club/2016_Master_Nationals_Riverfront_Club_A4__Heat_2_HD.mp4 206 - Roku/DVP-7.70%2520(087.70E04142A) - - Miss LpWTXnQLQ1IUQQLe9-PSoT746pfi--nrKBxgm_ikkPft0_QzVdqC_Q== xxx.cloudfront.net http 205 2.050 - - - Miss HTTP/1.1
2017-11-02 23:25:57 JAX1 1025641 xx.xxx.32.124 GET xxx.cloudfront.net /regattas/club/2016_Master_Nationals_Riverfront_Club_A4__Heat_2_HD.mp4 200 - Roku/DVP-7.70%2520(087.70E04142A) - - Error VErphxx-KF-v1aAZK6WMW98FuM5cD1G0wm4lb_Zsx_GkbgZB2jWKHQ== xxx.cloudfront.net http 180 0.383 - - - Hit HTTP/1.1
2017-11-02 23:28:49 MSP50 1048352 xx.xxx.32.124 GET xxx.cloudfront.net /regattas/club/2016_Master_Nationals_Riverfront_Club_A4__Heat_2_HD.mp4 200 - Roku/DVP-7.70%2520(087.70E04142A) - - Error 0o3N4njSi3i_OqDeo-ARuZEoZQu3GqhE5_RLm9Yp05G-mwaMS_IpBA== xxx.cloudfront.net http 180 1.147 - - - Miss HTTP/1.1
2017-11-02 23:28:47 JAX1 48060143 xx.xxx.32.124 GET xxx.cloudfront.net /regattas/club/2016_Master_Nationals_Riverfront_Club_A4__Heat_2_HD.mp4 200 - Roku/DVP-7.70%2520(087.70E04142A) - - Error ZOA3ysvVKazSfhOJF9d5kiZUs80omOB9Ly3CyMC27EVtRSMVds1Y4g== xxx.cloudfront.net http 180 174.101 - - - Miss HTTP/1.1
2017-11-02 23:30:10 MSP50 1970009 xx.xxx.32.124 GET xxx.cloudfront.net /regattas/club/2016_Master_Nationals_Riverfront_Club_A4__Heat_2_HD.mp4 206 - Roku/DVP-7.70%2520(087.70E04142A) - - Error BSPf964lVszjaJ9NqqrjeD0L8uYTtXf5WA09nRJ4sdFjaAliF8IVOw== xxx.cloudfront.net http 204 81.618 - - - Miss HTTP/1.1
2017-11-03 20:27:19 JFK1 946020 xx.xxx.228.223 GET xxxxx.cloudfront.net /misc/Roku_Channel_Help.mp4 200 - Roku/DVP-8.0%2520(298.00E04108A) - - Error PK2uDUrw7Y_sUgwzFYLtc3LaWcFngofD1Hd8cFcGdRHg52AL6-ogcQ== xxxxx.cloudfront.net http 136 0.074 - - - Hit HTTP/1.1
2017-11-03 20:27:19 JFK1 941328 xx.xxx.228.223 GET xxxxx.cloudfront.net /misc/Roku_Channel_Help.mp4 206 - Roku/DVP-8.0%2520(298.00E04108A) - - Error E1iykDnNjVgPL8_4c0XcIu5IJmS7v1fe5Lf3xyPR5WFciBExF7ExvA== xxxxx.cloudfront.net http 157 0.040 - - - Hit HTTP/1.1
2017-11-03 20:29:01 JFK1 49857208 xx.xxx.228.223 GET xxxxx.cloudfront.net /misc/Roku_Channel_Help.mp4 200 - Roku/DVP-8.0%2520(298.00E04108A) - - Hit jteprNW9dfnMeOgAB96IND8kvdkUOI3MinEcWWOjJ4gPD7L4Ee3MgQ== xxxxx.cloudfront.net http 136 102.339 - - - Hit HTTP/1.1
2017-11-03 20:29:22 JFK1 946078 xx.xxx.228.223 GET xxxxx.cloudfront.net /misc/Roku_Channel_Help.mp4 206 - Roku/DVP-8.0%2520(298.00E04108A) - - Error VfGjokSp5cCVkOEgRgXPtlhPHI43TbB3syfHP2Zvta_jT4KfEhQLqA== xxxxx.cloudfront.net http 156 122.885 - - - Hit HTTP/1.1
2017-11-03 20:34:10 JFK5 384794 xx.xxx.228.223 GET xxxxx.cloudfront.net /misc/Roku_Channel_Help.mp4 206 - Roku/DVP-8.0%2520(298.00E04108A) - - Error jesajhZRE1O3IUKbgq0Us7qlvJXRTsjbZyFxfsnaXIDT_We9vEUZAA== xxxxx.cloudfront.net http 157 0.118 - - - Miss HTTP/1.1
2017-11-03 20:34:10 JFK5 982211 xx.xxx.228.223 GET xxxxx.cloudfront.net /misc/Roku_Channel_Help.mp4 200 - Roku/DVP-8.0%2520(298.00E04108A) - - Error OsJceDTgVJPEd6gz4CKp9XCpUOpc__EwS7CfRhnwOjPaSSLIcu3xUg== xxxxx.cloudfront.net http 136 0.316 - - - Miss HTTP/1.1
rrirower
Posted 2017-11-02T18:53:23.103
Reputation: 642
1You need to capture logs for both the bucket and for CloudFront, observe both and compare them to each other. You need to know whether the Roku is making complete requests or
Rangerequests and whether they are going back to S3 or being served from the CloudFront cache, and which CloudFront edge locations are involved. There is a great deal of information to be learned from the logs, and until you have a grasp of that, it will be difficult for anyone to speculate about the possible nature of the problem. A local packet trace would also be valuable. – Michael - sqlbot – 2017-11-02T22:54:51.470Thanks. I have enabled logging for both the bucket and cf. I'll report back. – rrirower – 2017-11-02T22:56:56.620
@Michael-sqlbot I've captured some S3 and CF log files. On thing that jumps out at me from the CF logs, is that I get a 2xx status code with a 'x-edge-result-type' = ERROR. Does that seem plausible? Additionally, yesterday, most videos would buffer. Today, they seem fine. ISP problem? – rrirower – 2017-11-03T15:53:35.567
It sounds like it could be exactly that. 2XX + ERROR suggests that the request succeeded initially but something went wrong before the last byte left the edge. Can you include some log lines with only the sensitive bits redacted? Notice that the edge identifier is there, in XXXN or XXXNN format (e.g.
IAD6). Were the problems limited to one edge or multiple edges in the same locale or global? – Michael - sqlbot – 2017-11-04T02:51:15.153@Michael-sqlbot Edited question with posted logs. Problem appears in more than one edge. – rrirower – 2017-11-04T14:56:01.453
Everything between the S3 and CloudFront seems to have been okay. It's interesting that your user accessed edges in both Florida and Minnesota -- that's quite a geographic split, but might represent CloudFront trying to route traffic away from a problem. Yours both went to edges in NYC. Unfortunately, there's another factor to consider, which is what logs look like when things are working correctly. Here, it appears that the connection may be lost or the Roku abandons it (which might be normal in some cases). Does the Roku expose any playback debugging tools? – Michael - sqlbot – 2017-11-04T16:33:20.710