2

We are hosting videos in AWS S3 in HLS format (lots of .ts files, + playlists .m3u8).

We have configured AWS CloudFront with all Edge locations to serve those files.

We have 2 apps, iOS and Android, with a video player that plays the CloudFront URL. Android app is using Google’s Exoplayer, and iOS app is using AVPlayer. They both present the same issues.

The videos play smoothly from the same region as the source S3 bucket. We had some issues at the start but we fixed them by fixing the bitrate encoding.

However customers from remote countries complained about the buffering: average of 1 minute to initially load the video, then average of 12s of buffering every minute.

So we decided to add AWS CloudFront so that the files would be cached in the Edge locations, making the videos faster to load.

But the customers kept complaining, and we were aware that it would still take a while to load for the first user.

So we set up 2 non-AWS servers in other regions that automatically download the videos, in the aim to get them cached in those 2 regions, as soon as they are published.

But it doesn’t seem to help either.

I am wondering if AWS CloudFront is effective at all for video streaming via direct download, or if we are missing something.

How to make CloudFront work to get rid of video buffering? 
Or if CloudFront is not the solution (why?), what should we be looking at?

Side note: We set the TTL to 10 years, just in case… and we manage invalidation manually.

APE
  • 129
  • 5
  • What did AWS support say about this? – EEAA Jun 14 '17 at 01:36
  • It didn't actually cross our mind to contact them for this... We looked and we looked on the Internet but couldn't find anyone with the same sort of issue. Will do now! – APE Jun 14 '17 at 01:39
  • I'm frequently surprised by AWS support. It's not cheap, but they do a good job. – EEAA Jun 14 '17 at 01:41
  • Right, we are on the basic plan so we don't have access to Technical Support. I will have to check if we can pay for that. – APE Jun 14 '17 at 01:43
  • $29 for developer support. But given you set up two servers in other regions, and they're still slow, it could be last mile bandwidth. With CF the video can be evicted from their cache any time. You could replicate your bucket to different regions, which would increase your costs of course, but RRS/IA class storage would bring it down a little. – Tim Jun 14 '17 at 01:52
  • Hi @Tim in my understanding AWS offer to replicate the bucket to only one other region. How would you have it replicated in several regions? Then how would you have users point to the right bucket/region? – APE Jun 14 '17 at 01:58
  • It would be fiddly. You could chain replication between regions. You'd have to use manual geolocation and give the user the right URL. Getting CloudFront or another CDN working is probably a better approach. This is really just a fallback. – Tim Jun 14 '17 at 02:09
  • Got it, thanks! Indeed we'd much prefer getting CloudFront working - however concerned about the part where the video can be evicted from their cache any time... – APE Jun 14 '17 at 02:18
  • The first thing you need to do, if you haven't, is activate logging on both the CloudFront distribution and the bucket. Review and compare. It doesn't sound as if you have actually verified whether your attempts to warm the caches actually accomplished anything resembling your intentions -- what edges did they hit? Where do the users hit? CloudFront is a surprisingly transparent black box if you understand its topology and avail yourself to the headers and logs. – Michael - sqlbot Jun 14 '17 at 12:59
  • Thanks for the tip @Michael-sqlbot we had activated S3 logs but not CF logs. We'll let it spit out a bit and then will compare. I assume we should expect more logs in CF than in S3. – APE Jun 14 '17 at 22:22
  • 1
    Yes. The CloudFront logs will be more interesting, but also, you should be able to cross-coordinate cache misses and refresh-hits to S3 requests. Take the client IP from the S3 logs, do reverse lookups on them, and the hostname should reveal the CloudFront edge that actually did the fetch from S3 (which should line up with the CloudFront edge codes in the CF logs, but discrepancies may also be informative). – Michael - sqlbot Jun 15 '17 at 00:28
  • Did you ever figure out what was wrong? – mco Jan 08 '19 at 22:22
  • @mco We ended up duplicating the bucket in 3 other countries, around the world, in hot locations (where we have the most users), and we have CloudFront fetch the video files from the bucket nearest to the user. That almost entirely solved the problem. Of course some countries could still be improved if we had more buckets but we had to find a balance between cost, performances and density. – APE Jan 09 '19 at 23:09

0 Answers0