23

I'm using AWS CodeDeploy to deploy my sites, and I noticed it's not very consistent in speed; sometimes it's pretty fast, but other times each step of a deployment can take minutes. This is pretty annoying when a deployment should be performed fast, in case of bugs or outages.

I can't find any documentation on the speed of CodeDeploy though, and also I can't seem to find any logic in when it's slow or when it's fast. Is there any way to speed it up and is there any way to know what's taking so long?

Jasper Kennis
  • 369
  • 1
  • 3
  • 12

4 Answers4

38

BlockTraffic and AllowTraffic

Simply adjusting your target group's health check settings can shave off a couple minutes.

Before

enter image description here enter image description here

After

enter image description here enter image description here

Explanation

This works because BlockTraffic and AllowTraffic both wait for successful health checks. The default health check interval is 1 check every 30 seconds, and a successful health check needs 5 consecutive 200 responses. Thus it takes more than 2 min 30 seconds by default. And that's for every EC2 instance. Decreasing the health check interval and limiting the number of successful checks needed will improve deployment time.

Community
  • 1
Peter Tao
  • 481
  • 1
  • 4
  • 4
13

CodeDeploy does very little by default - it grabs the code from S3 or Github, then runs your scripts per the appspec.yml file's instructions.

If your deployments are grabbing gigabytes of data from S3, you'll find that takes some time for the data transfer (particularly on smaller EC2 instances with limited bandwidth), but other than that deployment delays are much more likely to be due to whatever you're doing in your deployment scripts.

The steps in a CodeDeploy deployment are:

  • ApplicationStop - you control this hook
  • DownloadBundle - CodeDeploy grabs code from S3/Github
  • BeforeInstall - you control this hook
  • Install - CodeDeploy copies code from a temp location to the final destination
  • AfterInstall - you control this hook
  • ApplicationStart - you control this hook
  • ValidateService - you control this hook

The bolded ones are up to CodeDeploy, the others are up to you. If you're seeing varying delays in the bolded ones, contact AWS support, but otherwise chances are you need to investigate your hooks.

ceejayoz
  • 32,469
  • 7
  • 81
  • 105
  • Yeah, my own scripts are simple and fast. It's the Download bundle step that often takes minutes to run, and my project isn't that big; just as often it just takes seconds. I don't have the dev support plan tho, and I'm not willing to pay for it just for this, so I'll just let it be for now. – Jasper Kennis Feb 16 '16 at 11:04
  • 1
    Checked it again, the problem has to be the bandwidth; we're using a small instance type for our staging server and bigger once for production, and it's always staging that is being slow. – Jasper Kennis Feb 17 '16 at 11:32
  • @JasperKennis Yeah, the smaller instance types can be fairly bandwidth-limited, particularly if you're on a host with noisy neighbors. That'd make sense. – ceejayoz Feb 17 '16 at 11:47
7

Another setting to check is the Target Group's "Deregistration delay". My health check settings were already low and this was the bottleneck in my case.

Aaron
  • 171
  • 1
  • 1
  • 1
    I set deregistration delay to 30 seconds, with 2 of 10 second health checks, but the BlockTraffic step is still consistently taking 1m30s, whereas it should be ~40s. AllowTraffic only takes 20 seconds. Any idea what else could cause it? – timetofly Aug 31 '20 at 15:18
  • @timetofly How many EC2 instances you had in the target-group when you experienced this delay? Have you found the reason yet? – rineez Sep 16 '20 at 05:48
  • Anywhere between 6 and 14 instances. I have not found the reason yet. Sometimes it's taking 1 minute, sometimes it's taking 1m20s. – timetofly Sep 21 '20 at 18:55
  • It worked for me. But remember that if you have some requests in your application that can take more than deregistration time, it'll be lost. In my case I can't set it lower than 60 secs – dm707 May 25 '21 at 14:56
0

don't enable Load Balancer on Code Deploy deployment group for Pipeline and you will get rid off that BlockTraffic and AllowTraffic steps.

Screenshot

Zviadi
  • 1