6

I would like to have a load balancer for my site and have the site up to date. The load balancer will take the AMI I select and spin up more of those instances when processing power hits a certain level. The problem is the AMI may not be up to date so I'll have some instances up to date and others not. When I deploy I can deploy to all the instances under the load balancer without issue, but I would need to know whenever the load balancer spins up a new instance in order to trigger this deployment. Also there would be a block of time when the updates happen greatly decreasing its responsiveness. So I've come up with a plan.

My plan:

After deploy:

    identify one of the instances and
        get instance id
    identify volume of instance id
    ec2-create-snapshot vol-yyyyyyyy
        get snapshot volume id
    ec2reg -s snap-zzzzzzzz -a x86 -d Description -n imagename
        get image id
    as-delete-launch-config existinglaunchconfig
    as-create-launch-config mylaunchconfig --image-id IMAGEID --instance-type m1.small --key mykey --group mysecuritygroup
    as-update-auto-scaling-group --launch-configuration mylaunchconfig

Before I go and spend however many hours trying to figure this out and scripting it all, testing, and everything else, my question is: Will this work? Is there a better way? Is there a tutorial or post that anyone knows of that would speed up my efforts on this issue? Thanks.

jchysk
  • 319
  • 4
  • 13

2 Answers2

7

A different approach that can work with AWS is to store the updated site/data/config somewhere like S3. Configure the AMI (or specify appropriate user-data script) so that when a new instance is spun up, it updates itself. You can prevent it from being added to the load balancer by not responding positively to the health check until the updates have completed.

Otherwise, if you need to have the instances come up faster, then your approach seems reasonable. Just make sure you suspend auto scaling before you start updating the live instances and resume after the new launch config. You don't want it starting instances that are not updated.

I'm also not sure if you can delete a launch config that is in use by an ELB. That step may have to wait until after it's been replaced. Post an update here when you find out.

Eric Hammond
  • 10,901
  • 34
  • 56
  • Thanks for the response. I'll definitely have to add in the suspend part before updating. I'll also comment when I find out about the deletion sequence for launch-config. – jchysk Mar 03 '12 at 18:13
  • I was not able to remove the existing launch configuration because it was being used. The procedure I ended up taking was: create new launchconfig-temp; add launchconfig-temp to my autoscalegroup; remove launchconfig; create launchconfig (same as launchconfig-temp); add launchconfig to autoscalegroup; – jchysk Mar 11 '12 at 23:59
  • @jchysk: As I suspected. Thanks for following up. – Eric Hammond Mar 15 '12 at 01:40
3

I ran into the same problem as you. I've got an auto scaling group that scales up an down with traffic and I push updates all the time even under peak load. My solution is quite a bit more involved since I need a staging area to be able to test my code updates before they hit live servers. At its simplest level here's how I build / update my servers.

Build base AMI - This is the AMI without any source code, this AMI can be bundled with 3rd party applications that you need or not. For example, one of my servers I need AppFabric, IIS, AWSSDK.NET and .NET 4.0. I install all of those tools and save the ami. My linux servers I do this at runtime since its much faster.

Store source code in S3 or SVN - When your server starts it grabs the latest code from SVN or S3. To update existing servers just terminate them one at a time or in groups and the Auto Scaling group will start another with the latest code.

Depending on how tolerant your service is to failures you can also implement a staging server like I have to test the new code before requests start getting sent. My setup requires two auto scaling groups. The first group is basically my security and authentication layer which uses php code. Once the request has been validated it gets forwarded to another auto scaling group.

Updates to the security authentication layer still require the steps explained by myself and Eric Hammond. However, I try not to update this code very much, it doesn't contain any business logic and almost never changes. Changes to this layer should always be backwards compatible so that if you do make updates you can have 2 versions of the servers online at once without any problems.

Updates to the business layer is where I apply all my code changes to upgrade features, fix bugs etc. The trick is to create another auto scaling group / loadbalancer when making updates with its own servers and code ( again from SVN or S3 ). Once you've tested it you update the local dns server ( assuming you have one ) and the security authentication layer automatically starts using the new servers.

Edit: Forgot to add this link http://www.slideshare.net/AmazonWebServices/aws-architectingjvariafinal. Has some very good information on solving this type of problem and a few other things that you may or may not know about aws.

bwight
  • 793
  • 1
  • 6
  • 14
  • I basically have a base AMI now that I deploy to through a continuous integration server which runs all the tests. The problem is the tests take a pretty long time (at least a few minutes) to run and then it all needs to be pushed to the new instance adding on at least that much time before another instance is active on the load balancer. If I have an AMI created whenever I push an update then newly spawned instances will always be up to date. – jchysk Mar 10 '12 at 23:32
  • 1
    I main reason i don't like AMI's with software on them is because they're more difficult to update. If you have an auto scaling group and not just a load balancer you have to put the ami in the configuration of the auto scaling group. Any time that you need to change the ami you have to delete the old configuration, create a new configuraiton then update the auto scaling group with the new configuration. Then cycle out all the old servers. Too much work for me... – bwight Mar 12 '12 at 14:18
  • Scripting it to create the new config each time was kind of a hassle, but not terrible. I don't need to cycle out all the old servers though because my build server will suspend processes, push everything to each of the individual boxes, clear out the old work space and put the new stuff in. So even old boxes will be up to date. – jchysk Mar 12 '12 at 19:08