1

For months I've been struggling to find the best workflow for building and testing my aws opsworks cookbooks locally prior to pushing to opsworks.

After a lot of stalled attempts I found a blog post by Mike Greiling and have since settled on an environment that works well for me. I'd like to share the setup/configuration because there are a lot of moving pieces.

I encourage anyone who has their own build / test environments for chef, that are specifically geared towards opsworks cookbooks, to also post answers here -- thanks!!.

Peter M
  • 953
  • 2
  • 14
  • 27

1 Answers1

4

I would HIGHLY recommend everyone check out Mike Greiling's blog post Simplify OpsWorks Development With Packer and his github repo opsworks-vm which help you to mock the entire opsworks stack including the install of the opsworks agent so you can also test app deploy recipes, multiple layers, multiple instances at the same time, etc. It is extremely impressive.

I've included all the setup needed to prep your environment before using Mike Greiling's Packer Virtualbox build.

Quick Start on Ubuntu 14.04

NOTE: This can NOT be done from an ubuntu virtual machine because virtualbox does not support nested virtualization of 64-bit machines.

  1. Install ChefDK
    1. mkdir /tmp/packages && cd /tmp/packages
    2. wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chefdk_0.8.1-1_amd64.deb
    3. sudo dpkg -i chefdk_0.8.0-1_amd64.deb
    4. cd /opt/chefdk/
    5. chef verify
    6. which ruby
    7. echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile && source ~/.bash_profile
  2. Install VirtualBox
    1. echo 'deb http://download.virtualbox.org/virtualbox/debian vivid contrib' > /etc/apt/sources.list.d/virtualbox.list
    2. wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
    3. sudo apt-get update -qqy
    4. sudo apt-get install virtualbox-5.0 dkms
  3. Install Vagrant
    1. cd /tmp/packages
    2. wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.deb
    3. sudo dpkg -i vagrant_1.7.4_x86_64.deb
    4. vagrant plugin install vagrant-berkshelf
    5. vagrant plugin install vagrant-omnibus
    6. vagrant plugin list
  4. Install Packer
    1. mkdir /opt/packer && cd /opt/packer
    2. wget https://dl.bintray.com/mitchellh/packer/packer_0.8.6_linux_amd64.zip
    3. unzip packer_0.8.6_linux_amd64.zip
    4. echo 'PATH=$PATH:/opt/packer' >> ~/.bash_profile && source ~/.bash_profile
  5. Build Mike Greiling's opsworks-vm virtualbox image using Packer
    1. mkdir ~/packer && cd ~/packer
    2. git clone https://github.com/pixelcog/opsworks-vm.git
    3. cd opsworks-vm
    4. rake build install
    5. This will install a new virtualbox vm to ~/.vagrant.d/boxes/ubuntu1404-opsworks/

To mock a single opsworks instance, create a new Vagrantfile like so:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu1404-opsworks"
  config.vm.provision :opsworks, type: 'shell', args: 'path/to/dna.json'
end

The dna.json file path is set relative to the Vagrantfile and should contain any JSON data you wish to send to OpsWorks Chef.

For example:

{
  "deploy": {
    "my-app": {
      "application_type": "php",
      "scm": {
        "scm_type": "git",
        "repository": "path/to/my-app"
      }
    }
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "path/to/my-cookbooks"
    },
    "recipes": [
      "recipe[opsworks_initial_setup]",
      "recipe[dependencies]",
      "recipe[mod_php5_apache2]",
      "recipe[deploy::default]",
      "recipe[deploy::php]",
      "recipe[my_custom_cookbook::configure]"
    ]
  }
}

To mock multiple opsworks instances and include layers see his AWS OpsWorks "Getting Started" Example which includes the stack.json below.

Vagrantfile (for multiple instances)

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu1404-opsworks"

  # Create the php-app layer
  config.vm.define "app" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/php-app.json'
    ]

    # Forward port 80 so we can see our work
    layer.vm.network "forwarded_port", guest: 80, host: 8080
    layer.vm.network "private_network", ip: "10.10.10.10"
  end

  # Create the db-master layer
  config.vm.define "db" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/db-master.json'
    ]

    layer.vm.network "private_network", ip: "10.10.10.20"
  end
end

stack.json

{
  "opsworks": {
    "layers": {
      "php-app": {
        "instances": {
          "php-app1": {"private-ip": "10.10.10.10"}
        }
      },
      "db-master": {
        "instances": {
          "db-master1": {"private-ip": "10.10.10.20"}
        }
      }
    }
  },
  "deploy": {
    "simple-php": {
      "application_type": "php",
      "document_root": "web",
      "scm": {
        "scm_type": "git",
        "repository": "dev/simple-php"
      },
      "memcached": {},
      "database": {
        "host": "10.10.10.20",
        "database": "simple-php",
        "username": "root",
        "password": "correcthorsebatterystaple",
        "reconnect": true
      }
    }
  },
  "mysql": {
    "server_root_password": "correcthorsebatterystaple",
    "tunable": {"innodb_buffer_pool_size": "256M"}
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "ops/cookbooks"
    }
  }
}

For those not familiar with vagrant you just do a vagrant up to start the instance(s). Then you can modify your cookbook locally and any changes can be applied by re-running chef against the existing instance(s) with vagrant provision. You can do a vagrant destroy and vagrant up to start from scratch.

Peter M
  • 953
  • 2
  • 14
  • 27