0

This has puzzled me for the past few months...I'm admittedly new to server administration and I took a stab running a production rails app over at EC2. While I've learned a lot and feel pretty comfortable using it, I don't know the reason why it takes ~ 1-2 minutes for a request to go through after deploying my code. From all I've gathered through google, it's something with passenger starting back up. The solution seems to be crafting a rolling restart script.

My server setup is:

  • Amazon EC2 Ubuntu 12.04 EBS small instance
  • Nginx
  • Passenger
  • PostgreSQL 9.2
  • Ruby 1.9.3
  • Rails 3.2.11
  • Deployment with Capistrano

All pretty basic stuff going on here... hardly any custom configuration in any part of the stack. I first deployed to Amazon using Rubber, but then built a server from scratch because I assumed Rubber was the cause of the slow spin up after a deploy... but it still hangs. Sure, I could do rolling restart, add another instance and load balancer along with a third instance to be a database instance, but for the time being it doesn't make sense for this project.

I have a few nearly identical server setups on other hosts (rails, nginx, passenger, and postgresql on centOS) and they deploy much quicker. They do not have nearly the same lag between the finish of a cap deploy and the first request going through.

adding my cap deploy file

require 'rvm/capistrano'
require 'bundler/capistrano'
require 'sidekiq/capistrano'

set :stages, %w(production staging)
set :default_stage, "staging"
require 'capistrano/ext/multistage'

set :application, "bundio"
set :repository,  "git@bitbucket.org:my/project.git"
set :scm, "git"
set :shared_children, shared_children + %w{public/uploads} # include public/uploads in the shared folder to be symlinked across releases

ssh_options[:forward_agent] = true
ssh_options[:keys] = [File.join(ENV["HOME"], ".ec2", "my-keypair")]
set :deploy_to, "/var/www/apps/project"
set :rails_env, "production"
set :rvm_type, :system
set :rvm_path, "/usr/local/rvm"
set :keep_releases, 3

set :deploy_via, :remote_cache

desc "check production task"
task :check_production do

  if stage.to_s == "production"
    puts " \n Are you REALLY sure you want to deploy to production?"
    puts " \n Enter the password to continue\n "
    password = STDIN.gets[0..12] rescue nil
    if password != 'doubleCheck'
      puts "\n !!! WRONG PASSWORD !!!"
      exit
    end

  end

end



#############
### Hooks ###
#############

before "deploy", "check_production"

# if you want to clean up old releases on each deploy uncomment this:
after "deploy:restart", "deploy:cleanup"

after "deploy:update_code", "deploy:symlink_shared"
after "deploy:update_code", "deploy:migrate"
after "deploy", "deploy:restart_daemons" 


#################
### End hooks ###
#################

set :bundle_without, [:development,:test]

namespace :deploy do
  namespace :assets do
    task :precompile, :roles => :web, :except => { :no_release => true } do
      begin
        from = source.next_revision(current_revision)
      rescue
        err_no = true
      end
      if err_no || capture("cd #{latest_release} && #{source.local.log(from)} vendor/assets/ app/assets/ | wc -l").to_i > 0
        run %Q{cd #{latest_release} && #{rake} RAILS_ENV=#{rails_env} #{asset_env} assets:precompile}
      else
        logger.info "Skipping asset pre-compilation because there were no asset changes"
      end
    end
  end
  task :link_db do
    #run "ln -s #{shared_path}/config/database.yml #{latest_release}/config/database.yml"
  end

  task :start do ; end
  task :stop do ; end
  task :restart, :roles => :app, :except => { :no_release => true } do
    run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
  end
  task :restart_daemons, :roles => :app do
    sudo "monit restart all -g daemons"
  end
  desc "Symlink shared/* files"
  task :symlink_shared, :roles => :app do
    run "ln -nfs #{shared_path}/config/database.yml #{release_path}/config/database.yml"
  end

end
Danny
  • 223
  • 2
  • 9

1 Answers1

0

I've experienced the same lag between deploys and first request coming. If you have a small amount of RAM Passenger could take even minutes to get up.

The Passenger enterprise version guarantees you can deploy with no downtime: http://www.modrails.com/documentation/Users%20guide%20Nginx.html#PassengerRollingRestarts

fsoppelsa
  • 457
  • 1
  • 6
  • 12
  • Good to know, thanks. Interesting point they make: "Any database schema upgrades you perform must therefore be backwards-compatible with the old application version." - I am going to investigate my other apps that are not on amazon and also use passenger fusion. I'll look at downtime and come back with some actual numbers + amount of ram. – Danny May 29 '13 at 15:43
  • 1
    my other apps are about 1/4 the load up time, but there's still a good amount of lag. After asking around, ppl recommended using Unicorn to avoid the slow start up time after deploy. From an engineyard blog post comparing this vs Unicorn: "All of those issues aside, there are three problems that are really major for people: The warm up time, the inability (or at best extreme hackishness) to do seamless deploys, and the fact that since it’s compiled into the web server, you can’t configure proxies in the detailed and custom ways that you can with other application servers like Unicorn." – Danny May 31 '13 at 05:06