random utterings...

Centralized multi-deployment approach with Capistrano

Chris Barnes — Tue, 26 Mar 2013 00:58:00 +0000

Capistrano is most often used for deploying a single Rails application but with a few libraries you can use it to deploy multiple applications or “services” in a service oriented architecture. In this article we’ll walk through how to build this type of deployment application with Capistrano.

From the docs:

Capistrano is a utility and framework for executing commands in parallel on multiple remote machines, via SSH. It uses a simple DSL (borrowed in part from Rake) that allows you to define tasks, which may be applied to machines in certain roles. It also supports tunneling connections via some gateway machine to allow operations to be performed behind VPN's and firewalls.

Normally with Capistrano your deployment code lives in the application repository. With multiple applications or services, there would be serious disadvantages to this approach. With a centralized deployment application, all of our deployment code is located in single repository. We can easily refactor it, secure it, and it's DRY.

Since Capistrano executes remote commands via SSH, user restrictions inside the deployment application shouldn’t be necessary. Individuals who have the permissions needed for a particular environment (production, qa, staging, etc) will be able to deploy to that environment. Additionally, you can restrict access to the deployment application repository so only specific people/departments have access to update the deployment application code.

Here's an example of a directory structure for deploying 3 services in an SOA application.

├── config
│   ├── deploy
│   │   ├── soa_app
│   │   │   ├── core_service
│   │   │   │   ├── production.rb
│   │   │   │   └── qa.rb
│   │   │   │   └── staging.rb
│   │   │   ├── user_service
│   │   │   │   ├── production.rb
│   │   │   │   └── qa.rb
│   │   │   │   └── staging.rb
│   │   │   └── billing_service
│   │   │       ├── production.rb
│   │   │       └── qa.rb
│   │   │       └── staging.rb
│   │   └── global.rb
│   └── deploy.rb
├── recipes
├── Capfile
└── Gemfile

You can create this structure manually or use a gem called CapHub which will generate the basic directory structure for you and install some needed gems. It will also install a couple that are out of scope for this article so we'll build the structure manually.

Add the following to your Gemfile

source "https://rubygems.org"

# Required: allows us to specify a separate configuration for each application/service
gem "capistrano-multiconfig"

# Optional: confirm deployment tasks if the local deployment application is not up to date
gem "capistrano-uptodate"     

# Optional: use rvm to install and manage RVM and Rubies remotely
gem "rvm-capistrano"

Add the following to your Capfile

require 'capistrano/multiconfig'
require 'capistrano/uptodate'
require 'rvm/capistrano'

# Optional: Use bundler for managing remote gems
require 'bundler/capistrano'

# Load capistrano's deploy recipes
load 'deploy'
load 'deploy/assets'

# Load all custom recipes
Dir['recipes/**/*.rb'].each { |recipe| load(recipe) }

# Load common configuration
load 'config/global'

The core functionality of our deployment application is in capistrano-multiconfig. It automatically builds Capistrano configurations from files located in config/deploy so creating config/deploy/soa_app/user_service/production.rb will make the corresponding configuration task available:

  $ cap -T
  ...
  cap soa_app:user_service:production     # Load soa_app:user_service:production configuration
  ...

Let's take a look at our common configuration.

# config/global.rb
# Put configuration here that is shared between ALL applications to be deployed

# some global defaults, yours may differ
set :rvm_ruby_string, '1.9.3'
set :scm, :git
set :scm_verbose, true
set :user, 'bob'
ssh_options[:keys] = [File.join(ENV["HOME"], ".ssh", "id_rsa")]
set :git_enable_submodules, 1

# keeps a cached copy of the application repo on each environment for faster deployments
set :deploy_via, :remote_cache

# set the application/service name and rails environment from the task naming convention
set(:application) { config_name.split(':').reverse[1] }    # core_service, user_service, billing_service
set(:stage) { config_name.split(':').last }                # production, qa, staging
set(:rails_env) { stage }
  
# we're deploying our database and server config also
before 'deploy:finalize_update', 'deploy:symlink_database_config', 'deploy:symlink_apache_config'

after(:deploy) do
  deploy.migrate        # migrate the database
  deploy.cleanup        # keep only the last 5 releases
end

Notice we're deploying server configuration as well as application code. This is optional of course but I like the approach. We're managing server configuration for each service in a git submodule called ‹service-name›-config. As configuration changes are introduced we track them in version control and we can test them against our staging environment. Each release of the service knows the commit reference it should pull from its config repo. Server configuration management (with rollback) is now part of our deployment strategy. We get access management for sensitive server configs built in and we can perform automated testing for configuration changes.

Anything that doesn't apply globally can be specified in individual environment configurations. Here's an example production configuration for the user_service.

# config/deploy/soa_app/user_service/production.rb
set :domain, 'users.soa_app.com'
role :web, domain
role :app, domain
role :db,  domain, :primary => true
set :repository,  "git@github.com:bobs-code/user_service.git"
  
# ask for a branch or tag to deploy for the user_service
# make sure to, git push origin --tags, first if you wish to deploy a tag
set :branch do
  branch = ENV['BRANCH'] || ENV['TAG']
  if branch.nil?
    Capistrano::CLI.ui.ask "Branch or Tag to deploy for the user service: "
  else
    branch
  end
end
set :deploy_to,   "/home/bob/#{application}-#{stage}"

You can override Capistrano’s default deploy tasks or write your own from scratch. In our example application we're going to mostly depend on Capistrano's deploy task. We'll override the restart task for our environment and add tasks to symlink our database and apache configurations.

namespace :deploy do
  desc "Symlink apache config"
  task :symlink_apache_config do
    run "sudo ln -nfs #{release_path}/config/submodule/apache/#{application}-#{stage} /etc/apache2/sites-enabled/#{application}-#{stage}"
  end

  desc "Restart apache"
  task :restart do
    run "sudo /etc/init.d/apache2 restart"
  end
end

To deploy our user_service we can execute

$ cap soa_app:user_service:production deploy

First time deployments to new servers will require some remote directory setup

$ cap soa_app:user_service:production deploy:setup

Deploying individual services one by one would be time consuming and error prone so we’ll need a way to deploy multiple services simultaneously. There is a caveat here, since configurations are actually tasks to Capistrano, we must be careful not to run them in parallel or they'll be merged. To deploy our entire application you could script the deployment tasks in sequence but let's create an additional recipe to handle this instead.

namespace :deploy_all do
 ...
 task :staging do
   Capistrano::CLI.ui.say "deploying core_service:staging"
   system("cap soa_app:core_service:staging deploy")
   
   Capistrano::CLI.ui.say "deploying user_service:staging"
   system("cap soa_app:user_service:staging deploy")
   
   Capistrano::CLI.ui.say "deploying billing_service:staging"
   system("cap soa_app:billing_service:staging deploy")
 end
 ...
end

We’ll need a (dummy) configuration environment for this so create an empty file config/deploy/soa_app.rb.

Now we can deploy to staging by executing

$ cap soa_app deploy_all:staging

Once you get your deployment application functional, I recommend reading http://jondavidjohn.com/blog/2012/04/cleaning-up-capistrano-deployment-output.

References

capistrano

capistrano-multiconfig

capistrano-uptodate

rvm-capistrano

caphub

Undo Git Commit

Chris Barnes — Sun, 18 Apr 2010 12:00:00 +0000

I just made a commit to a public project and pushed the changes up to github. Then I realized that I left some confidential information in one of the files. Searching online lead to a couple of different ways to fix this (git rebase, and git filter-branch), neither of which were working well for me.

I finally figured out that doing the following would work as long as the confidential information were only included in the last commit.

First update the file and remove the confidential info.

git add my-file-with-confidential-info

git commit --amend
(this will add what's in your current staging area to the last commit)

git push origin +master
(force push your changes to github)

Hope this helps someone else.

Open Source Wedding

Chris Barnes — Tue, 16 Mar 2010 08:37:00 +0000

I'm getting Married this October and so I've created a wedding site for my personal use (http://www.fairbarneswedding.com). I thought, why should I use it once and leave it to rot in source control somewhere? So I'm releasing it under the GNU GPL.

At the time of this writing the project has news articles, events calendar with rsvp, paypal integrated donation meter, wedding party members, wedding stories, and static pages. If you fork the project and make it better, send me a push request. You can find the source at http://github.com/randomutterings/wedding

Changing images with JQuery and the fade effect

Chris Barnes — Sun, 15 Feb 2009 10:14:00 +0000

Photo by me!

It took me a bit of digging to figure out how to fade out a picture, change the src, then fade it back in. I started out trying to use Scriptaculous but I got stuck trying to get the queue setup to wait until the image was faded out to change the src. I ended up using jQuery which also gave me a bonus, the solution is unabtrusive.

Here are the application.js and show.html.haml files from my project.

Pool-party, Auto-scaling with EC2

Chris Barnes — Sat, 31 May 2008 12:03:00 +0000

Photo by hrlndspnks

Having a great time at Railsconf08. Meeting some great people and learning a lot. I plan to post in more detail about a lot of this after the conference but I wanted to go ahead and mention one of the coolest presentations I've seen so far.

Ari Lerner has created a ruby gem called pool-party. It auto-scales your ec2 cluster based on criteria you set in the config file like average requests per second. Set your minimum requests per second and if the average of all the servers on your cluster goes over the maximum, pool-party will launch another instance for you. It scales down in the same way. It doesn't care what you're using your cluster for and a its going to support plugins soon so you can add in your own hook like functionality. Pool-party is using s3fuse to mount an s3 bucket on the instance at startup. Specify a bucket in pool-party and it gets mounted at /data on each instance. This solves the data persistence issue with ec2. You're not required to use it though, just don't specify a bucket in pool-party and nothing gets mounted.

Ari open-sourced Pool Party as of 2 days ago. Check it out at poolpartyrb.com. If you want to contribute, its on github at http://github.com/auser/pool-party.

Installing git on linux

Chris Barnes — Thu, 24 Apr 2008 12:32:00 +0000

Trying to install git from source on our linux server (we're running RHEL4 at work) and I was getting an error running make.

undefined reference to `libiconv'

Running make like this worked.

make install CFLAGS="-liconv"

But after the install was finished, running git barfed at me about loading a shared library libiconv.so.2.

I had to edit /etc/ld.so.conf and add /usr/local/lib and run ldconfig.

Credit Mark Turner for the ldconfig tip. Thanks man.

Making the jump to full time freelancing

Chris Barnes — Tue, 15 Apr 2008 20:47:00 +0000

Photo by Jack Spellingbacon

This week its all about changes. I've moved most of the site over to Radiant, except for the blog. I've also done some design changes and added a hire me page. This is in anticipation of May 15th when I'm planning to start full time freelancing. I've been doing development for 10 years now. I have experience running my own business and working closely with clients and end users so although its a new season for me, its not something I'm unfamiliar with.

Something that I am completely new to is teaching children. I'm planning to begin home schooling my kids in the coming months and I'll be writing about that in the blog as well. If you're interested in home schooling or freelancing or you just want to follow along to see how things go, subscribe to the blog.

Fixed Length Random numbers in Ruby

Chris Barnes — Wed, 09 Jan 2008 11:20:00 +0000

My latest project is almost complete and we're setting up a demo site with lots of fake data already included so I used the faker gem to generate most of it.

One issue I had that I couldn't do with the faker gem out of the box is get a fixed length random number. I need this for license numbers, credit card numbers, phone numbers and others. The faker gem does offer a random phone number but the ones in my project don't support extensions or punctuation so rather than generating the number then using string functions to strip out the stuff I didn't want, I decided to find an easier way.

I started by Googling for it assuming that someone else had already figured it out and while I'm sure its been done a hundred times before, I couldn't find it anywhere. So after much thought and digging through the ruby docs, this is what I came up with.

rand(9999999999).to_s.center(10, rand(9).to_s)

rand(9999999999) will give me a number between 0 and 9999999999. Convert it to a string and use the center method to make sure its at least 10 digits long and pad the rest with a random number from 0-9. rand(9)

If you need a number of say 7 digits in length, just change the first rand to rand(9999999) and change the center method length to 7 so it looks like this.

rand(9999999).to_s.center(7, rand(9).to_s)

Hope this is helpful to someone.

Subversion script for Rails developers

Chris Barnes — Wed, 19 Sep 2007 15:15:00 +0000

I read somewhere that good developers use version control. There's a good article on HowtoUseRailsWithSubversion.

So why should I repeat those steps for every new project I work on.

There is another article which attempted to tackle this problem. However, you still have to do the initial steps and then copy this script into your project directory and execute it.

The following script takes it a bit farther. You call it like this.

rails-svn app_dir repository username

So if I wanted to create a new rails project called toejam, I would execute:

rails-svn toejam svn://randomutterings.com/projects/toejam/trunk chris

Here's what happens under the hood.

After checking to make sure the directory doesn't already exist, your rails app is created with the standard rails command. (If the directory already exists, you will be asked if you want to continue. Choosing yes will allow the script to continue but if this script has been run on the same app twice, all of the svn commits will be duplicated.)
Your new app is imported to the subversion repository specified. Authentication is done with the username provided. You may be prompted for a password from your repository.
The original app directory is deleted and a working copy is checked out from the repository into that directory.
All the default rails log files are removed and subversion is instructed to ignore those files. (Log files can get large and we definitely don't need them in version control.)
The database.yml is moved to database.example to serve as a template for anyone who checks out the code and any newly created database.yml files are ignored by subversion. (Doing this will help if you have multiple developers working on the same project in different environments. It also prevents you from overwriting the database.yml file on the production server, oops!
Subversion is told to ignore everything in the tmp folder, the .htaccess files, and the dispatch.fcgi file.

Without further ado, here's the script. Just copy it and save it somewhere in your path. /usr/bin is a good choice. Don't forget to chmod 755 so you can execute it.

    #!/bin/bash

    if [ "$#" != "3" ]; then
      echo "Usage: rails-svn app_dir repository username"
      exit 1
    fi

    APPDIR=./$1
    SVN_TRUNK=$2
    SVN_USER=$3

    function check_if_exist () {
      if [[ -e $1 ]]; then
        echo ""
        echo "$1 already exists, overwrite? y or n"
        echo ""
        read OVERWRITE
        case "$OVERWRITE" in
        y)
          echo "Overwriting..."
          ;;
        *)
          echo "Action canceled"
          exit 1
          ;;
        esac
      fi
    }

    check_if_exist ${APPDIR}
    rails $APPDIR
    svn import $APPDIR $SVN_TRUNK -m "Import" --username $SVN_USER
    sudo rm -r $APPDIR
    svn checkout $SVN_TRUNK $APPDIR
    cd $APPDIR

    svn remove log/*
    svn commit -m "removing log files" 
    svn propset svn:ignore "*.log" log/
    svn update log/
    svn commit -m "Ignoring all files in /log/ ending in .log"
    svn move config/database.yml config/database.example
    svn commit -m "Moving database.yml to database.example to provide a template for anyone who checks out the code"
    svn propset svn:ignore "database.yml" config/
    svn update config/
    svn commit -m "Ignoring database.yml"
    svn remove tmp/*
    svn propset svn:ignore "*" tmp/
    svn update tmp/
    svn commit -m "ignore tmp/ content from now" 
    svn propset svn:ignore ".htaccess" public/
    svn update public/
    svn commit -m "Ignoring .htaccess"
    svn propset svn:ignore "dispatch.fcgi" public/
    svn update public/
    svn commit -m "Ignoring dispatch.fcgi"

I'm interested in converting this into a gem and possibly rewriting it in ruby. If anyone has the skills and wants to contribute, leave a comment or email me.

Rapt error during peepcodes test first development tutorial

Chris Barnes — Thu, 13 Sep 2007 12:47:00 +0000

OK, let me start off by saying, I love the tutorials from peepcode, you guys are doing a great job so keep doing it.

Alas, I got stuck while watching the Test-first-development tutorial. They were using Rapt as an alternative to script/plugin and when I tried to run it I kept getting an error.

Error: RaPT currently does not work outside of a Rails application directory. Please change to the top level of a Rails application and try again.

This error is misleading but at least it was obviously misleading, after all I know I'm in my Rails application directory.

After some googling, I found a post describing a work around for the problem, but it basically gave instructions on installing the plugin with script/plugin. But the guys at peepcode had a reason for using rapt over script/plugin (its faster). So I Google some more. I finally find a bug on the Rapt Trac posted in 2006 that explains that the user must first add repositories by executing the following.

rapt discover

If anyone knows how to have it default to yes without prompting you for every repository, please post a comment.

Alternatively, you can add only the repositories you want to use by executing the following.

rapt source http://path-to-repository