David Dollar

Introducing Foreman

2011-05-06T00:00:00-04:00

Lately I've noticed my web apps are getting complicated to run. My apps no longer consist of a single web process; I usually have have one or more types of background workers to run different types of jobs and a clock process to handle scheduling. Splitting an app up in this way is great for performance and scalability. The downside, however, is that it becomes much more complicated to get the app and all of its parts running.

Foreman is an attempt to make this easier. Using foreman you can declare the various processes that are needed to run your application using a Procfile.

web:    bundle exec thin start -p $PORT
worker: bundle exec rake resque:work QUEUE=*
clock:  bundle exec rake resque:scheduler

To get started, simply run gem install foreman. If your project is using both Bundler and Foreman, launching it should be as simple as

$ bundle install
$ foreman start

Foreman will start all of the processes associated with your app and display stdout and stderr of each process. Processes are color-coded by type to make them easy to read. $PORT will be automatically assigned by foreman and made available to each underlying process.

Exporting to Production

While foreman is great for running your application in development, there are some great existing tools for managing processes in production. Foreman can export to either upstart or standard unix init.

Exporting to upstart

$ foreman export upstart /etc/init
[foreman export] writing: /etc/init/testapp.conf
[foreman export] writing: /etc/init/testapp-web.conf
[foreman export] writing: /etc/init/testapp-web-1.conf
[foreman export] writing: /etc/init/testapp-worker.conf
[foreman export] writing: /etc/init/testapp-worker-1.conf
[foreman export] writing: /etc/init/testapp-clock.conf
[foreman export] writing: /etc/init/testapp-clock-1.conf

After exporting to upstart, the following commands become available:

$ start testapp
$ stop testapp-clock
$ restart testapp-worker-1

Exporting to init

$ foreman export inittab
# ----- foreman testapp processes -----
TE01:4:respawn:/bin/su - testapp -c 'PORT=5000 bundle exec thin start -p $PORT >> /var/log/testapp/web-1.log 2>&1'
TE02:4:respawn:/bin/su - testapp -c 'PORT=5100 bundle exec rake resque:work QUEUE=* >> /var/log/testapp/worker-1.log 2>&1'
TE03:4:respawn:/bin/su - testapp -c 'PORT=5200 bundle exec rake resque:scheduler >> /var/log/testapp/clock-1.log 2>&1'
# ----- end foreman testapp processes -----

Advanced Options

Concurrency

Foreman supports running more than 1 of each process type.

# run 1 of each process type, and 2 workers
$ foreman start -c worker=2

# do not run a clock process
$ foreman start -c clock=0

Port Assignment

Foreman starts port assignments at 5000 by default, and assigns them in blocks of 100 per process type in the order used in your Procfile

For the Procfile used at the top of this post, web.1 would be assigned port 5000, worker.1 would be 5100, and clock.1 would be 5200. If you were ran 2 web processes, web.2 would be assigned 5001.

You can specify an alternate starting port for foreman to use with the -p option.

Documentation

For more information about the available options, check out the man page.

Simulating the Heroku Environment in Development

2009-11-08T00:00:00-05:00

I've been using Heroku quite a bit lately for easy deployment. Things such as DATABASE_URL and S3 credentials are managed through heroku config and show up in your production app as ENV['DATABASE_URL']. This is a great way to keep these configuration details out of your repository.

One downside to this approach is that it can be annoying to get these environment variables set in your local development version of the app. Enter rack-environment. This piece of Rack middleware lets you manage your development environment in a small YAML file that will get loaded in front of your app. Simple, yet effective.

Using rack-environment with Rails

# config/environments/development.rb
config.gem 'rack-environment'
config.middleware.use 'RackEnvironment'

# config/environment.yml
VARIABLE1: value1
VARIABLE2: value2

Using rack-environment with any Rack-based framework

# config.ru
Rack::Builder.new
  use RackEnvironment if ENV['RACK_ENV'] == 'development'
  run MyApplication.new
end

Using Ubuntu's Upstart To Keep Services Running

2009-09-22T00:00:00-04:00

Ubuntu, since Intrepid, has replaced its init services with a daemon called Upstart. Similar to Apple's launchd, it provides a greater granularity to the events occurring on a modern computer.

We can use Upstart to start and stop services, and most importantly, to respawn them automatically if they die. The following is an Upstart script that I wrote to keep my IRC bouncer (bip) running.

The script has the following features:

Has a pre-start stanza to create the appropriate /var/run directory.
Starts and stops in the appropriate runlevels.
Uses sudo -u to run the process as the bip user.
Will respawn in case of process termination.

# /etc/event.d/bip
pre-start script
  mkdir -p /var/run/bip
  chown bip:bip /var/run/bip
end script

start on runlevel 2
start on runlevel 3
start on runlevel 4
start on runlevel 5

stop on runlevel 0
stop on runlevel 6

exec sudo -u bip /usr/bin/bip -n -f /etc/bip.conf -s /var/lib/bip
respawn

Static Pages In Rails

2009-08-22T00:00:00-04:00

In any web application there will be some static content. In the application I'm building for the Rumble, I needed a static landing page, as well as simple FAQ and Contact pages. I wanted simple static files that I could write in Haml (my choice of templating engine for this project).

The controller below allows you to keep static pages in app/views/pages/<page>.html.haml

# app/controllers/pages_controller.rb
class PagesController < ApplicationController

  rescue_from ActionView::MissingTemplate, :with => :invalid_page

  def show
    render params[:id]
  end

  def invalid_page
    redirect_to root_path
  end

end

A custom route allows us to link to static pages using page_url(:welcome).

The :requirements on the route prevent malicious user input.

# config/routes.rb
map.page 'pages/:id',
  :controller   => 'pages',
  :action       => 'show',
  :requirements => { :id => /[a-z]+/ }

Deploying a Rails Application with Capistrano

2009-08-22T00:00:00-04:00

In most of the apps I work on regularly, the Capistrano setup has become a ungainly monster that has evolved over time and over multiple Capistrano and Rails versions. Participating in the Rails Rumble has given me an opportunity to set up a Capistrano configuration from scratch.

This is what I came up with. It will automatically handle the following:

SQLite databases stored in shared/db will be carried across releases
Configuration files stored in shared/config will be symlinked into config
Automatically install all Rubygems needed by the application
Restart via Passenger's tmp/restart.txt

set :application, "application-name"
set :deploy_to,   "/srv/app/#{application}"
set :deploy_via,  :remote_cache
set :repository,  "git@example.org:application.git"
set :scm,         "git"
set :user,        "deploy"

default_run_options[:pty]   = true
ssh_options[:forward_agent] = true

role :app, "server.example.org"
role :db,  "server.example.org", :primary => true

namespace :deploy do
  desc "Restart Application"
  task :restart do
    run "touch #{current_path}/tmp/restart.txt"
  end
end

namespace :gems do
  desc "Install Gems"
  task :install do
    run "cd #{release_path} && sudo env RAILS_ENV=production rake gems:install"
  end
end

namespace :shared do
  desc "Link shared config files"
  task :configuration do
    run %{
      for FILE in $(ls #{shared_path}/config);
      do
        ln -sf #{shared_path}/config/$(basename $FILE) #{release_path}/config/$(basename $FILE);
      done
    }
  end

  desc "Link a shared database directory"
  task :database do
    run %{
      for FILE in $(ls #{shared_path}/db/*.sqlite3);
      do
        ln -sf #{shared_path}/db/$(basename $FILE) #{release_path}/db/$(basename $FILE);
      done
    }
  end
end


after 'deploy:update_code', 'shared:configuration'
after 'deploy:update_code', 'gems:install'
after 'deploy:update_code', 'shared:database'
after 'deploy:update_code', 'deploy:migrate'

ActiveRecord Enumerables

2009-08-22T00:00:00-04:00

As part of the requirements for my Poll model in the Rails Rumble, I have a few columns that need to be locked down to certain values. Rather than repeating validates_presence_of and validates_inclusion_of over and over, I threw together a quick plugin that does both of these as well as adds some convenience methods.

You can find the plugin at http://github.com/ddollar/active-record-enumerable.git

class Food < ActiveRecord::Base
  include ActiveRecordEnumerable

  enumerate :type,  :as => %w( fruit vegetable ), :default => 'foo'
  enumerate :color, :as => %w( red green )
end

If :default is not specified, the first enumerator in the list is used.

>> f = Food.new
=> #<Food id: nil, type: "foo", color: "red">
>> f.save
=> true

>> f.color.red?
=> true

>> f.color.turquoise?
=> false

>> f.update_attributes!(:color => "random")
=> ActiveRecord::RecordInvalid: Validation failed: Color is not included in the list

Deploying a Jekyll Site Via Git Hooks

2009-08-21T00:00:00-04:00

Hello World.

When deciding on Jekyll for this blog, I wanted a way to automagically deploy updates simply by pushing commits. As it turns out, this is quite easy to accomplish using a small post-update git hook.

$ cat /my/repo/hooks/post-update

echo "Deploying Jekyll Site"

# configure these
REPOSITORY_DIR="/my/repo"
DEPLOY_DIR="/var/www/blog"

# dont edit below here
DEPLOY_TMP_DIR=$(mktemp /tmp/jekyll.deploy.XXXXX)
rm -f $DEPLOY_TMP_DIR
git clone $REPOSITORY_DIR $DEPLOY_TMP_DIR
cd $DEPLOY_TMP_DIR
jekyll $DEPLOY_DIR
rm -rf $DEPLOY_TMP_DIR