Steve Lorek

Service Objects: What They Are, and When to Use Them

2013-03-13T00:00:00+00:00

There has been a lot of discussion within the Ruby community around the service objects pattern. The term "service objects" may be unfamiliar to even seasoned Rails developers, because Rails itself implements the Model-View-Controller design pattern. There is no concept of a 'service' in a new Rails application. So it's understandable that this concept may seem foreign, and indeed non-obvious. However, I believe that the service objects pattern is well worth considering for any non-trivial Rails application.

So before I discuss how to implement service objects in Rails, I'll first describe the concepts so you can decide whether it's worth doing so.

What are services?

A 'service' describes system interactions. Usually, these will involve more than one business model in our application.

As an example; we have a User model and this encapsulates a password. If a user has forgotten their password, the business rules dictate that we have to send them an e-mail with a link to reset it. This functionality is a service.

There are other scenarios in which the user's password could be reset. For instance, the system administrator may decide to reset their password on their behalf.

In the above example, the service behaviour fits into the MVC world as follows:

Model (User) - a user has a password. It must be present and a minimum of 8 characters. The model is a representation of the user and doesn't really have an opinion about the different circumstances in which the password could be reset.
View - the 'forgot password' form and success/failure states
Controller - responds to the form in the view being submitted by attempting to instantiate the 'forgotten password' service and then rendering the response (error or success) from the view
Service - responsible for locating the user, sending the reset password e-mail and reporting back to the originator (in this case the controller)

This is the general definition of a service, so it's up to you whether you think this makes sense.

The options

There's several approaches to encapsulating service logic within a Rails application:

Fat model, skinny controller - this principle encourages one to put the service logic in the model. Therefore the model very quickly becomes filled with methods which encapsulate system interactions rather than domain objects. It becomes difficult to identify which methods are domain and which are interactions. Inter-dependancy between model classes can quickly become a problem, which can make refactoring and testing very difficult.
Concerns - these attempt to address the 'overloaded model' syndrome where you have a very large model file, simply by separating methods into modules which are then mixed into the model. It's the approach encouraged in Rails 4 with its' 'models/concerns' folder which is created by default. This can sometimes make sense, but increases the level of abstraction which can make it difficult to figure out where a method is defined. Additionally, it doesn't address the potential inter-dependancy issues that can arise, and frequently you can end up with massive methods, or multiple methods doing similar things to handle several different scenarios. In my view, it's the equivalent of sweeping the dirt under the rug.
Observers and callbacks - these hook into the lifecycle of a model object. For instance, we could create and save a ForgottenPassword object, which has a callback or observer which then sends the e-mail. The disadvantage of this approach is that you are restricted to the lifecycle of persistent objects. So not only do you have to create and persist the ForgottenPassword object for it to work (is this necessary?), but the behaviours can very quickly become difficult to track, especially with observers which are running outside of the request cycle, and callbacks can make things very difficult to test or to predict behaviour.
Fat controller - use the controller to co-ordinate the interactions between the objects. This can be a good approach, but the drawbacks are the controllers can become large enough to be unwieldy, and very difficult to test. Additionally, if you're using the console or a Rake task or background job you'll have to duplicate the functionality there as well.
Service objects

Service Objects explained

Service objects implement individual services, as described above. Following the 'forgotten password' scenario, we might have a class called ForgottenPassword. Here's what it might look like:

class ForgottenPassword
  def initialize(user_id)
    user = User.find user_id
    mail = UserMailer.password_reset @user
    mail.deliver
  end
end

This is a very simple example - as I'm sure you can appreciate this logic can often be far more complex. We might also want to develop an exception handler here and return more logical errors to the controller (or Rake task or whatever is calling it).

Using this service object, we can remove the equivalent User model method, breaking the dependancy between User and UserMailer, and removing any assumption about how the password can be reset from the model. Our tests on the User model just got a lot simpler, and our tests on the ForgottenPassword service are straightforward and segregated too.

The ForgottenPasswordController might look something like this:

class ForgottenPasswordController
  def create
    begin
      ForgottenPassword.new(params[:user_id])
      # render the view
    rescue
      # Handle exceptions and render the view
    end
  end
end

Again, our tests on the controller will be very simple and you don't run the risk of turning them into integration tests.

When to use them

In my view, the full benefits of service objects are realised when modelling all interactions as discrete service objects. Doing so will enable you to segregate your testing into very discrete and logical areas.

However, you will see these benefits by adopting the approach for new functionality in an existing application, too. Developing and maintaining that functionality will be easier.

Conclusion

Services are a concept which Rails doesn't really recognise out of the box. But I feel that the benefits of doing so are well worth it. While it can result in slightly more code in a simple app (although the above example demonstrates this can be fairly trivial), I've found it helps write tests faster and produces more maintainable code. I hope you'll consider using them.

Installing Ruby 2.0.0 and Rails 4.0.0 beta 1 on Mountain Lion with RVM

2013-03-02T00:00:00+00:00

Celebrating Ruby's 20th anniversary, both Ruby 2.0.0 and Rails 4.0.0 beta 1 were released last week. Naturally, we will want to get playing with these new tools right away, but installation has not been straightforward for some.

Here's how I (eventually) managed to get the pair running on OS X Mountain Lion:

Install dependancies

Most of these problems seem to revolve around the openssl dependancy and RVM's default process for upgrading it. To install and use openssl 1.0.1e successfully, I found homebrew did the trick. Follow these steps to install it (and libyaml, the other dependancy).

$ brew update
...
$ brew install libyaml openssl

Update RVM

We need to upgrade RVM to install Ruby 2.0.0.

$ rvm get head

Install Ruby 2.0.0

You may be reading this because you've already tried installing it, so let's take some precautionary cleanup measures.

$ rvm remove ruby 2.0.0
...
$ rvm pkg remove

Now you should be ready to install and use Ruby 2.0.0:

$ rvm install ruby
...
$ rvm use 2.0.0

Check for any error messages in the log. At this point, there should be none.

Install Rails 4

As it's still in beta, you'll need to specify the version of the Rails gem to install:

$ gem install rails --version 4.0.0.beta1 --no-ri --no-rdoc

Again, this should now install faultlessly. Let me know if you have any problems.

How to Shrink a Git Repository

2012-05-11T00:00:00+00:00

Our main Git repository had suddenly ballooned in size. It had grown overnight to 180MB (compressed) and was taking forever to clone.

The reason was obvious; somebody, somewhere, somewhen, somehow, had committed some massive files. But we had no idea what those files where.

After a few hours of trial, error and research, I was able to nail down a process to:

Discover the large files
Clean them from the repository
Modify the remote (GitHub) repository so that the files are never downloaded again

This process should never be attempted unless you can guarantee that all team members can produce a fresh clone. It involves altering the history and requires anyone who is contributing to the repository to pull down the newly cleaned repository before they push anything to it.

Deep Clone the Repository

If you don't already have a local clone of the repository in question, create one now:

$ git clone remote-url

Now—you may have cloned the repository, but you don't have all of the remote branches. This is imperative to ensure a proper 'deep clean'. To do this, we'll need a little Bash script:

#!/bin/bash
for branch in `git branch -a | grep remotes | grep -v HEAD | grep -v master`; do
    git branch --track ${branch##*/} $branch
done

Thanks to bigfish on StackOverflow for this script, which is copied verbatim.

Copy this code into a file, chmod +x filename.sh, and then execute it with ./filename.sh. You will now have all of the remote branches as well (it's a shame Git doesn't provide this functionality).

Discovering the large files

Credit is due to Antony Stubbs here - his Bash script identifies the largest files in a local Git repository, and is reproduced verbatim below:

#!/bin/bash
#set -x 

# Shows you the largest objects in your repo's pack file.
# Written for osx.
#
# @see http://stubbisms.wordpress.com/2009/07/10/git-script-to-show-largest-pack-objects-and-trim-your-waist-line/
# @author Antony Stubbs

# set the internal field spereator to line break, so that we can iterate easily over the verify-pack output
IFS=$'\n';

# list all objects including their size, sort by size, take top 10
objects=`git verify-pack -v .git/objects/pack/pack-*.idx | grep -v chain | sort -k3nr | head`

echo "All sizes are in kB. The pack column is the size of the object, compressed, inside the pack file."

output="size,pack,SHA,location"
for y in $objects
do
	# extract the size in bytes
	size=$((`echo $y | cut -f 5 -d ' '`/1024))
	# extract the compressed size in bytes
	compressedSize=$((`echo $y | cut -f 6 -d ' '`/1024))
	# extract the SHA
	sha=`echo $y | cut -f 1 -d ' '`
	# find the objects location in the repository tree
	other=`git rev-list --all --objects | grep $sha`
	#lineBreak=`echo -e "\n"`
	output="${output}\n${size},${compressedSize},${other}"
done

echo -e $output | column -t -s ', '

Execute this script as before, and you'll see some output similar to the below:

All sizes are in kB. The pack column is the size of the object, compressed, inside the pack file.
size     pack    SHA                                       location
1111686  132987  a561d25105c79aa4921fb742745de0e791483afa  08-05-2012.sql
5002     392     e501b79448b9e970ab89b048b3218c2853fdfc88  foo.sql
266      249     73fa731bb90b04dcf79eeea8fdd637ba7df4c089  app/assets/images/fw/iphone.fw.png
265      43      939b31c563bd40b1ca70e4f4a9f7d67c27c936c0  doc/models_complete.svg
247      39      03514d9e84418573f26b205bae7e4e57057c036f  unprocessed_email_replies.sql
193      49      6e601c4067aaddb26991c4bd5fbddef003800e70  public/assets/jquery-ui.min-0424e108178defa1cc794ee24fc92d24.js
178      30      c014b20b6fed9f17a0b2809ac410d74f291da26e  foo.sql
158      158     15f9e56bc0865f4f303deff053e21909661a716b  app/assets/images/iphone.png
103      36      3135e15c5cec75a4c85a0636b154b83221020c97  public/assets/application-c65733a4a64a1a885b1c32694574b12a.js
99       85      c1c80bc4c09e692d5e2127e39c87ecacdb1e816f  app/assets/images/fw/lovethis_logo_sprint.fw.png

Yep - looks like someone has been pushing some rather unnecessary files somewhere! Including a lovely 1.1GB present in the form of a SQL dump file.

Cleaning the files

Cleaning the file will take a while, depending on how busy your repository has been. You just need one command to begin the process:

$ git filter-branch --tag-name-filter cat --index-filter 'git rm -r --cached --ignore-unmatch filename' --prune-empty -f -- --all

This command is adapted from other sources—the principal addition is --tag-name-filter cat which ensures tags are rewritten as well.

After this command has finished executing, your repository should now be cleaned, with all branches and tags in tact.

Reclaim space

While we may have rewritten the history of the repository, those files still exist in there, stealing disk space and generally making a nuisance of themselves. Let's nuke the bastards:

$ rm -rf .git/refs/original/

$ git reflog expire --expire=now --all

$ git gc --prune=now

$ git gc --aggressive --prune=now

Now we have a fresh, clean repository. In my case, it went from 180MB to 7MB.

Push the cleaned repository

Now we need to push the changes back to the remote repository, so that nobody else will suffer the pain of a 180MB download.

$ git push origin --force --all

The --all argument pushes all your branches as well. That's why we needed to clone them at the start of the process.

Then push the newly-rewritten tags:

$ git push origin --force --tags

Tell your teammates

Anyone else with a local clone of the repository will need to either use git rebase, or create a fresh clone, otherwise when they push again, those files are going to get pushed along with it and the repository will be reset to the state it was in before.

Configure Postfix on OS X Lion to Relay via Gmail

2011-12-28T00:00:00+00:00

Today I had to test e-mail delivery of an app I was working on from home. Annoyingly, my ISP (O2) block outgoing port 25 because they don't trust their users not to run open mail relays. This meant that e-mails sent from my app were not being delivered.

So I needed to set up Postfix (the native Sendmail implementation running on OS X Lion) to relay all outgoing mails to another mail server. It turned out to be quite difficult to configure Postfix to connect to Gmail.

Gmail requires an authenticated TLS session to relay mail. But the instructions I found across the web did not seem to work, in that the Gmail CA certificates were not trusted even after adding them.

The easy solution was to simply stop Postfix from requiring a trusted CA altogether. Below are instructions to replicate my setup.

First edit the Postfix configuration file as below:

$ sudo vim /etc/postfix/main.cf

Edit the file as below:

relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_mechanism_filter = plain
smtp_use_tls = yes
smtp_tls_security_level = may

The 'may' security level settings tells Postfix to ignore untrusted CAs and continue delivery.

Create a new file containing your Gmail account credentials:

$ sudo vim /etc/postfix/sasl_passwd

[smtp.gmail.com]:587 USERNAME@DOMAIN:PASSWORD

Then load the account credentials into Postfix:

sudo postmap /etc/postfix/sasl_passwd
sudo rm /etc/postfix/sasl_passwd

Then restart Postfix:

sudo launchctl unload -w /System/Library/LaunchDaemons/org.postfix.master.plist
sudo launchctl load -w /System/Library/LaunchDaemons/org.postfix.master.plist

Now you can send a test e-mail:

mail -s "Testing, Testing" steve@stevelorek.com

Type the body of your message then Ctrl-D to send the e-mail.

Check the /var/log/mail.log to see the status of your delivery.

Resque Worker Management with God

2011-05-06T00:00:00+00:00

God is a process management tool, which can watch your Resque workers, ensuring they're running and spawn additional workers as necessary in the background.

We install God as below:

$ sudo gem install god

God uses configuration files for each of the processes it monitors. The way we're going to set it up is to have a single directory, /etc/god, containing all of the scripts we want to be running. Any file named .god will be run by the initialisation script we create later.

$ mkdir /etc/god

Resque comes with an example configuration file which is a good basis for writing your own app-specific configuration.

When you've got your configuration file, you can run God with the following command:

$ rvmsudo god -c /etc/god/resque.god -D

Running God Automatically

In a production environment, you'll want God to be running all of the time on system boot. We will need an init script to get this working, in /etc/init.d/god:

#!/bin/bash
#
# God
#
# chkconfig: - 99 1
# description: start, stop, restart God (bet you feel powerful)
#

# source function library
. /etc/rc.d/init.d/functions

RUBY_PATH="/usr/bin/ruby"
PATH=$RUBY_PATH:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON="/usr/bin/god"
PIDFILE=/var/run/god.pid
LOGFILE=/var/log/god.log
SCRIPTNAME=/etc/init.d/god
CONFIGFILEDIR=/etc/god

#DEBUG_OPTIONS="--log-level debug"
DEBUG_OPTIONS=""

# Gracefully exit if 'god' gem is not available.
test -x $DAEMON || exit 0

RETVAL=0

god_start() {
  start_cmd="$DAEMON -l $LOGFILE -P $PIDFILE $DEBUG_OPTIONS"
  #stop_cmd="kill -QUIT `cat $PIDFILE`"
  echo $start_cmd
  $start_cmd || echo -en "god already running"
  RETVAL=$?
  if [ "$RETVAL" == '0' ]; then
    sleep 2 # wait for server to load before loading config files
    if [ -d $CONFIGFILEDIR ]; then
      for file in `ls -1 $CONFIGFILEDIR/*.god`; do
        echo "god: loading $file ..."
        $DAEMON load $file
      done
    fi
  fi
  return $RETVAL
}

god_stop() {
  stop_cmd="god terminate"
  echo $stop_cmd
  $stop_cmd || echo -en "god not running"
}

case "$1" in
  start)
    god_start
    RETVAL=$?
    ;;
  stop)
    god_stop
    RETVAL=$?
    ;;
  restart)
    god_stop
    god_start
    RETVAL=$?
    ;;
  status)
    $DAEMON status
    RETVAL=$?
    ;;
  *)
    echo "Usage: god {start|stop|restart|status}"
    exit 1
    ;;
esac

exit $RETVAL

Make sure you change the paths at the beginning of the file to match your server's environment. To install the script, issue the following commands:

$ sudo chmod +x /etc/init.d/god
$ sudo touch /var/log/god.log
$ sudo update-rc.d -f god defaults

Congratulations; your workers are now fully managed by God!

Installing Pygments 1.4 on MacOS X

2011-03-01T00:00:00+00:00

I'm using Jekyll to power my blog. It's a very simple framework which lends itself perfectly to developers used to coding files and the Git workflow, and comes without any of the bloat required for a client-facing CMS.

Jekyll comes with built-in support for Pygments; a syntax highlighter supporting hundreds of languages. This article details the officially supported method for installing Pygments 1.4 on Mac OS X Snow Leopard. You will need to have Python installed already—this is provided by the Mac OS X Developer Tools—and you'll need to be logged into an administrator account.

First, establish the version of Python you have:

$ python --version
Python 2.6.1

Download the Setuptools package which corresponds to your version of Python. In this case, it will be setuptools-0.6c11-py2.6.egg.

Run the Setuptools package as root:

$ sudo sh setuptools-0.6c11-py2.6.egg
...
Finished processing dependencies for setuptools==0.6c11

This will add a new command you can run: easy_install. With this we can install Pygments (again as root):

$ sudo easy_install Pygments
...
Finished processing dependencies for Pygments

Now, in our config.yml file for our Jekyll blog, we can add:

pygments: true

This will add support for syntax highlighting using the {% highlight %} liquid tags. Enjoy!

Incorporating ActionMailer in Cucumber Tests

2011-02-28T00:00:00+00:00

It is very common for Ruby applications to send transactional e-mails to users. What's not immediately obvious is how, as developers, we can test this functionality using our tools of choice: Cucumber and Rspec.

Ideally, the method should be re-usable so that we can simply add a single step to our scenarios, e.g:

And it should send me an "account confirmation" e-mail

One way to do this in a re-usable way is to add a new tag, @email, which sets up ActionMailer. In your features/support/env.rb file, add the following:

Around('@email') do |scenario, block|
  ActionMailer::Base.delivery_method = :test
  ActionMailer::Base.perform_deliveries = true
  ActionMailer::Base.deliveries.clear
  block.call
end

This ensures ActionMailer is in test mode (all e-mails will be added to an array we can inspect, rather than sent out to the real world), and it clears any other e-mails in the deliveries array before each step is executed. Now we can simply add the @email tag before any scenario which needs to test e-mail functionality and this will be done for us automatically.

Then we create a new features/step_definitions/email_steps.rb file and add the following step definition:

Then /^it should send me an? "([^\"]*)" e\-mail$/ do |arg1|
  @email = ActionMailer::Base.deliveries.last
  @email.to.should include @user.email
  @email.subject.should include(arg1)
end

This will test the subject line against the argument "account confirmation" in your scenario, so we could use this step for other types of e-mail; e.g. "password reminder".

We can also test that the application doesn't send an e-mail, for example:

Then /^it should not send an e\-mail$/ do
  ActionMailer::Base.deliveries.size.should eq 0
end

Here's an example of the e-mail testing functionality in action:

@email
Scenario: Creating an account
  Given I am logged out
  Then I should be able to register a new user account
  When I enter my details
  Then it should create a new account for me
  And it should send me an "account confirmation" e-mail

I hope this helps someone!

Associated Records with factory_girl

2011-02-12T00:00:00+00:00

factory_girl is a Ruby Gem which makes it easy to build re-usable data objects in a test suite. It works especially well with ActiveRecord models.

I was recently working on adding a new feature to a project which contained some factory implementations written by a colleague, but the Cucumber steps kept failing where it was checking for associated records. This was because the associated records were set up incorrectly in the factory.

Below is an example of how we build associated records the right way. First is the parent model:

Factory.define(:booking) do |f|
  f.budget 12.40
  f.date 2.weeks.from_now
  f.number 23
  
  f.after_build do |b|
    b.menu_items = [Factory.build(:menu_item, :booking => b)]
  end
end

Note the way we add child records in the after_build block, and how we're passing the current object to the factory. And now for the child model:

Factory.define(:menu_item) do |f|
  f.description "Roast Beef"
  f.sale_price 8.00
  f.association :booking
end

The problem with my colleague's implementation was the build message didn't include the booking attribute, therefore factory_girl was creating another Booking object to meet the MenuItem's association specification. So each time the factory was run, two Bookings were created, and the MenuItem was associated with the wrong record, not the one returned by the factory.

Now that we've specified our implementations correctly, we can build both Bookings and MenuItems individually.

Much head scratching was caused by that one—hopefully this refresher will save someone the same!

Speed Testing with Spork

2011-02-01T00:00:00+00:00

Test-Driven and Behaviour-Driven Development (TDD and BDD) have seen massive rates of adoption among the Ruby community, chiefly thanks to its close integration in the Rails framework and the fast pace of tool development. The benefits of these methodologies are plainly obvious to many experienced Software Developers and Project Managers.

One set of tools has established itself as the 'preferred' set among the Ruby community; Rspec, which tests object specifications, and Cucumber, which tests user interface behaviour. You may be using other tools in addition to these. Today I'm going to give you a tip to help you optimise your workflow by making testing faster, using another tool called Spork.

Spork is a small server application which essentially eliminates the long delay in starting up your test suite. Typically, running rake spec or rake features results in a delay of several seconds before testing actually begins. The reason for this is the application environment is being loaded from scratch each time. Spork eliminates this delay by loading it once and then 'forking' itself each time you run your tests. The result: tests begin immediately.

Installing Spork in your Application

This tutorial is based on a Rails 3 application. First, edit your Gemfile to include the Spork dependancy:

gem 'spork', '>= 0.9.0.rc2'

Note: we require at least version 0.9.0.rc2 because it addresses an issue when using Shoulda. If you're not using Shoulda, you can drop the version requirement.

Then update your bundle:

$ bundle install

The final thing we need to do is alter the test environment's behaviour. Unlike in development mode, the test mode will cache your classes, essentially meaning that any changes you make during development will not be reflected automatically when running tests with Spork. So, in the config/environments/test.rb file, change the following line as shown:

config.cache_classes = false

Using Spork with an RSpec Test Suite

We need to configure Spork to tell it what parts of the test suite initialisation it can handle. Spork provides a convenient command to make this easy:

$ bundle exec spork --bootstrap

This will add some extra lines to the spec/spec_helper.rb file. Basically, what you need to do now is move the existing code underneath into the relevant blocks; those which need to happen only once on initialisation, and those which need to happen each time the tests are run. In many simple cases you will be able to copy everything into the Spork.prefork block. But you must be careful here not the break anything.

Now you're ready to start the Spork server:

$ bundle exec spork

Open another shell session and rather than rake spec, use the following command:

$ bundle exec rspec --drb spec

Your tests should now run instantly without delay! Note we don't run rake because it will force a load of the Rails application anyway, rendering the savings we've made worthless.

You may also add the --drb switch to your .rspec file—in which case, you can drop this from your command.

Spork with Cucumber

Cucumber comes with support for Spork as well. You will need to do some editing of your features/support/env.rb file to add the two Spork blocks, as per the spec/spec_helper.rb file. The same rules apply.

Bear in mind that all hooks (e.g. Before, After) need to be in the each_run block. You can refer to this gist and my example below for further guidelines:

require 'spork'

Spork.prefork do
  ENV["RAILS_ENV"] ||= "test"
  require File.expand_path(File.dirname(__FILE__) + '/../../config/environment')
  
  require 'cucumber'
  require 'cucumber/formatter/unicode'
  require 'cucumber/rails/rspec'
  require 'capybara/rails'
  require 'capybara/cucumber'
  require 'capybara/session'
  require 'factory_girl/step_definitions'

  Capybara.default_selector = :css
end

Spork.each_run do
  require 'cucumber/rails/world'
  require 'cucumber/rails/active_record'
  require 'cucumber/web/tableish'
  require 'simplecov'
  require Rails.root.join('db/seeds')

  SimpleCov.start 'rails'
  
  Before do
    Seeds.load
  end

  ActionController::Base.allow_rescue = false

  Cucumber::Rails::World.use_transactional_fixtures = false

  if defined?(ActiveRecord::Base)
    begin
      require 'database_cleaner'
      DatabaseCleaner.strategy = :truncation
    rescue LoadError => ignore_if_database_cleaner_not_present
    end
  end
end

We can now run the Spork server. However, the command is different to the Rspec example:

$ bundle exec spork cuc

You must use the cuc argument to run Spork with Cucumber. Now, in a new shell session you can run Cucumber as below:

$ bundle exec cucumber --drb

Again, we don't run rake features because Rake would initialise our Rails application on its own anyway. Your features should now be tested instantly!

Security Issue in Mail Gem

2011-01-26T00:00:00+00:00

This is an important one for anyone using Rails 3 in a production environment who hasn’t yet updated the Mail gem to version 2.2.15 or later.

A project I’m working on at the moment sends transactional e-mails via the sendmail delivery method. Some of the recipients’ e-mail addresses contain the ampersand (&) character in the local part (the bit before the @ symbol).

We started being notified of strange exceptions when the application was sending to these addresses. A quick test in the console revealed this:

ruby-1.9.2-p0 > mail.to = 'test.&.test@stevelorek.com'
 => "test.&.test@stevelorek.com" 
ruby-1.9.2-p0 > mail.deliver
sh: .test@stevelorek.com: command not found

Clearly, at some point the e-mail address is being output to the shell directly, without any escaping. We can see that the ampersand is acting as a command splitter, so everything after the ampersand is treated as a new command.

It’s good practise to sanitise and validate any user input anyway – indeed, many validations will erroneously reject an ampersand in the local part anyway. However, this still represents a serious security risk to Rails 3 applications.

The culprit was the Mail gem which Rails depends on to handle e-mail delivery. The lines responsible for this error are in lib/mail/network/delivery_methods/sendmail.rb of version 2.2.14:

def Sendmail.call(path, arguments, destinations, mail)
  IO.popen("#{path} #{arguments} #{destinations}", "w+") do |io|
    io.puts mail.encoded.to_lf
    io.flush
  end
end

After discussing this with Andy Lindeman we reported the bug to the gem’s author, Mikel Lindsaar, who quickly applied Andy’s patch and issued an announcement which was copied to the Ruby on Rails security group.

The patch also had the nice effect of making my application work with all valid e-mail addresses.

I would advise that everyone in a production environment update the Mail gem to version 2.2.15 immediately:

$ bundle update mail

Introduction

2011-01-26T00:00:00+00:00

Firstly, a big welcome and many thanks for taking the time to check out my blog.

I’m a professional Web Developer working at Willoughby Stewart. I’ve been working in the industry for 16 years, starting when the Web was in its infancy and the term “Web design” meant Times New Roman or nothing! To begin with I used the Web for entrepeneurial benefit—from launching my first commercial project at the age of 14 to running a successful e-commerce business for 6 years. Web development was purely a means to an end – as it should be – but has always been a passion.

This is actually my first ‘proper’ personal site (if you don’t count the ‘Doom II’ fan site holding page I made in 1995). I’m not quite sure why that is—but as an entrepreneur I’ve always invested all of my energies into those ventures, learning new skills along the way. I’ve never really felt the need to express my personal views to the Web development community, nor have I felt confident enough to teach others as I’ve almost existed inside my own ‘bubble’. Now that I’ve been working as a Developer for a few years I feel that perspective has changed and I can offer something genuinely unique and helpful.

I hope to offer a considered view of the industry in this blog. While development – whether it’s programming, version control, or design, and everything else – is certainly art, my motivator has always been to ask myself “what do I need to learn to achieve X, Y and Z?”

I can give you two personal examples of this philosophy. When I was 16, I wanted to emulate the success of Amazon and other, now defunct retailers, so I looked into how to set up an online shop. My research showed that the off-the-shelf solutions of the time were not fit for purpose, so I learned how to build one myself.

I had no programming experience or training. I’d had some minor exposure to open source Perl scripts from a previous site (the online presence of a national magazine), so I started hacking one of the free open source cart scripts, little changes here and there, and then more fundamental ones. All the while I was wishing “if only I could do this”—and the more I dug through forums and HotBot search, the more it seemed PHP 3 was the answer to these problems. So I rewrote my shopping cart in PHP. About a dozen rewrites and a couple of years later I naturally ended up with something akin to a MVC design pattern. I also learned a hell of a lot about what it takes to set up a retail business online, which I’m now using for the benefit of my clients.

Where do I want to be in 5 years? I love Web development, but how can I satisfy my entrepreneurial urges? The answer is to make myself the most valuable asset I can be, by expanding my skills and hopefully stepping up the career ladder, and hopefully I’ll be in a great position when the next big idea arrives. What skills are most in demand? I’ve decided to build on my Ruby and Rails skills and see if I can transfer my experience to iOS development. But at the same time I need to keep my front-end design skills relevant, and that means HTML 5, jQuery and CSS 3.

This blog will be a mixture of technical articles on the subject of Web development, commentary on the industry, and hopefully some advice to other Developers hoping to start their careers. I hope that sounds good to you.

Also, a note on the design; this is the one area that’s held me back for so long. I’m a massive perfectionist. So I’m starting off simple on purpose, because it’s the content that really matters.