The Thoughts of Robots - Home

Add an ActiveRecord scope for easier eager loading

2009-10-28T22:18:00Z

In the optimization phase of a project, one of the first things I do is look for pages that could benefit from the use of ActiveRecord’s eager loading. Since named_scopes were introduced in Rails 2.1, most of my model loading has involved a chain of scopes, and it is always frustrating to have to change

@blogs = Blog.published.active

@blogs = Blog.published.active.scoped(:include => [:slugs, :topic => :slugs])

It’s a small difference, but one I’ve never really liked. It seems like an obvious choice would be to add a named_scope to all subclasses of ActiveRecord::Base and have that handle the :include parameter. I finally wrote the 11 lines of code required to do this today:

module ActiveRecord
  module EagerLoadScope
    def self.included(base)
      base.named_scope :eager_load, lambda { |*inclusion|
        {:include => inclusion}
      }
    end
  end
end

ActiveRecord::Base.class_eval { include ActiveRecord::EagerLoadScope }

And now that line looks like this:

@blogs = Blog.published.active.eager_load(:slugs, :topic => :slugs)

Again, it’s a small improvement, but one I consider worthwhile.

Hiding parts of a view while in development

2009-09-11T16:30:00Z

Most of my recent projects have involved working with a front end developer, who would provide HTML/CSS templates with dummy content to be integrated into a Rails backend. During the process of wiring up the views, I often wish to hide a section of a template or partial until the models and controllers are setup to support them.

I’ve started using this little view helper to hide code in a view, while providing a console message to remind me to go back and address the issue when the time is right:

def hide(&block)
  if Rails.env == 'development'
    match, file, line = caller.first.match(/([^:]+):(\d+).+/).to_a
    Rails.logger.info "** Hiding content on line #{line} of #{file}"
  end
end

It’s simple to use, especially in HAML:

- hide do
  = some_view_code_that_will_not_execute

The helper will print something like this to your console:

** Hiding content on line 14 of /Users/jimb/Projects/demo/app/views/objects/show.html.haml

Dollar Spec is a tiny JavaScript spec library

2009-09-04T06:20:00Z

Dollar spec is a tiny JavaScript spec framework that gets out of the way and lets you get things done. It supports nested describes and before/after blocks, but that is about it. The standard set of expectations is small right now, but they are easy to add.

Why another JavaScript spec framework?

I wanted a small, pure-JS framework that supported a syntax I didn’t have to think too much to use. I wanted to only add as much syntax as was neccessary, and leave the rest to JavaScript. I wanted to run JavaScript tests from the console. I wanted a really extensible way to add new expectations. And I wanted minimal namespace usage. DollarSpec creates a single global entity, $spec.

Also, it seemed like a good way to stretch my brain. I’ve been doing a lot of JavaScript development recently, and it was fun to apply some of the new techniques I’ve picked up.

OK Let’s See What You’ve Got

Here’s a sample spec for an awesome addition function:

$spec.describe('my awesome addition function', function(spec) {

    var add = function() {
       var total = 0;
       for (var i = 0, l = arguments.length; i < l; i++) {
           total += arguments[i];
       }
       return total;
    };

    spec.before(function() {
        // code in this block is run before each spec
        // and yes, describe blocks can be nested
    });

    spec.it('adds two numbers', function(expect) {
        expect(add(2,2)).to.equal(4);
    });

    spec.it('adds three numbers', function(expect) {
        expect(add(1,1,2)).to.equal(4);
    });

    // to show a failing spec
    spec.it('adds two numbers', function(expect) {
        expect(add(1,1,2)).to.equal(5);
    });

    // to show a pending spec
    spec.it('casts and adds strings');
});

$spec.run();

If you run this inside Firefox, you will get some messages in the Firebug console:

These message can be adjusted to your liking:

// To disable console output (defaults to true)
$spec.opts.console = false;

// Print a line to the console for each test (defaults to false)
$spec.opts.verbose = true;

If you don’t want to use the console reporting, the results of all specs are collected and returned by $spec.stats(). This makes it easy to do whatever you wish with the results.

Supported expectations

Keep in mind that any expectation can be negated using not(). For example:

expect(3).to.not().equal(8);

expect(actual).to.be(object)

Compares the expected and actual values using ===.

expect(actual).to.beA(Object)

Verifies that actual is an instance of Object.

expect(actual).to.equal(expected)

Compares actual and expected using ==.

expect(affectorFunction).to.change(affectedFunction)

Compares the value returned from calling affectedFunction before and after calling affectorFunction.

expect(affectorFunction).to.change(affectedFunction).by(amount)

Compares the difference between the value returned from calling affectedFunction before and after calling affectorFunction to amount.

expect(callbackFunction).to.raiseError([verifyFunction])

Verifies that callbackFunction raises an exception when called.

Optionally will call verifyFunction and pass in a raised exception for further inspection. verifyFunction must return true or false.

Custom expectations

DollarSpec’s expectations are built from two pieces- verbs and matchers.

Verbs

A verb is a function that can be used in an expectation to define something about the parameters of what is expected. The most basic verbs simply define which matcher to use. Verbs must always return this in order to allow chaining to work. Here is the verb definition for equal:

$spec.verb('equal', function(expected) {
    this.set('matcher', 'equal');
    this.set('expected', expected);
    return this;
});

This verb will cause the expectation to use the equal matcher, and stores the value of expected so it can be examined in the matcher later.

Matchers

Matchers are functions that determine if an expectation passes or fails. They can also define messages to be displayed to the user when a test fails. Here’s the beA matcher:

$spec.matcher('beA', function(result) {
  result.failure = "Expected an instance of " + this.klass.toString() + ", but it was " + typeof(this.actual);
  result.negatedFailure = "Expected instance of a class other than " + this.klass.toString() + ", but it was one";

  return this.actual instanceof(this.klass);
});

result.negatedFailure is the message to be displayed when the tests passes, but was used in a negated expectation using not().

You can see how the built-in verbs and matchers are implemented in the Github repository.

Dogfood for Dinner

Dollar Spec has a suite of specs, written using Dollar Spec and powered by diligence, a tiny remote JavaScript test runner library. It requires Node.js, which is awesome. Go check it out if you haven’t yet.

To run the specs:

cd spec
node suite.js

Open a browser and go to localhost:5678. You should see something like the following:

Diligence is still under development, but its working well so far on a few JavaScript projects I’m working on. I’ll post more about it as it matures.

More information

There is a bit more info in the README on Github. Feedback is always welcome!

Alternatives

You should probably take a look at ScrewUnit if you’re looking for a more complete/mature BDD framework.

Now go write some specs.

Install Passenger and nginx from source on Ubuntu

2009-06-19T20:11:00Z

Passenger 2.2.3 came out this week, and while up until now I was using the Ubuntu packages from Brightbox (great blokes, really), I was having some problems with serving caches files and I didn’t want to wait for the package to be updated. The package itself can be found on Github if you’d like to see it in more detail.

I used a combination of two guides to get everything working: one at Slicehost’s wiki and another blog post by someone whose name may or may not be Johnny Pez.

It can be a little confusing to jump back and forth between the two, so I’ve posted a compiled set of steps here. After working through this article, you can continue on with the second Slicehost article for setting up the rest of your environment.

Passenger

I’m going to assume you have Ruby and RubyGems installed. If not, do that first before continuing.

Install the Passenger gem:

sudo gem install passenger

Then you need to build the nginx extension, which will be needed when we compile it.

cd `passenger-config --root`
sudo rake nginx

nginx

You’ll need to install nginx’s prerequisites:

sudo apt-get install libpcre3 libpcre3-dev libpcrecpp0 libssl-dev

Then, download the nginx source and unpack it. Everyone seems to be using the legacy stable version of nginx, and I haven’t tried any of the newer ones yet.

wget http://sysoev.ru/nginx/nginx-0.6.37.tar.gz
tar xzvf nginx-0.6.37.tar.gz

Now we need to configure nginx.

cd nginx-0.6.37
./configure --sbin-path=/usr/local/sbin --with-http_ssl_module  \
--add-module=`passenger-config --root`/ext/nginx

You can change the configure flags if you want (see the comments on the Slicehost article), but I’ve left them alone so the rest of Slicehost’s articles can be followed without modification.

Assuming the configure succeeded, all that’s left is to compile and install.

make
sudo make install

All Systems Go

To make sure everything is working, start nginx:

sudo /usr/local/sbin/nginx

Visit your server’s IP, and you should see the nginx welcome screen. This is good! Kill nginx:

sudo kill `cat /usr/local/nginx/logs/nginx.pid`

Now continue on with the Slicehost series to complete your nginx configuration. You’ll also probably want to mirror the default configuration layout (these articles specify Intrepid, but the steps will be pretty close on most recent versions of Ubuntu).

Interrogation creates boolean accessors from named_scope declarations

2009-06-05T05:51:00Z

This is a little hack I came up with today when I got tired of duplicating logic in scopes and accessor methods. If I define a method in a model like this:

class Submarine < ActiveRecord::Base
  named_scope :on_surface, :conditions => {:location => 'surface'}
  named_scope :submerged, :conditions => {:location => 'under water'}
end

I think I should be able to interrogate the model to see if it is in these scopes without needing to define Submarine#on_surface? or Submarine#submerged? This is what I want to be able to do:

sub = Submarine.first => #<Submarine id: 1, location: 'surface'>
sub.on_surface? => true
sub.submerged? => false

Interrogation will allow for exactly this. The first version would fire a database query on each call, but I’ve since added in an attempt at parsing simple conditions against a model’s attributes, with a fallback to the database for complicated conditions.

One last thing

The largest problem with this approach is an object’s state is not always in sync with the database. I’ve thought about this a lot, and at this point I think it’s best to raise an exception when using these methods on dirty objects. I think handling dirty objects is beyond the scope of this little hack.

The code is on Github.

Duplicate a Mac OS X Terminal window with Ruby and Appscript

2009-05-30T17:07:00Z

Even though OS X’s Terminal now supports multiple tabs, I still find myself using lots of separate windows so I can see everything at once. A common pain point was opening a second Terminal window and changing to the current working directory (I do this a lot from the root of Rails apps: one window for running the server, and one for git and other work). I’ve since written a little Ruby script that essentially duplicates the current Terminal window. Save the following as dup and put it somewhere on your path:

#!/usr/bin/env ruby
#
# Duplicates a terminal window, optionally running the passed command
# % dup (creates new window)
# % dup command arg1 arg 2 (new window, plus runs passed command)

require 'rubygems'
require 'appscript'

include Appscript

def quote(command)
  command.gsub(/(\$`\\!")/){|m|"\\\\#{m}"}
end

path = ENV['PWD']
terminal = app('Terminal')
coords = terminal.windows.first.position.get
dimensions = terminal.windows.first.size.get

command = "cd \"#{quote(path)}\" && clear"
command << " && #{quote(ARGV.join(' '))}" unless ARGV.empty?

terminal.do_script(command)
terminal.windows.first.size.set([dimensions[0], dimensions[1]])
terminal.windows.first.position.set([coords[0] + dimensions[0] + 10, coords[1]])

Appscript is required, but it is only a sudo gem install appscript away. The script will pass and arguments passed to it to the new terminal window. One common use for me is dup script/server.

Reloading Rails plugins in development mode

2009-04-23T18:54:00Z

In Rails 2.3 you can cause your plugins to be reloaded with each request by adding this to your development.rb file:

config.reload_plugins = true

This was possible before, but it required altering Dependencies.load_once_paths and other black magic. I’ve always had issues with these approaches, and so it’s nice to see official support for such a useful feature.

Carmen: A Rails plugin for geographic names and abbreviations

2009-04-01T00:54:00Z

On a recent project I became tired of state select and country select, the ubiquitous Rails plugins for providing a list of state or country names in views. I didn’t like that their data was defined in code. Or that their data was defined in the view, so validating a model field against them was messy.

I started to refactor country select by moving the list of countries into a Geography module. But then I needed a list of states, and their abbreviations, and it struck me as odd that I needed two plugins that worked differently in order to provide what seemed to be some very basic functionality to my app.

Well she sneaks around the world, from Kiev to Carolina…

Carmen is the result of my frustration: a small unified library that handles country and state names and abbreviations. It stores its data lists in YAML, so it’s easy to edit and add new state lists.

Carmen’s primary use is providing a list of countries (or states within country), and, optionally, their abbreviations.

Carmen::states('US') => [['Alabama', 'AL'], ['Arkansas', 'AR'], ... ]
Carmen::state_names('US') => ['Alabama', 'Arkansas', ... ]
Carmen::state_codes('US') => ['AL', 'AR', ... ]

Similar methods are available for countries. It also has some convenience methods to assist in converting between names and abbreviations:

Carmen::country_name('US') => 'United States'
Carmen::country_code('Canada') => 'CA'
Carmen::state_code('Manitoba', 'CA') => 'MB'
Carmen::state_name('AZ', 'US') => 'Arizona'

Any of the state methods that require a country code will use a default country code if none is supplied. This defaults to ‘US’, but can be easily changed for those outside the US:

Carmen.default_country = 'CA'

And of course, it’s simple to use Carmen in your model validations:

class Address < ActiveRecord::Base
  validates_inclusion_of :country, :in => Carmen::country_codes
end

Carmen also supplies view helpers that work the same as those in country select or state select, so it should be a drop in replacement in most cases. I’ve posted the docs, and the code is on Github.

Using will_paginate with DataMapper and Sinatra

2009-02-17T06:52:00Z

Yes, it can be done. But not using prebuilt gems, since we need to use the agnostic branch of the will paginate repository for DataMapper support. Here’s what I did to install the necessary version of will_paginate:

git clone git://github.com/mislav/will_paginate.git
cd will_paginate
git checkout --track -b agnostic origin/agnostic # check out agnostic branch
gem build will_paginate.gemspec # build the gem
sudo gem install will_paginate-3.0.0.gem # install the gem

You could also create a fork, merge the code from agnostic into the master branch, push to GitHub, and have your own username-will_paginate gem to install. I decided this wasn’t worth the effort, at least for the project I was working on.

Models

When will_paginate is required, it does some checks to see if it is running under Rails or Merb, and if so, mixes the class methods and view helpers into the base classes for you. If you’re working with another framework (such as Sinatra), you’ll need to require the finders and view helpers on your own:

require 'will_paginate'
require 'will_paginate/finders/data_mapper'

At this point, YourModelName#paginate will return paginated collections. But we need to be able to render pagination links as well…

Views

will_paginate has abstracted out the view logic into a few modules, and all it depends on is the implementation of the WillPaginate::ViewHelpers::LinkRenderer#url method. To get the view helper working, I added the following to the bottom of my Sinatra app’s file (based on the supplied the merb adapter):

require 'will_paginate/view_helpers/base'
require 'will_paginate/view_helpers/link_renderer'

WillPaginate::ViewHelpers::LinkRenderer.class_eval do
  protected
  def url(page)
    url = @template.request.url
    if page == 1
      # strip out page param and trailing ? if it exists
      url.gsub(/page=[0-9]+/, '').gsub(/\?$/, '')
    else
      if url =~ /page=[0-9]+/
        url.gsub(/page=[0-9]+/, "page=#{page}")
      else
        url + "?page=#{page}"
      end      
    end
  end
end

helpers WillPaginate::ViewHelpers::Base

The code assumes the page number will be in the query string; it could easily be modified to work in other situations.

Git Error: fatal: reference is not a tree error

2009-02-07T15:34:00Z

If you do what I do and keep your Rails plugins in separate git repos using git submodules, you will encounter a git error if you’ve made changes to the local repositories for a plugin and try to deploy without pushing the changes up to the server:

fatal: reference is not a tree: 17f1db7f33c986f0d4a9dc0c5846322095cda96b

To resolve this, cd into the relevant directory (vendor/plugins/whatever/) and push to the repo. It seems obvious, but I’ve made this mistake once or twice.

Fitzgerald update: before filters and sendDownload

2009-02-03T03:27:00Z

Fitzgerald has been running at least two websites for a little while now, but I’ve had no time to do much with it for the last month. There were a few feature additions as a result of a client site I was working one, and now’s as good a time as any to mention them. Especially seeing as the official docs are a couple blogs posts for now.

Before Filters

One feature of Fitzgerald I added to the repo a while ago but failed to document is the ability to add before filters to actions. Anyone that’s used an MVC web framework will probably be familiar with this concept; for those who aren’t, the idea is to assign one or more methods to be called before a given action. The most common use case for this is probably ensuring that a use is logged in before viewing a given page.

It should be noted that adding after filter would be trivial; however, I’m not sure there really is a need for them in a framework of this size. So I’ve left them out for now.

Fitzgerald’s implementation of filters is a little different than most in order allow a filter to return text to the browser or redirect. Any time a filter returns anything, the filter chain is halted and the result is sent to the browser.

Filters are defined using a similar syntax to binding actions to routes, and are executed in the order they are defined:

<?php
class ApplicationWithAFilter extends Fitzgerald {

    protected function verify_user() {
        // redirect unauthorized users to the home page
        if(!$this->isAllowed()) {
            return $this->redirect('/');
        }
    }

    protected function get_protected() {
        // display some secrets
    }

    protected function post_protected() {
        // make some secrets
    }

}

$app = new ApplicationWithAFilter();

// Define a before filter to be executed before one or more actions
$app->before('get_protected|post_protected', 'verify_user');

?>

Just add a method with the correct name to your Application definition, and you are all set. I’ve updated example.php in the repo to include usage of a before filter.

sendDownload

sendDownload is really just a brute force version of sendFile that does everything it can to persuade a browser to download a file instead of opening it inline. There are no guarantees this technique will work all of the time, but in my testing, sendDownload contained the correct black magic to force Internet Explorer to behave itself.

Since I haven’t documented either of these methods yet, here’s how to use sendFile and sendDownload:

<?php
class ApplicationWithDownloads extends Fitzgerald {
    public function get_pdf() {
        $this->sendFile('filename_to_send_as.pdf', 'application/pdf', $this->path('pdf/awesome.pdf'));
    }
    public function get_pdf_for_dowload() {
        $this->sendDownload('filename_to_send_as.pdf', $this->path('pdf/awesome.pdf'));
    }
}
?>

sendFile takes three arguments: the filename to send the file as, the content type to set in the headers, and the path to the local file to send. Note the call to path, which simply appends its argument to the app root (which is defined as one level up from fitzgerald.php). Usage of sendDownload is the same, except it doesn’t take a content type argument.

Building Sphinx 0.9.9RC1 with PostgreSQL 8.3 support via MacPorts on Mac OS 10.5

2009-01-24T20:08:00Z

Up until today I was using the version of Sphinx provided by MacPorts, 0.9.8, as a lot of developers probably are. It installed painlessly, but I’m in the process of auditioning several different search engine options for a project and I wanted to make sure I was using the latest in my comparisons. I’m also using a PostgreSQL 8.3 database, which requires specifying a few additional options to configure.

Here are the commands I used to install Sphinx 0.9.9RC1 (run inside the Sphinx source directory after you download it):

export LDFLAGS="-L/usr/lib"
./configure --with-pgsql --with-pgsql-includes=/opt/local/include/postgresql83/ \
    --with-pgsql-libs=/opt/local/lib/postgresql83/
make
sudo make install

The first line ensures that make can find the system iconv libraries. Otherwise, it’s a pretty straight forward process.

Update

A simpler solution is to make sure that the pg_config executable can be found on your path. Add /opt/local/lib/postgresql83/bin/ to your path, and then you can simply use:

export LDFLAGS="-L/usr/lib"
./configure --with-pgsql
make
sudo make install

Obvious Tip 1: Short Circuiting Logic While Debugging

2009-01-20T07:11:00Z

Who hasn’t forced a conditional statement one way or the other to force part of a complicated system to always trigger during a bout of (potentially late night, potentially ill-advised) hackery? I’ve learned the hard way to always take the twenty seconds to add a warning so I don’t forget to remove the hack. Of course, specs are going to fail if I forget to, but it’s easy to spend time looking in the wrong places for a fix to something as sturdy as if statements.

puts('! OVERRIDING SOME LOGIC !')
if true || some_method_that_may_or_may_not_return_true?
  ...
end

Make sure to put the reminder outside the conditional. Depending on what you’re working on, you can use a framework logger if you have one. I used puts above because the code I’m working on (which sparked this entry) is standalone.

Using blocks in Rails custom date formats

2009-01-15T23:51:00Z

A not-so-well known feature of ActiveSupport is its ability to handle blocks as definitions for custom date and time formats. Here’s one I just added to a project to give nice short dates, stripping out any leading zeros:

ActiveSupport::CoreExtensions::Date::Conversions::DATE_FORMATS.merge!(
  :shorty => lambda {|date| date.strftime("%m/%d/%y").gsub(/0?(\d+)\/0?(\d+)\/(\d+)/, '\1/\2/\3')}
)

You can drop that in your environment.rb; a better option is to place it in a file inside config/initializers.

List Ordering with a JavaScript State Machine

2008-12-30T07:06:00Z

I’ve been thinking a lot about state machines recently- specifically, if the use of an event-driven state machine could be a useful pattern when creating JavaScript interfaces. I did a little poking around online, and the best resource I could find was a series of articles on IBM’s developerWorks. The articles are actually quite good, and I’d recommend reading them. They include some nice background information on the hows and whys of state machine usage.

The one valid criticism I have of the articles is they spend a lot of time developing special code to deal with browser idiosyncrasies and event handling although the articles date from early 2007, well after the rise of JavaScript frameworks. I decided to see what I could to do build a lightweight state machine model using mostly framework code, in this case Prototype and Low Pro.

A Somewhat Contrived Example

Since I want to mostly show the syntax I’m using, here is a basic example for a simple widget: a web radio. Here are the states and transitions this example will be using:

States

Here are the states shown in that diagram, converted to JavaScript (I’ve stripped most of the functionality out for brevity, a link to the full code is below). I’ve stolen the idea of using functions as event handlers like this from the developerWorks articles. I like the flexibility it offers, and also that in order to transition to a new state, one of these functions simply needs to return the name of the new state. I’ve also added a shortcut- if a event handling function is only going to return a state name, that string can just be used in the definition in place of the function.

The state ‘start’ and its event ‘init’ are simply there so we can define a place for the machine to start.

Radio.definition = {
    start: {
        init: 'Stopped'
    },

    Stopped: {
        play: 'Playing'
    },

    Paused: {
        play: 'Playing',
        stop: function(event) {
            return 'Stopped';
        }
    },

    Playing: {
        enter: function() {
            // called when we enter this state
        },
        pause: 'Paused',
        stop: 'Stopped',
        exit: function() {
            // called when we leave this state
        }
    }
};

Events

The next thing to consider is how to map DOM events to the events defined above. I decided to define them for each state using a simple CSS selector to event name mapping. Just as with the state definitions, each CSS selector is either mapped to a string (this time an event name) or a function that will return an event name.

Radio.events = {
    Stopped: {
        'button.play:click': 'play'
    },
    Playing: {
        'button.stop:click': 'stop',
        'button.pause:click': function(event) {
            // ...
            return 'pause';
        }
    },
    Paused: {
        'button.stop:click': 'stop',
        'button.play:click': 'play'
    }
};

All Together Now

The state and event mapping definitions are brought together in a fairly standard Low Pro behavior, inheriting from another behavior containing the state machine logic. Any functions that are used as a part of the state and event definitions above are evaluated in the context of a behavior instance, so this is a convenient place for utility and helper functions.

Radio.machine = Behavior.create(State.behavior, {
    definition: Radio.definition,
    events: Radio.events,
    incrementCounter: function() {
        // ...
    },
    updateCounter: function() {
        // ...
    }
});

The complete behavior is applied using the standard Low Pro Event.addBehavior().

Event.addBehavior({
    'div.radio': Radio.machine
});

Here’s a working example of the radio using the complete code.

Sort, Sort, Sort

Obviously, a state machine could be used to model a much more complicated interaction. A client application I’m working on had a need for some customizable sorting, and I decided to write the code from the ground up in order to have complete control over the sort lifecycle. After some thought, I decided to try to model the behavior of many desktop UI systems: a clone of the element being dragged would move with the mouse, and an insertion point would appear when and where a legal insertion could occur. I thought this was the best interface for most use cases as it allows the user to see both an object’s new and old positions throughout the drag process. I also like the clear indication of when a valid drop can occur.

The states and events used by the ordering widget are shown in the following diagram:

I won’t paste all of the code to the sorter here, but I’ve posted it on GitHub as well as a working example. It worked out rather well, and I found the use of a state machine helped in keeping the code well organized and extensible, despite the fairly complicated interaction being modeled.