opensoul.org - Code

Great Lakes Ruby Bash

2010-02-17T18:40:00Z

The Great Lakes Ruby Bash is now accepting talk proposals. The conference will be held on Michigan State University’s campus in East Lansing, Michigan on Saturday, April 17th.

We’re looking for passionate speakers to give 25 and 40 minute presentations about their experiences with Ruby and related technologies. Our goal is to engage attendees and inspire them to create great software, empower users, and continue learning with others.

Proposals are due Feb 28th, 2010 at 12:59pm EST. We hope to have all proposals reviewed and speakers chosen by March 8th, 2010. Visit the website for more info.

Active Resource in practice

2010-02-16T14:36:00Z

I’m working on app to integrate Pivotal Tracker and Harvest. There’s a great ruby wrapper around Harvest’s API, but there isn’t a decent Ruby wrapper for Tracker’s v3 API, so I thought I would just build one as I needed it.

If this app were read only, I would probably use HTTParty and HappyMapper, but since I also want to be able to update timers and stories, Active Resource seemed like the right tool for the job. Active Resource in theory is great. Active Resource in practice is not so great. I’ve toyed around with it in the past, but using it for something real I found it…lacking.

Fortunately the Harvest gem had solved a lot of these problems. I write about them here in hopes that they will be useful to you.

Challenge 1: Headers don’t inherit

Pivotal Tracker uses a token for authentication and looks for it in a header called “X-TrackerToken”. It would be nice if you could just set this once, and all Active Resource classes would use it. But unfortunately, headers don’t inherit.

So the trick is to define a base class for all of your models to inherit from, and in that override how Active Resource treats headers.

module PivotalTracker
  class Resource < ActiveResource::Base
    Resource.site = "https://www.pivotaltracker.com/services/v3"

    class << self
      # If headers are not defined in a given subclass, then obtain
      # headers from the superclass.
      def headers
        if defined?(@headers)
          @headers
        elsif superclass != Object && superclass.headers
          superclass.headers
        else
          @headers ||= {}
        end
      end
  end

  class Project < Resource
  end
end

Now we can set our token once and subclasses will inherit it:

PivotalTracker::Resource.headers['X-TrackerToken'] = "mytoken"
projects = PivotalTracker::Project.find(:all)

Challenge 2: “Associations”

I find it strange that Active Resource doesn’t support associations. Rails has a standard way of defining embedded resources, so you would think that Active Resource would have a standard way of consuming them (I know, I should get off my lazy duff and contribute a patch, but it’s so much easier to just complain about it).

So for APIs that have nested resources like Pivotal Tracker’s, Active Resource forces you to hard code the parent resource id. If you want to get the iterations for a project, then you have to set the project_id on the Iteration resource.

PROJECT_ID = 1738

module PivotalTracker
  class Iteration < Resource
    self.prefix = "/services/v3/projects/#{PROJECT_ID}"
  end
end

This is just not a scalable solution. I’m going to need to be able to access multiple projects in the app that I’m working on. So the Harvest gem had a really clever (and evil) solution, which I’ve modified a bit here.

It basically involves creating an anonymous subclass of our resource, and setting the prefix just for that subclass.

module PivotalTracker
  class Project < Resource
    def iterations
      Iteration.build_subclass.tap do |iteration|
        iteration.prefix = "/services/v3/projects/#{self.id}"
      end
    end
  end
end

Now we can access iterations for any project.

iterations = Project.find(x).iterations.find(:all)

The #build_subclass method is defined on the base resource and just creates an anonymous subclass and copies some settings that don’t inherit.

Onward Ho!

I don’t have a lot built out yet for the new Pivotal Tracker wrapper, but you can check out the latest progress on GitHub. I feel like I’ve overcome most of the bit barriers, so it shouldn’t take much to finish it up.

Do you have any other tips or tricks for working with Active Resource?

Passenger and browser testing in virtual machines

2010-02-09T04:33:00Z

If you’re running Passenger in development, here is how to make Windows running in a virtual machine connect to your app in Passenger.

Boot up the VM and open up the Windows command prompt (go to “Start->Run…”, enter “cmd” and press enter)
Type ipconfig to see the network configuration. Take note of the “Default Gateway” address.
Navigate to C:\WINDOWS\system32\drivers\etc and edit the hosts file. Add a line with the gateway address pointing to your app’s domain (you can even list multiple on the same line).
```
172.16.248.2    awesomeapp.local otherawesomeapp.local
```
Open up a browser in the VM and type in the address.

Tada! Now you can test out your app in those other browsers.

Things that will rock the (my) world in 2010

2010-01-05T02:31:00Z

Here are a few things that I’m really looking forward to using and abusing this year:

MongoDB

We’ve all been trying to shoehorn our Web 2.x applications into a 20 year old technology with an antiquated query language. If you haven’t looked into MongoDB, you need to, and you also need to check out MongoMapper.

I went out on a limb back in October and stated that a year from now, we’ll be using MongoDB for most new web apps. I think Mongo will be one of those things like Git that catches on like wildfire.

I have worked on a couple apps now with MongoDB, and it is a lot of fun. It blows my mind just how much our understanding of data modeling is tied to relational databases.

Sammy

Sammy is an awesome little Javascript framework for building RESTful and evented web applications. I’ve only build one little toy app with it so far, but I really enjoyed it and plan to update a couple existing apps an use it on some new ones this year.

Sunspot

If you need to do full-text searching, then Sunspot is your man. I have tried and given up on almost every single solution for doing real full-text searching in applications. I have a reputation around the office for enthusiastically touting several different search solutions as “the one”, only to watch all of them fall on their face in production.

But Sunspot has not let me down yet, and I have one app that has been using it in production for a few months with zero problems. I look forward to abusing it this year.

Rails 3

Two weeks ago, Rails 3 would not have made it on this list. In fact, it would have probably made it on my “things that are totally going to suck” list. I was afraid I was becoming a curmudgeon.

But the Rails core team has kicked it into high gear the past couple weeks. I spent some time playing with Rails (setting up a new app was no easy feat, although now there’s a guide) and digging through the internals and I can now say that I am officially excited.

What are you looking forward to?

acts_as_audited and authlogic

2009-10-29T15:52:00Z

For those using authlogic that have had issues with auditing your User model, version 1.0.2 of acts_as_audited should cure your woes.

All you need to do is exclude the last_request_at and perisable_token fields from being audited. We also excluded a few other fields that don’t need to be audited:

class User < ActiveRecord::Base
  acts_as_audited :except => [
    :crypted_password, 
    :persistence_token,
    :single_access_token,
    :perishable_token,
    :last_request_at,
  ]
end

How to Gemify your Rails Plugins

2009-10-05T17:33:00Z

Ever since Rails added support for declaring gem dependencies, there is really no (good) reason to use plain ol’ plugins. We’ve been slowly gemifying all of our plugins as we need them. There’s a few hoops you have to jump through to get Rake tasks and Capistrano recipes working, but it’s fairly straight forward.

First, you need something that will help you generate the gemspec and build the gem. You can do this by hand, but there’s several great plugins out there that make it easy. We recommend Jeweler. Follow the directions in the Jeweler README for “Using in an existing project”.

1. Move `init.rb` to `rails/init.rb`

Rails plugins have the magical init.rb that gets loaded when the plugin is initialized. To make this work in a gem, all you have to do is move it to rails/init.rb. Recent versions of Rails will look there whether you install it as a plugin or a gem, so you can just move it to the new location if you don’t care about ancient versions of Rails.

2. Move rake tasks to `lib/`

Rails will load tasks/*.rake defined in any plugins. Unfortunately, these don’t get loaded from Gems. To make your rake tasks work from a plugin, you will need to move them into the lib directory, and explicitly require them from your app’s Rakefile:

require 'mygem/tasks'

If you want your tasks to still be available when your code is installed as a plugin, you can just explicitly require the task from task/mygem.rake:

require File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib', 'mygem', 'tasks'))

3. Move Capistrano tasks to `lib/`

Capistrano recipes defined at recipes/*.rb in your plugin are also automatically loaded by Rails. Unfortunately, they suffer the same fate as Rake tasks and have to be move to the lib directory and be explicitly required from config/deploy.rb.

When moving the recipes to the lib directory, we have to jump through a hoop to get Capistrano to load it properly.

Capistrano::Configuration.instance.load do
  # put cap recipes here
end

And in your config/deploy.rb:

require 'mygem/recipes'

As with Rake tasks, if you want your recipes to still work when installed as a plugin, add the following to a file in the recipes/ directory of your plugin:

require File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib', 'mygem', 'recipes'))

4. Generators

You don’t have to do anything, they just work.

That’s it

Publish your gem and go buy yourself a drink. Check out one of our gems if you need more examples.

Capistrano, Git and SSH keys

2009-06-24T00:45:00Z

This trick has been around for a while, but I’ve talked with several people that didn’t know about it.

When deploying apps with Capistrano, your server needs to have access to the Git repository. Generating an SSH key for each server is a bit of a pain, but there’s an easier way: SSH agent forwarding enables you to use any of your local SSH keys on the server. It’s really easy to set up.

Enable SSH forwarding in deploy.rb:

set :ssh_options, {:forward_agent => true}

The only other thing you need to do is tell the SSH agent about your key.

$ ssh-add -K

The -K option only works on OS X and it adds your key to your keychain so you don’t have to run ssh-add after you reboot (and if you have a passphrase set, you don’t have to type it every time). You can also pass it the path to an SSH private key in a different location.

Now the server can pull from any Git repository that you have access to.

Cucumber scenarios that depend on Sphinx

2009-06-01T13:18:00Z

I love writing apps that make heavy use of search indexes, but testing them can be a bit of a pain. Here is how I got ThinkingSphinx to play nice with Cucumber.

Here is the relevant part of what I put in features/support/env.rb:

# Cucumber::Rails.use_transactional_fixtures

# http://github.com/bmabey/database_cleaner
require 'database_cleaner'
DatabaseCleaner.strategy = :truncation
Before do
  DatabaseCleaner.clean
end

ts = ThinkingSphinx::Configuration.instance
ts.build
FileUtils.mkdir_p ts.searchd_file_path
ts.controller.index
ts.controller.start
at_exit do
  ts.controller.stop
end
ThinkingSphinx.deltas_enabled = true
ThinkingSphinx.updates_enabled = true
ThinkingSphinx.suppress_delta_output = true

# Re-generate the index before each Scenario
Before do
  ts.controller.index
end

What’s going on here?

Start by commenting out the line about using transactional fixtures in env.rb. Using transactional fixtures will run each scenario inside of a transaction and roll it back at the end of the scenario to revert the database state. Thinking Sphinx uses an after_commit callback for kicking off the delta indexing, but the callback never gets run when transactional fixtures are enabled because the entire scenario is run inside of a big transaction.

Once we’ve disabled transactional fixtures, our test database will start to fill up, likely causing some problems. So we need to add a Before block that clears out the database before each scenario. I’m using database_cleaner, which gives you some different strategies for cleaning the database. Alternatively, the brute-force solution is just to reload the schema before each scenario, but this is slower than truncating the data.

Before do
  ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['test'])
  ActiveRecord::Schema.verbose = false
  load "#{RAILS_ROOT}/db/schema.rb"
end

Next, we start Sphinx when env.rb is loaded, and shut it down when the Ruby process exists. We also enable deltas and updates, which are disabled by default in test mode. Finally, we define a Before block that updates the index before each scenario so we don’t end up with a stale index.

Putting it all together

I’m using Sphinx’s delayed delta support, so whenever I update records, I need to have delayed_job process jobs. Instead of trying to get delayed_job to run in the background, I took the easy way out and defined a step: “When the system processes jobs”.

Scenario: Posting a new listing
  Given I am logged in as "MovinMan" 
  When I create a new listing titled "Lots of Boxes" near "49423" 
  And the system processes jobs
  And I browse listings near "49423" 
  Then I can see a listing titled "Lots of Boxes"

Which is just implemented as:

When 'the system processes jobs' do
  Delayed::Job.work_off
end

If you’re just using the default deltas, and not delayed deltas, then you can update the index like this:

When /^the system updates the index$/ do
  MyModel.sphinx_indexes.first.delta_object.index(MyModel)
end

I hope that helps. Post your suggestions in the comments for improving this.

Site-specific app for Rails docs

2009-05-06T02:57:00Z

Unless you’re a Rails genius, you probably need to frequently reference the Rails API docs. And if you haven’t discovered it yet, railsapi.com is awesome.

John Nunemaker suggested that I create a site-specific browser and point it to a local copy of the docs from railsapi.com. I did and have been loving it, so I’m suggesting that you do it too.

Download Fluid (or the comparable app for your platform if you’re not on a Mac). Then download a copy of the docs from railsapi.com and create a new app pointing to that local copy.

Better yet, head over to railslogo.com and grab the Creative Commons licensed logo to use as the icon.

Now my only complaint is that I don’t have docs for Ruby and other gems in this app, but I have a hunch that it won’t be long until that changes.

Keepin' Sphinx Indexes Fresh

2009-04-30T01:33:00Z

<infomercial-voice>Stale indexes got you down? Embarrassed that your users’ searches are coming up empty? Act now and you can avoid stale indexes with NEW and IMPROVED delayed delta indexing!!</infomercial-voice>

Ok, maybe it’s not new and improved. It’s actually been around since January, but it’s still awesome. Thinking Sphinx can use delayed_job to keep indexes fresh.

I was slow at jumping on the Sphinx bandwagon for one reason: the index has to be rebuilt to incorporate new data. Delta indexing alleviated some of this by storing frequent changes in a small separate index, but it still had to be occasionally reindexed. It also seemed to only index existing records in my trials with it. New records didn’t ever seem to show up until I rebuilt the whole index.

From what I can tell, delayed delta indexing makes everything Just Work™, and here’s how to use it…

After you’ve setup ThinkingSphinx, set the :delayed property to :delta in your index:

class Listing < ActiveRecord::Base
  define_index do
    indexes title
    indexes description
    indexes user.login, :as => :user

    set_property :delta => :delayed
  end
end

The delayed delta support depends on delayed_job, but if you’re using the gem version, it’s already bundled in. I’m using delayed job for some other things in my project, so I still have it installed separately and that seems to be working just fine.

delayed_job uses a table to keep track of the jobs that need run, so create a migration containing:

create_table :delayed_jobs, :force => true do |table| 
   table.integer  :priority, :default => 0 
   table.integer  :attempts, :default => 0 
   table.text     :handler 
   table.string   :last_error 
   table.datetime :run_at 
   table.datetime :locked_at 
   table.datetime :failed_at 
   table.string   :locked_by 
   table.timestamps 
end

And lastly, all you need to do is fire up the worker process:

$ rake thinking_sphinx:delayed_delta

Now whenever changes are made to your models, the index will be updated moments later. And that’s how you keep it fresh!

Location-based search with Sphinx and acts_as_geocodable

2009-04-15T00:48:00Z

Sphinx, everybody’s favorite search engine, has support for location-based search, giving you geo-aware, full-text searching. So now you can find all of the garage sales on Saturday within 20 miles that have LPs and a reel mower.

All you need to do is add latitude and longitude (in radians) to the index, allowing you to limit the results to records within a distance of a point. The hardest part is getting the coordinates, but acts_as_geocodable makes that really easy.

To start, install acts_as_geocodable. Once you have that configured properly, install ThinkingSphinx, define an index on your geocodable model and add the coordinates to the index:

class Listing < ActiveRecord::Base
  acts_as_geocodable

  define_index do
    indexes title
    indexes description

    has geocoding.geocode(:id), :as => :geocode_id
    has 'RADIANS(geocodes.latitude)', :as => :latitude, :type => :float
    has 'RADIANS(geocodes.longitude)', :as => :longitude, :type => :float
  end
end

The three lines that start with has add the geocode id, and the latitude and longitude in radians to the index. Our index doesn’t need the geocode id, but we have to add it so that ThinkingSphinx properly joins the geocodes table.

After you rebuild the index and start the daemon, you can search for records by location. Here’s an example of taking a zip code from a user and finding all records with in 20 miles. (Note: you will need to grab the latest version, 0.2.9, of Graticule for this next bit of code to work)

def search
  location = Geocode.geocoder.locate(params[:zip]).coordinates.map(&:to_radians)
  @listings = Listing.search(params[:q], :geo => location,
    :with => {'@geodist' => 0.0..(20 * 1610.0)})
end

After looking up the coordinates of the zip code that the user entered, we do a search with the :geo parameter, limiting the results using the special @geodist condition. We have to pass in a range of floats that represent the distance of the points in meters (and since the US is in the stone age, I’m converting from miles).

That’s all there is to it. Now go write some cool location-based search and comment here about it.

Testing Facebook with Cucumber

2009-03-06T02:57:00Z

For those that haven’t heard: Cucumber is pretty much the greatest thing since sliced bread. It dramatically improved the quality and stability of our applications, and the outside-in approach that it encourages forces you to stay focused on what’s important.

When we started working on a Facebook application a few months ago, we couldn’t fathom not using Cucumber. So we had to figure out a way to test it. It took us a few months to evolve it to a point where we could extract it, but this week we pushed a change to Facebooker to make life a little easier. So grab the latest version of Facebooker and keep on reading…

First, in features/support/env.rb, replace the default Rails world with the one in Facebooker:

# require 'cucumber/rails/world'
require 'facebooker/rails/cucumber'

Given I am logged in as a Facebook user

Most of our Facebook application requires that a user be logged in. So most of our scenarios started with “Given I am logged in as a Facebook user”.

Scenario: Uploading a video
  Given I am logged in as a Facebook user
  When I upload a video
  Then I can see a video on my blog

And here is our implementation for that step:

Given "I am logged in as a Facebook user" do
  @current_user = User.create! :facebook_id => 1
  @current_user.facebook_user.friends = [
    Facebooker::User.new(:id => 2, :name => 'Bob'),
    Facebooker::User.new(:id => 3, :name => 'Sam')
  ]
  @integration_session.default_request_params.merge!(
    :fb_sig_user => @current_user.facebook_id,
    :fb_sig_friends => @current_user.facebook_user.friends.map(&:id).join(',')
  )
end

Our application has a User model with a facebook_id attribute and a #facebook_user method which returned an instance of Facebooker::User. Due to how the Facebook session is being mocked, it is important that we set our fake user’s id to 1 for now (I’ll try to figure out a way around this). We also manually add some friends for our application to use. Lastly, we merge in our user’s id and friend ids into the default request params so that any requests we make include those parameters.

Drop your…canvas

There were some places in our app where requests don’t go through the canvas. For example, we have a few multipart forms, which have to submit directly to your application. To mimic this, wrap webrat calls in #without_canvas:

When "I upload a video" do
  visit root_path

  without_canvas do
    fill_in 'Title', :with => 'A video'
    fill_in 'Description', :with => 'Caption for video'
    attach_file 'Video', "#{RAILS_ROOT}/features/support/sample.mpg", "video/mpeg"
    click_button 'Upload Video'
  end
  follow_redirect!
end

Note that if your action redirects to a URL with :canvas => true, webrat will see that as an “external” redirect and won’t follow it. Just call #follow_redirect! and it’ll go on it’s merry way.

Accessing the Facebook API

Instead of making requests to the Facebook API in your tests, Facebooker will try to read a canned response fixture from features/support/facebook/. It will give you a friendly error whenever this happens, so just follow the directions and you should be on your way.

Feedback & Patches

I’m sure there are things that don’t work right, so let me know if you run into any troubles. If you have any ideas for making the Cucumber support better, please share them here or on the Facebooker mailing list.

Showing Ajax activity with CSS

2009-03-04T12:00:00Z

You know when you have a form that submits with Ajax, and you want to show some type of activity on the form? There’s several ways to do this, like including a hidden image that you show and hide while there is activity, or inserting and removing an image with javascript. These all just feel too dirty to me. I don’t want to have to change the markup just because the form is submitting via Ajax.

As usual, there’s a cleaner way to do it with just CSS. I’ve used this simple but effective trick on a couple projects now, so I thought I would share.

First, here is a demo of what it looks like: (sorry, you’ll have to leave your feed reader.)

Click the save button.

or Cancel

#loading_example { margin: 1em 3em; padding: 1em; border: 4px solid #CCC; border-radius: 10px; -moz-border-radius: 10px; -webkit-border-radius: 10px; } form.example .buttons { text-align: center; } form.example.loading .buttons { text-indent: -2000px; overflow: hidden; background: url(/assets/2009/3/4/loading.gif) no-repeat center center; } form.example.loading .buttons * { visibility: hidden; }

This technique is very simple. It adds a class on the form with Javascript while it is loading, and then removes it once it’s done. The loading class hides the submit button and replaces it with a background image, usually an animated gif.

First, with Javascript, add a class to the form while the Ajax request is loading. It would look something like this in Prototype:

$('my_form').observe('submit', function(event) {
  event.stop();
  this.request({
    evalScripts: true,
    onLoading:  function() { this.addClassName('loading'); }.bind(this),
    onComplete: function() { this.removeClassName('loading'); }.bind(this)
  });
});

Next, wrap an element around your buttons or whatever you want to hide.

<form>
  <div class="buttons">…</div>
</form>

Lastly, there’s just a little CSS to hide buttons and any elements inside them, and set the background image:

form .buttons { text-align: center; }
form.loading .buttons {
  text-indent: -2000px;
  overflow: hidden;
  background: url(../images/loading.gif) no-repeat center center;
}
form.loading .buttons * { visibility: hidden; }

There you have it. It’s really simple, but it’s worked well.

Plugging Rack into Rails

2009-03-03T13:33:00Z

The interwebs are all a’buzz about Rack. For those that haven’t been following along, Rack is a specification and a library for connecting web servers to Ruby libraries (a.k.a. “Middleware”). It’s basically the Web 2.0 version of CGI, except that it doesn’t suck and it’s just for Ruby.

Rails 2.3 has Rack baked in. It uses Rack for things like sessions and parameter parsing. But what if you want to add your own middleware to a Rails app?

It’s really easy! In the init.rb of a plugin, or just in a Rails initializer:

ActionController::Dispatcher.middleware.use MyMiddleware

This will append your middleware to the end of the “stack”, so it will be executed after all of Rails’ middleware, but before anything else in the Rails framework.

But what if you need to massage request parameters before Rails parses them? All you need to do is insert your middleware in the stack before Rails parses the params. You can find the current list of middleware by inspecting ActionController::Dispatcher.middleware. In there you’ll find ActionController::ParamsParser, which is what we want to insert our middleware before. Unfortunately, there’s not an insert_before method (yet), so we’ll need to use insert_after, giving it a middleware earlier in the stack:

ActionController::Dispatcher.middleware.insert_after 'ActionController::Failsafe', MyMiddleware

If you don’t like the way that one of the existing pieces of middleware handles things, you can swap it out for your own version:

ActionController::Dispatcher.middleware.swap 'Rails::Rack::Metal', HeavyMetal

So there you have it. Eat, drink, and go write middleware.

Force Absolute URLs

2009-02-21T05:03:00Z

Need to force all URLs in a controller or action to be absolute URLs, but don’t want to use the *_url named routes or, even worse, :only_path => false everywhere?

Background

As we were putting the finishing touches on an awesome Facebook app today (yes I said “awesome” in the same sentence as “facebook app”…you’ll see), we needed to make it so that users can add the app as a tab in their profile. Facebook makes this very easy, but there’s an interesting ~~quirk~~ feature. From the developer’s wiki:

Absolute URLs on a tab take the user to your application’s canvas page.

Relative URLs are treated as if they are relative to your home canvas page and AJAX-loads in the tab.

We needed all of the URLs on the page to be absolute so that they would take the user out of the tab and into our application. So we were faced with two options:

Change all of our named routes from *_path to *_url and use explicit paths instead of polymorphic urls (you know, that thing that lets you do form_for @model, or link_to h(@model.name), @model)
Figure out some quick hack that forced Rails to generate absolute URLs on this one page and go on about our lives

The page reused a lot of components from all over the site, so we weren’t excited about option 1. And, like all good developers, we were just adverse to that much work.

So we started looking for a good fit for option 2. We needed to force :only_path to be false on any URL generation. There’s a way to specify default_url_options, but as the name implies, those are only defaults. Using the *_path named routes forces :only_path to true.

Solution

It turns out, Rails passes all URL generation through a method on your controller called #rewrite_options. So we just overrode that method to force :only_path to false on our one action. Here’s the code we used:

class ContentController < ApplicationController
  # …

  def profile
    # profile action
  end

private
  # Force absolute URLs on the profile tab.
  def rewrite_options(options)
    super(action_name == 'profile' ? options.merge(:only_path => false) : options)
  end
end

It’s slightly evil, but it worked beautifully.