Life at the Speed of Green

callback_skipper for faster factories

2011-11-28T09:00:00.000-06:00

The Rails community has taken a strong stance that test fixtures are evil and factories are the new hotness. The most compelling reason for this shift being that factories greatly reduce maintance costs as your application grows over time.

The largest drawback when replacing fixtures with factories is the additional performance overhead when initializing your test state. This is due to the fact that factories utilize the full ActiveRecord lifecycle (initialize object, validate, save to database) compared to fixtures which are a glorified database bulk import which bypass object creation/validation.

Take this example ActiveRecord model and testcase (using FactoryGirl factories):

class Foo < ActiveRecord::Base
  after_create :do_something_expensive
end

def test_new_instance_has_bar
  foo = Factory.create :foo

  assert foo.bar?
end

This is a classic example where using factories will be much slower than fixtures. It may not seem like a big deal, but it becomes a real issue when you have a sizable application testsuite, and I'm a firm believer that if a testsuite isn't fast, it doesn't get run.

Sending email with after_save callback is another great usecase. How can we get the benefits of factories without the additional overhead? Can we have our cake and eat it too?

Since 99% of our testcases rely on the basic factory instance and are not dependent upon the expensive callback being fired, it would be ideal to skip the expensive callback for most of your testcases and only fire it for testscases that are explicitly asserting it's behavior.

The newly released callback_skipper gem fulfills this exact usecase. The goal is to make it trivial to skip a particular model callback for a specific instance. This is a homerun for test factories to setup a default factory which skip non-critical callbacks and still have the flexibility to create a factory that does fire the slow-ass callbacks for specific tests.

# spec/factories/foo_factory.rb
Factory.define :foo do |f|
  f.after_build do |o|
    o.skip_callback :save, :after, :do_something_expensive
  end
end
Factory.define :foo_with_expensive_callback, :class => 'Foo' do |f|
end

Oh yes, we can have our cake and eat it too...

The callback_skipper gem is equivalent to the core ActiveRecord.skip_callback method with the added benefit of only skipping the callback for a specific instance instead of globally for all invocations.

As always, the gem is 100% opensource and suggestions are welcome!

Cron Backgrounded Resque Jobs

2011-11-09T09:55:00.000-06:00

Cron is still the de facto standard when it comes to scheduling execution of jobs. It does one thing, and it does it well.

Using cron to fire off heavyweight Ruby/Rails jobs is fairly trivial to get up and running. When each cron job fires up a full Rails process, it won't be long until you accrue enough jobs that your cron server will be brought to it's knees. Since you already have a pool of Resque background workers running to process async jobs, why not leverage dormant workers to process jobs that are kicked off via cron?

Here is a simple solution that can be used to have cron enqueue jobs into Backgrounded Resque jobs without loading the entire Rails environment.
This is an excellent optimization to increase the availability of your cron server and avoid running out of memory when multiple jobs fire up at the same time.

crontab invocation:

script/backgrounded enqueue Foo.bar --queue baz

script/backgrounded:

#!/usr/bin/env ruby
require 'rubygems'
require 'thor'

module Backgrounded
  class CLI < Thor

    desc 'enqueue', 'enqueue a clazz.method invocation for resque backgrounded workers'
    method_option :queue, :aliases => "-q", :desc => "resque queue to enqueue the operation to", :default => 'backgrounded'
    method_option :rails_env, :aliases => '-e', :desc => 'control which rails env used to load the redis config', :default => 'production'
    # operation Clazz.method to enqueue into resque backgrounded queue (ex: Foo.bar)
    def enqueue(operation)
      require 'bundler'
      Bundler.setup
      require "thread"
      require "active_support/inflector"
      require "resque"
      require "yaml"
      clazz, method = operation.to_s.split('.')
      raise 'invalid operation' unless clazz && method
      Resque.redis = YAML.load_file(File.join("config", "resque.yml"))[options[:rails_env]]
      Resque::Job.create(options[:queue], 'Backgrounded::Handler::ResqueHandler', clazz, -1, method)
    end
  end
end

Backgrounded::CLI.start

Use cron for it's excellent scheduling capabilities and take advantage of your background processing infrastructure to handle the incoming request!

Cleaner RSpec/FactoryGirl Integration

2011-10-04T21:00:00.000-05:00

For the past few days, I've been re-thinking how to integrate FactoryGirl fixtures into my RSpec tests. The transition from Test::Unit to RSpec has taken me some time and I'm still "Unlearning what I have learned" and adapting to the RSpec way of doing things. It's not that Test::Unit style tests are wrong, it just that I need to think a bit differently in order to take full advantage of the expressiveness of RSpec tests.

One common pattern for Rails application tests is to instantiate fixture data and perform assertions on the instances:

describe User do
  context 'basic user' do
    before do
      @user = Factory.create :user
    end
    it { @user.should be_inactive }
    it { @user.should_not be_happy }
  end
end

According to The Anatomy of an Effective Unit Test the before block in this example is playing the role of both fixture setup and test execution. The ambiguous definition of the before method is analogous to how horrible it is to use API's with incredibly useful methods such as run or execute. Sure, we know when the block is executed, but method names are key to understanding their purpose as well.

I'm a strong proponent of expressive method names as a way to self document your code and that's exactly what we need in this scenario. We need a clear and concise way to encapsulate our fixture data initialization outside of the before method and hook it into the RSpec test lifecycle. This will help remove the ambiguous purpose of the before method in this example.

Luckily, The RSpec DSL can be easily be extended. The new factory_girl_rspec gem adds the with method which is a seriously clean and concise DSL for initializing your fixtures:

describe User do
  context 'basic user' do
    with :user
    it { user.should be_inactive }
    it { user.should_not be_happy }
  end
end

The with method is a simple utility that maps directly to the FactoryGirl fixtures that you have defined for your test environment. Each execution of with will create your fixture instance and define a local method within your test context retrieve the instance. Compare the readability of this test with the previous incarnation. It reads like a story right? Watch out Cucumber, who needs text processing stories when you've got human readable ruby code!

Deviations of fixture data are extremely common in unit tests as well, and the with method expresses differences in an equally concise manner:

describe User do
  context 'user with first_name == "Jim"' do
    with :user, :first_name => "Jim"
    it { user.should be_inactive }
    it { user.should be_happy }
  end
end

Guard JSLint on Rails Released!

2011-09-14T22:30:00.000-05:00

Continuous testing has changed my life and Guard is top dog when it comes to continuous testing for Rails development. It's simple, reliable and extremely easy to extend. Autotest and Watchr don't hold a candle to Guard.

It is imperative when building Javascript heavy applications that you add JSLint to your workflow. The excellent jslint-on-rails gem makes it absolutely trivial, so if you haven't done so already, do it now! (Don't worry. I'll wait for you. Seriously, it only takes a few minutes.)

Some people whine and complain that JSLint is evil and there may be some legitimacy to that claim, but the simple fact is that JSLint has saved my ass more than once. Before JSLint, a single missing semicolon would take down an entire application for IE6/7 users, but ever since we've introduced JSLint into our development flow, we've had zero issues.

Integrating JSLint into your continuous testing flow is now possible with the new guard-jslint-on-rails plugin. Catch those pesky lint issues before you deploy to production. Catch them even before your continuous integration process runs. Heck, now you'll catch them before you commit them to your version control repository!

First: install the gem...

$ gem install guard-jslint-on-rails

Next: configure guard...

$ guard init jslint-on-rails

And voila! Each time you tweak your Javascript files, your changes will be validated against your JSLint config!

JSLint Rails Javascript Responses

2011-06-27T23:37:00.002-05:00

Unit testing Javascript has quickly become an absolute requirement for building a modern web application. It's not uncommon for most web applications to have more Javascript code/libraries than serverside logic. There are plenty of frameworks javascript testing frameworks available for unit testing your standalone javascript components (JSUnit, QUnit, etc), but how do you test Javascript responses generated by Rails?

Let's start by describing problem in a bit more detail. Here are a few snippits of code that you'll find in pretty much any AJAX heavy Rails application.

Example Rails form for creating a new blog post via AJAX:

# app/views/blogs/new.html.haml
= form_for Blog.new, :remote => true do |f|
  = f.text_area :body
  = f.submit

Example Rails controller to create blog post:

# app/controllers/blog_controller.rb
class BlogController < ApplicationController
  def create
    @blog = Blog.create params[:blog]
    respond_with @blog
  end
end

Example Rails Javascript ERB response to return new blog post:

# app/views/blogs/create.js.erb
$('#new_blog_form').trigger('reset');
$('body').append('<h2>New Blog Post has been created!</h2>');
$('body').append('<%= escape_javascript(@blog.body) %>');

Now this is a very simple example, but how can you effectively assert that the Javascript response is doing what you expect? Sure, you can use selenium or some other heavyweight testing library to boot up a browser and exercise your Javascript, but there's got to be a quicker way to get the job done.

With a few simple solutions, you can get a lot of bang for your buck without introducing more complex and finicky end to end testing frameworks.

Move Javascript Logic to Static Component

Refactoring your Javascript response to use the "Fat Component, Skinny Response" pattern is the same pattern we've been using for building our controllers, and it moves all of the logic out of your ERB templates and into a standalone (and testable) Javascript component.

Example refactored Javascript response:

# app/views/blogs/create.js.erb
myApp.createBlog(<%= @blog.to_json %>);

Example Javascript component:

# public/javascripts/application.js
myApp = {
  createBlog: function(data) {
    $('#new_blog_form').trigger('reset');
    $('body').append('<h2>New Blog Post has been created!</h2>');
    var content = $('p').text(data.body);
    $('body').append(content);
  }
};

Now you can effectively unit test your component using traditional Javascript testing libraries.

JSLint the Rails Response

JSLint is a simple and very effective smoketest to ensure that any Javascript response is valid and without syntax errors. This prevents any extra semi-colons or simple syntax errors from breaking your user's browser. All "modern" browsers are pretty forgiving when dealing with Javascript errors, but one browser in particular *cough* IE *cough* can really freak out if it encounters invalid Javascript

Rspec users can drop this be_lintable helper into there application for a quick sanity check that their JS responses should be operational. Not using RSpec? The JSLint on Rails plugin is pretty simple to integrate into any application.

describe 'xhr post :create' do
  before { xhr :post, :create, :blog => {:body => 'test'} }
  it { response.body.should be_lintable }
end

Now go forth and test those Javascript Rails responses!

Email Preview + Transactions and Factories

2011-06-15T08:52:00.004-05:00

The email_preview gem has undergone a serious overhaul in the last week. After some extensive real-world project use, a new version has been released with some major improvements.

Transactional Fixtures

At a high level, email_preview fixtures are very similar in nature to Rails test fixtures. They are a snapshot of data with a particular configuration and are not intended to be long lived outside of the email_preview. The setup and teardown of previewed emails should follow the same setup/teardown of fixture data used during unit testing. Now each email_preview action is wrapped in a database transaction and rolled back after completion to avoid polluting your sandbox with unnecessary data!

# config/initializers/email_preview.rb
EmailPreview.register 'User activation email' do
  u = User.create :email => 'foo@example.com'
  UserMailer.activation(u)
end

Factories

Why duplicate the setup of fixture data when building your email preview templates? You already use a factory library for your unit tests right? The email_preview gem now works great with any factory framework you choose (FactoryGirl, Machinist, etc).

# config/initializers/email_preview.rb
# example FactoryGirl fixture
EmailPreview.register 'User activation email' do
  u = Factory.create :user
  UserMailer.activation u
end

Realtime Search: Solr vs Elasticsearch

2011-05-31T11:09:00.000-05:00

Who will win the crown for the King of Realtime Search?

Oh the suspense! The results are available on the Socialcast Engineering blog...
http://engineering.socialcast.com/2011/05/realtime-search-solr-vs-elasticsearch/

Multitenant Ruby Gem

2011-03-03T23:04:00.002-06:00

Multitenant: locking down your app and making cross tenant data leaks a thing of the past...since 2011

Designing multi-tenant applications has increasingly becoming the norm for software developers. The benefits of building an application with a multi-tenant architecture are almost too many to list (cost savings, data mining, simplified operations, etc). But, let there be no doubt that building a multi-tenant application increases the overall complexity of the application.

The Multitenant gem helps alleviate that complexity and ensures that all SQL queries executed adhere to the scoping of the current tenant. It works with any Rails3 compatible application without any modifications to your existing schema.

To configure, add the belongs_to_multitenant declaration to any model that has an association to a particular tenant. This will inject a new SQL scope into your model.

class User < ActiveRecord::Base
  belongs_to :tenant
  belongs_to_multitenant
end

Now, all queries executed within the Multitenant.with_tenant block will automatically be scoped to the current tenant. No longer must you worry about a stray MyModel.all call exposing data to the wrong users! Also, any new records created within this block will automatically be associated to the current tenant. Another win for ensuring data separation!

# use an Rails around_filter to setup this block
Multitenant.with_tenant current_tenant do
  # queries within this block are automatically
  # scoped to the current tenant
  User.all

  # records created within this block are
  # automatically assigned to the current tenant
  User.create :name => 'Bob'
end

Unintentionally leaking data between tenants should be the NUMBER ONE concern when building multi-tenant apps and it surprises me how few resources are available to assist developers in building multi-tenant apps and ensuring data separation between tenants. So, if you're building a multi-tenant Rails app, make sure the Multitenant gem is in your toolbox!

Source code is 100% opensource and available on Github:
https://github.com/wireframe/multitenant

Gem is available on rubygems:
https://rubygems.org/gems/multitenant

Stay Connected with the ensure_connected gem

2011-01-31T21:41:00.000-06:00

This is your application when "MYSQL has gone away"

Most developers working with long running background processes will eventually run into the infamous "MySQL server has gone away" error. This error is thrown when the background processes are idle for an extended period of time and lose their active connection to the database.

There are a few workarounds floating around the interwebs to deal with this exception, but the ensure_connected gem is a clean and programmatic solution to take care of this issue once and for all.

The ensure_connected gem wraps any method with a check to ensure that you are connected to the database before performing work.

class Worker
  def do_something
    #do your work here.  
    #you're connected to the database baby!
  end
  ensure_connected :do_something
end

And Resque users have it even better with the resque-ensure-connected plugin. Simply drop this gem into your project and all of your background job invocations will be wrapped with the ensure_connected check. This works with the Backgrounded Resque integration as well!

As always, sourcecode is 100% opensource and available on Github.
https://github.com/wireframe/ensure_connected
https://github.com/wireframe/resque-ensure-connected

Gems are available for download through Gemcutter.

Send Mail with Backgrounded Mail

2011-01-18T23:38:00.001-06:00

This is what life was like before backgrounded_mail...

Don't tie up your precious resources blocking on a synchronous call to sendmail or SMTP! Use the backgrounded_mail gem to make your application more responsive and deliver email using your background processing library.

The background_mail API is very straightforward. Instead of invoking 'deliver' on the mail object, simply use the 'deliver_backgrounded' method to push the email into your background processing queue.
ex:

UserNotifier.welcome(user).deliver_backgrounded

The backgrounded_mail gem is compatible with Rails3 or any project using the mail gem. It is built on the world class backgrounded library to enqueue email delivery using your background processing library of choice (ex: resque, delayed job, workling, etc).

Sourcecode is available on Github.
https://github.com/wireframe/backgrounded_mail

The 3 E's of Engineering

2011-01-14T21:25:00.000-06:00

Anyone that calls themself an engineer knows "the look". The one that strangers give when you first tell them that you're a Software Engineer. The blank stare, the glazed eyes and the usual lame response about how smart, lucky or rich you must be.

I've had the great privilege over the years to train new developers and mentor developers of the future, and the most important event of someone's first day is for me to burst that great big pink bubble and bring them back to reality.

Software development is hard. It's equal parts art and science, and being a real engineer is anything but easy. Engineer has three e's and every single engineer needs to master The 3 E's of Engineering.

Estimate

A good software engineer needs to estimate tasks accurately. It takes a lot of practice and gets easier the more experience and domain knowledge you have with a given product or area of expertise.

Estimation is about giving a level of effort "guesstimate" for how difficult a task will be. An estimate should be stated in difficulty, not duration. Estimating using hours is an extremely common and disastrous mistake and all estimates should be given in points or a it is as simple as stating a low, medium, or high level of effort for a given task.

An estimate is almost always resource agnostic meaning that if a task that is estimated as "difficult", it will be difficult regardless of who actually does the work. For example, an experienced developer may be able to complete a difficult task quicker than a less experienced developer, but this is explained by each developer having a different personal velocity and does not change the fact that the task was labeled as "difficult".

An estimate is an agreement for how difficult a task will be, but it is not a contract. It is not intended to answer, "How long will this take?" or "When will this be done?". The estimate is only one part of the equation necessary for building an effective product roadmap and you need to understand all the other variables before you can see the whole picture (prioritization and resource allocation are the other critical components, but that topic is for another blog post!). :)

Evaluate

A good software engineer needs to evaluate risk and more importantly to minimize unnecessary risk whenever possible. This is another area where practice and experience are the best ways to improve, but there are a few rules of thumb that should help every developer.

Be fluid, react quickly, and over-communicate changes or risks as they occur. If waterfall software development has taught me anything (other than how horribly broken it is), it is that you can never plan every aspect of a project before starting implementation. Even the best planned and estimated project will still encounter bumps along the way and need adjust accordingly. Over-communicating possible risks to the rest of your team and business is the perfect place for Socialcast. Avoid excessive meetings and endless email threads when raising a red flag.

Use feature flags to roll out new features and test changes incrementally. Support turning on (and off) features at a granular level for an individual user or subset of users before going live for everyone.

Without monitoring, success is just a guess. How do you know that releasing a feature actually improved your product? How do you rollback a change if it negatively impacts your user experience?

Beware the word "re-write" and never ever listen to someone that says you need to "re-write and start over". It's a death sentence.

If something doesn't feel right, it probably isn't. Put the breaks on when changes make the task no longer relevant. Failing fast is actually a huge success! Don't linger on the time you've invested on something that get's thrown away. Dwell on the time saved not finishing the irrelevant project.

Execute

A good software engineer needs to execute and complete tasks quickly and efficiently. Excellence in this area boils down to following a few very simple rules.

Build the absolute simplest thing that could possibly work. No more. No less

It's not done until it's tested and if it's not tested, it's already broken (you just don't know it yet).

Don't check it off as complete until it's in production. Minimize risk to get that shit done and out the door!

Anatomy of an Effective Unit Test

2010-12-08T17:05:00.001-06:00

Writing an effective and expressive unit test is just as important as writing clean application code. Unfortunately, there are not nearly enough resources for how to write good unit tests.

Dr. Sonnek is in the house today with a prescription. Let's start by defining the "Anatomy of an Effective Unit Test".

Every single unit test should be composed of 4 parts: Description, Setup, Execution, Assertion.

To be perfectly clear, I'm not encouraging writing complex unit tests in any way, shape or form. These 4 aspects of a unit test can, and should be, fulfilled in as few as 5 lines of code. Each testing library will have a slightly different feel, but the majority of old school testing frameworks rely on procedural execution or conventions to express these rules (JUnit, JSUnit, TestUnit, etc).

#this is teh old school way to write tests
#test description
def test_creating_blog_sends_email
  #setup
  user = User.create!

  #execute
  blog = user.blogs.create!

  #assert
  assert Mail.sent?
end

Several modern testing frameworks have introduced a clearer separation between these important aspects and use structural separation between setup/execution and assertion which makes it easier to scan tests and read what they are trying to accomplish. Here's a basic example, and I'll break down each section afterwards.

#this is teh new hotness for writing tests
#describe the execution context (data, environment, etc)
context "creating blog post with new user" do
  setup do
    #setup
    user = User.create!

    #execute
    blog = user.blogs.create!
  end
  #assert
  should 'send email to author' do
    assert Mail.sent?
  end
end

I'll use shoulda syntax for these examples, but the same principles apply to RSpec, JSspec, etc. Now, let's dive into the Anatomy of an Effective Unit Test...

Description

Separating the setup/execution/assertions into separate blocks creates self documenting and highly readable tests. The "description" component for each part of your test (setup, execute, assertion) is used to construct a sentence for your test. This is a welcome change from the uglified test naming convention used by legacy test frameworks. The previous example would read:
"creating blog post with new user should send email to author"

Setup

Test setup is used to initialize environmental or data dependencies before execution and assertion. The description for your setup block should clearly describe these dependencies.

context "with new user" do
  setup do
    @user = User.new
  end
end

Execute

Execution is just another form of setup, and lives alongside other environmental or data setup. The description for your execution context should match the method you are invoking and what parameters are being used for this particular invocation.

context "creating blog" do
  setup do
    blog = @user.blogs.create!
  end
end

Assert

One assertion per test. No more...no less. Old school unit tests very commonly have multiple assertions per test but Shoulda makes it super simple to encapsulate each assertion and re-use common setup/execution contexts.

should 'send email to author' do
  assert Mail.sent?
end

Writing effective unit tests is never easy, but structuring your unit tests correctly and understanding the importance of each aspect should make your job just a little bit easier.

Backgrounded Class Methods

2010-10-30T22:22:00.002-05:00

Stay "Class-y" Backgrounded!

Backgrounded was originally intended to only run ActiveRecord instance methods in the background but as the library has matured, its focus has broadened into an elegant proxy to arbitrary background processors.

There are lots of usecases where it's necessary to run a background method on a non-ActiveRecord instance, and the latest Backgrounded release now supports running any class level method with backgrounded! All out of the box back-ends have been updated to respect this new configuration.

Backgrounded class methods work exactly the same as instance methods and there are no client API changes required to use them. Just reference your class level method with a class level backgrounded declaration and you're all set.

class User
  class << self
    backgrounded :my_instance_method
    def my_instance_method
    end
  end
end

#run the instance method in the background
=> User.my_instance_method_backgrounded

Sourcecode on Github.
http://github.com/wireframe/backgrounded

Latest gem on Gemcutter.
https://rubygems.org/gems/backgrounded

New Relic + Backgrounded + Resque = Awesome

2010-10-14T18:15:00.000-05:00

If you're writing a rails app, you're already using new relic to monitor your app right? Wouldn't it be great to get the same performance monitoring of your Backgrounded operations as well?

Fork it up, now you can!

I've published a fork of the rpm_contrib gem that will automatically instrument and monitor your Backgrounded operations with absolutely no code changes! That's right, all you need to do is include the wireframe-rpm_contrib gem and your Backgrounded Resque tasks will automatically be tracked!

Sourcecode available on github (fork of rpm_contrib):
http://github.com/wireframe/rpm_contrib

Gem available on gemcutter:
https://rubygems.org/gems/wireframe-rpm_contrib

The default rpm_contrib gem adds instrumentation for Resque jobs, but is not aware of Backgrounded jobs so all of the jobs were recorded as Backgrounded::Handler::ResqueHandler#perform. My little patch fixes that so that Backgrounded Resque jobs have the correct classname/method context.

Advanced Backgrounded Configuration

2010-10-11T18:00:00.003-05:00

Mr. Sparkle says, "Wow! Backgrounded is super terrific!" (loose translation)

In the wake of the backgrounded ruby gem release, only two complaints have been raised about the library...

First: the backgrounded API is just too damn intuitive. It is literally so beautiful that every other background processing API feels like a giant turd. Using backgrounded abstracts implementation details between various frameworks so developers can focus on writing kick ass apps. Sorry folks, but I have no intention of fixing this issue. :)

Second: The elegance and simplicity of the backgrounded abstraction has proven too limiting for developers that need greater control 'under the hood'. The 0.6.0 release of backgrounded should satisfy these power hungary devs by supporting an optional configuration hash for each backgrounded method. Any options specified in your backgrounded class will be passed through to tweak the specific background processing handler in use.

The first backgrounded handler to honor the optional configuration is the Resque integration which now supports configuration of the particular queue the backgrounded method will be run in. The default behavior is to run all backgrounded methods in a single default queue (named backgrounded) and this can be overridden by passing in the optional :queue name for any backgrounded method.

It is vitally important that the introduction of these advanced configuration options not break the existing API. The beauty and simplicity of the original backgrounded API is still very much intact and all existing clients will continue with the default behavior with no change. Here's an example to show how the original backgrounded methods can be configured right alongside with advanced backgrounded methods.

class Blog
  backgrounded save_to_filesystem, :email_followers => {:queue => :low}

  def email_followers
  end
  def save_to_filesystem
  end
end

Sourcecode on Github.
http://github.com/wireframe/backgrounded

Latest gem on Gemcutter.
https://rubygems.org/gems/backgrounded

Backgrounded Resque Support

2010-09-27T19:14:00.001-05:00

After an extended rest, Backgrounded is back in action with support for Resque.

Resque is one of the newer background processing frameworks for ruby applications created by the amazing folks at Github. A weekend of hacking with Resque easily persuaded me to add official support for Resque to the Backgrounded 0.4.1 release.

After getting Resque up and running, the Backgrounded configuration is simple (as always).

#config/initializers/backgrounded.rb
require 'backgrounded/handler/resque_handler'
Backgrounded.handler = Backgrounded::Handler::ResqueHandler.new

The benefits of Backgrounded really shine when new libraries like Resque pop up. One of the primary goals of Backgrounded is to make it trivial to swap out different backend implementations and not change a single line of your application code. Just update your configuration file and restart. It's a thing of beauty...

Sourcecode on Github.
http://github.com/wireframe/backgrounded

Latest gem on Gemcutter.
https://rubygems.org/gems/backgrounded

email_preview Rails Plugin

2010-06-09T23:44:00.000-05:00

As a developer, working with email pretty much sucks. Crafting emails is a time consuming task where you constantly refine and tweak markup to get the perfect layout for your message. Heck, it's easier to do cross browser testing of your application for IE6!

The email_preview engine is a new Rails 3 plugin I've released to help developers build kick ass emails for your Rails 3 app. It provides a quick and easy way to preview sample emails from within your web browser and see how they will look in the end user's email client.

The developer API is extremely simple. You can use the standard Rails ActionMailer interface to configure each instance, or manually configure each email using the excellent mail library.

#config/email_preview.rb

EmailPreview.register 'user activation email' do
  u = User.new :email => 'foo@example.com'
  UserNotifier.activation
end

EmailPreview.register 'simple example email' do
  Mail.new do
    to 'tom@example.com'
    from 'me@foo.com'
    body 'check this out'
  end
end

EmailPreview.register 'multipart email (html + text)' do
  Mail.new do
    from 'mikel@test.lindsaar.net'
    to 'you@test.lindsaar.net'
    subject 'This is a test email'

    text_part do
      body 'This is plain text'
    end
    html_part do
      content_type 'text/html; charset=UTF-8'
      body '<h1>This is HTML</h1>'
    end
  end
end

Once you've configuring your sample emails, startup your Rails 3 application and preview them in any web browser at http://localhost:3000/email_preview. All email templates are re-rendered with each request so you can tweak/save/refresh to instantly preview how your emails will look for your users. Talk about quick turnaround time!

Need more assurance that your emails will look correct? If your dev environment is configured to send emails, you can send any of your email previews to your inbox by filling out the convenient form at the top of each page.

This plugin is 100% opensource and the code is available on github at http://github.com/wireframe/email_preview. Patches are *always* welcome, and if you have any questions or suggestions, please drop me a line.

Zillow Google Chrome Extension

2010-03-22T22:00:00.001-05:00

Incase you haven't heard, I love me some Google Chrome extensions. And now the Zillow Google Chrome extension just made looking for a new house a heck of a lot easier.

Grab an address, pop open the Zillow Google Chrome extension and find the estimated property value in no time!

The Zillow Google Chrome extension is a mashup using the Google Maps API and the Zillow API to quickly find as much detail about a property while avoiding the cluttered and slow ass real-estate websites. Consider this the "official" successor to my original Zillow Ubiquity plugin. Ubiquity was cool but this extension is the cat's meow!

The Zillow Google Chrome extension shows the estimated property value along with the estimated high/low range value. A direct link to the detailed estimate on Zillow is also available if you want more information about the property.

The property location is displayed with an embedded Google map along with a direct link to a full property map to dive into the street view (on Google Maps of course). Using Google's geolocation service also makes it possible to quickly find a location even if you don't know the full address!

The code is 100% opensource and available on github and here are a few ideas for future improvements. fork away!

an extension browser action could detect when the user selects an address on the page
prefill the popup address based on the user selected address on the page
hovering over address could display an inline popup bubble with the estimate information

Rails "Good Touch"/"Bad Touch"

2010-03-12T21:57:00.002-06:00

Rails is bit like a dirty old man.

Consider this example

class User < ActiveRecord::Base
  after_save :do_something_expensive
end

Rails makes it trivial to update timestamps, and I found out the hard way that the underlying implementation leaves a lot to be desired.

User.first.touch

Easy, right? But, did you know that this useful little method will fire all of your after_save callbacks? This led to some nasty performance problems recently, and now you can now stop Rails from having it's way with you! Bad Touch Rails!

The good_touch project is now available on github and gemcutter and eliminates the overhead for updating a simple timestamp attribute. It's as easy as using the standard/bad touch method without all the horseplay.

User.first.good_touch

Let the touching continue...

Improvements for apn_on_rails

2010-02-08T20:00:00.011-06:00

If you're using Rails and integrate with Apple Push Notifications, you are more than likely familiar with the apn_on_rails gem. It's a fantastic library and has been critical for getting off the ground with push notifications quickly. The project is currently in flux and has been missing some critical features, so I decided to dig into the source myself and hack away.

Using custom metadata to each push notification was needed right off the bat. Basic push notifications support a "badge", "alert", and "sound", but custom attributes can be used to push metadata that would be useful for your native application. The original apn_on_rails plugin does not support custom attributes, but after building on the work done by lmarburger, custom payload attributes are working great.

Truncation of alert text is another critical feature that needed some love. Apple limits push notifications to a maximum of 255 characters which includes the alert text and all custom attributes so it is very important to optimize the data you're sending down and not go over the 255 character limit. The initial gem used a very simple implementation for truncating the text, which worked fine until you started to push custom attributes as well. Those valuable characters got eaten up in a hurry and the truncation of the alert text needed to be aware of your entire payload length.

With a few hours of work, the entire truncation algorithm has been rewritten (with tests) and is working great. This new algorithm will dynamically truncate your alert text based on your entire payload body. This is a much better algorithm for text truncation and I've seen a 20% increase in the amount of text sent with push alerts before the truncation kicks in (average from 179 to 215 characters).

I also fixed a pretty major bug when truncating notifications with multibyte characters. Anyone using apn_on_rails for an internationalized app should definitely consider the update.

My fork of the original apn_on_rails gem is available on github and the updated gem is published and available on gemcutter.

Mailinator Google Chrome Extension

2010-02-07T21:40:00.000-06:00

Have you ever visited a website that required an email registration form, but you really didn't want to give out your primary email address? Mailinator is a wonderful service that helps rid the world of junk mail, one garbage email address at a time.

Using a one-time use email inbox is a great way to keep unsolicited email out of your way. And now the Mailinator Google Chrome extension makes it even easier to generate a one-time email address to get you through those annoying site registration forms.

Features include:

one click generation of a random email address
popup widget to check your mailinator email inbox

Under the covers, the plugin is very simple. Mailinator provides a pretty decent iframe integration out of the box, and generating a random email inbox in Javascript is pretty trivial.

The code is 100% open source and available on github. I'd love to add in a "Copy address to clipboard" feature, but it looks like Flash is the only way to accomplish that. Let me know if you have any other suggestions.

I have no affiliation with Mailinator. I'm just a hacker that is very sick of sites requiring email addresses and not offering open ID integration. :)

#airfail for HTML Developers

2010-01-10T22:07:00.003-06:00

Opinions on ADOBE AIR

Incase you haven't heard, Adobe AIR is a complete and utter failure for developing desktop applications with HTML/CSS/Javascript technologies. It's primary purpose is to provide Flash/Flex developers the ability to exploit the desktop. It's secondary purpose is to simply frustrate and antagonize HTML/CSS developers past the point of frustration.

Here is a short list of the very real issues I've encountered while building and deploying AIR applications. This is not simply a list of problems dug up from an issue tracker or forum. Every one is an issue I've worked through to the point of pulling my hair out. This list should serve as a warning to any potential developers considering Adobe AIR. You'll regret it...mark my words. The fact that Adobe advertises AIR as a viable runtime for HTML/CSS developers is one of the greatest crimes against humanity I have witnessed in recent history.

No HTTP Headers sent with file uploads
Dealing with HTTP headers is an extremely standard requirement especially when working with secured services that require Basic Authentication. The AIR SDK has fairly well documented methods to configure sending HTTP headers with outgoing requests, BUT for some reason the SDK fine print states that "custom HTTP request headers are not supported in uploads or downloads". Excuse me? You're kidding right? Multipart HTTP multipart headers are already being sent with file uploads so why the hell prevent other headers from being added? Why support HTTP headers for every remote request except for file uploads? The AIR SDK gives you direct access to a Socket for God's sake. Do I have to write my own fucking multipart upload?

Every single application that relies on Flash technology, which includes web based SWF projects as well as HTML and Flex AIR apps, suffer from this bug. This blatant hole is yet another reason Flash will be dead as soon as HTML5 is more widely adopted.

random "corrupt" installation file
Try to install an app and receive a "corrupt file" error. reinstall the app and it works fine. Awesome. Our QA team has encountered this issue frequently enough that it's become standard issue to just ignore the errors.

Integration with Native OS (Or lack there of)

AIR offers so little integration with "native" features, that it's hard to call out any particular failures and I guess that's one of my biggest gripes with AIR. If you're building a runtime for desktop applications, having integration with the native desktop OS is a pretty necessary and compelling feature.

No Integration with Native Notification Systems
Adobe supposedly was working on this back in 2008. Seriously? It takes two years to integrate with one of the most standard core features provided by modern day operating systems? Growl has been around on Mac for years. Linux has libnotify and Windows has Snarl. Titanium has had this built into their SDK since practically day one.

Sure, you can emulate these notifications with AIR, but it is such a horrible, horrible hack and forces every single app to reinvent the wheel.

no integration with dock icons
most commonly used on Mac, but applicable to other platforms as well. Fluid is a perfect example of exposing a simple api to control dock icons. This is a minor issue and not directly a "bug" but this came up frequently in our business requirements and AIR was unable to perform the expected task.

sharing cookies with browser
This is just plain disgusting. Why in the world would you build a runtime that is coupled to the underlying OS session/cookie management system? This extremely unexpected behavior which has cost me tons of troubleshooting time where a user could log into a site using their web browser and essentially "hijack" the credentials of the AIR application.

Window Handcuffs
The AIR windowing API is clunky and frustrating pile. Not only that, but it also is broken.

lightweight windows stealing window focus
hidden application windows on OSX will be activated and focused when receiving popup notification. (blog post coming soon)

No global mouse move events
With no way to detect mouse pointer location when outside of your application, it is absolutely impossible for you to build headless applications that listen and react to mouse gestures.

Crippled Webkit

Webkit is an extremely fast moving project. There are new features being added every day, and when Adobe chose to include Webkit into their runtime, I'd expect them to include info on the version of Webkit that is included. Is CSS3 available? is HTML5 available? Well, it's up to the developer to find out with good old trial and error. Here are some of the more "common" things I was unable to do with the specific version of webkit included in AIR 1.5.

doesn't support webkit custom scrollbars
Sporadic support for HTML5 attributes. placeholder attribute available for text input, but not textarea
doesn't support title attribute for very standard browser tooltips.

I consider these Adobe's issues because the version of Webkit included in the AIR runtime is ridiculously out of date. I've been implementing these features in actual web apps for nearly a year now (and there's no reason the title attribute doesn't work). AIR2 will "supposedly" be updating it's version of Webkit, but I highly doubt that Adobe will be smart enough to provide reference documentation for developers to know what features are available.

Cross Platform Issues

No multiplatform runtime is perfect, but web browsers are by far the best technology available to provide a consistent cross platform experience. Webkit was an excellent starting point for AIR, but the underlying runtime has plenty of cross platform issues. Here are some of the major issues I've run into

Windows

Linux

Images hosted on SSL CDN cause system alerts

Mac

Keychain prompts for PrivateEncryptedData

Performance

I'm very aware that there is a cost to run an application in a runtime environment. I've built Java applications for years and had to deal with the memory bloat for apps running in a Java runtime, but I don't think I'm asking a lot here.

The majority of AIR applications are mashup/widgets that are intended to be always on background services, but nearly every AIR application sits idle at ~10% CPU on a dual core Macbook Pro. It's simply ridiculous and again all fingers point to the underlying flash runtime. One common trick is to manually override the flash framerate to improve performance. It's pretty ridiculous that the runtime isn't smart enough to handle this when your application is minimized or otherwise hidden.

AIR Lightweight Windows Steal Focus By Default

2009-12-23T23:31:00.000-06:00

Lightweight windows are intended for AIR applications to show simple popup windows and notifications (like my emulated growl notification system). This is pretty standard stuff and there are a number of documented tutorials and examples that go over this.

var options = new air.NativeWindowInitOptions();
options.type = air.NativeWindowType.LIGHTWEIGHT;
options.systemChrome = air.NativeWindowSystemChrome.NONE;
var visible = true;
var scrollbars = false;

var bounds = new air.Rectangle(200, 250, 300, 400);

var htmlLoader = air.HTMLLoader.createRootWindow(visible, options, scrollbars, bounds);
htmlLoader.window.nativeWindow.alwaysInFront = true;
htmlLoader.load(new air.URLRequest("myfile.html"));

Executing this simple example will show one very odd and obvious interaction issue. Your supposedly "lightweight" window will steal the focus from your main application window. Huh, that doesn't seem very lightweight to me.

The issue is that any window that is created with "visible" set to "true" will automatically be activated once content is loaded. What a nice little gotcha. Thanks for the info Adobe!

In order to work around this issue and get your lightweight window to behave as expected, you need to initialize it to hidden and then flip it to visible after the content is loaded using a completely unnecessary extra event listener.

htmlLoader.addEventListener(air.Event.COMPLETE, function() {
  htmlLoader.window.nativeWindow.visible = true;
});

#airfail

Delicious Google Chrome Extension

2009-12-16T18:22:00.002-06:00

I use Delicious. A LOT.

It's one of the few web services that I have used religiously for years. One of the main reasons I continue to use Delicious is sheer number of great integrations with their developer API. The Delicious plugin for Firefox is quite good and there is nothing equivalent available for my new default browser, Google Chrome...until now!

I'm excited to officially announce the release of my Delicious Chrome extension!

The Delicious Chrome extension adds a browser action that will save your current page as a Delicious bookmark. The extension will also show popular and recommended tags for the current page and you can add these tags to your bookmark with a simple click.

But that's not all! The Delicious Chrome extension also integrates with the standard Chrome bookmark stars. Starring a page will automatically save the page on Delicious as well! This should greatly help users add Delicious to their existing browsing behavior without having to learn a new tool.

This extension is 100% opensource and contributions are always welcome! Please leave any feedback here or on the extension install page.

Batch Javascript Execution

2009-11-25T16:57:00.001-06:00

Let's say you're in a bit of a mess with the performance of your fancy web application's Javascript. Your application needs to do a lot of work on the front end. I'm talking a lot of repetitive work that takes a long time. Since Javascript is single threaded the end user's experience is pretty horrible. Their web browser essentially "locks up" while you do all of your work and Firefox users will even be prompted with "Unresponsive script" alerts. How are you going to optimize this?

The "correct" answer is to either reduce the number of times you need to do the work, or to make each execution as fast as humanly possible. Although that is a great suggestion, it's also potentially expensive and time consuming. There comes a point of diminishing returns when you've fixed all the low-hanging fruit and you're only shaving off milliseconds with each change. What if we could get a great user experience without going through all that work? Well, now you can!

The trick is to allow the browser to breathe between executions. Executing your work in batches gives the browser time to re-draw and keeps the UI from locking up. Although we're actually increasing the total execution time for doing your work due to the extra delays, the browser remains much more responsive and it greatly improves the end user experience.

As an example, consider an application that is constructing a *huge* table via javascript. (Let's ignore the fact that the unit of work is not optimized like it should be)

$.each(myHugeArray, appendTable);
function appendTable() {
  $('<tr><td>test</td></tr>').appendTo($('#myTable'));
}

This is your Javascript...

Given enough iterations, this implementation will cause the browser to grind to a halt until the work is complete. Instead of doing all the work at once, we can break it up into batches to help the end user experience. Now, let's look at this solution that does the exact same work, just broken up into batches.

doInBatchesWithDelay(myHugeArray, appendTable, 5, 10);
function appendTable() {
  $('<tr><td>test</td></tr>').appendTo($('#myTable'));
}
function doInBatchesWithDelay(array, callback, batchSize, delay) {
  for (var x = 0; x < batchSize; x++) {
    if (array.length >0) {
      callback(array.pop());
    }
  }
  if (array.length > 0) {
    setTimeout(function() { 
      doInBatchesWithDelay(array, callback, batchSize, delay) 
    }, delay);
  }
}

This is your Javascript in BATCHES! It's still slow, but it'll kick your face in a punch fight!

This might be considered a hack, but not so! The end user experience is all that matters, and they have no concern for what tricks we have to use to keep the UI snappy. The cost to profile and optimize applications is usually quite high and this is something you can easily drop in today and quiet the complaints of application sluggishness. Addressing their problem with this small change will give you some headroom to work on rearchitecting your slow-ass implementation if needed.

Now, go forth and batch!

Life at the Speed of Green

callback_skipper for faster factories

Cron Backgrounded Resque Jobs

Cleaner RSpec/FactoryGirl Integration

Guard JSLint on Rails Released!

First: install the gem...

Next: configure guard...

JSLint Rails Javascript Responses

Move Javascript Logic to Static Component

JSLint the Rails Response

Email Preview + Transactions and Factories

Transactional Fixtures

Factories

Categories

Realtime Search: Solr vs Elasticsearch

Who will win the crown for the King of Realtime Search?

Multitenant Ruby Gem

Stay Connected with the ensure_connected gem

Send Mail with Backgrounded Mail

The 3 E's of Engineering

Estimate

Evaluate

Execute

Anatomy of an Effective Unit Test

Description

Setup

Execute

Assert

Backgrounded Class Methods

Stay "Class-y" Backgrounded!

New Relic + Backgrounded + Resque = Awesome

Advanced Backgrounded Configuration

Mr. Sparkle says, "Wow! Backgrounded is super terrific!" (loose translation)

Backgrounded Resque Support

email_preview Rails Plugin

Zillow Google Chrome Extension

Rails "Good Touch"/"Bad Touch"

Improvements for apn_on_rails

Mailinator Google Chrome Extension

#airfail for HTML Developers

Integration with Native OS (Or lack there of)

Crippled Webkit

Cross Platform Issues

Performance

AIR Lightweight Windows Steal Focus By Default

Delicious Google Chrome Extension

Batch Javascript Execution