The issue

Quoting the Lighthouse ticket, imagine the following scenario in Rails 2.3:

class Page < ActiveRecord::Base
  default_scope :conditions => { :deleted_at => nil }
 
  def self.deleted
    with_exclusive_scope :find => { :conditions => "pages.deleted_at IS NOT NULL" } do
      all
    end
  end
end

If you rewrite it to the new 3.0 syntax, your first attempt would probably be:

class Page < ActiveRecord::Base
  default_scope where(:deleted_at => nil)
 
  def self.deleted
    with_exclusive_scope :find => where('pages.deleted_at IS NOT NULL') do
      all
    end
  end
end

However, if you try it out on console, you will find out it does not work as expected:

Page.all         #=> SELECT "pages".* FROM "pages" WHERE ("pages"."deleted_at" IS NULL)
Page.deleted.all #=> SELECT "pages".* FROM "pages" WHERE ("pages"."deleted_at" IS NULL) AND ("pages"."deleted_at" IS NOT NULL)

To understand why it does not work, let’s take a look at the source code!

Investigating the issue

With Active Relation, Active Record is no longer responsible to build queries. That said, ActiveRecord::Base is not the one that implements where() and friends, in fact, it simply delegates to an ActiveRecord::Relation object. From ActiveRecord::Base source code:

delegate :select, :group, :order, :limit, :joins, :where, :preload, :eager_load, :includes, :from, :lock, :readonly, :having, :create_with, :to => :scoped

And the scoped implementation is shown below:

def scoped(options = nil)
  if options.present?
    scoped.apply_finder_options(options)
  else
    current_scoped_methods ? relation.merge(current_scoped_methods) : relation.clone
  end
end
 
def relation
  @relation ||= ActiveRecord::Relation.new(self, arel_table)
  finder_needs_type_condition? ? @relation.where(type_condition) : @relation
end

As you can see, scoped always returns an ActiveRecord::Relation that you build your query on top of (notice that ARel::Relation is not the same as ActiveRecord::Relation).

Besides, if there is any current_scoped_methods, the scoped method is responsible to merge this current scope into the raw relation. This is where things get interesting.

When you create your model, current_scoped_methods returns by default nil. However, when you define a default_scope, the current scope now becomes the relation given to default_scope, meaning that, every time you call scoped, it returns the raw relation merged with your default scope.

The whole idea of with_exclusive_scope is to be able to make a query without taking the default scope into account, just the relation you give in as argument. That said, it basically sets the current_scope_methods back to nil, so every time you call scoped to build your queries, it will be built on top of the raw relation without the default scope.

With that in mind, if we look again at the code which we were trying to port from Rails 2.3, we can finally understand what was happening:

def self.deleted
  with_exclusive_scope :find => where('pages.deleted_at IS NOT NULL') do
    self
  end
end

When we called where('pages.deleted_at IS NOT NULL') above, we were doing the same as: scoped.where('pages.deleted_at IS NOT NULL'). But, as scoped was called outside the with_exclusive_scope block, it means that the relation given as argument to :find was built on top of default_scope explaining the query we saw as results.

For example, the following syntax would work as expected:

def self.deleted
  with_exclusive_scope do
    where('pages.deleted_at IS NOT NULL').all
  end
end

Since we are calling where inside the block, the scoped method no longer takes the default scope into account. However, moving the relation inside the block is not the same as specifying it to :find, because if we were doing three queries inside the block, we would have to specify the same relation three times (or refactor the whole code to always do a query on top of this new relation).

That said, it seems the previous with_exclusive_scope syntax does not suit very well with ActiveRecord’s new API. Maybe is it time for change? Can we provide a better API? Which are the use cases?

Identifying the use cases

The with_exclusive_scope method has mainly two use cases. The first one, which we just discussed above, is to allow us to make a query without taking the default scope into account inside our models:

def self.deleted
  with_exclusive_scope do
    where('pages.deleted_at IS NOT NULL').all
  end
end

While this code looks ok, if we think about relations, we will realize that we don’t need to give a block to achieve the behavior we want. If the scoped method returns a raw relation with the default scope, couldn’t we have a method that always returns the raw relation? Allowing us to build our query without taking the default scope into account?

In fact, this method was already implemented in Active Record and it is called unscoped. That said, the code above could simply be rewritten as:

def self.deleted
  unscoped.where('pages.deleted_at IS NOT NULL').all
end

Much simpler! So, it seems that we don’t need to support the block usage at all, confirm?

Deny! Going back to the Page example above, it seems we should never see deleted pages, that’s why we set the default_scope to :deleted_at => nil. However, if this application has an admin section, the admin may want to see all pages, including the deleted ones.

That said, what we could do is to have one controller for the normal User and another for the Admin. In the former, we would always use Page.all, and Page.unscoped.all in the latter.

However, if these controllers and views are very similar, you may not want do duplicate everything. Maybe it would be easier if we do something like this:

def resource_class
  if current_user.is_admin?
    Page.unscoped
  else
    Page
  end
end

And, instead of always referencing the Page class directly in our actions, we could call resource_class. While this solution is also ok, there is a final alternative, that would require no changes to the current code. If you want to use the same controller for different roles, but changing the scope of what they are allowed to see, you could simply use an around_filter to change the model scope during the execution of an action. Here is an example:

class PagesController < ApplicationController
  around_filter :apply_scope
 
  # some code ...
 
  protected
 
  def apply_scope
    if current_user.admin?
      Page.with_exclusive_scope { yield }
    else
      yield
    end
  end
end

That said, being allowed to give a block to with_exclusive_scope is actually useful and since we want to deprecate with_exclusive_scope in favor of unscoped in the future, we brought this very same syntax to unscoped as well:

def apply_scope
  if current_user.admin?
    Page.unscoped { yield }
  else
    yield
  end
end

Tidying it up

Well, after the behavior in with_exclusive_scope was properly ported to the new API, we need to be sure we are not forgetting about anything… wait, actually we are.

with_exclusive_scope has an evil twin brother called with_scope which behaves very similarly, except that it always build the query on top of the scoped relation. It works like this:

class Page < ActiveRecord::Base
  default_scope where(:deleted_at => nil)
end
 
Page.with_scope :find => { :conditions => { :active => true } } do
  Page.all #=> Bring all active pages that were not deleted
end

However, this feels way too hash-ish. Of course, we could use relations to make it a bit prettier:

Page.with_scope :find => where(:active => true) do
  Page.all #=> Bring all active pages that were not deleted
end

This is ok, but it seems that we could improve it even more. That said, we added a new method to relations, called scoping:

Page.where(:active => true).scoping do
  Page.all #=> Bring all active pages that were not deleted
end

Yeah! Sign me up ’cause this looks way better than the previous syntax! And, if you check the original commit, you will notice the unscoped method with a block simply delegates scoping:

def unscoped
  block_given? ? relation.scoping { yield } : relation
end

So, with unscoped and scoping implemented, we just need to commit, git push and be happy, confirm? Deny! There is one last case to check.

create_with

If you payed attention properly, you can notice that every time we called with_exclusive_scope and with_scope, we always passed { :find => relation } as hash, instead of simply giving the relation. This happens because these methods accept two hash keys: find and create.

As you may expect, one specifies the behavior for create and the other for finding. In most of the cases, they are exactly the same and work with the new syntax:

page = Page.where(:active => true).new
page.active #=> true

However, for obvious reasons, this only works if the conditions are given as a hash. Consider this case:

page = Page.where("active = true").new
page.active #=> nil

That said, there may be a few scenarios where you want to specify the creation conditions on its own, explaining the :find and :create options in with_exclusive_scope and with_scope methods. So, how can I achieve it with the new syntax? Easy!

page = Page.create_with(:active => true).new
page.active #=> true

If you provide both conditions as a hash and create_with, create_with always have higher priority:

page = Page.where(:active => false).create_with(:active => true).new
page.active #=> true

Note this syntax already existed, we are just making it explicit now as part of the new API! That said, commit, push and be happy!

Wrapping up

All in all, with_exclusive_scope and with_scope are now part of the old ActiveRecord API giving place to the new, strong and vibrant unscoped and scoping methods!

However, they are not going to be deprecated now. They will follow the same deprecation strategy as all the current methods.

And you? What do you think about this new scoping API?

SimpleForm inputs

From the README:

Forms made easy (for Rails)!

SimpleForm aims to be as flexible as possible while helping you with powerful components to create your forms. The basic goal of simple form is to not touch your way of defining the layout, letting you find the better design for your eyes. Good part of the DSL was inherited from Formtastic, which we are thankful for and should make you feel right at home.

As the README says, SimpleForm is a tool to help you build forms easily in Rails. Let’s see some examples:

<%= simple_form_for @user do |f| %>
  <%= f.input :username, :label => 'Your username please' %>
  <%= f.input :password, :hint => 'No special characters.' %>
  <%= f.input :remember_me, :as => :boolean %>
  <%= f.button :submit %>
<% end -%>

There are plenty of things going on here: we create a form using simple_form_for helper, then we use the :input method to create input elements based on column type. For instance, :username will create a default text input, while :password attribute will render an input type password. For the :username attribute, we are specifying a label manually. For :password, the label will be taken from I18n, and we are adding a hint message to the field. For :remember_me, we are explicitly saying to render it as a checkbox, using the :as => :boolean option (that is the default for boolean attributes). Also, there is a button helper that simply delegates to Rails helpers, in this case submit.

The output for a new @user would be:

<form action="/users" class="simple_form user" id="new_user" method="post">
  <div class="input string required">
    <label class="string required" for="user_username"><abbr title="required">*</abbr> Your username please</label>
    <input class="string required" id="user_username" maxlength="255" name="user[username]" size="50" type="text" />
  </div> 
  <div class="input password required">
    <label class="password required" for="user_password"><abbr title="required">*</abbr> Password</label>
    <input class="password required" id="user_password" name="user[password]" size="30" type="password" />
    <span class="hint">No special characters.</span>
  </div> 
  <div class="input boolean optional">
    <label class="boolean optional" for="user_remember_me"> Remember me</label>
    <input name="user[remember_me]" type="hidden" value="0" />
    <input class="boolean optional" id="user_remember_me" name="user[remember_me]" type="checkbox" value="1" />
  </div> 
  <input id="user_submit" name="commit" type="submit" value="Create User" /> 
</form>

You may have noticed there is some additional css classes added to the markup, like string and required. They are added automatically by SimpleForm to help us style and plug some javascript in. There are specific css classes for each available input type. Also, pay some attention to the label: inside it there is an abbr tag with an asterisk (*) showing that the field is required. SimpleForm uses the new validations reflection API from Rails 3 to check if the attribute has the presence validator, and mark the field as required if so. And we are able to say that a field is required or disable the required mark, by passing the option :required => true|false.

Furthermore, there is the hint tag for the :password attribute that SimpleForm creates based on the :hint option we have defined. Also notice that the gem has automatically added a div wrapper to each input, with the same css classes. SimpleForm allows us to configure this wrapper as well, using for instance p instead of div. We are going to see more about configuration later.

SimpleForm is already prepared to generate some of the new HTML 5 input tags, such as email, url and number inputs:

<%= simple_form_for @user do |f| %>
  <%= f.input :website, :as => :url %>
  <%= f.input :email %>
  <%= f.input :age, :hint => "This defaults to 'number' input based on field type" %>
  <%= f.button :submit %>
<% end -%>

Based on the attribute name, SimpleForm will generate url or email input types, and we can always set a specific type with the :as option. Numeric attributes will always be rendered as input type number.

Working with associations

SimpleForm adds a custom and straightforward method to render select tags for associations, called association. For now, consider our User belongs to a Company, and has and belongs to many Roles. Let’s go straight to the example:

<%= simple_form_for @user do |f| %>
  <%= f.input :name %>
  <%= f.association :company %>
  <%= f.association :roles %>
  <%= f.button :submit %>
<% end -%>

It will detect the association type and render a select tag for choosing the company, listing all companies in the database, and another select for roles, with multiple option enabled.

SimpleForm also has some add-ons, letting us render associations as a collection of radios or check boxes. Using the same example:

  f.association :company, :as => :radio
  f.association :roles, :as => :check_boxes

Now we are rendering a collection of radios for choosing the Company, and another collection of check boxes for choosing Roles.

Configuration

SimpleForm lets us do some customizations by running its install generator:

rails generate simple_form:install

# Output
  create  config/initializers/simple_form.rb
  create  config/locales/simple_form.en.yml
  create  lib/templates/erb/scaffold/_form.html.erb

As we can see, running this generator will copy an initializer file, responsible for configuring SimpleForm; a locale file, to let us change some I18n messages; and a form template inside our lib dir. This template will be used instead of the default Rails scaffold form template, so it will create our form already using SimpleForm. Easy, right?

Let’s take a look at some configuration options:

• components: defines the components used by the form builder. We can remove any of them, change the order, or add new ones. Defaults to [ :label, :input, :hint, :error ].
• hint_tag: tag used for hints, defaults to span.
• error_tag: tag used for errors, defaults to span.
• wrapper_tag: tag used as wrapper to all inputs, defaults to div
• label_text: determines how the label text should be generated altogether with the required text. It must be a lambda/proc that receives both label and required texts. Defaults to "required label".

There are a lot more options available in the initializer file, such as default input size and priority countries for generating country selects. Also, the locale file lets us determine the required text and mark, or even the entire required html tag.

Internationalization

SimpleForm is ready for I18n, supporting labels and hints. In addition, it lets us set different content for each action, new and edit. Here is an example locale file:

  en:
    simple_form:
      labels:
        user:
          username: 'User name'
          password: 'Password'
          edit:
            username: 'Change user name'
            password: 'Change password'
      hints:
        user:
          username: 'User name to sign in.'
          password: 'No special characters, please.'

Simple, right? If it does not find any specific translation using I18n for the label, it will fallback to human_attribute_name.

Here we go!

SimpleForm has much more to offer. We would like to invite you to take a better look at the examples and possibilities. Remember, SimpleForm aims to be flexible and powerful to help you easily build forms, without saying how you should create your markup.

Also, feel free to explore the source code and extend SimpleForm even further. Since it’s based on components, creating a new component which moves the current hints to inside the input (using javascript or the new placehoder attribute in HTML 5), should be easy!

It’s worth saying SimpleForm is Rails 3 compatible in the master branch. If you are using Rails 2.3.x, there is a v1.0 branch and version that you might want to take a look.

SimpleForm has been helping us a lot so far, we hope you enjoy it. Moreover, we would like to enjoy other tools that help your productivity day by day, please leave a comment and let us know, we would appreciate a lot!

Without further ado, here follows my talk video:

DSL or NoDSL by José Valim from Krakow Tech Conferences.

Also, I uploaded my slides (even though they do not make a lot of sense by themselves) as a way to complete the awesome list of resources being wrapped at http://euruko2010.heroku.com/! I’m definitely looking forward to Euruko 2011 in Berlin!

José Valim, Lead Developer at Plataforma Tecnologia and Rails Core Member, will be in Barcelona from 12nd to 19th June. Between trying some Tapas and Paellas, José Valim will be at Spain to open new business opportunities.

So if you are running a company in Barcelona, Madrid or nearby, this is a great opportunity for a Training Session in Rails 3 or a tech-talk about Plataforma Tec’s development tools – with Valim in person. Or if you want to start a new project as well, this would be a great opportunity for a kick-off meeting!

If you’re interested or have other ideas that you want to discuss, please contact us.

And last but not least, Valim will also give a talk at Barcelona on Rails on the 17th, so don’t forget to stop by and say hi! If you are not from Barcelona and would like to talk with us, you are welcome as well!

name = "George Guimarães"

This is annoying for brazilians and anyone who uses non-ASCII characters. The problem is that the default Ruby in Mac OS X isn’t linked against readline. A simple solution is to compile readline on your system, and then compile your own ruby binaries. A better one is to use tools that automate this process.

This post by Christopher Sexton uses homebrew to compile readline-0.6 but compiles ruby by hand. Since we love RVM, we’ll use it too.

If you use Homebrew (and you *should*), just do

brew install readline
brew link readline

Beware: linking readline into your system may break other tools that depends on readline source to compile. It was harmless on my system.

Ok, so now we want to build a new ruby and irb binaries. I wanted to use ruby 1.8.7-p248, so:

rvm install 1.8.7-p248 -C --enable-shared,--with-readline-dir=/usr/local

This tells the configure script to enable shared library linking (it is the default), and to search for readline in /usr/local (homebrew has just linked readline there). You may have to use --force if you already have this ruby version compiled. Newer versions of RVM do not need --force.

Now you can use accented and unicode characters on keyboard input in IRB using ruby 1.8.7.

And you? Do you have any tricks with IRB that you may want to share? Do you use Wirble, Utility Belt or others?

In this post, I want to tell you which are the three test suites that I admire the most and why.

Integration award: Railties

Rails 3 has several improvements and not all of them may be visible to the application developer. One of the hidden unicorns is Railties test suite. As Yehuda stated in a blog post during the refactoring of version 2.3 to 3.0:

“Although the Rails initializer tests covered a fair amount of area, successfully getting the tests to pass did not guarantee that Rails booted.”

This happened because, in order to have fast tests, Rails 2.3 suite stubbed and mocked a significant part of the booting process. The new test suite is able to create a new application using the application generator, change configuration options, add plugins and engines, boot it and even make HTTP requests using Rack::Test.

For instance, take a look at this test which ensures that app/metals inside plugins are successfully added to the application middleware stack:

def test_plugin_metals_added_to_middleware_stack
  @plugin.write 'app/metal/foo_metal.rb', <<-RUBY
    class FooMetal
      def self.call(env)
        [200, { "Content-Type" => "text/html"}, ["FooMetal"]]
      end
    end
  RUBY
 
  boot_rails
  require 'rack/test'
  extend Rack::Test::Methods
 
  get "/not/slash"
  assert_equal 200, last_response.status
  assert_equal "FooMetal", last_response.body
end

The most important lesson here is: whenever mocking or stubbing in our tests, we still need to add tests without the mocks and stubs to ensure all API contracts are respected.

Readability award: Capybara

Capybara is a tool to aid writing acceptance tests for web applications. Capybara can use several drivers to interact with a web application, as Selenium, Celerity or even Rack::Test. Each driver needs a different setup and has different features. For instance, both Selenium and Celerity can handle javascript, but not Rack::Test.

As you may imagine, all these different drivers can make a test suite become a real spaghetti. However, Jonas Nicklas was able to transform a potential problem into a very elegant and readable test suite with Rspec help. Here is, for instance, the tests for selenium:

describe Capybara::Driver::Selenium do
  before do
    @driver = Capybara::Driver::Selenium.new(TestApp)
  end
 
  it_should_behave_like "driver"
  it_should_behave_like "driver with javascript support"
end

Each behavior group above (“driver” and “driver with javascript support”) is inside Capybara library allowing everyone to develop its own extensions using a shared suite. For instance, if a driver has javascript support, it means the following tests should pass:

shared_examples_for "driver with javascript support" do
  before { @driver.visit('/with_js') }
 
  describe '#find' do
    it "should find dynamically changed nodes" do
      @driver.find('//p').first.text.should == 'I changed it'
    end
  end
 
  describe '#drag_to' do
    it "should drag and drop an object" do
      draggable = @driver.find('//div[@id="drag"]').first
      droppable = @driver.find('//div[@id="drop"]').first
      draggable.drag_to(droppable)
      @driver.find('//div[contains(., "Dropped!")]').should_not be_nil
    end
  end
 
  describe "#evaluate_script" do
    it "should return the value of the executed script" do
      @driver.evaluate_script('1+1').should == 2
    end
  end
end

Capybara test suite is one of the best examples of using tests as documentation. By skimming the test suite you can easily know which features are supported by each driver! Sweet, isn’t it?

Friendliness award: I18n

When you are a big Open Source project, your test suite needs to be easy to run in order to new developers can create patches without hassle. The I18n library for Ruby definitely meets the big Open Source project requirement since it’s widely used and provides several extensions.

However, some of these extensions depends on ActiveRecord, some in ruby2ruby, others in ruby-cldr… and soon it will even support a few Key-Value stores, as Tokyo and Redis. Due to all these dependencies, you would probably imagine that running I18n test suite would require several trials and a lot of configuration before it finally works, right?

WRONG! If you don’t have ActiveRecord, I18n will say: “hey, you don’t have ActiveRecord” but still run the part of test suite that does not depend on it. So if a developer wants to fix or add something trivial, he doesn’t need to worry with installing all sorts of dependencies.

Besides, as mentioned a couple months ago, the I18n library allows you to create several combinations of backends. In other words, the I18n test suite needs to ensure that all these different combinations work as expected.

This problem is quite similar to the one in Capybara which needs to test different drivers. However, I18n uses Test::Unit thus it cannot use shared examples groups as in Rspec. So how were I18n developers able to solve this issue? Using Ruby modules!

Here are the tests for the upcoming KeyValue backend:

require 'test_helper'
require 'api'
 
class I18nKeyValueApiTest < Test::Unit::TestCase
  include Tests::Api::Basics
  include Tests::Api::Defaults
  include Tests::Api::Interpolation
  include Tests::Api::Link
  include Tests::Api::Lookup
  include Tests::Api::Pluralization
  # include Tests::Api::Procs
  include Tests::Api::Localization::Date
  include Tests::Api::Localization::DateTime
  include Tests::Api::Localization::Time
  # include Tests::Api::Localization::Procs
 
  STORE = Rufus::Tokyo::Cabinet.new('*')
 
  def setup
    I18n.backend = I18n::Backend::KeyValue.new(STORE)
    super
  end
 
  test "make sure we use the KeyValue backend" do
    assert_equal I18n::Backend::KeyValue, I18n.backend.class
  end
end

Each included module above adds a series of tests to the backend. Since key-value backends cannot store procs, we don’t include any test related to procs.

Wrapping up

These three are my favorite test suites and also part of my favorite open source projects!

We’ve adopted Capybara as the official testing tool at PlataformaTec for some time already and I18n is one of the subjects of my upcoming book about Rails 3. In one specific chapter, we will build a tool that stores I18n translations into TokyoCabinet, which allows us to create and update translations through a web interface, similarly to ActiveRecord. The only difference is that TokyoCabinet is waaaay faster.

Finally, the fact you can mimic several of Rspec features using simple Ruby (like Capybara using shared example groups and I18n simply using modules) will be part of my talk in Euruko 2010 entitled DSL or NoDSL: The power is in the middle. The talk will show cases where DSLs mimics much of the behavior provided by Ruby and discuss what we are winning and/or losing in such cases.

Keep following us and, until the next blog post is out, we would love to hear in the comments which are your favorite test suites!