has_many :bugs, :through => :rails

Active Record batch processing in parallel processes

2011-08-29T12:16:38Z

Active Record provides find_each for batch processing of large number of records. However, when you are dealing with REALLY larger number of records ( I’m talking millions here ), find_each can become quite slow.

One obvious solution is to use something like Resque:

User.find_each {|user| Resque.enqueue(MyJob, user) }

But this solution can feel a little heavy in certain cases. Enter forking!

if GC.respond_to?(:copy_on_write_friendly=)
  GC.copy_on_write_friendly = true
end

jobs_per_process = 100
process_count = 10

User.find_in_batches(:batch_size => jobs_per_process * process_count) do |group|
  batches = group.in_groups(process_count)

  batches.each do |batch|
    Process.fork do
      ActiveRecord::Base.establish_connection

      # Do the actual work
      batch.each {|user| .. }
    end
  end

  Process.waitall
end

The above code fetches 1000 records from the db, forks 10 processes and processes 100 records in each process, in parallel, before moving on to the next 1000 records. This should be significantly faster than the usual sequential processing.

It’s advisable to use REE for doing something like this when memory usage is a concern.

Using ruby-debug with Pow

2011-07-29T16:45:11Z

After moving to Sam Stephenson’s awesome Pow, not being able to use ruby-debug was the primary obstacle I had adjusting to my new development environment. But as it turns out, it’s very simple to use ruby-debug with Pow.

Make sure you have ruby-debug in your Gemfile.

gem 'ruby-debug19' # 'ruby-debug' if using Ruby 1.8

Add the following to the bottom of your application’s development.rb config file:

unless $rails_rake_task
  require 'ruby-debug'

  Debugger.settings[:autoeval] = true
  Debugger.settings[:autolist] = 1
  Debugger.settings[:reload_source_on_change] = true
  Debugger.start_remote
end

This will start a debugger server.

Restart Pow and connect to your debugger server from a terminal:

[lifo@null ~]$ rdebug -c
Connected.

You should now see a ruby-debug console whenever the application hits a breapoint.

Being unobtrusive

If you don’t want to bother your co-workers with the ruby-debug stuff, you can configure a global .gitignore file and setup ruby-debug from an initializer that’s ignored via the global .gitignore.

Setup global .gitignore:

$ git config --global core.excludesfile ~/.gitignore
$ echo my_super_secret_ruby_debug_initializer.rb >> ~/.gitignore

Add an initializer for setting up ruby-debug:

# config/initializers/my_super_secret_ruby_debug_initializer.rb
if (Rails.env.development? || Rails.env.test?) && !$rails_rake_task
  require 'ruby-debug'

  Debugger.settings[:autoeval] = true
  Debugger.settings[:autolist] = 1
  Debugger.settings[:reload_source_on_change] = true
  Debugger.start_remote
end

You’ll still need to have ruby-debug in Gemfile. But I imagine that’ll go unnoticed!

If you are not religiously using ruby-debug already, you are certainly missing out. Here are a few resources that should help you get started:

UPDATE 1: Added $rails_rake_task checks to make sure not to start the debugger for rake tasks

Cramp 0.12

2011-02-13T16:59:58Z

Moved to http://cramp.in

Bulk ALTER TABLE with Rails 3 and MySQL

2011-01-31T13:34:18Z

With this commit, Rails migrations will support adding/removing/changing multiple columns via a single ALTER TABLE statement for MySQL.

Let’s say you need to add 2 new columns to a table and modify data type of another column. Normally, you’d do something like this from a migration:

change_table(:users) do |t|
  t.string :im_handle
  t.belongs_to :company
  t.change :birthdate, :datetime
end

The above translates to 3 separate ALTER TABLE statements:

ALTER TABLE `users` ADD `im_handle` varchar(255)
ALTER TABLE `users` ADD `company_id` int(11)
ALTER TABLE `users` CHANGE `updated_at` `updated_at` datetime DEFAULT NULL

Now if you read the relevant MySQL documentation, it’s a little fucked up how ALTER TABLE works. But in a nutshell, here’s what MySQL does.

1. Write lock the table. So no more INSERT or UPDATE.
2. Make a temporary copy of the table, including all its data.
3. Do all that alteration on the temporary table.
4. Make the temporary table new primary. Get rid of the original primary table.

Whew! Imagine doing this 3 times for a table containing millions of rows. Hint: It’s not pleasant.

This commit adds a new :bulk => true option to change_table.

At the time of writing this, only ‘mysql’ adapter supports the :bulk option.

change_table(:users, :bulk => true) do |t|
  t.string :im_handle
  t.belongs_to :company
  t.change :birthdate, :datetime
end

:bulk => true makes sure the change_table runs only 1 ALTER TABLE statement.

ALTER TABLE `users` ADD COLUMN `im_handle` varchar(255), ADD COLUMN `company_id` int(11), CHANGE `updated_at` `updated_at` datetime DEFAULT NULL

When you are dealing with massive tables, :bulk => true will not only make your migration run faster, but also make it more atomic.

Nested layouts in Rails 3

2010-12-23T12:36:11Z

This an update/rewrite of my previous post nested layouts – as that version no longer works with Rails 3.

Code

def parent_layout(layout)
  @_content_for[:layout] = self.output_buffer
  self.output_buffer = render(:file => "layouts/#{layout}")
end

Add the above helper method in your ApplicationHelper or wherever you see fit.

Example

Let’s say your application has an admin section. And for all the views using admin.html.erb layout, you want to fetch stylesheets, javascript, footer etc. from the application.html.erb layout. One way to do this would be to keep the shared code in partials. However, using inheritance style layouts feels more natural to me.

Here’s how you can use application.html.erb as the layout for the admin.html.erb layout :

# layouts/admin.html.erb
<h1>Admin Panel</h1>
<div class="admin_navigation">Home | Posts | Users | Assets</div>
<%= yield %>

<%= parent_layout 'application' %>

Your application.html.erb layout is just another regular layout that can be used independently.

# layouts/application.html.erb
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head><title>Hello World</title></head>
  <body>
    <%= yield %>
  </body>
</html>

Using admin.html.erb layout is now affectively same as using the following layout :

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head><title>Hello World</title></head>
  <body>
    <h1>Admin Panel</h1>
    <div class="admin_navigation">Home | Posts | Users | Assets</div>
    <%= yield %>
  </body>
</html>

Signed and Permanent cookies in Rails 3

2010-02-05T12:37:00Z

David added a very cool feature to Rails recently – Signed cookies and permanent cookies This lets you set permanent and/or signed cookies very easily.

Before this, you’d have to write :

cookies[:user_preference] = {
  :value => @current_user.preferences,
  :expires => 20.years.from_now.utc
}

Now just becomes :

cookies.permanent[:user_preference] = @current_user.preferences

In case you happen to have seen my Railssummit presentation I had talked about using ActiveSupport::MessageVerifier for implementing “Remember me” functionality. The above commit makes that a whole lot easier.

In your model User.rb :

# User.rb
def self.authenticated_with_token(id, stored_salt)
  u = find_by_id(user_id)
  u && u.salt == stored_salt ? u : nil
end

And when the user checks “Remember me” box, make sure the following gets run :

cookies.permanent.signed[:remember_me] = [current_user.id, current_user.salt]

This will set a permanent and signed cookie using the secret specified in ActionController::Base.cookie_verifier_secret. If you don’t have the cookie_verifier_secret defined, you might want to do that in one of the initializers.

Now when you want to login using the cookie :

user = User.authenticated_with_token(*cookies.signed[:remember_me])

In this specific case, it’s very important to use the salt in the cookie value. That makes sure the cookie gets invalidated if the user changes his password.

Active Record Query Interface 3.0

2010-01-22T18:28:00Z

I’ve been working on revamping the Active Record query interface for the last few weeks ( while taking some time off in India from consulting work, before joining 37signals ), building on top of Emilio’s GSOC project of integrating ARel and ActiveRecord. So here’s an overview of how things are going to work in Rails 3.

What’s going to be deprecated in Rails 3.1 ?

These deprecations will be effective in Rails’ 3.1 release ( NOT Rails 3 ) and will be fully removed in Rails 3.2, though there will be an official plugin to continue supporting them. Consider this an advance warning as it involves changing a lot of code.

In short, passing options hash containing :conditions, :include, :joins, :limit, :offset, :order, :select, :readonly, :group, :having, :from, :lock to any of the ActiveRecord provided class methods, is now deprecated.

Going into details, currently ActiveRecord provides the following finder methods :

find(id_or_array_of_ids, options)
find(:first, options)
find(:all, options)
first(options)
all(options)
update_all(updates, conditions, options)

And the following calculation methods :

count(column, options)
average(column, options)
minimum(column, options)
maximum(column, options)
sum(column, options)
calculate(operation, column, options)

Starting with Rails 3, supplying any option to the methods above will be deprecated. Support for supplying options will be removed from Rails 3.2. Moreover, find(:first) and find(:all) ( without any options ) are also being deprecated in favour of first and all. A tiny little exception here is that count() will still accept a :distinct option.

The following shows a few example of the deprecated usages :

User.find(:all, :limit => 1)
User.find(:all)
User.find(:first)
User.first(:conditions => {:name => 'lifo'})
User.all(:joins => :items)

But the following is NOT deprecated :

User.find(1)
User.find(1,2,3)
User.find_by_name('lifo')

Additionally, supplying options hash to named_scope is also deprecated :

named_scope :red, :conditions => { :colour => 'red' }
named_scope :red, lambda {|colour| {:conditions => { :colour => colour }} }

Supplying options hash to with_scope, with_exclusive_scope and default_scope has also been deprecated :

with_scope(:find => {:conditions => {:name => 'lifo'}) { ... }
with_exclusive_scope(:find => {:limit =>1}) { ... }
default_scope :order => "id DESC"

Dynamic scoped_by_ are also going to be deprecated :

red_items = Item.scoped_by_colour('red')
red_old_items = Item.scoped_by_colour_and_age('red', 2)

New API

ActiveRecord in Rails 3 will have the following new finder methods.

where (:conditions)
having (:conditions)
select
group
order
limit
offset
joins
includes (:include)
lock
readonly
from

¹ Value in the bracket ( if different ) indicates the previous equivalent finder option.

Chainability

All of the above methods returns a Relation. Conceptually, a relation is very similar to an anonymous named scope. All these methods are defined on the Relation object as well, making it possible to chain them.

lifo = User.where(:name => 'lifo')
new_users = User.order('users.id DESC').limit(20).includes(:items)

You could also apply more finders to the existing relations :

cars = Car.where(:colour => 'black')
rich_ppls_cars = cars.order('cars.price DESC').limit(10)

Quacks like a Model

A relation quacks just like a model when it comes to the primary CRUD methods. You could call any of the following methods on a relation :

new(attributes)
create(attributes)
create!(attributes)
find(id_or_array)
destroy(id_or_array)
destroy_all
delete(id_or_array)
delete_all
update(ids, updates)
update_all(updates)
exists?

So the following code examples work as expected :

red_items = Item.where(:colour => 'red')
red_items.find(1)
item = red_items.new
item.colour #=> 'red'

red_items.exists? #=> true
red_items.update_all :colour => 'black'
red_items.exists? #=> false

Note that calling any of the update or delete/destroy methods would reset the relation, i.e delete the cached records used for optimizing methods like relation.size.

Lazy Loading

As it might be clear from the examples above, relations are loaded lazily – i.e you call an enumerable method on them. This is very similar to how associations and named_scopes already work.

cars = Car.where(:colour => 'black') # No Query
cars.each {|c| puts c.name } # Fires "select * from cars where ..."

This is very useful along side fragment caching. So in your controller action, you could just do :

def index
  @recent_items = Item.limit(10).order('created_at DESC')
end

And in your view :

<% cache('recent_items') do %>
  <% @recent_items.each do |item| %>
    ...
  <% end %>
<% end %>

In the above example, recent_items</tt> are loaded on <tt>recent_items.each call from the view. As the controller doesn’t actually fire any query, fragment caching becomes more effective without requiring any special work arounds.

Force loading – all, first & last

For the times you don’t need lazy loading, you could just call all on the relation :

cars = Car.where(:colour => 'black').all

It’s important to note that all returns an Array and not a Relation_. This is similar to how things work in Rails 2.3 with namedscopes and associations.

Similarly, first and last will always return an ActiveRecord object ( or nil ).

cars = Car.order('created_at ASC')
oldest_car = cars.first
newest_car = cars.last

named_scope → scopes

Using the method named_scope is deprecated in Rails 3.0. But the only change you’ll need to make is to remove the “named_” part. Supplying finder options hash will be deprecated in Rails 3.1.

named_scope have now been renamed to just scope.

So a definition like :

class Item
  named_scope :red, :conditions => { :colour => 'red' }
  named_scope :since, lambda {|time| {:conditions => ["created_at > ?", time] }}
end

Now becomes :

class Item
  scope :red, :conditions => { :colour => 'red' }
  scope :since, lambda {|time| {:conditions => ["created_at > ?", time] }}
end

However, as using options hash is going to be deprecated in 3.1, you should write it using the new finder methods :

class Item
  scope :red, where(:colour => 'red')
  scope :since, lambda {|time| where("created_at > ?", time) }
end

Internally, named scopes are built on top of Relation, making it very easy to mix and match them with the finder methods :

red_items = Item.red
available_red_items = red_items.where("quantity > ?", 0)
old_red_items = Item.red.since(10.days.ago)

Model.scoped

If you want to build a complex relation/query, starting with a blank relation, Model.scoped is what you would use.

cars = Car.scoped
rich_ppls_cars = cars.order('cars.price DESC').limit(10)
white_cars = cars.where(:colour => 'red')

Speaking of internals, ActiveRecord::Base has the following delegations :

delegate :find, :first, :last, :all, :destroy, :destroy_all, :exists?, :delete, :delete_all, :update, :update_all, :to => :scoped
delegate :select, :group, :order, :limit, :joins, :where, :preload, :eager_load, :includes, :from, :lock, :readonly, :having, :to => :scoped
delegate :count, :average, :minimum, :maximum, :sum, :calculate, :to => :scoped

The above might give you a better insight on how ActiveRecord is doing things internally. Additionally, dynamic finder methods find_by_name, find_all_by_name_and_colour etc. are also delegated to Relation.

with_scope and with_exclusive_scope

with_scope and with_exclusive_scope are now implemented on top of Relation as well. Making it possible to use any relation with them :

with_scope(where(:name => 'lifo')) do
  ...
end

Or even use a named scope :

with_exclusive_scope(Item.red) do
  ...
end

That’s all. Please open a lighthouse ticket if you find a bug or have a patch for an improvement!

UPDATE 1 : Added information about deprecating scoped_by_ dynamic methods.

UPDATE 2 : Added information about deprecating default_scope with finder options.

Websockets made easy with Cramp

2010-01-15T18:42:00Z

Moved to http://cramp.in

Introducing Cramp

2010-01-07T14:17:00Z

Moved to http://cramp.in

Railssummit Slides

2009-10-18T18:47:00Z

Here are the slides of my presentation at Railssummit 2009. Huge thanks to Locaweb and Fabio Akita for organizing the conference and having me there.

My talk was about Rails focused tips/tricks.

Lessons Learnt in 2009

View more documents from pratiknaik.

Or you can download the pdf from here.

Make your shoulda tests faster with fast_context

2009-09-20T17:36:00Z

I’ve been using Shoulda for a while. And for my current project, I decided to go fixtureless with Shoulda + Factory Girl. All good, except one problem. Slow as fuck tests. So here’s fast_context as a solution for it. fast_context compiles all the ’should’s within a context into a single test.

For example :

class UsersControllerTest < ActionController::TestCase
  fast_context "#new" do
    setup do
      @user = Factory(:user)
      get :new
    end

    should_render_template :new
    should_render_with_layout :users
  end
end

This will be compiled into something like :

class UsersControllerTest < ActionController::TestCase
  test "test: #new should run_fast"
    @user = Factory(:user)
    get :new
    should_render_template :new
    should_render_with_layout :users
  end
end

As your setup method just once per fast_context, the tests run much faster.

To install :

script/plugin install git://github.com/lifo/fast_context.git

This was a result of a short time of hacking. So there may be bugs. Please comment here if you run into any issues.

save! > save

2009-08-07T14:24:00Z

Thoughtbot folks have a great article on not expecting exceptions – save bang your head, active record will drive you mad. I’ll admit, just like the poster, I used to use save! in controllers to DRY my code. And have a global rescue_from in application.rb. But over the time, I changed the camp and now I’m fully in that “Don’t expect expectations” camp. Some things are more important that DRYing 3 lines of code.

But I’d want to take this a step further. When you’re not expecting something to fail, always use the methods that raise exceptions on failure.

So I strongly disagree with the poster on this :

I think ActiveRecord::Base#save! and ActiveRecord::Base.update_attributes! should be pulled from the public API

I would advocate just the opposite for certain cases. In many of the code reviews we’ve done via ActionRails, the following pattern was seen in many of the models :

def do_something
  self.foo = 'bar'
  save
end

def create_items
  names.each {|n| self.items.create :name => n }
end

In the snippets above, it’s not checking for cases where the save fails. And for good reasons that they’re not likely to fail as code is changing some very minor. But in these scenarios, a failure would be an unexpected situation. Hence you should always use save! or create!.

There could be easily be any unexpected reasons the above save could fail. And using save! protects you from those situations and help catch those minor programming mistakes early, which otherwise could prove to be very costly in terms of time/efforts. So the above code should really be :

def do_something
  self.foo = 'bar'
  save!
end

def create_items
  names.each {|n| self.items.create! :name => n }
end

However, if you’re using exceptions for flow control, this practise won’t always help you :

def create
  @user = User.create! params[:user]
  redirect_to @user
rescue ActiveRecord::RecordNotSaved
  flash[:notice] = 'Unable to create user'
  render :new
end

As this catches the exception ActiveRecord::RecordNotSaved, unexpected save! failures from your model methods/callbacks will get caught too. And hide the real error.

Moral of the story :

Don’t expect exceptions
Use methods throwing exceptions when you’re not expecting a failure. For example, everywhere you’re not checking if save or create fails when working with Active Record objects, always use save! and create! instead.

USE INDEX with Active Record finders

2009-08-06T12:16:00Z

MySQL doesn’t always pick the right index for your queries. Hence, sometimes you must tell it which index to use. Consider the example :

Activity.all(:conditions => ['created_at >= ? AND country_id = ?', 10.days.ago, 79])

Running EXPLAIN on the above query :

EXPLAIN SELECT * FROM `activities` WHERE (created_at >= '2009-07-27 12:58:44' AND country_id = 79);

Possible keys : index_activities_on_created_at,index_activities_on_created_at_and_country_id
Using the key : index_activities_on_created_at

As you can see, even though the table has index on both the fields involved in the query – index_activities_on_created_at_and_country_id, MySQL still uses index_activities_on_created_at. You can explicitly ask MySQL to use the index you want by supplying USE INDEX

SELECT * FROM `activities` USE INDEX(index_activities_on_created_at_and_country_id) 
  WHERE (created_at >= '2009-07-27 12:58:44' AND country_id = 79);

Active Record does not have any finder option to specify the index hint. Hence the solution is to exploit the :from option :

from = "#{quoted_table_name} USE INDEX(index_activities_on_created_at_and_country_id)"
Activity.all(:from => from, 
             :conditions => ['created_at >= ? AND country_id = ?', 10.days.ago, 79])

Ruby I don't like #3 - Object#freeze

2009-08-05T14:19:00Z

Object#freeze annoys me. Not a lot, but enough to bitch blog about it. So, freeze lets you make sure no one else modifies your precious little object :

>> a = "hello"
>> a.freeze
>> a << "wtf"
TypeError: can't modify frozen string
        from (irb):23:in `<<'
        from (irb):23

However, freeze does not protect the variable. It only protects the value.

>> a = "hello"
>> a.freeze
>> a += "wtf"
=> "hellowtf"

The weird behaviour is even more visible when you’re dealing with arrays :

>> x = ["hello", "freedom"]
>> x.freeze
>> x << "world"
TypeError: can't modify frozen array
        from (irb):6:in `<<'
        from (irb):6
>> x[0] << "wtf"
>> x
=> ["hellowtf", "freedom"]

It’s even weird with hashes :

>> a = {:x => 1}
>> a.freeze
>> a[:x] = 2
TypeError: can't modify frozen hash
        from (irb):11:in `[]='
        from (irb):11

However, above are not really the primary reasons I don’t like freeze. It’s the fact that you cannot unfreeze an object without using something like evil.rb. And this goes against a lot of things Ruby stands for in my book. Ruby is never about defensive programming. Even where it tries to save you from yourself, there are always proper ways you can overcome the restriction. For example, private methods and send. If you want to restrict programmers, Ruby is not for you. Use Java/Python/whatever. Not Ruby. Ruby is not meant for preventing idiots from shooting their leg.

Taking a real example :

module ActiveSupport
  module Testing
    module Performance
      DEFAULTS =
        if benchmark = ARGV.include?('--benchmark')  # HAX for rake test
          { :benchmark => true,
            :runs => 4,
            :metrics => [:process_time, :memory, :objects, :gc_runs, :gc_time],
            :output => 'tmp/performance' }
        else
          { :benchmark => false,
            :runs => 1,
            :min_percent => 0.01,
            :metrics => [:process_time, :memory, :objects],
            :formats => [:flat, :graph_html, :call_tree],
            :output => 'tmp/performance' }
        end.freeze

Here’s a code from Rails performance tests. As you can see, it defines a hash with config variables based on benchmark or profile mode. And then freezes the hash. Assuming you’re benchmarking, DEFAULTS[:runs] determines how many times Rails should run the test in a loop :

DEFAULTS[:runs].times { run_test_with_benchmarking "whatever test" }

Now many times when I’m benchmarking and want to increase the number of times a test is ran, I just want to do something like :

class SpeedTest < ActionController::PerformanceTest
  DEFAULTS[:runs] = 1000

  def test_some_method
    Model.some_method
  end
end

However, that’s not possible thanks to the freeze. I do know that changing DEFAULTS[:runs] is not a public API yada yada yada. But it’s Ruby and I’ll change whatever the fuck I want to. I can understand certain cases where people use freeze to prevent silly errors. That’s probably OK to a certain extent. But remember, if you design your software for idiots, only idiots will use it.

Ruby I don't like #2 - catch(:wtf) { throw :wtf }

2009-08-04T17:28:00Z

The 1960s and 1970s saw computer scientists move away from GOTO statements in favor of the structured programming programming paradigm. Some programming style coding standards prohibit use of GOTO statements. – Wikipedia

Ruby takes the whole GOTO nonsense to an entirely new heights. Ruby’s version of GOTO/LABEL is called throw/catch. The lunacy goes further as Ruby’s throw is equivalent to GOTO with a return value.

def hello
  throw :done, "wtf"
end

catch(:done) { hello }
=> "wtf"

Not only it makes the flow control hard to follow, it also shows your lack of fundamental programming skills. I’d love to see a case where you use throw/catch because there’s no other way. Only place I’ve ever used throw/catch is in my evil middleware Rack::Evil. And the name says it all.

Let’s take a real example from Rails :

def find_with_associations(options = {})
  catch :invalid_query do
    join_dependency = JoinDependency.new(self, merge_includes(scope(:find, :include), options[:include]), options[:joins])
    rows = select_all_rows(options, join_dependency)
    return join_dependency.instantiate(rows)
  end
  []
end

Just by looking at this method, you’ll have absolutely no idea who’s gonna be throwing :invalid_query. It could be any method subsequently called while the block is being executed. Only way to know is by doing a global search for throw :invalid_query.

Rails uses throw/catch here because it wants to return an empty array when something somewhere goes wrong. And the thing that can possibly go wrong is so deep down inside, throw/catch provides an easy way out without much refactoring. However, easy is not always the best way or the proper way.

If we look at the relevant code from the involved methods :

def select_all_rows(options, join_dependency)
  connection.select_all(
    construct_finder_sql_with_included_associations(options, join_dependency),
    "#{name} Load Including Associations"
  )
end

def construct_finder_sql_with_included_associations(options, join_dependency)
  scope = scope(:find)
  sql = "SELECT #{column_aliases(join_dependency)} FROM #{(scope && scope[:from]) || options[:from] || quoted_table_name} "

  ....
  if !using_limitable_reflections?(join_dependency.reflections) && ((scope && scope[:limit]) || options[:limit])
    add_limited_ids_condition!(sql, options, join_dependency)
  end
  ....

  sanitize_sql(sql)
end

def add_limited_ids_condition!(sql, options, join_dependency)
  unless (id_list = select_limited_ids_list(options, join_dependency)).empty?
    sql << "#{condition_word(sql)} #{connection.quote_table_name table_name}.#{primary_key} IN (#{id_list}) "
  else
    throw :invalid_query
  end
end

This doesn’t seem that bad on the first look. But think again. Apart from the control flow retardness, the method add_limited_ids_condition adds an extra responsibility to the caller – catching invalid_query. And this is very easy to miss too – as seen with the very same method in question here – calculations.rb. Add a few of more throw/catch and you get a proper spaghetti code.

I think the better way to write the above code is :

def find_with_associations(options = {})
  join_dependency = JoinDependency.new(self, merge_includes(scope(:find, :include), options[:include]), options[:joins])
  rows = select_all_rows(options, join_dependency)
  rows ? join_dependency.instantiate(rows) : []
end

def select_all_rows(options, join_dependency)
  finder_sql = construct_finder_sql_with_included_associations(options, join_dependency)
  connection.select_all(finder_sql, "#{name} Load Including Associations") if finder_sql
end

def construct_finder_sql_with_included_associations(options, join_dependency)
  ....
  limitable = !using_limitable_reflections?(join_dependency.reflections) && ((scope && scope[:limit]) || options[:limit])

  unless limitable && add_limited_ids_condition!(sql, options, join_dependency).blank?
    ....
    sanitize_sql(sql)
  end
end

def add_limited_ids_condition!(sql, options, join_dependency)
  id_list = select_limited_ids_list(options, join_dependency)
  sql << "#{condition_word(sql)} #{connection.quote_table_name table_name}.#{primary_key} IN (#{id_list}) " if id_list.present?
end

I’d normally say that you should be flexible about following such rules about using a pattern or not using some. But this is an exception. Using throw/catch is just fucking wrong. Plain and simple.

Ruby I don't like #1 - Explicit 'return'

2009-08-03T17:16:00Z

In Ruby, you don’t have to specify an explicit return value from a method. Ruby will just return the last evaluated statement. Similarly, if an explicit return statement will make itself the last evaluated statement – i.e return control to the caller with the specified return value.

However, I’m not a big fan of explicit return statements. In my experience, the only place where they make sense is in the first line of the method, where the control is returned to the caller if the supplied arguments are not valid/expected. Consider the following method :

def read(file_name, options = nil)
  return nil unless File.exist?(file_name)
  ....
end

I think the above is the only case where I feel it’s ok to use an explicit return as it’s much better than the alternative – wrapping the entire method in a big if block. Also, you don’t really need to specify nil. The above can be rewritten as :

def read(file_name, options = nil)
  return unless File.exist?(file_name)
  ....
end

Now the real problem is visible when you look at the full method :

def read(file_name, options = nil)
  return nil unless File.exist?(file_name)

  if expires_in(options) > 0
    return nil
  end

  File.open(file_name, 'rb') { |f| Marshal.load(f) }
end

A much simpler version of the above method is :

def read(file_name, options = nil)
  if File.exist?(file_name) && expires_in(options) <= 0
    File.open(file_name, 'rb') { |f| Marshal.load(f) }
  end
end

Of course there’s no one ring to rule them all. It might be desirable to use multiple returns in a method. But every time you do that, take a moment to make sure it’s making the code easier to read.

Stop returning false from your before filters

2009-07-31T18:17:00Z

In the past version of Rails you had to explicitly return false from before filters to halt the filter chain and make sure the action doesn’t get run. The code looked somewhat like :

class AdminController < ApplicationController
  before_filter :check_admin

  private

  def check_admin
    unless current_user.admin?
      redirect_to home_path
      return false
    end
  end
end

But since Rails 2.0, If you call render, head or redirect_to from a before_filter, the filter chain will be halted. So the above controller will now look like :

class AdminController < ApplicationController
  before_filter :check_admin

  private

  def check_admin
    redirect_to home_path unless current_user.admin?
  end
end

You should go ahead and update all your before filters to reflect this change. return false is nothing but code smell.

Running Rails performance tests on real data

2009-07-29T19:37:00Z

It’s not quite straight forward to run Rails performance tests on real production data. By default, performance tests will use the test database and wipe it before using it, thus making it impossible to put real data in the test database.

To run performance tests on real data:

Create a file called performance_test_helper.rb inside your test/ directory with the following:

# START : HAX HAX HAX
# Load Rails environment in 'test' mode
RAILS_ENV = "test"
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
# Re-establish db connection for 'performance' mode
silence_warnings { RAILS_ENV = "performance" }
ActiveRecord::Base.establish_connection
# STOP : HAX HAX HAX

require_dependency 'application'

require 'test/unit'
require 'active_support/test_case'
require 'action_controller/test_case'
require 'action_controller/integration'

require 'performance_test_help'

Replace the following lines in performance tests:

require 'test_helper'
require 'performance_test_help'

with

require 'performance_test_helper'

Supply configuration for ‘performance’ environment in your database.yml:

performance:
  adapter: mysql
  encoding: utf8
  database: database_with_real_data
  pool: 5
  username: root
  password:
  socket: /tmp/mysql.sock

The above steps will also make sure that the performance tests use the database database_with_real_data. Also, the database will not get wiped on every run. You’ll also have to ensure the test database – database_with_real_data – is up-to-date with the schema changes.

Now you can just run your performance tests using the usual rake tasks:

$ rake test:benchmark
$ rake test:profile

Nested Layouts

2009-07-07T22:21:00Z

Don’t think the nested layouts get any simpler in Rails.

module ApplicationHelper
  def parent_layout(layout)
    @content_for_layout = self.output_buffer
    self.output_buffer = render(:file => "layouts/#{layout}")
  end
end

Now using the parent_layout helper method inside your layouts for nesting :

# items.html.erb
<h1>Just my items</h1>
<%= yield %>

<% parent_layout 'master' %>

and the parent layout of items – the master layout :

# master.html.erb
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head><title>Hello World</title></head>
  <body>
    <%= yield %>
  </body>
</html>

Now, the items layout will always be wrapped under the master layout. Just make sure that the parent_layout call is always on the last line using <%. This technique also works for nesting deeper than a single level.

Default Scopes and Inheritance to the rescue

2009-03-24T10:28:00Z

On my one of the current projects, there are two primary models each with a flag called approved_. 99% of the front end part deals with only approved items. Unapproved items are usually only in the admin panel side of the story. So I started with using a namedscope called approved:

class Item < ActiveRecord::Base
  has_many :tags

  default_scope :order => 'items.name ASC'
  named_scope :approved, :conditions => { :published => true }
end

And now I’d have to use Item.approved. everywhere in my application. But that became a bit too cumbersome sooner than later. Playing around with this a bit, I came up with the solution using default_scope and the good ol’ inheritance:

class Item < ActiveRecord::Base
  has_many :tags

  default_scope :order => 'items.name ASC'
end

class PublishedItem < Item
  set_table_name 'items'
  set_inheritance_column nil # hax?

  default_scope :conditions => { :published => true }, :order => 'items.name ASC'
end

Checking this on console :

>> p = PublishedItem.first
  SELECT * FROM `items` WHERE (`items`.`published` = 1) ORDER BY items.name ASC LIMIT 1

>> i = Item.first
  SELECT * FROM `items` ORDER BY items.name ASC LIMIT 1

Seems to work just fine.

You could do it the other way around too:

class RawItem < ActiveRecord::Base
  set_table_name 'items'
  has_many :tags

  default_scope :order => 'items.name ASC'
end

class Item < RawItem
  set_table_name 'items'
  set_inheritance_column nil # hax?

  default_scope :conditions => { :published => true }, :order => 'items.name ASC'
end

Whichever one works for you.

Please note that the above code is NOT using STI_. It’s using set_inheritancecolumn nil workaround to bypass the Active Record STI stuff and rely just on the ruby inheritance.

My Rails guides

2009-03-18T12:42:00Z

Instead of blogging some of the articles, I decided to write those as Rails guides. So go ahead and check them out !

Rails on Rack – Overview of Rails’ Rack integration and middleware/metal interfaces
Performance Testing Rails Applications – Using Rails’ built in performance test suite for profiling and benchmarking your Rails application

I also wrote a lot of the Active Record Query Interface guide. Of course my guides aren’t really eligible for the guides hackfest prizes. But hey, you could still win them by writing a new guide !

assert_* and assert_not_*

2009-03-17T15:49:00Z

Not sure I like this or not, but I’m gonna give it a shot in the “real world” nevertheless.

module AssertionExtensions
  def method_missing(method_id, *arguments, &block)
    return super unless method_id.to_s =~ /^assert_(not_)?(.*)$/

    method = "#{$2}?"
    object = arguments.first

    if $1
      arguments.each do |object|
        assert ! object.send(method), "#{method} is not false for #{object}"
      end
    else
      arguments.each do |object|
        assert object.send(method), "#{method} is not true for #{object}"
      end
    end
  end
end

class ActiveSupport::TestCase
  include AssertionExtensions
end

Now using this in your tests:

def test_is_admin
  # [people(:admin), people(:superuser)].each {|p| assert p.admin?}
  assert_admin people(:admin), people(:superuser)

  # assert ! people(:foo).admin?
  assert_not_admin people(:foo)
end

Make Test::Unit display errors earlier

2009-03-14T12:51:00Z

Just a monkey patch to make Test::Unit display errors as soon as they happen.

require 'test/unit'
require 'test/unit/ui/console/testrunner'

class Test::Unit::UI::Console::TestRunner
  def add_fault(fault)
    hax_output(fault)
    @faults << fault
    output_single(fault.single_character_display, 1)
    @already_outputted = true
  end

  def hax_output(fault)
    @io.puts("\n")
    output_single(fault.short_display, 1) # fault.long_display for the full trace
    @io.puts("\n")
  end
end

Useful when your tests take too long to run.

The Evil calls back

2009-01-15T17:54:00Z

Happy new year!

Just an update on the two middlewares I contributed to rack-contrib. In case you don’t already know what is rack-contrib, it’s a project started by Ryan Tomayko as a playground for experimental Rack middlewares from the ruby community. There are 15 middlewares there already, so do check it out!

Rack::Evil

The name says it all. It’s pure evil, as it enables the rack application to return the response from any place during while it’s serving the request by doing throw :response, [status, header, body].

Example:

app = lambda do |env|
  template = <<-TEMPLATE
  Hello Template
  <%= throw :response, [404, {'Content-Type' => 'text/html'}, 'From Template'] %>
  Never
  TEMPLATE
  result = ERB.new(template).result(binding)

  [200, {'Content-Type' => 'text/plain'}, "Ran the template"]
end

And the response reaching the client will not be “Ran the template” with 200 status. But it’ll be “From Template” with 404 status!

Rack::Callbacks

Now that more people are getting on board with the concept of middlewares, people have started using middlewares for pure before/after filter kind of things. And I think don’t think that’s the best way moving forward. That’s the problem Rack::Callbacks tries to alleviate.

Rack::Callbacks lets you wrap your rack application with a series of before and after callbacks.

Example :

class TimeZone
  def initialize(default)
    @default = default
  end

  def call(env)
    env['rack.timezone'] = find_timezone(env) || @default
  end
end

class CompressBody
  def call(response)
    status, headers, body = response

    compressed_body = zip_body(body)
    [status, headers, compressed_body]
  end
end

app = Rack::Callbacks.new do
  before TimeZone, "default"

  run YourPrimaryRackApp.new

  after CompressBody
end

The flow of execution is very apparent here. All the before callbacks are run first, then the actual application and after callbacks at last.

before Callback, arg1, arg2….argx

before() takes Callback class as the first argument. Optional arguments are passed to Callback#initialize. This is very similar to the regular rack middlewares. However before callbacks are quite different from middlewares.

Callback#call accepts a single argument : env. However, the return value of Callback#call is simply discarded.

run Application

Runs the primary rack application.

after Callback, arg1, arg2….argx

after() takes Callback class as the first argument. Optional arguments are passed to Callback#initialize. Just like before() and middlewares. However, after callbacks are substantially different from before callbacks and middlewares:

Callback#call accepts a single argument : response, which is the rack response returned by the application or other after filters. Callback#call also must return a valid rack response. Return value of Callback#call is supplied to the next after filters in the stack, and the final after filter returns the response to the client.

Rails templates

2008-12-04T00:50:00Z

So now that Edge Rails got templates ( Thanks to Jeremy ) I just wanted to give a top level overview.

Templates are simple ruby files containing DSL for adding plugins/gems/initializers etc. to your freshly created Rails project. To apply the template, you need to provide rails generator with location of the template you wish to apply, using -m option :

rails blog -m ~/template.rb

Thanks to the magic of open-uri, the template location can be a URL too :

rails blog -m http://gist.github.com/31208.txt

You can even apply templates to your existing Rails application using rails:template rake task and supplying LOCATION environment variable :

rake rails:template LOCATION=~/template.rb

A very simple template would look like :

# template.rb
run "rm public/index.html"
generate(:scaffold, "person name:string")
route "map.root :controller => 'people'"
rake("db:migrate")

git :init
git :add => "."
git :commit => "-a -m 'Initial commit'"

That’s very self explanatory. Here are the key methods for the template DSL :

gem(name, options = {})

Adds a config.gem entry for the supplied gem to generated application’s config/environment.rb

So if your application depends on bj and hpricot :

gem "bj"
gem "hpricot", :version => '0.6', :source => "http://code.whytheluckystiff.net"

Please note that this will NOT install the gems for you. So you may want to run rake gems:install:

rake "gems:install"

And let Rails take care of installing the required gems if they’re not already installed.

plugin(name, options = {})

Installs a plugin to the generated application.

Plugin can be installed from Git :

plugin 'authentication', :git => 'git://github.com/foor/bar.git'

You can even install plugins as git submodules :

plugin 'authentication', :git => 'git://github.com/foor/bar.git', :submodule => true

Please note that you need to git :init before you can install a plugin as a submodule

Or use plain old SVN :

plugin 'wtfsvn' :svn => 'svn://crap.com/wtf/trunk'

initializer(filename, data = nil, &block)

Adds an initializer to the generated application’s config/initializers directory.

So personally, I like using Object#not_nil? and Object#not_blank? :

initializer 'bloatlol.rb', <<-CODE
class Object
  def not_nil?
    !nil?
  end

  def not_blank?
    !blank?
  end
end
CODE

Similarly lib() creates a file in lib/ directory and vendor() creates a file in vendor/ directory. There is also file(), which accepts a relative path from RAILS_ROOT and creates all the directories/file needed :

file 'app/components/foo.rb', <<-CODE
class Foo
end
CODE

That’ll create app/components directory and put foo.rb in there.

rakefile(filename, data = nil, &block)

Creates a new rake file under lib/tasks with the supplied tasks :

rakefile("bootstrap.rake") do
  <<-TASK
    namespace :boot do
      task :strap do
        puts "i like boots!"
      end
    end
  TASK
end

And that creates lib/tasks/bootstrap.rake with a boot:strap rake task!

generate(what, args)

Runs the supplied rails generator with given arguments. For example, I love to scaffold some whenever I’m playing with Rails :

generate(:scaffold, "person", "name:string", "address:text", "age:number")

run(command)

Executes an arbitrary command. Just like the backticks. My main use case is to remove public/index.html :

run "rm public/index.html"

rake(command, options = {})

So you scaffolded, but who’s gonna run the db:migrate rake task !? Here’s who :

rake "db:migrate"

Simple enough.

You can also run rake tasks in a different rails environment :

rake "db:migrate", :env => 'production'

Or even use sudo :

rake "gems:install", :sudo => true

route(routing_code)

This adds a routing entry to config/routes.rb file. In above steps, we generated a person scaffold and also removed public/index.html. Now to make PeopleController#index as the default page for the application :

route "map.root :controller => :person"

Voila!

inside(dir)

I have my edge rails lying at ~/commit-rails/rails. So every time i have to manually symlink edge from my new app. But now :

inside('vendor') do
  run "ln -s ~/commit-rails/rails rails"
end

So inside() runs the command from the given directory.

ask(question)

ask gives you a chance to get some feedback from the user and use it in your templates. Lets say you want your user to name the new shiny library you’re adding :

lib_name = ask("What do you want to call the shiny library ?")
lib_name << ".rb" unless lib_name.index(".rb")

lib lib_name, <<-CODE
class Shiny
end
CODE

yes?(question) or no?(question)

And you can even ask questions from templates and decide the flow based on user’s answer. Lets say you want to freeze rails only if the user want to :

rake("rails:freeze:gems") if yes?("Freeze rails gems ?")

no?(question) acts just the opposite.

git(:must => “-a love”)

As we all love git/hub, Rails templates let you do the git stuff too !

git :init
git :add => "."
git :commit => "-a -m 'Initial commit'"

And bort ?

Here’s what a bort template would look like :

# bort.rb
inside('vendor') do
  run "ln -s ~/commit-rails/rails rails"
end

plugin 'rspec', 
  :git => 'git://github.com/dchelimsky/rspec.git'
plugin 'rspec-rails', 
  :git => 'git://github.com/dchelimsky/rspec-rails.git'
plugin 'exception_notifier', 
  :git => 'git://github.com/rails/exception_notification.git'
plugin 'open_id_authentication', 
  :git => 'git://github.com/rails/open_id_authentication.git'
plugin 'asset_packager', 
  :git => 'http://synthesis.sbecker.net/pages/asset_packager'
plugin 'role_requirement', 
  :git => 'git://github.com/timcharper/role_requirement.git'
plugin 'restful-authentication', 
  :git => 'git://github.com/technoweenie/restful-authentication.git'

gem 'mislav-will_paginate', :version => '~> 2.2.3', 
  :lib => 'will_paginate',  :source => 'http://gems.github.com'
gem 'rubyist-aasm'
gem 'ruby-openid'

rake("gems:install", :sudo => true)

generate("authenticated", "user session")
generate("rspec")

Shamelessly inspired/yanked from Jeremy’s templates repo

Save that code in bort.rb and :

[lifo@null Rails]$ ruby ~/commit-rails/rails/railties/bin/rails bortapp -m bort.rb

Contribute

If you like this template stuff and want to share your templates with the rest of us, please contribute to Jeremy’s rails-templates project – which will be a collection of Rails templates.

And as usual, any bugs/feature requests can go to Rails lighthouse

UPDATE 1 Add an example of installing plugins as a git submodule. Thanks to Peter Cooper for the patch.
UPDATE 2 Add an example of rake rails:template LOCATION=foo task