the rails loft

Find or find_by_id

Renzo Borgatti — Tue, 27 Jan 2009 20:28:00 -0500

If you are a Rails dev, you probably know that there are two "finder" behaviors in Rails. There is User.find(1) that raises an ActiveRecord::RecordNotFound if the record with the given ID was not found. There are also generated finders like User.find_by_id(1) that return nil instead. Inconsistency? I don't think so. Let's take the typical controller update method:

 1  def update
 2    @user = User.find(params[:id])
 3    respond_to do |format|
 4      if @user.update_attributes(params[:user])
 5        flash[:notice] = 'User was successfully updated.'
 6        format.html { redirect_to(@user) }
 7      else
 8        format.html { render :action => "edit" }
 9      end
10    end
11  end

The find method will raise RecordNotFound if for any reason the searched user cannot be found. Here is the same thing but checking if the returned user instance is null:

 1  def update
 2    @user = User.find_by_id(params[:id])
 3    if @user
 4      respond_to do |format|
 5        if @user.update_attributes(params[:user])
 6          flash[:notice] = 'User was successfully updated.'
 7          format.html { redirect_to(@user) }
 8        else
 9          format.html { render :action => "edit" }
10        end
11      end
12    end
13  end

I think the two options both have their place based on the following consideration:

Probability that the record will be found or not. Maybe the object is not a User but something more volatile like an Object which deleted from the database when it's "consumed".
There is a general catch mechanism by Rails when an exception raises up to the last possible moment. If you decide not to catch something you know a courtesy page can be presented and the error will be written in the log
If you decide to catch the exception you need to know what to check and maybe you need to look at the documentation.
Finally, there are cases where you really need to do something different if the record was not found than leave the error to rise. In this case you have the two options: find with catch or find_by_id with null checking.

Find should be used without catch in 90% of the cases. Because most of the time the ID you're searching for is just the result of a previous request on an object that is still there almost for sure. You can accept the 0.01% of cases where the find call rises an error that you can later track in the error log. If find had the same behavior of find_by_id (let's say for consistency reason), you'll be forced to check for null. If you let it go you end up with "nil when you didn't expect it" which is not as clear as a RecordNotFound.

If you need to catch that the record doesn't exist, then you have two options: catch the RecordNotFound or use find_by_id and check for null. I'm lazy and I prefer the second one: I don't need to check the documentation to remember what to catch.

Happy coding coders!

What a difference a line makes

Renzo Borgatti — Fri, 12 Dec 2008 23:24:00 -0500

I’m all for keeping the business logic in places where it can be easily isolated and eventually reused. I’m sure I’m not saying anything new, but there are cases where it makes a lot of sense to delegate. Examples of those cases are stored procedures and complex queries. Relational databases are pretty smart in parsing and optimizing SQL and when the database is a separate process on another machine it’s even more efficient to use a different CPU than the main application process.

Today I had the pleasure to apply one fast and simple change that drastically reduced the computation load. Here’s the “before”:

Song.find(:all).select(&:rating).sort_by(&:rating).reverse.first(10)

and here’s the “after”:

Song.find(:all, :joins => [:ratings], :order => "rating DESC", :limit => 10)

Assuming N is the number of songs in the database in the first case you need to load N object instances and issue N + 1 queries to load information about the rating of a specific song. In the second case 1 query is needed and all the filtering is delegated to the database. The nice thing is also the way the SQL string is hidden beautifully in the DSL-ish method call which is very similar to the first version. I remember the old days (for me!) in 2002 when this was just something like connection.executeStatement(“SELECT * BLAH…”) or something and I had to create my own SQLBuilder to reuse and test some of that SQL. O/R mappings tools and fluent interfaces of today are just a huge leap forward.

The take away for the Rails developer: think twice whenever you write or see a find(:all) with no options. Look carefully the console output to see what SQL Rails is generating and how long does it take. If the result set it operates on is reasonably small, no big deal. If the collection is huge, please delegate to the database.

Hey, by the way, do you see what the query is doing? :)

Poor Men Story Runner

Renzo Borgatti — Thu, 30 Oct 2008 10:26:00 -0400

Say that for some reasons like for example time constraints on the project, lack of customer collaboration or lack of business analysts you can’t maintain a full fledged automated story base. In that case one option could be to maintain the story runner format to describe the feature to implement but not try to make that executable. I know, I know, what a waste for such a great opportunity. But I’m talking about a desperate case.

As funky as it sounds, you can try to implement the plain text story in rspec as controller specs with integrated view. For example:

 1  describe BlogsController, "blog details page" do
 2    integrate_views
 3    
 4    before(:each) do
 5      Given "the user is logged in"
 6      When "I click on a blog post"
 7    end
 8    
 9    it { Then "the blog detail view is displayed" }
10    it { And "the full text of the blog is shown" }
11    it { And "the comment list section is shown" }
12      
13  end
14  
15  [:Given, :When, :Then, :And].each do |method_name|
16    self.class.send :define_method, method_name do |what|
17      self.send(what.gsub(' ','_'))
18    end
19  end

Yes, very very not-readable. It just destroy all the effort to create stories in plain text. But it’s something a developer can read. The next step is to create a method for each of the descriptive texts, replacing blanks with underscores:

 1  def the_user_is_logged_in
 2    @user = login
 3  end
 4  
 5  def I_click_on_a_blog_post
 6    @blog = stub_model(Blog, 
 7      :title => "Random thoughts.",
 8      :comments => comments, 
 9      :text => "MIDWAY upon the journey of our life")
10    Blog.should_receive(:find_by_id).with(1).and_return(@blog)
11    get :show, :id => 1
12  end
13  
14  def the_blog_detail_view_is_displayed
15    response.should have_tag("div#blog") do
16      with_tag("strong", @blog.title)
17    end
18  end
19  
20  def the_full_text_of_the_blog_is_shown
21    response.should have_tag("div#blog") do
22      with_tag("div#text", @blog.text)
23    end
24  end
25  
26  def the_comment_list_section_is_shown
27    response.should have_tag("div#blog") do
28      with_tag("div#comments") do
29        with_tag("div#comment", 5)
30      end
31    end
32  end

As you probably guessed already, this is the first scenario for a blog section on some web page. You can click on the “more” link to read more and access comments to the blog post. The plan is to implement the other scenarios in the same spec file changing the describe block to change the target controller. The specs are just scratching the surface of the view, then you’ll need to change spec to implement other part of the system.

Another interesting thing is that for this to be called “integration” test, I shouldn’t have used any stub_model instruction. Correct. First step I want to create it fast, no database thinking. Second step I’ll remove all stubs and using some way of talking to the database (FixtureReplacement for example) I’ll be using real connections to the model and therefore to the database. At the end, a kind of acceptance testing.

I know what you’d like to ask now: is this really any better than just plain run the Story Runner itself? Not at all. It’s absolutely a bad practice to use RSpec like this, I don’t want to suggest in any way that this can substitute the story runner. But in case of emergency (read, fixed scope, fixed price, fixed time projects) where the customer is not helping writing stories, where there are no business analysts and everything should be done by developers, dropping the story runner is probably the less of all possible pains. You should also consider that if you are using Watir or Selenium on Rails connected to the story runner, the effort of maintaining such an environment sensible acceptance framework is big.

Specifically:

You don’t need the infrastructure to run stories which is usually a separated task from running specs
You don’t need to maintain steps either
You don’t need to attach the story runner binary to the continuous integration, stories will run as normal specs.
You are free to maintain story texts outside the project, on a wiki or online somewhere.
Coverage with rcov is just part of the package. For stories you can’t know how well are you covering that given aspect of the application. At least, I thought about it and I don’t know how to deal with the problem.

Well, hope you can forgive me for this post but find the idea useful for your project.

matchers_in_have_tag_blocks.should be(/with_tag/)

Phil Matarese — Wed, 15 Oct 2008 10:00:00 -0400

The Setup

When writing test code for Rails apps, I prefer Rspec. For me the spec definition syntax feels more natural. Though, the other day I got tripped up by the Rspec equivalent of assert_select. With standard rails tests, the assert_select method can be nested to help ensure properly structured html elements.

1  # This tests that both users show up in the yes_list.
2  # As opposed to the no_list I guess.
3  assert_select 'div#yes_list' do
4    assert_select 'div#user', /#{@user1}/
5    assert_select 'div#user', /#{@user2}/
6  end

Rspec uses have_tag as a wrapper to this method, but uses with_tag and without_tag to wrap the method when called inside the nesting. The without_tag allows for simpler negative assertions, in case we wanted to test that user3 was not in the list for example.

1  # This also tests that both users show up in the yes_list.
2  response.should have_tag('div#yes_list') do
3    with_tag('div#user', {:text => /#{@user1}/})
4    with_tag('div#user', {:text => /#{@user2}/})
5  end

Once you know this, I think the spec code looks very clear. Unfortunately, calling have_tag within a have_tag block won't raise an error. There's no mechanism that will warn you that your inner have_tag is ineffective. The results of the inner have_tag will simply be ignored.

The Pitfall

When I first wrote the above spec, I used have_tag and have_text instead of with_tag. And to my naïve eyes, everything looked great. The code ran and the tests passed. Done and done.

 1  # This merely asserts the existence of a yes_list,
 2  #   no matter who's on the list.
 3  # Here at the Rails Loft, we're more discerning than that.
 4  response.should have_tag('div#yes_list') do
 5    have_tag('div#user') do
 6      have_text(/#{@user1}/)
 7    end
 8    have_tag('div#user') do
 9      have_text(/#{@user2}/)
10    end
11  end

This was completely wrong. The tests were passing, but they still passed when I removed @user1 from the yes_list one. When used inside another have_tag block, a have_tag's assertions was completely ignored. I think this had something to do with the inner have_tag setting up a new AssertSelect object that then got ignored by the outer AssertSelect object. The reason for my misunderstanding doesn't really matter, though. What matters is that I thought everything was fine; all the tests were passing after all. Luckily, I fiddled with the code a bit before committing, and discovered that reversing an important condition in my application code didn't affect my test one bit. Whoops. This was a mistake that continuous integration and testing metrics could have never caught, which emphasized to me how important it is to be thorough and careful when writing tests.

The Lesson

Always, always, always make sure your tests fail before you make them pass. Your tests are there to ensure proper functionality of application code, and this is the only way you can be certain your test code is doing what you expect. This doesn't necessarily mean you always have to use test-first development, though you should certainly give TDD a try. Making your tests fails could be as simple as toggling a conditional in your application code or changing the id of a tag against which you're matching. This way you'll be confident that your tests are truly working the way they appear to work.

Thinking in Legacy

Scott Roth — Wed, 21 May 2008 14:15:00 -0400

If you get my reference in the title of this post, then you have probably been around for a while and may have fallen into the thought trap I did the other day. Those of you that don't know Bruce Eckel's book probably are free from the "experience" (more accurately described as "baggage") that causes people to think in legacy patterns.

I was working on the metrics reporting section of a Rails application recently. We needed to display a value which was derived by summing up a subset of the rows in a metrics table. The metaphorical lightbulb went off over my head - I can do all this work in the database and just return the final result to my application!

So, I got busy and when I was done I had a a perfectly ugly SQL string using the sum function hardcoded into my model. I then used the always nice call of "ActiveRecord::Base.connection.execute(hardcoded_sql_string)" to get my result back. I thought job well done and moved on with my life.

Well, one of my less scarred co-workers saw the commit email go through and asked me why I was tainting the project with such hideous code. Well, I proudly proclaimed, thinking I was about to teach a lesson, you always want to push the processing as far down the stack as possible - after all, those database boxes are Big Iron. In addition, if you do the calculation in the database, you don't have to push all the rows over the wire in order to do a simple calculation in the app.

But, Scott, my co-worker exclaimed, the database is on the same box as the application server. And as he said it, I realized I had pre-optimized based on a set of assumptions that didn't make sense for this application. This application wasn't in some bloated enterprise infrastructure where there was a huge Oracle box lurking in the background with a firewall or two between it and the application server. So, we rewrote the code in a more Railsy way, pulling back the data and populating the objects and iterating through them to arrive at our results, and, lo and behold, there wasn't a performance difference between the two methods - and the Rails way was much easier on the eyes and for any future developers to understand.

I'm not advocating that you never drop down to SQL in your Rails apps. ActiveRecord is definitely a leaky abstraction (though a handy one) and it is well known that you can get into performance problems quickly if you don't understand the queries that are being made behind the scenes. However, in this particular case, I had no reason to drop down an abstraction level to direct SQL. I gained nothing and made the code less maintainable. This specific example really just made me realize that we all carry around internalized assumptions that can turn out to not be valid for all situations. It's good to remember this as we move from project to project, language to language, infrastructure to infrastructure, and environment to environment.

Setting up Multiple Site Instances on Joyent Accelerators

Scott Roth — Wed, 14 May 2008 14:13:00 -0400

I recently set up a new Rails site on a Joyent Accelerator for a client. The accelerator size selected had more than enough capacity, so we decided to set up the staging instance of the site on the same box. There is a great set of instructions on the Joyent wiki for how to properly set up a Rails site that will be deployed via Capistrano to an Accelerator, so start there if you've never set up a site on Joyent before. There are actually very few tweaks to those instructions you need to follow to set up multiple site instances on one Accelerator - in my case a "staging" and a "production" environment for a single application.

mongrel_cluster.yml
The default place to store this file is under /config. We are actually going to set up a separate mongrel file for each environment, so create a file called mongrel_cluster_[environment].yml (e.g. mongrel_cluster_staging.yml) under /config/accelerator/environments in your Rails application. Set the number of servers, the port, and the environment variables as you would normally. We'll be moving this file back to its expected location during the deployment process.

Now copy over the three files mentioned on the wiki into the /accelerator folder and make the needed changes from the instructions. We'll just be adding a few additional modifications.

accelerator_tasks.rb
Since we have moved the location of the mongrel configuration file we need to let the "create_vhost" task know where to read in the YAML file from as it uses variables from that file to properly configure apache. So update this line:

1  cluster_info = 
   YAML.load(File.read("config/accelerator/environments/mong
   rel_cluster_#{application}.yml"))

apache_vhost.erb
No changes needed. W00t!

smf_template.erb
This file is the template used to set up all the dependant services that your Rails app will need. The problem with the default version is that the mongrel service needs to have a distinct name for each application that is set up on the box otherwise bad conflicts happen. (This is also mentioned on the Joyent wiki page.) Update this line to make sure you have a distinct name for each environment:

1  <dependent name='mongrel_multi-user_<%= application %>' 
   restart_on='none' grouping='optional_all'>

Now we move onto the final step. We have moved the location of the mongrel configuration file so that we can support multiples of that file in our source control system. However, the mongrel service on the deployed server is still looking for that file in the old location with the old name. So, we are going to add a simple task to deploy.rb to fix this at deploy time. At the bottom of your file add a new custom task:

1  namespace :custom do
2    task :movemongrelconfig, :roles => :app do
3      # Move the env specific file to the general location
4      run "mv 
       #{release_path}/config/accelerator/environments/mongr
       el_cluster_#{application}.yml 
       #{release_path}/config/mongrel_cluster.yml"
5    end
6  end

And call the new task at the end of the deployment process:

1  after :deploy, 'custom:movemongrelconfig'

I'm sure that there are other ways to get multiple site instances running, but this one was relatively simple and has been working well for us for some time now.

Making FTP Code Compatible With SFTP Servers

Phil Matarese — Wed, 30 Apr 2008 17:30:00 -0400

Why?

We recently needed to toggle between support for SFTP and traditional FTP in one of our applications. In a Rails app, this is more of an annoyance than necessary because of the interface differences between Net::FTP and Net::SFTP. Not only are the transfer and navigation methods arbitrarily different, but the way the current directory on the server is dealt with is not consistent. I suspect these differences grow from the underlying differences in the technologies, but it would still be nice to code just one branch of code that does file transfers for either type of server. This is Ruby, right?

What?

This was the impetus for writing the sftp-compat plugin to extend Net::SFTP, giving it all the same interfaces as Net::FTP. The plugin is very new, but it was extracted from a real, working project we developed. It's functional, and it's being used in production right now as we speak (rather as I write and as you read - nobody is speaking right now - it's actually very quiet here - nice and tranquil - mmm.)

How?

All of the documentation can be found at http://sftp-compat.rubyforge.org/. There may be some functionality missing from the interface depending on your use case. The current version was designed for writing to remote FTP hosts, so there are a few methods for reading that I never needed to implement. If you are interested in using sftp-compat, but need the missing functions or would like to commit extra functions, please drop me an email.

Cinema Redux Redux

Phil Matarese — Wed, 30 Apr 2008 16:45:00 -0400

After the Cinema Redux presentation at GoRuCo was posted at ConFreaks, I couldn't resist reduxing the redux. I mean, this is totally meta. And, meta goes with Ruby like analogies go with blog posts. So, here it is... Cinema Redux Redux.

Cinema Redux at GoRuCo

Cinema Redux (Detail)

Create Your Own Cinema Redux Poster with Ruby and RMagick

John Berry — Fri, 25 Apr 2008 21:44:00 -0400

A few years back, the talented Brendan Dawes created Cinema Redux, which lays out a film as a series of stills captured at 1 frame per second. The result is a matrix of images resembling a DNA print of the film. By arranging these stills into rows of 1 minute, you get an interesting overview of how a film flows from start to finish.

While there are still some prints available for sale - 2001: A Space Odyssey is available through Coudal Partners - I wanted to do my own. Here are a few:

2001: A Space Odyssey

There Will Be Blood

Dr. Strangelove

Helvetica
by Gary Hustwit

Super Mario 1, 2, 3,
World & 64 speed runs

The Incredibles

Detail view of 2001: A Space Odyssey

What the Script Does

montage.rb has 2 major processes. First, it reads each of the rendered stills and resizes them to computed size (based on configuration options), and writes them to a subdirectory. Next, it reads in these scaled images into an RMagick ImageList - using the montage method - then writes the resulting file. The montage method would handle resizing for you, but the memory required to handle all frames from the movie at full resolution is prohibitively high. Manually scaling the images allows for more efficient memory management.

Step 0

Before you get started, you'll of course need RMagick 2 (and thus ImageMagick). While I'm sure you could hack the script to run with RMagick 1.x, I strongly advise you don't. RMagick 2 does a much better job of memory management, and supports the .destroy! method, allowing the script to free memory in the image resize loop. RMagick 1.x caused all sorts of problems for me early on.

Step 1: Download montage.rb

That's it, just download montage.rb. Well, then unzip it.

Step 2: Export The Movie as a Series of Stills

Create a new directory and place in it your movie and montage.rb. This next export step will create thousands of still images, so you might want to get a little organized.

If you don't have QuickTime Pro:
You'll need FFmpeg. The command you'll use will be similar to:

ffmpeg -i YOUR_MOVIE.avi -r 1 -f image2 movie_%4d.png

Once this is done, you can skip to Step 3.

If you do have QuickTime Pro:
Select "File -> Export..."

From the Export select menu, chose "Movie to Image Sequence"

Click the Options button, and use the following settings

(For these options, I chose Best Depth and Medium Quality)

Choose your target directory and save. This step will create those few thousand still images, so make sure you're putting them in the newly created directory.

Step 3: Choose montage.rb options

Run montage.rb without any arguments to get a listing of the usage options. The only required argument is the file pattern that will match against the movie stills.

What Now?

I went with QOOP as a printing service, and chose a 20" x 30" format. Some jpeg compression was in order to get the filesize below their threshold, but not much. The print is high quality and I'm quite pleased.

Note that if you're into Processing, you can find a sketch that appears to do similar work here.

Rspec Without db:test:prepare

Phil Matarese — Wed, 23 Apr 2008 14:30:00 -0400

First Things First

Once upon a time, after deploying an Rspec'd app for the first time, we wanted to run all the specs on our staging server to see that everything was running as expected. Only, there was a problem with dependencies.

At the time we had a pretty gross, gem-dependency soup cooking up in our app. We had rspec as a frozen gem and rspec_on_rails as a plugin, and this was causing (unconfirmed) load-order problems for rake. After checking the rspec documentation again, we changed rspec from a gem to a plugin, which straightened everything out (I think).

The Basics

Basically, our current setup is to have piston (essential for plugin management), rcov (keep the coverage above 50% or the bus blows up), and ZenTest (because autotest is fun) installed locally as gems on all development machines - there is no reason for us to use these on our production or staging servers. Within the app we have rspec_on_rails, rspec_autotest, and rails_rcov (which uses magic) installed in the plugins directory (under the watchful eye of piston.) These are part of the app, not only because the rspec docs like it that way, but because we'd like to ensure all the tests pass after a deployment. Everything is all nice and neat now. Shouldn't have anymore problems. Nope. No sir. No way. Uh-uh.

The Problem

When running rake spec, the database can't be found. But... we installed it... right? Of course, it's right there. And... the permissions? They're good, too. Uhh... Hey, did that say the "development" database can't be found? What the...

Unbeknownst to us, the spec task had as a prerequisite db:test:prepare, which takes a schema_dump from the development database and uses it to create a test database. This is something we don't need. Unfortunately, there's no option or flag that you can use to disable this prerequisite.

The Solution

So this one got a little hairy. The load order for rake tasks is rails, custom code in lib/tasks/*, and then rspec. So, I couldn't drop the spec prerequisites with my custom code, because they didn't exist yet. My first attempt to solve this involved wiping out db:test:prepare completely, no matter what. I was a bit hesitant to do this, because it had nothing to do with what I was really trying to accomplish – I've got nothing against db:test:prepare, I just want it decoupled from rspec. Here is our current solution which only wipes out db:test:prepare when the spec task is invoked. Not only does this solve the problem of testing on a staging server, but it also speeds up testing in our local development environments. Enjoy.

 1  # bock_db_test_prepare.rb
 2  
 3  # without this, you could never
 4  # remove elements of a task
 5  class Rake::Task
 6    def clear_it
 7      @actions.clear
 8      @prerequisites.clear
 9    end
10  end
11  
12  # set up the blocking as a rake task,
13  # so we can add it as a prereq to spec
14  namespace :ap do
15    namespace :spec do
16      task :skip_db_test_prepare do
17        Rake::Task['db:test:prepare'].clear_it
18      end
19    end
20  end
21  
22  # block it!
23  task :spec => 'ap:spec:skip_db_test_prepare'

The final line works even though the spec task doesn't exist yet. Essentially, our custom code creates an empty spec task with our single prerequisite. When rspec loads, the real task and prerequisites will be appended after ours. Success.