Revolution On Rails

[PLUGIN RELEASE] ActsAsOverflowable

noreply@blogger.com (John Constable) — Wed, 09 Jul 2008 17:10:00 +0000

acts_as_overflowable

Written by Nicholas Lega

== DESCRIPTION:

Allows a column to overflow data into a secondary column if the data size exceeds the character limit. This is useful for fast indexing.

Instead of trying to index text blobs, you can specify a varchar column to be used for indexing. Any characters that do not fit into the base column will be automatically saved into or retrieved from an overflow field.

== SYNOPSIS:

Create overflow column migration:


 class CreateOverflowModels < ActiveRecord::Migration   
   def self.up         
     create_table :overflows do |t|       
       t.integer :overflowable_id       
       t.string :overflowable_type       
       t.string :overflow       
       t.timestamps     
     end

     add_column(:my_class_that_uses_overflows,
                :has_overflow,                 
                :boolean)     

     add_column(:my_class_that_uses_overflows,                 
                :overflowable_text,                 
                :string,                 
                :limit => 255)
   end

   def self.down
     drop_table :overflows

     remove_column(:my_class_that_uses_overflows,
                   :has_overflow)

     remove_column(:my_class_that_uses_overflows, 
                   :overflowable_text)
   end
 end

Add to model:


 class MyClassThatUsesOverflow < ActiveRecord::Base   
   acts_as_overflowable :overflow_column => :overflowable_text,
                        :overflow_limit => 255,
                        :overflow_indicator => :has_overflow,
                        :overflow_table => :overflows
 end

To get and set the value of the "overflowable_text" field as described in the test model, use:


 # #{my_field_name_here}_with_overflow is the method name
 long_text = overflowable_obj.overflowable_text_with_overflow

 overflowable_obj.overflowable_text_with_overflow = very_long_text

Simply using the getters and setters for "overflowable_text" without appending "_with_overflow" to the method name will return the fragment of the text contained in the base column.

== INSTALL:

sudo gem install revolutionhealth-acts_as_overflowable -s http://gems.github.com

== SOURCE:

http://github.com/revolutionhealth/acts_as_overflowable/tree/master

== FEATURES/PROBLEMS:

* Only tested on mysql and sqlite3

[PLUGIN RELEASE] ActsAsSeoFriendly

noreply@blogger.com (Jeffrey) — Thu, 19 Jun 2008 16:21:00 +0000

ActsAsSeoFriendly

== DESCRIPTION:

Create an SEO friendly field for a model automatically based on a given field.

So if you have a Blogs model, and you would like create an SEO friendly version
of the 'title' field, you would just add this to your model and then be able to
use the SEO friendly id as the unique id to the resource. The plugin will only
append an integer to the SEO id if there is a collision.

== SYNOPSIS:

Create seo column migration:


class CreateSeoTestModels < ActiveRecord::Migration
  def self.up
    create_table :seo_test_models do |t|
      t.string :name
      t.timestamps
    end
  SeoTestModel.create_seo_friendly_column()
  end

  def self.down
  SeoTestModel.drop_seo_friendly_column()
    drop_table :seo_test_models
  end
end

Add to model:


 class SeoTestModel < ActiveRecord::Base
  acts_as_seo_friendly :resource_id => :name,
              :seo_friendly_id_field => :seo_id, # default is :seo_friendly_id
              :seo_friendly_id_limit => 100 # default is 50
 end

To lookup the resource in the controllers use:


 SeoTestModel.find_by_seo_id(params[:id])

== INSTALL:

sudo gem install revolutionhealth-acts_as_seo_friendly -s http://gems.github.com

== SOURCE:

http://github.com/revolutionhealth/acts_as_seo_friendly/tree/master

To see the plugin in action see our recipes section, for example:

http://www.revolutionhealth.com/recipes/thai-miang-khem-style-salad

== FEATURES/PROBLEMS:

* Only tested on mysql and sqlite3

ThreadsafeBenchmark

noreply@blogger.com (Sean) — Thu, 13 Dec 2007 21:44:00 +0000

When testing products such as services which need to be stress-tested prior to release, it's necessary to use multi-threading to get as close to real world usage as possible. This gem, though not intended to be a replacement for full-fledged testing suites such as LoadRunner, can provide instantaneous results to facilitate TDD programming. To reduce the duplication of code, the gem utilizes Ruby's built-in Benchmark module for the base functionality while preventing the output from clobbering through the use of thread-specific IO buffers.

require 'threadsafe_benchmark'
include ThreadsafeBenchmark

threads = []
max_num = 5000

5.to_i.times { |i|
 threads << Thread.new(max_num) { |n|
   threadsafe_bm(6) { |x|
     x.report("for:")   { for i in 1..n; a = "1"; end }
     x.report("times:") { n.times do   ; a = "1"; end }
     x.report("upto:")  { 1.upto(n) do ; a = "1"; end }
   }
 }
}

threads.each { |t| t.join }

Using the standard Benchmark, the results would be printed haphazardly making it difficult to read and interpret. But ThreadsafeBenchmark cleans everything up giving us nicely laid out columns.

	user	system	total	real
for	0.000000	.000000	.000000	( 0.002889)
times	0.000000	.000000	.000000	( 0.002477)
upto	0.000000	.000000	.000000	( 0.002479)
for	0.000000	.000000	.000000	( 0.002401)
times	0.010000	.000000	.010000	( 0.002586)
upto	0.000000	.000000	.000000	( 0.002413)
for	0.000000	.000000	.000000	( 0.002205)
times	0.010000	.000000	.010000	( 0.002245)
upto	0.000000	.000000	.000000	( 0.002272)
for	0.000000	.000000	.000000	( 0.001822)
times	0.010000	.000000	.010000	( 0.001958)
upto	0.000000	.000000	.000000	( 0.001943)
for	0.010000	.000000	.010000	( 0.010090)
times	0.010000	.000000	.010000	( 0.009225)
upto	0.010000	.000000	.010000	( 0.007986)

The gem and source files are available at Rubyforge.

Enhanced Migrations v1.2.0

noreply@blogger.com (Sean) — Tue, 06 Nov 2007 18:38:00 +0000

The original release of this highly useful plugin marked a turning point in collaborative Rails development by freeing the developer to commit their database migration without fear of having it ignored because of a higher placed migration number. This latest release includes some minor bug fixes plus a useful method for stepping through migrations one at a time without the need for copying and pasting long version numbers.

Fixed bug where an empty migrations_info table would create a non-parseable schema.rb.
Made plugin database independent.
Added capability to step through migrations using VERSION=[previous, next, first and last].
dump_schema_information now returns all migrations, not just latest (credit to François Beausolei).
Added tests which use SQLite and a task (enhanced_migrations:clean_sqlite_db) to help with testing.

As an example of item number three, consider the following situation. Using the enhanced migrations plugin, you've just created a migration that adds a new table to your database. Upon running it, you discover that you used the wrong data type for a column. Rather than having to copy and paste the previous migration's version number, simply using 'previous' in the VERSION number will now suffice.


rake db:migrate VERSION=previous

This also works using prev along with next, first and last for their respective operations. Keep in mind that first will go to the first migration and is not the same as VERSION=0.

Migrate as usual


shell> rake db:migrate
== CreateRecipesTable: migrating ==============================================
-- create_table(:recipes)
-> 0.0902s
== CreateRecipesTable: migrated (0.0904s) =====================================

== AddRecipesForUser1: migrating ==============================================
-- execute("INSERT INTO recipes (name, owner) VALUES ('Lemon Meringue Pie', 'user1')")
-> 0.3684s
== AddRecipesForUser1: migrated (0.5302s) =====================================

== AddRecipesForUser2: migrating ==============================================
-- execute("INSERT INTO recipes (name, owner) VALUES ('Steak and Kidney Pie', 'user2')")
-> 0.2574s
== AddRecipesForUser2: migrated (0.3962s) =====================================

Migrate to the previous version


shell> rake db:migrate VERSION=previous
== AddRecipesForUser2: reverting ==============================================
-- execute("DELETE FROM recipes WHERE owner = 'user2'")
-> 0.4512s
== AddRecipesForUser2: reverted (0.4516s) =====================================

Migrate to the first version


shell> rake db:migrate VERSION=first
== AddRecipesForUser1: reverting ==============================================
-- execute("DELETE FROM recipes WHERE owner = 'user1'")
-> 0.1676s
== AddRecipesForUser1: reverted (0.1678s) =====================================

Migrate another previous version, essentially the same as VERSION=0 since we are already at the first migration.


shell> rake db:migrate VERSION=previous
== CreateRecipesTable: reverting ==============================================
-- drop_table(:recipes)
-> 0.1680s
== CreateRecipesTable: reverted (0.1683s) =====================================

How to get
Download the gem from Rubyforge and install it like so:

sudo gem install enhanced_migrations-1.2.1.gem

Update - 11/15/2007
The previous version had a small bug that was discovered during use here that sometimes caused an app's database yaml file to be overwritten by the gem's database config which is used only for testing. The chances of this happening outside of our particular setup are small but we felt it warranted an immediate fix. We've also added a gzip'd source file for the plugin fans out there.

Links in the post have been updated to reflect the location of this new release.

Weblog update

noreply@blogger.com (Unknown) — Fri, 24 Aug 2007 21:09:00 +0000

Hey all I wanted to give you an update:

Three of our active contributors are moving on: Aaron, Eddie, and Val. They plan to stay active in the Rails community, as well as new areas like Facebook. You can follow their work at blog.hungrymachine.com. It has been a great pleasure to work with them and we wish them the best.

FragmentFu - Fun with Fragments

noreply@blogger.com (Unknown) — Mon, 06 Aug 2007 21:16:00 +0000

First, please read the Advanced Rails Caching.. On the Edge for a primer on ESI, caching, and fragments.
To get started developing with ESI, I'm going to walk through a simple tutorial building an ESI enabled Rails application.
Get Mongrel-ESI
Find the latest mongrel ESI here: http://code.google.com/p/mongrel-esi/downloads/list


   wget http://mongrel-esi.googlecode.com/files/mongrel-esi-0.0.5.gem

(Does anyone have a better solution of ruby projects hosted on Google Code? )

Install Mongrel ESI


     sudo gem install mongrel-esi-0.0.5.gem

Create a new rails application (Currently using rails 1.2.3)


   rails fragment_fu_demo

     create  
     create  app/controllers
     create  app/helpers
     create  app/models
     .....

   cd fragment_fu_demo

Create a home page for your application

Delete the public/index.html page


     rm public/index.html

Edit the config/routes.rb and uncomment the following line:


     map.connect '', :controller => "welcome"

Create a welcome controller


     ./script/generate controller welcome index

Install the FragmentFu plugin

   
  ./script/plugin install http://mongrel-esi.googlecode.com/svn/trunk/plugin/fragment_fu

Start your application

 
  ./script/server

visit http://localhost:3000 in your browser. You should see a "Welcome#index" page.

Start MongrelEsi
From the command line, run:


 mongrel_esi start
    ** Starting Mongrel listening at 0.0.0.0:2000

visit http://localhost:2000 in your browser. You should see a "Welcome#index" page.

Create a fragment to cache
Edit the app/controllers/welcome_controller.rb
Add an action called now

 
     def now
        render :text => "#{Time.now} is #{Time.now.usec} nano-seconds"
     end

Edit the app/views/welcome/index.html.erb and replace with the following:


     <h1>Welcome</h1>
     <p><%= render :esi => "/welcome/now" %></p>

Page without ESI parsing
visit http://localhost:3000 and you should see: "Time:" in the browser. If you view the source, you'll see


    <h1>Welcome</h1>
    <p><esi:include src="/welcome/now" max-age="0"/></p>

This is the simplest of esi tags, and the default max-age is 0, which means do not cache.

Page with ESI Parsing

visit http://localhost:2000 and you'll see:

Welcome
Date: Mon Aug 06 00:00:00 -0400 2007 which is 62972ns

Caching modules
Edit the index.html.erb and replace with the following:


   <h1>Welcome</h1>
   <p><%= render :esi => "/welcome/now" %></p>
   <h2>What time is it again?</h2>
   <p><%= render :esi => "/welcome/now"%></p>

If you refresh http://localhost:2000, you'll notice they nanoseconds are different

Lets add a ttl to first call


  <h1>Welcome</h1>
  <p><%= render :esi => "/welcome/now", :ttl => 45.seconds %></p>
  <h2>What time is it again?</h2>
  <p><%= render :esi => "/welcome/now"%></p>

If you refresh http://localhost:2000, you'll notice they are now the same. Refresh again in the next 45 seconds, and it will not change.

The next tutorial will cover a small TODO application, with inline invalidation and exception handling. Coming Soon!

Advanced Rails Caching.. on the Edge

noreply@blogger.com (Unknown) — Mon, 06 Aug 2007 17:15:00 +0000

When trying to scale a portal to millions of users with complex personalization, we had to rethink the application design. We needed partial page cacheability, and we wanted that close to the Edge... and recently the idea has been catching on. First with Components Are the New Black, and the idea of Nginx, SSI, and Memcache, what I'd like to briefly describe below is a partial page caching strategy that has worked well for the past year.

First, we use a technology called ESI. ESI stands for Edge Side Includes, which is a simple markup language for describing dynamic assembly of applications. At the core, its similar to SSI, but its a more versatile spec that has been accepted/implemented by a half dozen cache servers, both open source (Squid, Mongrel-ESI) and commercial (Oracle Web Cache, Akamai, among others). It also includes an invalidation protocol, exception handling on the edge, and a few other features.

The ESI W3C specification has been out for 6+ years, so these ideas are not new, but ESI is a diamond in the rough. No one seems to be using it.

A simple example of ESI in your rails application is including a common header. If the header is static, using a shared layout, or rendering a shared partial across applications, could be sufficient. But if you use the common idiom of a sign-in bar, like "Welcome Bob - Signout", on a page you want to fully cache, or a common header you want to share across multiple applications, then you may consider another approach.

If you leverage ESI, you can put the below in your application layout.

<div id="header">
  <esi:include src="/header" max-age="300"/>
</div>

The <esi:include> tag will inject the response from /header into the page and cache that fragment for 300 seconds. You can also add a variety of options, request header parameters (which can be used to personalize the request) along with a dozen other optional parameters, some of which I will outline below. This is simple SSI (Server Side Includes) .

But if we take it a step further, lets look at how we can apply it to a site like Twitter.

If you look at the image above, the majority of the page can be fully cached. I've outlined two green boxes, which appear to be dynamic content, which can have different TTLs. Using ESI, your markup could be:

<div id="latest">
  <esi:include src="/latest" max-age="5"/>
</div>
...
<esi:include src="/featured" max-age="3600"/>

The ESI enabled cache server would parse this markup, make 2 separate HTTP requests (/latest, and /featured) and cache those with their corresponding TTLs. You can furthermore cache the wrapping template with a Surrogate-Control header, which tells the cache server to keep a cached copy of the template. A request to this page 4 seconds later would make 0 requests to your rails infrastructure. 8 seconds later would only hit the /latest, returning the rest of the page from cache. You can also envision a application pool of servers just to handle /latest, but I'll get into sharding applications via ESI in a later article.

Exception Handling

If you take this a step further, you can also define timeouts and exception behavior. As defined below, If the /latest request takes more than 1s, The cache server will give up, and retrieve the static snippet defined in the except block from your static web host.

<esi:try>
  <esi:attempt>
    <esi:include src="/latest" max-age="5" timeout="1"/>
  </esi:attempt>
  <esi:except>
    <esi:include src="http://static.foo.com/latest" max-age="5" timeout="1"/>
  </esi:except>
</esi:try>

We call these "Sorry" modules, because they often say "Sorry, this feature is having trouble", but the rest of the page may surface meaningful content. Even better, write out more meaningful content to disk once a day and serve that from Apache.

Invalidation

The other benefit of ESI is Invalidation support. There are various mechanisms to invalidate content, but my favorite is Inline Invalidation. Consider the common rails idiom of updating data. You post to a controller which redirects to a view of that data. Since the HTTP redirect (301) bubbles all the way back to the browser, any content in the body of the redirect can be parsed by the ESI server. Therefore you can put the invalidation xml is the redirect response body and not conditionally dirty your view logic.

<esi:invalidate>
  <?xml version="1.0"?>
  <!DOCTYPE INVALIDATION SYSTEM "internal:///WCSinvalidation.dtd">
  <INVALIDATION VERSION="WCS-1.1">
    <OBJECT>
      <BASICSELECTOR URI="/foo/bar/baz"/>
      <ACTION REMOVALTTL="0"/>
    </OBJECT>
  </INVALIDATION>
</esi:invalidate>

The above is a simple single URL example, but the specification supports regex, which would work well for restful resources (e.g. /people/#{person.id}/*), among other things.

FragmentFu

FragmentFu is a plugin that enables ESI support in rails applications. At this point it is an extraction from our internal plugin for modules, caching, and ESI. Its in active extraction/development, but I wanted to involve the community in gathering ideas, suggestions and comments.

Some sample snippets:

<% render :esi => widget_url(1) %>

<%= render :esi => latest_url, :ttl => 5.seconds, :timeout => 1.second %>

<%= render :esi => featured_url, :ttl => 10.hours, :except => '/some/static/path' %>

We've also toyed around with the idea of fragment respond_to's. Since ESI supports adding request header parameters, we can mimic Prototype's
X-Requested-With behavior to implement:

def latest
  ...
  respond_to |wants| do
    wants.html { do something }
    wants.fragment { do something }
  end
end

How to get started

Mongrel-ESI is a great starting place. It was built in-house and released a few months ago on Google Code. It supports a large subset of ESI, ESI Invalidation, and its a great tool for developing applications that utilize ESI. You can grab Mongrel-ESI, download the Oracle Web Cache Standalone or even Squid 3.0... and when you're site gets really big, its time to give Akamai a call. :)

Code Digest #2

noreply@blogger.com (Unknown) — Sun, 24 Jun 2007 13:06:00 +0000

When you program for a living, you write lots of code. There is often some code that you are fond of. We started the Code Digest series to present such code written by the RHG developers. We encourage other teams and individual developers to share similar snippets in their blogs so we all can learn from each other and become better rails developers.

Mai Nguyen

Simple AJAX error messaging

When simple javascript validation is not enough and your product managers insist on ajax for error messaging, guess what. You have to implement ajax error messaging. One such case is 'username availability'. This simple example displays an error message on blur of a the username field. It doesn't hit the server unless the value of the field is well-formed (at least that will save you *some* network traffic ...)

Controller would look something like:

class FooController < ::ApplicationController
  
  def is_username_taken
    
    unless Person.find_by_username(params[:username])
      render :nothing => true
      return
    end

    message_html_id = params[:message_html_id]
    render :update do |page|
      page.replace_html message_html_id ,"Username is not available, please choose another."
      # if you want error styling associated with the failed state
      page << "document.getElementById('" + message_html_id + "').className = 'failed'"
    end

  end
  
end

View would look something like:

<dl>
  <dt><label>Username: </label></dt>
  <dd id="username_input">
    <%= text_field :person, :username, :class => "input", :type => 'text', :id => 'person_username_input' %>
  </dd>
  <dd id="username_messaging"><%= @person.errors.on(:username)%></dd>
</dl>

Your javascript would look something like:

var UserNameUnique = Class.create();
UserNameUnique.prototype = {
    initialize: function( field_id, message_id) {
        this.message_id = message_id;
        this.field = document.getElementById( field_id );
        if (typeof this.field == 'undefined') {return;}
        // Observe blur on field    
        Event.observe(this.field, 'blur', this.checkName.bindAsEventListener(this));
    },
    checkName: function() {
    if( typeof this.field != 'undefined' )
    {
            var name = this.field.value;
            // don't hit server unless username is well-formed
            var re = /^[A-Za-z])[a-zA-Z0-9]{2,25}$/;
            if( name.match( re ))
                new Ajax.Request('/foo/is_username_taken', {asynchronous:true, parameters:'username='+name+'&message_html_id='+this.message_id});
            }
    }
};

new UserNameUnique('person_username_input', 'username_messaging');

You could also use Rails helper observe_field instead of the writing your own javascript, but there is more flexibility in writing your own javascript (such as the need to check well-formed values before hitting the server).

Mark Brooks

Creating the options list for select tags can be annoying, especially if there is a requirement that the current "option" be displayed as the default option. With javascript, it isn't such a big deal, but one must take care of the degraded case as well.

In any event, the base object is an array of two-item hashes representing both the name of a video channel and the number of videos in that channel. By way of example:

@channelinfo = [
  {"name"=>"Fitness","count"=>4},
  {"name"=>"Diabetes", "count"=>1},
  {"name"=>"Pregnancy", "count"=>11}
]

The option values are the name fields of each hash. The key is, we want the current option value to be the first one in the option list, and the rest to be in alpha order by name.

So let's create an option builder from the bottom up.

First, we only need the channel names for the select tag. This gives us what we need:

options = @channelinfo.collect { |channel| channel['name'] }

That gives us a list of channel names. Using the list above, it would be ["Fitness", "Diabetes", "Pregnancy"].

However, we need to make sure that the current channel is first in the list. Let's say that current channel is Diabetes. Since we already have that value in @current_channel, we can exclude it from our options list:

options = @channelinfo.collect do |channel|
  channel['name']
end.reject do |channelname|
  channelname == @current_channel
end

Now the resulting list will look like ["Fitness", "Pregnancy"]. However, we still need the current channel to be at the front of the lift, so we add it back as the first element:

options = @channelinfo.collect do |channel|
  channel['name']
end.reject do |channelname|
  channelname == @current_channel
end.unshift(@current_channel)

Now our options list looks like ["Diabetes", "Fitness", "Pregnancy"].

Two points. First, we want to make sure that, while the current channel is at the head of the list, the rest of the list items are in alpha order, so we add a sort directive in the appropriate place. Also, it is probably a good idea to exclude any nils that might pop up in the collection on the original data object, since it comes from a service and it is possible, however unlikely, that a hash might get spit out without a 'name' property. The more rigorous code looks like this now:

options = @channelinfo.collect do |channel|
  channel['name']
end.compact.reject do | channelname |
  channelname == @current_channel
end.sort.unshift(@current_channel)

Now we can generate our options list using:

options.collect do |channelname|
  "<option>" + channelname + "</option>"
end.join(",")

although if you want to, you can simply combine the whole thing as follows:

@channelinfo.collect do |channel|
  channel['name']
end.compact.reject do |channelname|
  channelname == @current_channel
end.sort.unshift(@current_channel).collect do |channelname|
  "<option>" + channelname + "</option>"
end.join(",")

to get the same result, and eliminate the unnecessary options binding.

Todd Fisher

Here is an extension trying to execute a block multiple times before giving up. Handy for network operations and such.

module Kernel
  def could_fail(retries = 3, &block)
    tries = 0
    begin
      yield
    rescue Exception
      tries += 1
      if tries < retries
        retry
      else
        raise
      end
    end
  end
end

Call it as result = could_fail { some_operation_that_might_fail_first_time }

Todd Fisher

When you are writing a script that needs to auto install gems, you are likely to run into a problem that it stops because there are multiple platform versions available (jruby, win32, etc) and the gem command expects you to pick one that matches your platform. This patch forces to use a specific platform so no user interaction is needed. Original idea from Warren updated to support rubygems 0.9.4.

module GemTasks
  def setup
    
    return if $gems_initialized
    $gems_initialized = true
    Gem.manage_gems

    # see => http://svn.bountysource.com/fishplate/scripts/debian_install.pl
    Gem::RemoteInstaller.class_eval do
      
      alias_method :find_gem_to_install_without_ruby_only_platform, :find_gem_to_install
      
      def find_gem_to_install( gem_name, version_requirement, caches = nil )
        if caches # old version of rubygems used to pass a caches object
          caches.each {|k,v| caches[k].each { |name,spect| caches[k].remove_spec(name) unless spec.platform == 'ruby' } }
          find_gem_to_install_without_ruby_only_platform( gem_name, version_requirement, caches )
        else
          Gem::StreamUI.class_eval do
            
            alias_method :choose_from_list_without_choosing_ruby_only, :choose_from_list
            def choose_from_list( question, list )
              result = nil
              result_index = -1
              list.each_with_index do  |item,index|
                if item.match(/\(ruby\)/)
                  result_index = index
                  result = item
                  break
                end
              end
              return [result, result_index]
            end
            
          end
          
          find_gem_to_install_without_ruby_only_platform( gem_name, version_requirement )
        
        end
      end
    
    end
  
  end
end

Val Aleksenko

Since class-level instance variable are not inherited by subclasses, you need to go some extra steps when writing a plugin using them. Depending on the amount of such variables, I have been either defining a method instead of class-level instance variables or forwarding them to subclasses.

Example #1. acts_as_readonlyable needs to provide a single class level instance variable. Defining a method instead.

def acts_as_readonlyable(*readonly_dbs)
  define_readonly_model_method(readonly_models)
end

def define_readonly_model_method(readonly_models)
  (class << self; self; end).class_eval do
    define_method(:readonly_model) { readonly_models[rand(readonly_models.size)] }
  end
end

Example #2. acts_as_secure uses a bunch of variables. Forwarding them to subclasses.

def acts_as_secure(options = {})
  @secure_except = unsecure_columns(options.delete(:except))
  @secure_storage_type = options.delete(:storage_type) || :binary
  @secure_class_crypto_provider = options.delete(:crypto_provider)
  @secure_crypto_providers = {}
  extend(ActsAsSecureClassMethods)
end
    
module ActsAsSecureClassMethods
  def inherited(sub)
    [:secure_except, :secure_storage_type, :secure_class_crypto_provider, :secure_crypto_providers].each do |p|
      sub.instance_variable_set("@#{ p }", instance_variable_get("@#{ p }"))
    end
    super
  end
end

[PLUGIN RELEASE] Metrics

noreply@blogger.com (Anonymous) — Fri, 22 Jun 2007 17:57:00 +0000

From Jeffrey Damick

Introduction

This gem provides a metrics collecting for controllers, database queries, and specific blocks of code or methods. It is designed to be light-weight and have minimal impact on production builds while providing performance indicators of the running application.

Disclaimer

This software is released to be used at your own risk. For feedback please drop us a line at rails-trunk [ at ] revolution DOT com.
Using this plugin should not be your first step in application optimization/scaling or even the second one.

Example

class SomeClassToTest
  collect_metrics_on :my_method

  def my_method(blah = nil)
    true
  end
end

Output:

[ERROR] [2007-06-21 23:21:19] [trunk] [Metrics]|[76716]|[MysqlAdapter.log]|0.012727|args=["root localhost trunk_test", "CREATE DATABASE `trunk_test`"]
[ERROR] [2007-06-21 23:19:56] [trunk] [Metrics]|[35158]|[Request to [Test::SomeControllerWithMetricsId]]|0.001373|action = index|path =some?
[ERROR] [2007-06-21 23:19:56] [trunk] [Metrics]|[33676]|[SomeClassToUseModuleMixin.another_method]|0.000020|args=["also"]

for more samples and test cases see test/metrics_test.rb

Usage

The metrics are written to: logs/_metrics.log

Configuration can be updated in metrics/config/metrics.yml, you may copy this file to your RAILS_ROOT/config/metrics.yml and customize for your application, the RAILS_ROOT will be checked first.

Sample metrics.yml

production:
  min_real_time_threshold: 1.0
  single_line_output: true
  some_module/test_controller: 0.0

Installation

As plugin:
script/plugin install svn://rubyforge.org/var/svn/metrics/trunk/vendor/plugins/metrics

License

metrics is released under the MIT license.

Support

The plugin RubyForge page is http://rubyforge.org/projects/metrics

Code Digest #1

noreply@blogger.com (Anonymous) — Sun, 17 Jun 2007 14:45:00 +0000

When you program for a living, you write lots of code. There is often some code that you are fond of. We start the Code Digest series to present such code written by the RHG developers. We encourage other teams and individual developers to share similar snippets in their blogs so we all can learn from each other and become better rails developers.

Daniel Silva

<%= link_to(image_tag("/images/icon.gif", :style=>"padding-right:0px;"), {:controller => '/registration', :action => 'login', :dest => request.request_uri}, {:style => ""}) %>

With this line, I can create an image link using Rails code, to take advantage of the power of routes in Rails while still creating this kind of structure: <a href="#"><img border="0"/></a>

Jack Dempsey

Ok, so here's something useful for using keyboard short cuts to navigate through a small supporting application:

def include_keyboard_shortcuts
  
  hotkey = 'Ctrl+Shift'
  code_string = ''
  keys = {
    'm'=>'/account_info/show',
    's'=>'/account_info/index',
    'c'=>'/menu/go?menu_action=new_customer',
    'p'=>'/products',
    'o'=>'/account_info/orders',
    't'=>'/order_tracker',
    'x'=>'/auth_sessions/destroy',
    'r'=>'/current_users/reset_password'
  }

  current_user_needed = %w{m o r}

  keys.each_pair do |k,v|
    next if current_user_needed.include?(k) && current_user.nil?
    code_string << "shortcut('#{hotkey}+hm#{k}',function() { document.location.href='#{v}' });\n"
  end

  javascript_tag(code_string)

end

Then in a layout file just:

<%= javascript_include_tag 'shortcuts' %>
<%= include_keyboard_shortcuts %>

Makes use of a nice shortcuts.js lib.

Val Aleksenko

This is a fragment of a conversation on our internal IRC with my solution for refactoring a piece of code:

May 07 14:07:56 <Anthro> There has to be a better way to do this (x and y are non-negative integers): increment = (x!=0) ? ( (y!=0) ? 0 : 1 ) : -1
May 07 14:08:22 <Anthro> I think it can be done with math instead of logic.
May 07 14:16:56 <jack>  Anthro: have you thought about using sin and cos? ;-)
May 07 14:18:42 <muzzy> (x <=> 0) <=> (y <=> 0)

Granted, it is not as clear as the original, but it is hard to resist it from the aesthetic point of view.

Val Aleksenko

I love writing dynamically generated methods and classes in Ruby as the next guy. When I was writing acts_as_readonlyable, I needed to manage multiple connections. I found that the easiest solution for managing active connections was generation of an AR class for each read-only definition and borrowing the connection from it. The usage is acts_as_readonlyable :read_only_entry_in_database_config.

def acts_as_readonlyable(readonly_db)
  define_readonly_class(readonly_db) unless ActiveRecord.const_defined?(readonly_class_name(readonly_db))
end

def readonly_class_name(db)
  "Generated#{ db.camelize }"
end

def define_readonly_class(db)
  ActiveRecord.module_eval %Q!
    class #{ readonly_class_name(db) } < Base
      self.abstract_class = true
      establish_connection configurations[RAILS_ENV]['#{ db }']
    end
  !
end

Warren Konkel

The ConfigFile class is a quick and easy way to load YAML files located in your Rails application's config directory. Simply place a YAML file into your config directory and then access it like a hash. For example this will read your config/database.yml: ConfigFile['database']['production']['adapter'].

class ConfigFile
  def self.[](arg)
    @@cached_configs ||= {}
    @@cached_config_mtimes ||= {}

    base_name = "config/#{arg}.yml"
    filename = File.join(RAILS_ROOT, base_name)
    raise "ERROR: Config not found: #{base_name}" unless File.exists?(base_name)

    if @@cached_configs[arg].nil? || @@cached_config_mtimes[arg] < File.stat(filename).mtime.to_i
      @@cached_configs[arg] = YAML.load(File.open(filename))
      @@cached_config_mtimes[arg] = File.stat(filename).mtime.to_i
    end

    @@cached_configs[arg]
  end
end

Our Contribution to Advanced Rails Recipes

noreply@blogger.com (Unknown) — Thu, 07 Jun 2007 14:16:00 +0000

We submitted a few recipes, answering the call for sharing what is happing in the rails community, and a half a dozen of them were accepted as entries to the upcoming book. If you enjoyed reading the blog, you would soon have a chance to see some of it on paper.

Revolution Health Traffic Doubles in 4 Weeks

noreply@blogger.com (Unknown) — Mon, 04 Jun 2007 14:08:00 +0000

Hitwise has recently publish traffic statistics on our growth over the past month.

This Wednesday AOL Founder Steve Case spoke at D5 about his new venture, Revolution Health. Revolution Health is setting out to be far more than just a health information source on the web, but it is already succeeding on that front. For the week ending May 26, 2007, Revolution Health was ranked #11 in the Health & Medical - Information category, and traffic increased by 113% since the week ending April 28, 2007, when it was ranked #23.

Ruby vs. JRuby - The path to consistent behavior + OpenSSL

noreply@blogger.com (Unknown) — Mon, 04 Jun 2007 03:42:00 +0000

In Ola Bini's recent post There can be only one, a tale about Ruby, IronRuby, MS and whatnot, he stresses the importance of documenting Ruby's API so that other implementations can match MRI's behavior. I have to strongly agree. We've recently been bitten by this, which I'll outline using OpenSSL as an example.

Ruby's OpenSSL library does not strictly enforce a ciphers's initialization vector (IV) size specification. It simply truncates to the appropriate size.

Ola's JRuby OpenSSL library (which is awesome btw!) leverages the Bouncy Castle's JCE implementation which strictly enforces IV specifications.

Below is a simple example that works fine on MRI, and fails on JRuby.

require 'openssl'
text = "abc123"
cipher = OpenSSL::Cipher::Cipher.new("des-ecb")

#The DES Cipher specifies a length of 8 bytes, the below IV exceeds that
key, iv = "1234567890", "1234567890"

cipher.encrypt
cipher.key, cipher.iv  =  key, iv
encrypted = cipher.update(text) << cipher.final

cipher.reset
cipher.decrypt

#I double the length of the cipher here, just to prove that MRI truncates it.
cipher.key, cipher.iv  = key*2, iv*2
decrypted = cipher.update(encrypted) << cipher.final
puts "#{text} == #{decrypted}"


> jruby test_openssl_iv.rb 
java.security.InvalidKeyException: DES key too long - should be 8 bytes
        at org.bouncycastle.jce.provider.JCEBlockCipher.engineInit(Unknown Source)
        at org.bouncycastle.jce.provider.JCEBlockCipher.engineInit(Unknown Source)
        at javax.crypto.Cipher.init(DashoA12275)
        at javax.crypto.Cipher.init(DashoA12275)
        at org.jruby.ext.openssl.Cipher.doInitialize(Cipher.java:416)
        at org.jruby.ext.openssl.Cipher.update(Cipher.java:432)

> ruby test_openssl_iv.rb 
Result is: abc123 == abc123

Yes, this is obviously an edge case, and one could argue that the Java implementation is more correct than MRI's, but the goal is any piece of code should run the same on either platform.

Charles Nutter (Headius) has started a RubySpec Wiki where I've noted this issue there.

I've submitted a bug against JRuby to track the issue, but arguably its not a JRuby bug. Its a difference between BouncyCastle and SSLeay's implementation. I'd be interested in some community feedback on how to correctly implement this. For now, we've worked around this by simply truncating if the RUBY_PLATFORM =~ /java/, which I know if horrible, but it allows me to move on.

Ruby-esque JMX -- Part 2

noreply@blogger.com (Unknown) — Tue, 29 May 2007 03:29:00 +0000

Below is the code from the JMX tinkering. I snake-ized the keys to be more Ruby-esque, per Ed's suggestion.

require 'java'
include Java

# Obviously stolen from Rails ActiveSupport for underscoring strings
class String
  def underscore
    self.to_s.gsub(/::/, '/').
    gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
    gsub(/([a-z\d])([A-Z])/,'\1_\2').
    tr("-", "_").
    downcase
  end
end

module JMX
  class MBean
    include_class 'javax.management.ObjectName'
    include_package "java.lang.management"
    include_package "javax.management"
    include_package "javax.management.remote"
    include_class "java.util.HashMap"
    attr_accessor :name
    def initialize(object_name)
      @object_name = object_name 
      @name = @object_name.to_s
    end

    def attributes
      attrs =  MBean.connection.getMBeanInfo(@object_name).attributes rescue []
      attrs.inject({}) do |list, a|
        list[a.name.underscore] = MBean.connection.getAttribute(@object_name, "#{a.name}") rescue "Unknown"
        list
      end
    end

    def self.find_all_by_name(name)
      object_name = ObjectName.new(name)
      beans = MBean.connection.queryMBeans(object_name,nil )
      beans.collect {|bean| MBean.new(bean.get_object_name)}
    end

    def self.find_by_name(name)
      #obviously inefficient
      find_all_by_name(name).first
    end

    def self.connection
      @@mbsc ||= begin
        #load from some config file later maybe
        url = "service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi"
        connector = JMXConnectorFactory::connect(JMXServiceURL.new(url), HashMap.new)
        connector.getMBeanServerConnection
      end
    end
    protected

    def method_missing(method, *args, &block)
      attributes.keys.include?(method.to_s) ? attributes[method.to_s] : super
    end
  end
end

#Find all the MBeans matching some object name
mbeans = JMX::MBean.find_all_by_name("java.lang:*")
puts "Found #{mbeans.size} beans"

#Find a bean, and get its attribute
bean = JMX::MBean.find_by_name("java.lang:type=ClassLoading")
puts "There are #{bean.loaded_class_count} classes loaded in that VM"

Ruby-esque JMX

noreply@blogger.com (Unknown) — Fri, 25 May 2007 21:06:00 +0000

The topic of JMX on JRuby came up recently and I decided to play around. I found a great starter on Jeff Mesnil's blog, but I decided I hated the syntax.

Ruby has spoiled me. ActiveRecord has spoiled me.

So I cooked up this little (fully working) example:

#Find all the MBeans matching some object name
mbeans = JMX::MBean.find_all_by_name("cacheStatistics:*")

mbeans.each do |bean|
puts "#{bean.name} "

 #Either use methods on the bean object
 puts "  - CacheHits: #{bean.CacheHits}"

 #Or access the attributes hash.
 puts "  - CacheMisses: #{bean.attributes["CacheHits"]}"
end

The code is ~50lines which I'll post at some point.

I never thought working with java objects could be made to "feel" nice.

I was also chatting with "headius" on #jruby, and he mentioned that Rob Harrop, of Spring fame, had a talk at JavaOne about about something similar called MScript. I'd love to get my hands on those slides.

Acts As Fast But Very Inaccurate Counter

noreply@blogger.com (Unknown) — Fri, 25 May 2007 15:54:00 +0000

Introduction

If you have chosen the InnoDB MySQL engine over MyISAM for its support of transactions, foreign keys and other niceties, you might be aware of its limitations, like much slower count(*). Our DBAs are in a constant lookout for slow queries in production and the ways to keep DBs happy so they recommended that we should try to fix count(). They suggested to check SHOW TABLE STATUS for an approximate count of rows in a table. This morning I wrote acts_as_fast_counter which proved that the speed is indeed improved but the accuracy might be not acceptable. The rest of the post just records details of the exercise.

The approach

I created a model per engine and seeded each with 100K records. Then I run count on each model for a thousand times and measured the results.

The code:

module ActiveRecord; module Acts; end; end 

module ActiveRecord::Acts::ActsAsFastCounter
  
  def self.included(base)
    base.extend(ClassMethods)
  end
  
  module ClassMethods
    
    def acts_as_fast_counter
      self.extend(FastCounterOverrides)
    end
    
    module FastCounterOverrides
      
      def count(*args)
        if args.empty?
          connection.select_one("SHOW TABLE STATUS LIKE '#{ table_name }'")['Rows'].to_i
        else
          super(*args)
        end
      end

    end
    
  end
  
end

ActiveRecord::Base.send(:include, ActiveRecord::Acts::ActsAsFastCounter)

# create_table :myisams, :options => 'engine=MyISAM'  do |t|
#   t.column :name, :string
# end
# 100_000.times { Myisam.create(:name => Time.now.to_s) }
# 
# create_table :innodbs, :options => 'engine=InnoDB' do |t|
#   t.column :name, :string
# end
# 100_000.times { Innodb.create(:name => Time.now.to_s) }

class Bench
  
  require 'benchmark'
  require 'acts_as_fast_counter'

  def self.run
    measure
    show_count
    convert_to_fast_counter
    show_count
    add_records
    show_count
    destroy_records
    show_count
    measure
  end

  def self.measure
    puts "* Benchhmarks:"
    n = 1_000
    Benchmark.bm(12) do |x|
      x.report('MyISAM') { n.times { Myisam.count } }
      x.report('InnoDB') { n.times { Innodb.count } }
    end
  end
  
  def self.convert_to_fast_counter
    Innodb.send(:acts_as_fast_counter)
    puts "* Converted Innodb to fast counter"
  end
  
  def self.add_records
    @myisam = Myisam.create(:name => 'One more')
    @innodb = Innodb.create(:name => 'One more')
    puts "* Added records"
  end
    
  def self.destroy_records
    @myisam.destroy
    @innodb.destroy
    puts "* Destroyed records"
  end
  
  def self.show_count
    puts "* Record count:"
    puts "    MyISAM: #{ Myisam.count }"
    puts "    InnoDB: #{ Innodb.count }"
  end
  
end

The results:

* Benchhmarks:
                  user     system      total        real
MyISAM        0.180000   0.040000   0.220000 (  0.289983)
InnoDB        0.430000   0.070000   0.500000 ( 35.102496)
* Record count:
    MyISAM: 100000
    InnoDB: 100000
* Converted Innodb to fast counter
* Record count:
    MyISAM: 100000
    InnoDB: 100345
* Added records
* Record count:
    MyISAM: 100001
    InnoDB: 100345
* Destroyed records
* Record count:
    MyISAM: 100000
    InnoDB: 100345
* Benchhmarks:
                  user     system      total        real
MyISAM        0.250000   0.030000   0.280000 (  0.350673)
InnoDB        0.250000   0.040000   0.290000 (  0.977711)

Final thoughts

The MySQL manual has a clear warning about inaccuracy of the amount of rows in the SHOW TABLE STATUS results:

Rows - The number of rows. Some storage engines, such as MyISAM, store the exact count. For other storage engines, such as InnoDB, this value is an approximation, and may vary from the actual value by as much as 40 to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate count.

The test confirms it by showing 345 more records then expected thus making it not very useful but for some edge cases. If you know a way to improve the speed of count() on InnoDB with some other approach beyond using a counter table, please share.

Javascript: Event.onElementReady

noreply@blogger.com (Unknown) — Thu, 24 May 2007 21:26:00 +0000

We deal with a lot of Javascript at RHG. Rather then create functional one-offs that become hard to maintain and very duplicative, we prefer to have collections of behavior that can be assigned to parts of the document. Our CSS designers especially like this because it makes our markup clean and easy to read. It also means we can have tests that exercise a series of interactions.

However, there is a problem we run into when loading the page over a slow connection: the Javascript is not run until window.onload, meaning our users suffer as they can see part of the page rendered but not use it until it's fully loaded. For those who don't know, window.onload will not execute until all images, CSS, and Javascript files have loaded. Our first solution to this problem was to use onDOMReady. This has worked fairly well and kept our site running reasonably quickly. We've run into problems with onDOMReady in IE6 however, and as a result disabled it in favor of having our pages render all the time. In the IE family of browsers if a script tries to access a part of the DOM before it has been completely processed the browser will raise an exception "operation aborted". After this alert message pops up, the web page becomes unusable and the browser could even crash.

After some careful thinking we decided we would attach our behavior objects in smaller chunks of the DOM. Rather then wait until the window.onload or onDomReady events fire we can use a few inline script tags that call a function that figures out how to attach itself to its most immediate parentNode.

For example, we do this:


<div class="parent">
   <div class="bvr-blah">
   </div>
   <script type="text/javascript">
     RHG.Behavior.attach();
   </script>
</div>

There could be multiple behaviors in the above block of markup that are all contained within the hypothetical div.parent element.

Now, to allow this to work the div.parent must be ready. Otherwise, IE will cancel the whole page rendering with the "Operation Aborted" alert.

Event.onElementReady checks for features of the DOM element to determine whether or not the element is really ready to be manipulatd. Thankfully, IE doesn't mind if you read from an element before it's ready only if you try and modify it. This method will poll the DOM element until either the nextSibling or the textContent is non null.


Object.extend(Event,{
 // check whether or not the DOM element is ready
 onElementReady: function(element,callback)
 {
   if( element && (element.nextSibling || element.textContent) ){
     callback();
   }
   else{
     setTimeout( this.onElementReady.bind(this,element,callback), 1 );
   }
 }
});

Now we can attach our behaviors as the page is rendering and avoid "Operation Aborted" alerts from IE.

DRYing Up Polymorphic Controllers

noreply@blogger.com (Unknown) — Wed, 23 May 2007 01:33:00 +0000

Polymorphic routes allow drying up the controller implementation when functionality is identical, regardless of entry point. A good example is comments for articles and blogs. There is a challenge to balance the implementation of the comments controller reflecting the multiple incoming routes. Let's look at the way it could be written.

Routing is straightforward with blogs and article models acting as commentable and both the comment model and comment controllers being polymorphic:

ActionController::Routing::Routes.draw do |map|
  map.resources :articles, :has_many => [ :comments ]
  map.resources :blogs, :has_many => [ :comments ]
end

This means that a comment can be created via post to either /articles/1/comments/new or /blogs/1/comments/new. The comments controller can be implemented to handle both:

class CommentsController < ApplicationController
  
  def new
    @parent = parent_object
    @comment = Comment.new
  end

  def create

    @parent = parent_object
    @comment = @parent.comments.build(params[:comment])

    if @comment.valid? and @comment.save
      redirect_to parent_url(@parent)
    else
      render :action => 'new'
    end

  end

private

  def parent_object
    case
      when params[:article_id] then Article.find_by_id(params[:article_id])
      when params[:news_id] then News.find_by_id(params[:news_id])
    end    
  end  

  def parent_url(parent)
    case
      when params[:article_id] then article_url(parent)
      when params[:news_id] then news_url(parent)
    end    
  end
    
end

This method works fine and there is not much drive to start refactoring it right away. This changes, though, if there is a need to add another commentable or allow some other polymorphic route. Instead of adding more 'when' clauses the whole functionality can be extracted and abstracted based on the idea of having fixed naming conventions for resources that allow movement from a controller name to a model. The refactored example has the parent functionality extracted to the application controller to share it as-is with other polymorphic routes:

class ApplicationController < ActionController::Base
  
protected

  class << self
  
    attr_reader :parents
  
    def parent_resources(*parents)
      @parents = parents
    end
  
  end

  def parent_id(parent)
    request.path_parameters["#{ parent }_id"]
  end

  def parent_type
    self.class.parents.detect { |parent| parent_id(parent) }
  end
  
  def parent_class
    parent_type && parent_type.to_s.classify.constantize
  end
   
  def parent_object
    parent_class && parent_class.find_by_id(parent_id(parent_type))
  end
  
end

class CommentsController < ApplicationController
  
  parent_resources :article, :blogs
  
  def new
    @parent = parent_object
    @comment = Comment.new
  end

  def create

    @parent = parent_object
    @comment = @parent.comments.build(params[:comment])

    if @comment.valid? and @comment.save
      redirect_to send("#{ parent_type }_url", @parent)
    else
      render :action => 'new'
    end

  end
    
end

The parent_resources call declares resources that are parent for a current controller. An alternative approach is to guess such parent resources from the request URI and routes. Aaron is currently working on a patch on Edge implementing it. We'll update this post later.

If you currently use multiple polymorphic resources and have if clauses in the controller code, you might want to rethink how it could be DRYed up using this approach. In some cases views are very parent type specific. Then it might be better to have different templates and partials rendered via render :template => "/controller/#{ parent_type }_action".

Capistrano Off the Beaten Path

noreply@blogger.com (Unknown) — Mon, 14 May 2007 16:10:00 +0000

Introduction

If you use Capistrano, most likely you use it to deploy rails applications by running it from the project directory. The plugem management tool piggy-backs on Capistrano to execute some recipes without using the current path because recipes operate on gems. Recipes do not, however, completely ignore the current directory, but instead use it for optional customization. Customization is not limited to a deployment recipe from the current directory but can be environment-specific across multiple sites if they each have a special extension gem installed.

Background

We use a bunch of ruby scripts at RHG for deployment and delivery of our numerous applications and shared component. When we started preparing some of that functionality for a public release, we needed to provide a way to customize the scripts. Warren Konkel suggested to migrate them to Capistrano since it was the most familiar tool for rails developers and it has a recipes hierarchy that can be used for customization. So, we wrote the plugem command tool that feeds its parameters to Capistrano via Capistrano::CLI.new(plugem_converted_arguments).execute!
after loading its own recipes.

The Code

    1 module Capistrano
    2   class Configuration
    3
    4     alias :standard_cap_load :load
    5
    6     def load(*args, &block)
    7
    8       standard_cap_load(*args, &block)
    9      
   10       if args == ["standard"]
   11
   12         load_plugem_deploy_recipes(File.dirname(__FILE__) + '/../..')
   13
   14         begin
   15           require 'plugems_deploy_ext'
   16           load_plugem_deploy_recipes(PLUGEMS_DEPLOY_EXT_DIR) # Overriding from extensions
   17         rescue Exception
   18           # No extension is loaded
   19         end
   20
   21       end
   22
   23     end
   24    
   25     def load_plugem_deploy_recipes(dir)
   26       Dir[File.join(dir, 'recipes', '*')].each { |f| standard_cap_load(f) }
   27     end
   28      
   29   end
   30 end

Ignoring lines 14-19 for now, all it does is inject the plugem deployment gem recipes to Capistrano. They are now available for execution by Capistrano. So when I call plugem update my_app it is being translated to cap plugem_update -s plugem_name=my_app(we use the plugem_ namespace for tasks and variables to avoid clashing with standard recipes). Since loading of plugem recipes is purposely not handled via the -f flag of capistrano, the standard deployment recipes like config/deploy.rb are still being loaded.

Customization

The package is extending Capistrano but we wanted it to be customized too. For example, you might want to define your own list of gem servers to download gems from. Capistrano allows to define recipes per project, user, or host. The plugems_deploy takes it a step further and provides a way to customize it per deployment environment, which might contain many hosts. It does this by expecting an optional plugems_deploy_ext gem to be installed on a system (lines 14-19). If it finds the extension gem, it loads recipes from there overriding the default ones.

Conclusion

Capistrano is not only a great deployment tool, it can be used as a base of highly-customizable general purpose tools.

Plugin I Cannot Live Without

noreply@blogger.com (Unknown) — Fri, 11 May 2007 18:14:00 +0000

The Enhanced Rails Migrations plugin was written to end the constant battle we had with clashing names in db migrations within our large development team. We tried everything: special commit policies, rake tasks, even claiming the next migration number in subversion. Nothing worked and CI server was sending 'broken build due to conflicting migration number' messages almost daily. Since the plugin was introduced to all our rails applications around six months ago, I have not heard of a single case of conflicting migrations. Seemingly, the goal was well accomplished.

What I found over time, is that the plugin is not only useful for large projects. Any rails development effort with more than one programmer involved benefits from using it. If you ever had to renumber your new migration after doing svn up you know what I am talking about. It makes sense to install this plugin as the very first one in your project since an amount of migrations at the beginning tends to grow much faster then later in the game.

The plugin works for rails versions 1.1.6 up to the latest edge. When you start your next project with multiple developers, use it and you should be able to forget that you ever had problems with clashing migrations.

Moving models to a different database

noreply@blogger.com (Unknown) — Wed, 09 May 2007 14:26:00 +0000

There many reasons to use multiple databases (DBs) and when this is done, there is often a case when a model needs to be moved from one DB to another. The impetus could be that part of the data is referential and this is being reflected by moving it to a read-only DB. Another possibility is that we want to protect some data with an additional layer of security, so we extract it to a secure DB. In all cases, the challenge is to migrate the existing data. When the amount of data is considerably large, there is no choice but to do it via SQL data loaders or similar techniques. On the other hand, if it is acceptable to leverage rails db migrations and you prefer to do any data manipulation through them, there are some challenges to face.

Often, you need to have an access to old and new models during data migration. One solution is to move or copy an existing model to a separate namespace and put the new model instead at the old namespace. Let's look at a couple of examples:

Extracting referential data

I have a model Fruit in our main DB which gets its data from an external source, so we only access it read-only. We want to enforce it by moving the data to a DB which we access with a read-only account. First, I create a referential_db entry in database.yml:

dbs:
  database: main

  referential_db:
    database: referential

Then, I copy the original model, Fruit, to a dedicated namespace, so the model becomes RetiredModels::Fruit. I add establish_connection to the original namespace model:

# create_table :fruits do |t|
#   t.column :name, :string
# end
class Fruit < ActiveRecord::Base
  establish_connection configurations[RAILS_ENV]['referential_db']  
end

Everything is set for migration. Since it is a referential data, the migration needs to preserve data integrity so the models belonging to Fruit can still reference it by an old id:

def self.up
  RetiredModels::Fruit.find(:all).each do |old_record|
    Fruit.new(old_record.attributes) { |new_record| new_record.id = old_record.id }.save!
  end
end

After the successful migration run, all data is replicated to a new DB. The retired model can be removed during next deployment, and the original table dropped.

There is one caveat for development and test modes. If you don't want to bother with multiple databases in those modes, you need to take care of having no table name clashing. So, the new model would have to use different table names via set_table_name.

Securing sensitive data

One of the models belonging to Fruit is SecretFruit. It contains a secret name for every fruit out there. Our legal department asked the development team to protect that data in case our DB is stolen. We decided to migrate the existing SecretFruit data to a protected DB and keep sensitive data encrypted with help from Acts As Secure. First, I create a secure_db entry in database.yml:

dbs:
  database: main

  secure_db:
    database: secure
    host: protected_host

Then, I copy the original model, SecretFruit, to a dedicated namespace, so the model becomes RetiredModels::SecretFruit. I modify the model in the original namespace to reflect the new requirements:

# create_table :secret_fruits do |t|
#   t.column :name, :binary
#   t.column :fruit_id, :integer
# end
class SecretFruit < ActiveRecord::Base
  establish_connection configurations[RAILS_ENV]['secure_db']
  acts_as_secure :crypto_provider => MasterKeyProvider
  belongs_to :fruit
end

Since data encryption is done on-the-fly and there are no data integrity requirements, the migration is straightforward:

def self.up
  RetiredModels::SecretFruit.find(:all).each { |old| SecretFruit.create!(old.attributes) }
end

I can now safely delete RetiredModels::SecretFruit and associated data.

DRYing Models via Acts As

noreply@blogger.com (Unknown) — Tue, 08 May 2007 04:44:00 +0000

ActsAs is an idiom familiar to every Rails developer, which makes it a good candidate for a shared functionality between models. Using it as early in the game as possible allows one to work on its functionality without a need to touch the code in multiple models. Let's look at a couple of examples.

Acts As Unique

I have some models that I want to have uniqueness across my application. I use some UUID mechanism (initially, a db call) to set a field (:token) after creation. Since I have multiple models, I decide to extract it the code for uniqueness setting to acts_as_unique. After refactoring, my model Fruit looks like:

# create_table :fruits do |t|
#   t.column :name, :string
#   t.column :token, :string
# end
class Fruit < ActiveRecord::Base
  acts_as_unique
end

My acts_as_unique might look like:

module ActiveRecord; module Acts; end; end
module ActiveRecord::Acts::ActsAsUnique
  
  def self.included(base)
    base.extend(ClassMethods)
  end
  
  module ClassMethods
    def acts_as_unique(field = :token)
      validates_uniqueness_of field 
      before_validation_on_create do |o|
        o.send("#{ field }=", connection.select_one('SELECT UUID() AS UUID', "#{name} UUID generated")['UUID'])
      end
    end    
  end
end

ActiveRecord::Base.send(:include, ActiveRecord::Acts::ActsAsUnique)

Let's try it:
>> f = Fruit.create(:name => 'apple')
>> p f.token
"0a4d7c46-4df0-102a-a4b9-59b995bffdb7"

Now I can work on acts_as_unique to replace the DB call with a UUID gem or some other implementation without affecting the rest of the code.

Acts As Trackable

I have some models for which I want to keep track of when instances are created or updated. I have a polymorphic Event model for storage of such events. Since there are multiple models I want to track, I extract the functionality to acts_as_trackable. After refactoring, my models look like:

# create_table :fruits do |t|
#   t.column :name, :string
# end
class Fruit < ActiveRecord::Base
  acts_as_trackable
end

# create_table :events do |t|
#   t.column "action",              :string
#   t.column "created_at",          :datetime, :null => false
#   t.column "trackable_type",      :string
#   t.column "trackable_id",        :integer
# end
class Event < ActiveRecord::Base 
  belongs_to :trackable, :polymorphic => true
end

module ActiveRecord; module Acts; end; end 
module ActiveRecord::Acts::ActsTrackable
  
  def self.included(base)
    base.extend(ClassMethods)
  end
  
  module ClassMethods
    def acts_as_trackable
      has_many :events, :as => :trackable, :dependent => :destroy
      after_update { |o| o.events.create(:action => 'updated') }
      after_create { |o| o.events.create(:action => 'created') }
    end
  end
  
end

ActiveRecord::Base.send(:include, ActiveRecord::Acts::ActsTrackable)

Let's see what we got:
>> f = Fruit.create(:name => 'apple')
>> p f.events.collect(&:action)
["created"]
>> f.name = 'passionfruit'
>> f.save!
>> p f.events.collect(&:action)
["created", "updated"]

The Event model is likely to evolve but it would be easier to support it since the only place where I need to reflect the changes is acts_as_trackable. The goal is achieved.

CSS: CSS Browser Selector

noreply@blogger.com (Unknown) — Wed, 02 May 2007 20:52:00 +0000

Each browser has its quirks. Having a clear way of organizing the work arounds
for those quirks is a challenge. Using CSS selectors is not a new idea, but I thought it might be helpful to others to give an example of the technique and how we've been able to successfully deploy it for revolutionhealth.com.

First, for IE (the browser that usually requires a hack), we can rely on conditional comments. This is good because it means we don't need to depend on Javascript. For other browsers we'll have to rely on a document.write solution. For Safari, Opera, and Firefox, we rely on the script from http://rafael.adm.br/css_browser_selector/ and for IE, conditional comments.

Here's what we include at the top of our document. (The browser_detect_start partial.)


<!--[if lt IE 7.]>
<div class='ie ie6'>
<![endif]-->
<!--[if IE 7]>
<div class='ie ie7'>
<![endif]-->
<script type="text/javascript">//<![CDATA[
 var d = browserCSSDetection();
 if( d.browser != "ie" ){ document.write( "<div class='" + d.browser + " " + d.os + "'>" ); }
//]]></script>

And here's what we do for the end of the document. (The browser_detect_end partial.)


<!--[if IE ]>
</div>
<![endif]-->
<script type="text/javascript">//<![CDATA[
 var d = browserCSSDetection();
 if( d.browser != "ie" ){ document.write( "</div>" ); }
//]]></script>

The browser detection in Javascript. This could be enhanced further, but for us this allowed us to get the site working relatively easy in Konqueror. As well it enabled us to fix our menu's so that they float over flash in Linux using this techinque.


function browserCSSDetection()
{
 // see: http://rafael.adm.br/css_browser_selector/
 var ua = navigator.userAgent.toLowerCase();
 var is = function(t){ return ua.indexOf(t) != -1; };
 var b = (!(/opera|webtv/i.test(ua))&&/msie (\d)/.test(ua)) ?
             ('ie ie'+RegExp.$1) :
               is('gecko/') ? 'gecko' :
               is('opera/9') ? 'opera opera9' :
               /opera (\d)/.test(ua) ? 'opera opera'+RegExp.$1 :
               is('konqueror')?'konqueror' :
               is('applewebkit/') ? 'webkit safari':
               is('mozilla/')?'gecko':'';
 // see: http://www.mozilla.org/docs/web-developer/sniffer/browser_type.html
 var os = (is('x11')||is('linux'))?' linux':is('mac')?' mac':is('win')?' win':'';
 var css = {browser:b,os:os};
 return css;
}

Finally, to make all this fit nicely into a layout:


<head>
<%= javascript_include_tag 'browser_detect' %>
</head>
<body>
<%= render :partial => "browser_detect_start" %>
<%= @content_for_layout %>
<%= render :partial => "browser_detect_end" %>
</body>

Update:
Here's a simple example of what this enables. The advantage of this over a conditionally included file is it keeps everything about the login box isolated to one place. You don't need to worry about openning up a separate file to make your IE fixes. We do use iefix specific CSS for our site, but only for very large features like menus


#login {
   padding:0px;
}
.ie6 #login {
   padding: 2px;
}

Plugems - Why views in plugems?

noreply@blogger.com (Unknown) — Wed, 02 May 2007 13:36:00 +0000

When we started to develop Plugems we had multiple applications that all shared a similar look and feel. One concept that emerged recently for me was that layouts can act like objects. In our suite of applications, we have a base layout that each application can derive from to satisfy it's unique set of presentation requirements. For example our UI plugem has the following:


 ui/views/layouts/default.rhtml

Using a little magic from fora.pragprog we can extend our default layout writing the following:


<% inside_layout 'default' do -%>
     <%= @content_for_layout %>
  <% end -%>

As well as with layouts, there are also smaller blocks of UI that make up a page that can be shared. I've always thought of these as widgets, which in many cases can be defined as a set of partials and helpers. An application may place a widget on a view, feeding it the data needed to bring it to life. This is different from components because there is no controller logic. Really the presentation side of a widget can be expressed by a set of partials, Javascript, CSS and a healthy dose of ruby [e.g. helpers]. In this way partials are the basic building block on which many UI components can be built. By themselves they are incomplete, but within the context of an application they are alive...

Plugems enables you to share common UI widgets across multiple applications. Common assets (css, js, images) can live in a plugem, but be shared across applications at deployment time. I hope to post more in depth on this topic in near future.

Captcha on Rails

noreply@blogger.com (Unknown) — Sat, 28 Apr 2007 13:49:00 +0000

Thinking about how Captcha handling could be implemented in Rails is a good mental exercise. Leaving aside the issue with entry barriers it creates for some people, and possible underlying technology implementations (like not-so-simple mathematical questions), let's formulate typical Captcha assumptions and requirements for a medium-sized Rails application:

Image based
Image/code combination could not be reused
On an unsuccessful attempt, a new Captcha image is displayed
Potentially multiple Rails application servers
The attacker potentially knows the details on how you implemented Captcha

Let's pretend that we know how to generate a perfect Captcha image based on a supplied code string. All we need now is to meet the rest of the requirements.

Image/code combination could not be reused, means a need to prevent a human from entering the correct code, capturing a post sent to a server, and then replaying it many times via a robot.

Another point to remember is that a form with a link is generated on the first browser request, an image is retrieved with another request, and the Captcha validation goes with the third one. Potentially all three might arrive to different application servers.

Now, if you like challenges, try to think about possible solutions. The rest of this post talks about a few I have come up with by myself, though some might have been influenced by brainstorming sessions with the Revolution On Rails team.

In many of the solutions, a shared storage is used to overcome the multiple servers limitation. It could be a DB (ActiveRecord), memcached (Session), or some other solution. There is a clean up issue for cases when users abandon a Captcha form without entering a code. For some storage types (like memcached) it is taken care of by the engine itself, while for others a periodic cleaning job is required.

Unbreakable

Image/Code Generation on Request #1

Request #1. The application generates a code and an image and puts them into a Storage. A unique Captcha id (for example a captcha record id from ActiveRecord) is used in an image URL as well as in a hidden form field.
Request #2. Based on the Captcha id, the image is retrieved and deleted from the Storage. The image is sent to a browser via send_file.
Request #3. The entered code is compared with code retrieved from the Storage, based on the captcha id. Regardless of the result, the record is deleted from the Storage.

Code Generation on Request #1, Image Generation on Request #2

Request #1. The application generates a code and puts it into Storage. A unique captcha id is used in an image URL as well as in a hidden form field.
Request #2. Based on the Captcha id, the code is retrieved from Storage and used for generation of an image. The image is being displayed via send_file.
Request #3. The entered code is compared with one retrieved from the Storage based on the Captcha id. Regardless of the result, the record is deleted from the Storage.

Image/Code Generation on Request #2

Request #1. A unique captcha id is generated and is used in an image URL as well as in a hidden form field.
Request #2. The application generates a code and puts it into a Storage. An image, based on the code, is generated and is being showed via send_file.
Request #3. The entered code is compared with one retrieved from the Storage based on the captcha id. Regardless of the result, the record is deleted from the Storage.

Image/Code Generation on Request #1 with Code Hash

Request #1. The application generates a code and puts a hash of it in a hidden field form. An image is generated and is put into a Storage. A unique Captcha id is used in an image URL as well as in a hidden form field.
Request #2. Based on the Captcha id, the image is retrieved and deleted from the Storage. The image is sent to a browser via send_file.
Request #3. The entered code is compared with the hashed one. Regardless of the result, the record is deleted from the Storage.

Potentially Breakable

Some of the potentially breakable solutions are perfectly acceptable for a practical usage since the risk could be very low while benefits of doing image generation offline or having no storage involved could be significant.

Pre-Generated Code/Image

There are several scenarios with code/image pairs pre-generated off-line but none that can sustain an attack based on calculation of an image hash, noting the correct code, and calling the form till the same image comes again. The volume of images and frequency of their regeneration defines the level of risk.

No Storage

It is probebly not possible to come up with a generic solution involving no storage of any kind, while meeting all the requirements. There are some special cases though, with very low security risk. For example, a two-step login process with a standard login screen presented first, and the captcha next:

Request #1: The login form is sent.
Request #2: The application generates a code, encrypts it is with login name and password, and puts the encrypted string in a hidden field form. The image url generates using an encrypted code with some salt (timestamp.)
Request #3. The code is retrieved from the requested url. An image based on the decrypted code is generated, and is displayed via send_file.
Request #4. The entered code is compared with the decrypted and the account is then verified if the correct code was entered.

The obvious attack vector is to crack/steal the key, though in practice it is a well-understood problem to address.

Conclusion

Most likely there are papers on Captcha describing all the aforementioned solutions and many more, so this post could be seen as reinventing the wheel. Still, it was fun thinking about them. It would be great to hear about a perfect generic no-storage solution, feel free to share if you have an idea.