STEPHEN CHU . com

Speed up your MySQL data load

2008-12-01T18:22:00.004-06:00

Perhaps you have some large datasets you would like to load into MySQL, such as a big text file generated from mysqldump that contains many sql INSERT statements. You can obviously populate your empty database by redirecting the content of this file into the mysql command line like such:


  $ mysql -u root -p your_database < /tmp/table.dump

Another way to load data into your database or table is to use the sql syntax LOAD DATA INFILE. It requires you to provide a file that is in some delimiter format such as csv or tab-delimited values. One can use the mysqldump command line switches --fields-*-by and --tab to dump the content of the database to such format.

After you started either of these data load, you may want to kick off your shoes and take a nap, cos this will take a while if your MySQL server is not tuned and your dump file is pretty big (in this example, an InnoDB table of 790,000 rows [~500 MB dump] took 45 mins on my MBP using a Macports [mysql5 @5.0.67_1] installation with default settings). If you have to reload your database/tables often, this is unbearably slow.

Fortunately, there are a few MySQL tuning settings that you can specify at mysql server start-up time to tremendously speed up your data load time. Keep in mind, it is advisable to flush your MySQL logs (like the following) prior to tinkering with any log file size settings to avoid any log file corruptions:

(Before you proceed, I recommend you backup your data first)


  $ sudo mysqladmin flush-logs
  $ ... then shutdown your MySQL server ...
  $ sudo rm /path/to/mysql/datadir/ib_logfile*
  $ ... then start your MySQL server ...

So, now it's time to put in our magic sauce. On my Macports MySQL installation, the settings were meagerly defaulted to:


  +---------------------------------+---------+
  | Variable_name                   | Value   |
  +---------------------------------+---------+
  | innodb_additional_mem_pool_size | 1048576 |
  | innodb_buffer_pool_size         | 8388608 |
  | innodb_log_buffer_size          | 1048576 |
  | innodb_log_file_size            | 5242880 |
  | bulk_insert_buffer_size         | 8388608 |
  +---------------------------------+---------+

Add the following settings in your appropriate my.cnf file. Your settings may vary with the kind of machine you have (mine is a MBP 2GB RAM):


  [mysqld]
  innodb_additional_mem_pool_size=256M
  innodb_buffer_pool_size=512M
  innodb_log_buffer_size=256M
  innodb_log_file_size=512M
  bulk_insert_buffer_size=256M

Our settings will now look like:


  +---------------------------------+-----------+
  | Variable_name                   | Value     |
  +---------------------------------+-----------+
  | innodb_additional_mem_pool_size | 268435456 |
  | innodb_buffer_pool_size         | 536870912 |
  | innodb_log_buffer_size          | 268435456 |
  | innodb_log_file_size            | 536870912 |
  | bulk_insert_buffer_size         | 268435456 |
  +---------------------------------+-----------+

Then, as I reload my table, it only takes just 4' 55", a whopping 900% performance boost.

Bonus Tip:

You might notice that you cannot use SELECT COUNT(*) to track your data load progress while LOAD DATA INFILE is being executed (it always returns a count of zero). Instead, you can use the SHOW INNODB STATUS command. Find the line that says "undo log entries" under section TRANSACTIONS, and that number is the number of rows inserted so far. You can also look at how many inserts were performed per second by finding the line "inserts/s" under the ROW OPERATIONS section.

Go to Ruby ! and ? method definition with TextMate CTags

2008-09-26T12:18:00.003-05:00

If you are not yet using the TextMate CTags bundle in your Ruby/Rails development, you are missing out big time. This bundle allows you to go to a method definition as easy as pointing your cursor at the method and hit a keyboard shortcut. It has served me well and I highly recommend it.

But soon you might realize that this "go to definition" shortcut only works most of the time. Specifically, whenever you try to go to the definition of a Ruby method that ends with a bang (!) or question-mark (?), CTags will fail open to the line that defines that method for you.

Here's a fix for you. The perl script that got executed underneath this little utility actually uses a TextMate variable TM_CURRENT_WORD. This variable points at the "word" your cursor is currently laying on top of in TextMate. But since by default, TextMate does not consider ! or ? to be part of a word, your CTags fails on you. All you need to do, is go to Preferences / Text Editing, and change Word Characters to include ! and ?.

Object-Oriented Programming with Rails ActiveRecord

2008-06-22T23:02:00.005-05:00

Have you seen this before?

@posts = Post.find :all, :conditions => { :user_id => @user.id }

This happens to new Rails developers quite a bit. This code retrieves correctly an array of @posts and send them to the view for rendering. But what about it?

The key thing here is that this can also be done in a better way:

@posts = @user.posts

When you use the former method to retrieve @posts, you are actually thinking in database terms along the lines of "in the database table 'posts' and therefore ActiveRecord model 'Post', retrieve the rows whose foreign key column 'user_id' has the value @user.id." ActiveRecord is a pattern for accessing your data from objects, but you also have to combine it with the power of the object-oriented-ness of Ruby to create the chimes of beautiful code. Using Rails and Ruby does not make anyone a better programmer automagically. One can still write code in Java/C# in the same procedural style as if you were writing C. It is how to leverage the best of all worlds makes you a better problem solver.

Therefore, think like an object bigot. Think in objects, and not in database foreign key column values. Whenever you see any Rails code in the pattern of this:

:model_id => @model.id

Just stop. "Objects on Rails" sounds a lot better than "Ruby on Foreign Keys."

Learning encapsulation should come before learning Rails

2008-06-18T01:39:00.006-05:00

Do you see anything wrong in this one line of code?

puts 'Drink up your milk!' if @milk.expiration_date < 2.days.from_now

I think there is. In particular, the programmer can do better. It is about a programming concept that we have all heard and should be familiar with: encapsulation. If you are into testing, which you ought to be, you can probably identify such encapsulation violations by how smelly your tests are, as I believe in Test Driven Design. Going back to the line of code. What's wrong with it?

The pattern of this code is, you have code somewhere that yanks states (expiration_state) out of an object (@milk) and then interrogate against those states (< 2.days.from_now). By no means this is a rule, as exceptions do exist. But, when this happens, your programming siren in your head should go off as if the CI build is broken: can this be a method on the object @milk itself?

Ruby is a powerful programming language. It allows you to do all sorts of fancy programming: dynamically altering classes, add methods to only the selected few object instances, while being duck-typed and object-oriented all at the same time. There are examples galore in Rails itself. But to enable all such magic, all of your ActiveRecord domain models have getters and setters on all of their states (i.e. data). While that is convenient to access their states, sometimes you have to be careful. With great power comes great responsibility, and that responsibility comes down to you.

You might not think this applies to you. But have you ever written code like this in Ruby?

MyMailer.deliver_product_returned_notification if @product.state == State::Returned

total = @line_items.sum { |line_item| line_item.price * line_item.quantity }

if [:admin, :superuser].include?(@user.role)
  ...
end

Now, let's look at these rewritten examples:

puts 'Drink up your milk!' if @milk.expiring?

MyMailer.deliver_product_returned_notification if @product.returned?

total = @line_items.sum(&:subtotal)

if @user.superuser?
  ...
end

Forget about the loops, the blocks, the hype, and everything about Ruby for a sec. Code is code. Not only does the code become more readable, when you try to enrich your domain by naming things correctly, you could also very well be opening up new business concepts that wasn't previously clear or accurate. If you have a domain model, you are modeling it against domain concepts so that your app can interact with. You'd better be sure it is right, or else lots of time will go wasted on code that solves only half of the business problems all the time.

Launching apps in command line (Mac)

2008-06-09T16:03:00.003-05:00

In most cases, to launch an application on a Mac is just a matter of using the awesome QuickSilver keyboard shortcut and type in what you want to open. But, sometimes it is useful to be able to launch/open an application in command line or terminal in a scripting context.

So, instead of going through all the typing $ /Application/MyFavouriteApp/Contents/MacOS/MyFavouriteApp in your terminal, you can do:

$ open -a MyFavouriteApp

This will open any applications in your /Applications folder by name.

Rails composed_of validation

2008-05-21T03:10:00.005-05:00

ActiveRecord allows you to declaratively write validations (e.g. validates_presence_of) in any ActiveRecord::Base models. In addition, the errors will be stored in the @model#errors collection and be used by various view helpers and methods like FormBuilder, error_messages_for, and error_message_on, wrapping a nice error CSS around the offending html form elements. But when you have a normal domain model object that is non-ActiveRecord::Base such as a composed_of type Value Object, you will not have access to these declarative validation magic methods. Today let me try to elaborate on how to use ActiveRecord::Validation methods in your composed_of objects in Rails 2.0.

(The following entry revolves around the assumption that you are using f.fields_for to create and assign value object domain models onto your ActiveRecord::Base models. Using this approach eliminates most value object creation code in your controllers, achieving "Skinny Controllers". To learn about, visit, check out the Rails composed_of &block conversion blog entry.)

Using the same example in the previous entry, let's include our ActiveRecord::Validation module into our Money class like such:

class Money
  include ActiveRecord:Validations
  
  attr_reader :balance, :currency
  
  validates_presence_of  :currency
  validates_inclusion_of :currency, :in => ['USD', 'EUR'], :if => :currency?
  validates_presence_of     :balance
  validates_numericality_of :balance, :if => :balance?
  
  def initialize(balance, currency)
    @balance, @currency = balance, currency
    @errors = ActiveRecord::Errors.new self
  end
  
  def new_record?
    true
  end
  
  def currency?
    !@currency.blank?
  end
  
  def balance?
    !@balance.blank?
  end
  
  def self.human_attribute_name(attr)
    attr.humanize
  end
  
  def balance_before_type_cast
    @balance
  end  
end

You will notice a couple things. One, I have to define the #new_record? method. This method is defined on all ActiveRecord::Base objects, but since our PORO object is not a record per se, we just stub it out. Also, we need to store a collection @errors of type ActiveRecord::Errors.

Depending on what validation routine you will end up using in Money, you may have to stub out different methods. For example, I am showing error messages with error_messages_for (more on this later), and it requires stubbing out self.human_attribute_name (as of Rails 2.0, but no longer needed in future Rails). Using validates_numericality_of requires me to stub out balance_before_type_cast. Also, the validation :if conditions requires me to add the question-mark methods balance? and currency?. Remember, this approach does not give you all the validation magic. For example, validates_uniqueness_of will not work because it assumes too much about your object being a normal AR model and needs access to a database. But in practice, your Value Objects should not need such validations, and in most cases they contain only simple one-off validations, and provide simple functionalities such as formatting like this and this.

After all these, let's see our ActiveRecord Book class.

class Book < ActiveRecord::Bases
  composed_of :balance, :class_name => 'Money', :mapping => [%w(balance balance), %w(currency currency)] do |params|
     Money.new params[:balance], params[:currency]
  end
  validates_presence_of :name
  validates_associated  :balance
end

The composed_of conversion block remains. You will notice the validates_associated :balance line as well. This tells your book instances that they should not be persisted should there be any balance validations failing, just like any normal validations you would write. By default, any failing balance validation will add an error message 'Balance is invalid' in your @book#errors collection. If you want to suppress that message from showing up, you can pass in option :message => nil.

So, to put it all together, here is the view and the controller code:

<h1>New book</h1>

<%= error_messages_for :object => [@book, @book.balance] %>

<% form_for(@book) do |f| %>
  <p>Name: <%= f.text_field :name %></p>

  <% f.fields_for :balance, @book.balance do |ff| %>
  <p>Balance: <%= ff.text_field :balance %></p>
  <p>Currency: <%= ff.text_field :currency %></p>
  <% end %>

  <p>
    <%= f.submit "Create" %>
  </p>
<% end %>

<%= link_to 'Back', books_path %>

To show error messages from multiple objects on the view, I am using the view helper method error_messages_for(*args). The :object option actually allows you to pass an array of objects (c'mon, you should know this trick about ActiveRecord by now. If not, check it out here and here).

def create
  @book = Book.new params[:book]

  if @book.save
    flash[:notice] = 'Book was successfully created.'
    redirect_to @book
  else
    render :action => "new"
  end
end

Again, a skinny, thin, sexy-looking controller action.

A normal Rails un-RESTful page as a resource... or can it?

2008-05-06T14:53:00.006-05:00

An index page comes with Rails scaffolding. It is used to show a list of the "thing" you are CRUD-ing on. However, all too often we are tasked to show some pages that we aren't doing any CRUD operations on. Just a few examples:

Dashboard page after user logs in

The "Forgot your password?" page

Tabs or subtabs of multiple lists

List page showing multiple lists of different entities

A normal dashboard page isn't something you would normally CRUD on. It is more like a place holder page that a user sees the first thing after s/he logs in, showing many items of all sorts valuable to a user. In many Rails app, depending on what those items are, the dashboard page will end up being rendered by one of the items' index action. Worse yet, next time when you actually need that index page by that model, you have to call it something else, maybe 'list', and then play with the routes to get it wired up correctly.

class CustomersController < ApplicationController

  # The dashboard page
  def index
    @customers = Customer.find :all
    @products = Product.find :all
    @tasks_of_the_day = Task.find :all
  end

  # A list of customers
  def list
    @customers = Customer.find :all
  end
  
end

ActionController::Routing::Routes.draw do |map|

  map.resources :customers, :products, :tasks
  map.with_options :controller => 'customer' do |r|
    r.dashboard '/customers/list', :action => 'list'
  end

end

Here's a suggestion: how about put it in views/dashboards/show.html.erb, and, while you are at it, give it a DashboardsController? Then, put it in your routes.rb as a (singular) map.resource :dashboard, like such:

class DashboardsController < ApplicationController

  def show
    @customers = Customer.find :all
    @products = Product.find :all
    @tasks_of_the_day = Task.find :all
  end
  
end

ActionController::Routing::Routes.draw do |map|

  map.resources :customers, :products, tasks  
  map.resource :dashboard

end

By rendering the dashboard page in a completely different controller, you now have a very readable GET dashboard_path named route (GET: http://localhost:3000/dashboards), and you will not contaminate the index action of your other models' controllers with instance variables of all kinds. You also have a more readable routes.rb file.

One of the examples above for non-RESTful pages is the "Forgot My Password" page. Can you think of a good way to do it the Rails REST-ful way? Please go to Seeing Rails Resources Clearly to share some of your thoughts.

Pragmatic Rake tasks organization

2008-04-05T13:43:00.007-05:00

Do you have a lot of rake tasks on your Rails projects? I am sure you do. Rake tasks are handy, and developing your next-big-idea Rails app is a lot less attractive without it. Most of the time you drop a new foo.rake file in your lib/tasks folder, and you write code in it and get out. Over the course of development, that folder is likely to look like your favorite Halloween party house (ahem, Neal's...), except on an every day basis.

If you want your rake tasks to be more manageable, here are a few tips:

1) Namespace your rake tasks.

Namespaces in rake is nice. They make sure all tasks in your namespace are cohesive as a area of work. Still, your job is to keep your rake tasks not too crowded in a single namespace. For example, the db namespace that Rails provides very easily become crowded with tasks of all sorts, because, well, on a Rails application what you do almost always is related to your database: drop/create/migrate your database, importing reference/application data, database integration points with other applications, db related background tasks, backups... Phew! How about create a new namespace for import, integration, background, and backup for each of them?

2) Name your rake files by their namespaces.

Even if you namespace all your rake tasks, still, you are likely to end up with a bunch of .rake files in lib/tasks. But, now that you have your tasks grouped by their meaningful and not-so-crowded namespaces, you should name your files by your namespace. For example, db.rake, data.rake, import.rake, integration.rake, and backup.rake. Can you guess where that rake db:awesome is implemented? You certainly don't know what it does, but you got that right, in lib/tasks/db.rake.

3) Try not to nest more than 2 namespaces.

This tip is not a rule, the point being don't over do it. Sometimes, that third-level namespace really makes sense, For example, db:create:all from Rails creates databases for all environments. It isn't for everything, though. For example, I prefer rake data:load better than rake db:data:load. Not only is it shorter, but also the namespace db is simply implied. Short nested-ness forces you to name your namespaces more descriptively, and thus easier to navigate to their implementation.

4) Move your rake file and its external ruby files into a sub-folder, named by that namespace.

When your rake tasks are complicated enough, you obviously will use objects to do the job for you. When that happens, instead of having lib/tasks/integration.rake, you will instead want to have a sub-folder lib/tasks/integration/, and put files like integration.rake and integration_awesome_klass.rb in it. There really is no need to contaminate your lib folder with rake specific files.

Shameless Plug: Still using NAnt? Check out how to organize your NAnt scripts as well, on this blog post.

params[:fu] #5 ) Update multiple models in update action atomically.

2008-03-27T22:25:00.003-05:00

Updating multiple models is hard? It sounds complicated, but with Rails it actually isn't, if you know how to take advantage of it. Knowing what you know about Rails params, let's take a look at today's topic: the update action.

<% form_for @reader do |f| %>
  <%= f.text_field :name %>
  
  <% for @subscription in @subscriptions %>
    <p>
    <% fields_for @subscription do |ff| %>
      <%= ff.collection_select :magazine_id, Magazine.find(:all), :id, :name, {}, :index => @subscription.id %>
    <% end %>
    </p>
  <% end %>

  <%= f.submit 'Save' %>
<% end %>

generates:

<p>
  <select id="subscription_4_magazine_id" name="subscription[4][magazine_id]">
    <option value="101">PC Magazine</option> 
    <option value="102">IT Pro</option> 
    <option value="103" selected="selected">WIRED</option>
  </select>
</p>

<p>     
  <select id="subscription_5_magazine_id" name="subscription[5][magazine_id]">
    <option value="101">PC Magazine</option> 
    <option value="102">IT Pro</option> 
    <option value="103" selected="selected">WIRED</option>
  </select>
</p> 
  
... (and more)

Processing ReadersController#update (for 127.0.0.1 at 2008-01-14 21:12:56) [PUT]
Parameters: { "commit"             => "Update",
              "reader"             => { "name" => "stephen chu" },
              "subscriptions"      => { "4" => { "magazine_id" => "101" },
                                        "6" => { "magazine_id" => "102" }, 
                                        "7" => { "magazine_id" => "103" } },
              "authenticity_token" => "238ba79b8282882ba01d840352616c2cc79280f0",
              "action"             => "create",
              "controller"         => "readers" }

See the pattern? The POST-ed parameters are of the same structure as in one of our last example, hash of hashes. The sub-hashes are keying off of the subscription id, because this time around we are updating existing subscriptions. So last time we used params[:subscriptions].values, what would it look like this time? Let take a look.

def update
  @reader = Reader.find params[:id]
  @reader.attributes = params[:reader]
  @reader.subscriptions.update params[:subscription].keys, params[:subscription].values
  
  if @reader.save
    flash[:notice] = 'Reader was successfully updated.'
    redirect_to @reader
  else
    flash[:notice] = 'Failed.'
  end
end

Again, another ActiveRecord model method utilizes the array of hashes pattern! The update method source code on RDoc looks like it is just updating one at a time. But a peek at the source code says otherwise:

def update(id, attributes)
  if id.is_a?(Array)
    idx = -1
    id.collect { |id| idx += 1; update(id, attributes[idx]) }
  else
    object = find(id)
    object.update_attributes(attributes)
    object
  end
end

Again, the update method recognizes array! So, where to get the arrays that we will use in our controller action? They come from .keys and .values of course:

params[:subscriptions].keys    # => [ "4", "6", "7" ]
params[:subscriptions].values  # => [ { "magazine_id" => "101" }, { "magazine_id" => "102" }, ... ]

So in essence, our controller code is free from all those ugly params-munging activities. Remember, controller actions should not shuffle around their params, or otherwise it fails to abide the "Skinny Controller, Fat Model" principle, and they will stink.

Now, if you are thinking about by using update, we run the risk of not atomically saving all of our models should any of our models fail validation, you are correct. This is where rescue_from in controller saves the day. Just transact our update action using AR transaction, and re-render the edit page should it catches ActiveRecord::RecordInvalid error, you should be able to make your update action atomic. Given how lean our controller action looks like, having a transaction block that wraps around our code is not so much a nuisance anymore.

This also wraps up our params[:fu] series. Remember, how you assemble your views form elements have a lot to do with how thin and skinny your controllers look like. Thanks for reading!

params[:fu] #4 ) Use the magical <association_name>_ids=([...array of ids]) association proxy method.

2008-03-26T22:53:00.003-05:00

In the last blog entry, we looked at an example of having to build a new parent model and build its join models all in two lines of code. But if you are trying to just assign a list of has_many or has_and_belongs_to_many models, you don't have to use the fields_for and index trick. You can just use an association proxy method. Let's check it out:

class Reader < ActiveRecord::Base
  has_and_belongs_to_many :blogs
end

<% form_for @reader do |f| %>
  <%= f.text_field :name %>

  <%= f.collection_select :blog_ids, Blog.find(:all), :id, :name, {}, :multiple => true %>

  <%= f.submit 'Create' %>
<% end %>

generates:

<select id="reader_blog_ids" multiple="multiple" name="reader[blog_ids][]">
  <option value="1">Martin Fowler</option>
  <option value="2">Stephen Chu</option>
</select>

Processing ReadersController#create (for 127.0.0.1 at 2008-01-14 21:12:56) [POST]
Parameters: { "commit"             => "Create",
              "reader"             => { "name" => "stephen chu"
                                        "blog_ids" => ["1", "2"] },
              "authenticity_token" => "238ba79b8282882ba01d840352616c2cc79280f0",
              "action"             => "create",
              "controller"         => "readers" }

And now, in my controller action, the creation of the reader only takes one line of code:

def create
  @reader = Reader.new(params[:reader])

... thanks to the following has_many (or habtm) association assignment method:

@reader.blog_ids = [...many ids...]

So, stop doing any more params-munging code in your controller action like these:

def create
  ...
  params[:blog_ids].each do |id|
    @reader.blogs << Blog.find(id)
  end
end

Get on the Rails bandwagon!

params[:fu] #3 ) Using fields_for and the index option to create a new parent model with multiple child models.

2008-03-25T21:19:00.003-05:00

Alright, the last couple days were easy. Today, let's take a look at a slightly more complicated example, but one that occurs on almost every single Rails project out there: saving multiple models on one POST. Let's say your new form allows you to create a new reader and attach 3 subscriptions to it. Here's the code:

class Reader < ActiveRecord::Base
  has_many :subscriptions
  has_many :magazines, :through => :subscriptions
  validates_associated :subscriptions
end

<% form_for @reader do |f| %>
  <%= f.text_field :name %>
  
  <% (1..3).each do |index| %>
    <% fields_for :subscriptions do |ff| %>
      <p><%= ff.collection_select :magazine_id, Magazine.find(:all), :id, :name, {}, :index => index %></p>
    <% end %>
  <% end %>

  <%= f.submit 'Create' %>
<% end %>

generates:

<select id="subscriptions_1_magazine_id" name="subscriptions[1][magazine_id]">
  <option value="101">PC Magazine</option>
  <option value="102">IT Pro</option>
  <option value="103">WIRED</option>
</select>  

<select id="subscriptions_2_magazine_id" name="subscriptions[2][magazine_id]">
  <option value="101">PC Magazine</option>
  <option value="102">IT Pro</option>
  <option value="103">WIRED</option>
</select>

... (and more)

Processing ReadersController#create (for 127.0.0.1 at 2008-01-14 21:12:56) [POST]
Parameters: { "commit"             => "Create",
              "reader"             => { "name" => "stephen chu" },
              "subscriptions"      => { "1" => { "magazine_id" => "101" },
                                        "2" => { "magazine_id" => "102" }, 
                                        "3" => { "magazine_id" => "103" } },
              "authenticity_token" => "238ba79b8282882ba01d840352616c2cc79280f0",
              "action"             => "create",
              "controller"         => "readers" }

By using the :index html option in our form builder generated fields, we are essentially inserting a unique index key to the posted value of that field. Why should we care? Well, here is how the controller code would look like:

def create
  @reader = Reader.new params[:reader]
  @subscriptions = @reader.subscriptions.build params[:subscriptions].values
  if @reader.save
    flash[:success] = 'Good.'
  else
    flash[:error] = 'Bad.'
  end
end

I just persisted a reader with multiple subscriptions, and the notable differences I added in the create action, was these characters: params[:subscriptions].values. Nothing much changed from the last has_one :computer example besides association related differences. There is no looping, map/collect-ing, gsub-ing, etc., in my action while creating and attaching these multiple subscriptions onto the new reader. The controller action is just doing its same-old routine: receives posted parameters, and shove them into the corresponding models. So how is this done?

Looking at the source of the #build method on the association proxy classes (e.g. HasManyAssociation), you will notice something interesting:

def build(attributes = {})
  if attributes.is_a?(Array)
    attributes.collect { |attr| build(attr) }
  else
    build_record(attributes) { |record| set_belongs_to_association_for(record) }
  end
end

The method recognizes array! If you pass in an array of hashes, it will process them one by one! So how do we get an array of hashes? In our case the way to get array of hashes is by calling Hash#values, because they are organized in sub-hashes. Calling .values will yield us the following array:

params[:subscriptions].values  # => [ { "magazine_id" => "101" }, { "magazine_id" => "102" }, ... ]

By organizing your params on your view in ways that they can be directly consumed by your models, you end up with a lot less code to write. No more bastardizing your action code!

(back to the TOC of the params[:fu] series)

params[:fu] #2 ) Put attributes into a different params key using fields_for if they belong to different models.

2008-03-24T23:03:00.002-05:00

fields_for is very useful for separating out your params:

<% form_for @reader do |f| %>
  <%= f.text_field :name %>
  
  <% fields_for :favourite_computer, @computer do |ff| %>
    <%= ff.text_field :name %>
  <% end %>
  <%= f.submit 'Create' %>
<% end %>

Processing ReadersController#create (for 127.0.0.1 at 2008-01-14 21:12:56) [POST]
Parameters: { "commit"             => "Create",
              "reader"             => { "name" => "stephen chu" },
              "favourite_computer" => { "name" => "macbook pro" },
              "authenticity_token" => "238ba79b8282882ba01d840352616c2cc79280f0",
              "action"             => "create",
              "controller"         => "readers" }

def create
  @reader = Reader.new params[:reader]
  @computer = @reader.build_computer params[:favourite_computer]
  if @reader.save
    flash[:success] = 'Good.'
  else
    flash[:error] = 'Bad.'
  end
end

class Reader < ActiveRecord::Base
  has_one :computer
  validates_associated :computer
end

Here, I am on a page that creates the reader and its favorite computer. Two models sounds hard? Not really, if you key your params to a different key for each of the model, your controller is still nice and thin. You do need to add one line to the action to create the computer though, but that maps nicely to the fact that you are operating on two models from the page post. I would not want to add more than one line of code to create one additional model, so forget those params-munging/looping.

Also, you can see that I am using validates_associated here. The point being I do not want to call save or valid? once for each model in my action, making my controller code fat.

Also, fields_for is not only just a helper method, but also a method available on your FormBuilder as well (you know, the f in your form_for is an instance of a FormBuilder. To see an example of how to call it off of a form builder, check out this blog entry.

(back to the TOC of the params[:fu] series)

params[:fu] #1 ) Wrap all relevant attributes in a single params key.

2008-03-23T22:47:00.009-05:00

This is the simplest form of using form_for and params:

<% form_for @reader do |f| %>
  <%= f.text_field :name %>
  <%= f.text_field :birth_place %>

  <%= f.submit 'Create' %>
<% end %>

Processing ReadersController#create (for 127.0.0.1 at 2008-01-14 21:12:56) [POST]
Parameters: { "commit"             => "Create",
              "reader"             => { "name" => "stephen chu", "birth_place" => "hong kong" },
              "authenticity_token" => "238ba79b8282882ba01d840352616c2cc79280f0",
              "action"             => "create",
              "controller"         => "readers"}

def create
  @reader = Reader.create! params[:reader]
end

By doing this, you eliminate the need to explicitly spell out each parameters posted, like the ugly Reader.new :name => params[:name], :birth_place => params[:birth_place].

Notice, Active Record is extremely hash-happy. Let's take a few examples and see how it is so hash-hungry (or rather, hash-happy).

@reader = Reader.new params[:reader]  # => { 'first_name' => 'stephen', 'last_name' => 'chu' }

@readers = Reader.find(:all, :conditions => params[:search])  # => { 'name' => 'chu' }

@reader.attributes = params[:reader]  # => { 'first_name' => 'martin', 'last_name' => 'fowler' }

@reader.update_attributes(params[:reader])  # => { 'birth_place' => 'hong kong' }

Notice the find with condition one. No one says I cannot have a hash or form_for that must map to a database table!

All of the above will work in just one line of code, and I am sure you can find more examples as well. Remember, hashes are one of the most underrated Rails toolset to reduce code you need to write. Stay tuned on how we can optimize the use of these AR one-liners.

(back to the TOC of the params[:fu] series)

Boost your Rails Controller params[:fu]

2008-03-23T22:46:00.010-05:00

In many ways, in a Rails controller POST action, what gets POST-ed in params are often overlooked. In many straight-forward tutorials, we slap a forms_for @customer in our view, and then the params will come out to be directly consumable by our ActiveRecord model. Unfortunately, many new Rails developers cannot see beyond the usefulness of hashing the posted parameters accordingly. In the next few blog posts, I intend to show you that if you take the time to assemble your params the way Rails likes it, you can dramatically reduce the amount of code you have to write in your controller actions and models, simply because Rails takes care of it for you. The end result is getting it done in less code, fewer tests, less trouble.

Here it goes, the params[:fu] series:

params[:fu] # => "1" Wrap all relevant attributes in a single params key.

params[:fu] # => "2" Put attributes into a different params key using fields_for if they belong to different models.

params[:fu] # => "3" Using fields_for and the index option to create a new parent model with multiple child models.

params[:fu] # => "4" Use the magical <association_name>_ids=([...array of ids]) association proxy method.

params[:fu] # => "5" Use ids to update multiple models in update action atomically.

update:

These tips have been chosen as the 8th runner-up in the Railscasts 100th episode contest. And for that, let me add another parmas[:fu] bonus to this list to all readers:

params[:fu] # => "bonus" Passing multiple params in an observe_field, now that you know your parmas[:fu].

Enjoy!

Rails composed_of &block conversion

2008-03-12T21:14:00.010-05:00

If you had read Domain Driven Design, you will remember the author, Eric Evans, talks about various types of model objects in your domain model. Two of which are Entity Objects and Value Objects.

Entity objects are those that we care about their identity. For example, on Amazon.com, two users obviously have different identities, despite the possibility that they have the same name. Value objects, on the other hand, are those that we only care about their internal states' equality. Two money objects each has amount 100 and currency 'US' are treated as equal in the entire application. These two types of objects are similar in that they can all be defined as a class, but their usage is quite different.

Usually, a big bulk of your domain objects will be Entity objects. Almost all of your ActiveRecord::Base subclasses belong to that category, since each object has a different 'id' to uniquely identify themselves.

Rails provides composed_of as a way for your Value Objects to be referenced by an ActiveRecord Entity object. Rails 2.0 even added some syntactic sugar to composed_of, allowing you to pass a block for value object conversion. For Rails 1.x users, you can achieve the same with the plugin called composed_of_conversion. Let's see how to beautify your Rails code.

class Account < ActiveRecord::Base
  composed_of :balance, :class_name => 'Money', :mapping => [%w(balance amount), %w(currency currency)] do |params|
    Money.new params[:balance], params[:currency]
  end
end

class Money
  attr_reader :amount, :currency
  
  def initialize(amount, currency)
    @amount, @currency = amount, currency
  end
end

class AccountsController < ApplicationController
  def create
    @account = Account.new params[:account]

    if @account.save
      flash[:notice] = 'New account created...'
      redirect_to @author
    else
      render :action => "new"
    end
  end
end

Notice how my @account creation does not require any additional Controller code for value object creation and assignment, keeping my controller away from snacks and candy (i.e. thin). But since I have redefined Account#balance and Account#balance= through the use of composed_of, what should my form_for fields in my view look like? The answer lies with a use of fields_for.

<% form_for(@account) do |f| %>

  <% f.fields_for :balance, @account.balance do |ff| %>
  <p>
    Balance:  <%= ff.text_field :balance %>
  </p>  
  <p>
    Currency: <%= ff.text_field :currency %>
  </p>
  <% end %>

  <p><%= f.submit "Create" %></p>
  
<% end %>


POST Parameters:
{ 
  "commit" => "Create",
  "account"   => { "balance" => { "balance"=>"400", "currency"=>"us" } }
}

By calling f.fields_for on the form builder (and not just fields_for), you are creating a new hash scope 'balance' under params[:account]. Since your composed_of is called balance, only the sub-hash 'balance' will be used in your composed_of block conversion. Then, your model will use the sub-hash to convert it into the Money object you want.

At this point, your Money value object need not be confined with a constructor that takes parameters by ordinal position anymore. You can have it take a hash just like any ActiveRecord::Base classes, so that any future addition of parameters to Money does not require changes to your conversion block. Just drop an html element using your fields_for form builder and it will just work.

Why do this? Now, you have a different class to handle money related operations, such as addition, subtraction, validations, money comparison, and even formatting your money amounts everywhere. Should you have another model that needs money, you know where to share code from.

Also, do not get into thinking that a value object must abstract at least 2 columns on a table. It doesn't have to be. A value object can abstract just one column but contains its own validation and formatting logic and just be fine.

Value objects are very important in any application, even though a lot of times they are simple. Make good use of them and your code quality will improve. Now it's time to roll up your sleeves and clean up those value object creation code inside your controllers!

A little #to_s can do wonder in DRY-ing up your views

2008-03-10T23:03:00.001-05:00

Do you do this every day?

<h1>Show - Blog <%= @blog.name %></h1>
<dl>
  <dt>Author:</dt>
  <dd><%= @blog.author.full_name %></dd>
</dl>

<ul>
  <% for comment in @blog.comments do %>
    <li>Posted by <%= comment.created_by.full_name %>: <%= comment.description %></li>
  <% end %>
</ul>

I am lazy. Having to spell out in my views what field to render for every object when it is that field 90% of the time in my application gives me too much typing. When I render a @comment, you bet I want its description field most of the time; similarly, when I render a @user, you bet I want his/her full_name.

Don't forget all these erb tags at the end of day only render #to_s anyways, so you can just define your default field there to get rid of some typing as well:

class Blog < ActiveRecord::Base
  def to_s
    name
  end
end

class User < ActiveRecord::Base
  def to_s
    first_name + ' ' + last_name
  end
end

class Comment < ActiveRecord::Base
  def to_s
    description
  end
end

Now, let's see how our view looks, 90% of the time:

<h1>Show - Blog <%= @blog %></h1>
<dl>
  <dt>Author:</dt>
  <dd><%= @blog.author %></dd>
</dl>

<ul>
  <% for comment in @blog.comments do %>
    <li>Posted by <%= comment.created_by %>: <%= comment %></li>
  <% end %>
</ul>

See, it's a lot DRY-er and less verbose.

You will also notice that by doing this I eliminated a need to use a possible Object#try(:method) case:

<dd><%= @blog.author.try(:full_name) %></dd>

is no longer necessary, because a nil @blog.author will automatically give an empty string on the view. The end result is we all get to be lazier every day :-)

Format your data using Rails composed_of Value Objects

2008-01-11T03:58:00.000-06:00

A requirement comes in as usual says, "I want prices to be formatted like '$2,000.00' on all rhtml's."

Your programming brain then associates the word "formatted" with "presentation". Afterall, the price column is simply stored in a decimal column in the database. And off you go, wrapping number_to_currency(@book.price) on all of your views, just like any tutorial says.

It's all good until the same formatting is required for all emails being sent, all web services requests being issued, all Excel/CSV reports being generated, all everything. Since you took advantage of formatting the prices with an ActionView helper method number_to_currency, you cannot do the same in places where ActionView helper methods are not available to you. You start creating wrapper classes around your Book class, maybe a presenter, to keep this "presentation" logic outside of your pristine business domain model.

class Book < ActiveRecord::Base
end

class BookPresenter < SimpleDelegator
  include ActionView::Helpers::NumberHelper
  
  def price
    nummber_to_currency "#{__getobj__.price}"
  end
  
  def isbn
    ... ...
  end
end

class BooksController < ApplicationController
  def show
    @book = BookPresenter.new Book.find(param[:id])
  end
  
<p>
  ISBN: <%= @book.isbn %>
  Price: <%= @book.price %>
</p>

To present formatted data is extremely common on any applications, be it a price, date/time, name, address, SSN, ISBN, or even a baseball game score. A lot of these information are so simple that they can be stored in a single database column and be represented by a primitive type like String, DateTime, or Fixnum in Ruby. Since they are so simple in nature, programmers think that all it takes to format them is wrap a helper method around it on the view. But this approach may backfire like the example above.

Alternatively, one can create a value object that has a #to_s method that does all the formatting for you. That way, everywhere that needs formatting can ask a book object for it, and not have to wrap a presenter around it for something as formatting.

class Book < ActiveRecord::Base
  composed_of :isbn, :class_name => 'ISBN'
  composed_of :price, :class_name => 'Money'
end

class ISBN
  def to_s
    ... ...
  end
end

class Money
  include ActionView::Helpers::NumberHelper

  def to_s
    nummber_to_currency @price
  end
end
  
class BooksController < ApplicationController
  def show
    @book = Book.find param[:id]
  end
  
<p>
  ISBN: <%= @book.isbn %>
  Price: <%= @book.price %>
</p>

So what's the difference? The primarily difference is, every Book object now knows how to format its information correctly, and now when a book needs to be used by various sources, be it on an rhtml page, a web service request, or an Excel/CSV report, those sources can interrogate a book object, but not "first fetch a book object, then wrap it with a presenter".

Some people will yell now, "but aren't you putting presentation logic into your domain model?" Yes, kinda. One of the toughest decisions that a developer always has to make is to distinguish between domain logic and non-domain logic. Now if I may ask, if my formatting is happening in multiple places, is this domain logic? I tend to say it is. Also, from experience, while it is extremely rare for me to put such presentation/formatting logic in an Entity Object domain model class (e.g. Book) because such classes are almost always complicated enough in just domain behaviors, having formatting logic in these usually less complicated Value Objects (e.g. Money) makes a lot of sense to me. And in Rails, composed_of provides me that convenience to me almost effortlessly.

The way Rails has these handy ActionView::Helpers is super useful, but every once a while we must step back and ask ourselves the question, "how can I use them beyond the tutorial way on my really complicated application?"

Tip #1: You can also have your #to_s methods to take an additional parameter to customize the formatting behavior. Take a look at how DATE_FORMATS are used in Time#to_s.

Tip #2: You may not need a new class for the sake of using composed_of every time you have a new formatting requirement either. Sometimes you may be able to get away with just extending the Ruby primitive types with a custom method/format, for example Price: <%= @book.price.to_s :money %>.

Go to mysql command shell using rake

2007-12-26T00:01:00.001-06:00

Tired of typing mysql -u foo -pbar -h db01.firewalled.com baz_development yet? Add to that when you have database environments galore like test, QA, selenium, staging, etc.

Here's a rake task for you. Just point it to a Rails environment and execute and you are in the corresponding mysql command prompt.

e.g.

$ rake mysql RAILS_ENV=qa

Loading mysql in RAILS_ENV=qa...
    Executing: mysql -u foo -pbar -h localhost baz_qa_test

Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.0.51 Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql>

Code:

desc "Loads mysql command prompt"
task :mysql do
  puts "\n\nLoading mysql in RAILS_ENV=#{RAILS_ENV}...\n"

  database_yml_path = "#{RAILS_ROOT}/config/database.yml"
  database_yml = YAML.load(ERB.new(File.read(database_yml_path)).result)

  raise "Could not find environment #{RAILS_ENV} in database.yml" unless database_yml[RAILS_ENV]

  config = database_yml[RAILS_ENV]
  username = config['username']
  password = config['password']
  database_name = config['database']
  host = config['host'] || 'localhost'
  port = config['port'] || 3306
  socket = config['socket'] || '/tmp/mysql.sock'

  raise "Failed. Setup requires a user and database for environment #{environment} in '#{database_yml_path}'.\n\n#{database_yml.to_yaml}" unless username and database_name

  args = "-u #{username}"
  args << " -p#{password}" unless password.to_s.strip.empty?
  args << " -h #{host}" unless host.to_s.strip.empty?
  args << " #{database_name}"

  command = []
  command << "mysql #{args}"

  puts <<-EOS
    Executing: #{command.join(' ')}\n
  EOS
  system command.join(' ')
end

Rid of those TextMate "ctags timed out!" error

2007-12-17T02:04:00.000-06:00

If you use TextMate for your Ruby on Rails development, you must use the ctags bundle CTags.tmbundle. It gives you a keyboard shortcut CTRL-] to jump to files that defines the class/method your cursor is on. Similar to "go to file and line" in some IDE. Super handy.

Trouble is when your project gets large, it gets longer and longer to update the tmtags index file to a point where it exceeds the default 15 seconds timout and gives a "ctags timed out!" error.

What you can do is edit the file ~/Library/Application\ Support/TextMate/Pristine\ Copy/Bundles/CTags.tmbundle/Support/bin/tmctags and increase the timeout $CtagsTimeout value to something more reasonable. Problem solved.

p.s. CTags suffers from another shortcoming, and that is it cannot index ruby methods named ending with a ? or !. Anyone knows of a fix/workaround? Please comment if you do!

Why create database migrations, when you don't need to?

2007-12-14T00:15:00.000-06:00

Rails out of the box supports database migration. It allows Rails programmers to be more Agile, do less database BDUF and instead change the database schema as the requirements change and the business wants it. Writing a migration is also as easy as running a command line script/generate migration and then executing it with rake db:migrate.

Database migration is useful for two reasons. Number one is to allow developers to change the database schema. This can be as simple as a single ALTER TABLE statement to add a new column. Number two, more importantly, is to migrate the all-important data exist in the database. But if your application is still under development for initial release, then migration may not be buying you too much good, because chances are no one cares how your table structure got to where it is now, and you may not have a whole lot of data.

We all use migrations feverishly probably because most of the Rails books/references/tutorials begins with: "Let's start by creating a database migration, in it we insert some data into the newly created tables, and learn how to do XYZ." Thus, your db/migrate folder is stuffed with migrations that does everything: create new tables; alter existing tables; create data for those tables; update data created from previous migrations; and perhaps all of the above, while without justifiable reasons on what those migrations are trying to migrate. In practice, it is quite time-consuming for a developer to run 200+ migrations every time he blows away a database, which is not uncommon. Not only that, sometimes you have multiple migrations that basically cancel out each other's changes as our customers change their minds back and forth. As a result, you could be creating migrations left and right when there may not be any real beneficiaries: a database loaded with data.

Sometimes your QA team may have their own data set that tests your app, and thus their loaded database is a beneficiary. While that is true, I prefer their data to be scripted and be freshly generated by ActiveRecord models (using create!) every time instead of keep migrating them, because as my application domain model expands, I want not just QA data but all datasets to be cleansed and validated by my model validations. There is no guarentee that after running a drop_column migration the QA dataset does not violate any business logic. Keeping all these data valid while database migrations are being rapidly created is very hard.

To take advantage of the fact that your development environment has nothing to lose until you have an initial release, while maximizing the benefits of keeping all your data valid (for all enviroments) the whole time while your app is under development, it's a simple steps 1-2-3:

Step 1:

Have one migration file, 001_release_one_schema.rb, that captures all database object creations. For example, all your create_table, create_index, views, triggers (*yikes*), etc. After this migration, your database should contain all database objects for your Rails app but in a "blank", data-less state.

$ cat db/migrate/001_release_one_schema.rb


  class ReleaseOneSchema < ActiveRecord::Migration
    def self.up
      create_table "foos" do |t|
      end

      (... and many others ...)
    end

    def self.down
      (... ... ...)
    end
  end

Step 2:

Create a rake task to populate all reference data that your application requires to run with. Reference data meaning all data that your application cannot change through its screens, but are essential for your app to run. For example, all currencies that are used to populate a drop-down on your app. A lot of drop-down lists data are reference data.

$ cat lib/tasks/data.rake


  namespace :data do
    desc "Loads a default dataset of both reference and user data into database."
    task :load => [ :environment,
                    :configuration,
                    :reference_data,
                    :user_data ]

    private

    task :configuration do
      ENV['DATASET'] ||= 'slim'
    end

    task :reference_data do
      require "#{RAILS_ROOT}/db/data/reference_data/#{ENV['DATASET']}"
    end
  end

$ cat db/data/reference_data/slim.rb


  @us_dollar  = Currency.create! :name => "US Dollar"
  @yen        = Currency.create! :name => "Yen"
  @euro       = Currency.create! :name => "EURO"
  ... ... ...

Step 3:

Create a rake task to populate all user data that your app requires to run with. User data are data that a user can create/update within your application. They are also required for the Rails app to function properly the first day when it launches. For example, for your flashy Paypal application, the fees structure on how it charges its users. An application administrator is allowed to raise or drop fees in your app.


  namespace :data do

    private

    task :user_data do
      require "#{RAILS_ROOT}/db/data/user_data/#{ENV['DATASET']}"
    end
  end

$ cat db/data/user_data/slim.rb


  Fee.create! :amount => 1_000, :currency => @us_dollar
  Fee.create! :amount => 1_500, :currency => @yen
  Fee.create! :amount => 2_500, :currency => @euro
  ... ... ...

There are several benefits of managing your database schema and data this way:

It is easier and faster to re-populate your entire database to the latest schema from scratch with data, since there are no extraneous migrations.

Faster to locate and update data needed for application. They are always in your dataset generation scripts.

No need to worry about outdated/removed ActiveRecord classes and declare them inside the migration file itself. They is no "legacy" ActiveRecord models.

All data are valid all the time because they are created through Model.create! sanitized by your model validations.

Easy to specify datasets to load by preference for DEV (slim, loaded), BA (story sign-off), QA (scenario-based), or demo (full) environments. e.g. rake data:load DATASET=slim RAILS_ENV=qa

No worries about broken/incomplete migrations. Fewer code, fewer trouble.

Now, after your Rails app goes to a 1.0 production release, you should switch this back to the normal Rails database migration style. I suspect your application users won't be too happy if you blow away their data every time you roll out a minor update or a major release... (or not?)

The last "D" in TDD means more than just "Development"

2007-12-04T21:50:00.000-06:00

When asked "Do you write tests?", a lot of developers these days will say "of course" as their answers. However, not everyone can admit to doing TDD (Test Driven Development) correctly. Test Driven Development says, a developer will write a test that fails first, then write code to make the test pass, and refactor when possible, and repeat. This is what most people's TDD rhythm is. For the most part this is fairly easy to do. But to reach the next level, one has to understand TDD as a tool: TDD means more than just test your own code. Here is a couple tips on what the last "D" means:

Discipline

It takes a great deal of discipline to even write a failing test before writing the actual code. Sometimes, we write a little seudo-code here, or move a method definition there, or changing code else where trying to see if more new code needs to be written after it, and sooner than you think you are writing the actual implementation of the methods you wanted to test (Test Afterwards Development anyone?). Other times you write the test, but you are too anxious to even run it and see it fails. And other times you want to jump into the actual code immediately when you see your new test fails, but failing for the unexpected reasons.

Don't fall into these traps. If anything is true, testing is hard, but it is at the same time rewarding and fun. What's also true is, it will pay off. Write the failing test, draw up your list of tests you will need to write, and satisfy them one by one. Having discipline is the cornerstone of becoming a better programmer.

Design

It takes too long to write a test? Tests are running too slowly? Are your tests difficult to read? Are they too brittle and fail all the time? Hang in there! You ever had the feeling you saw code in the codebase that irks the living hell out of your mind written by someone else on your team? Well, it is time for you to get some of these feedback about your own code. Yay, your code sucks! Your tests are telling you that! Let's address each of these one by one.

Slow running tests? You shouldn't be hitting a database or web service in your unit tests, because you can mock/stub them out. Difficult to mock/stub it out? There probably is a better way to design your classes your tests are hitting. Ever heard of Inversion of Control (or Dependency Injection)? Master them. True TDD masters use them extensively.

Unreadable tests? Is it because of too many mocks/stubs? Or is it the code is 500 lines long and doing high octane 720-double-backflip logic? Either way, you have to learn to like small objects. Check this blog post of mine out.

Hard to test something? Tests too brittle? Perhaps you have encapsulation issues in your class design. If your classes are referencing 15 other neighbors, of course they are hard to mock/stub. Chances are, you have to spend time to debug your tests to find out what's wrong! Heard of Law of Demeter? Even if you have, take a look at this highly entertaining yet informative post. It might change your perspective a little.

The bottom line is, TDD is a way to guide you to writing good code, but only if you know how to use it as a tool. Now that you know, hopefully you will have a new perspective next time you write a test.

Passing multiple parameters in observe_field

2007-12-02T09:35:00.000-06:00

Sometimes you just want to pass multiple parameters on your ajax observe_field web request for processing. If you Google around, you will find out there are multiple ways of doing it.

And here is yet another way:

    observe_field :target_dom_id,
      :url => some_named_route_url,
      :method => :get,
      :with => "Form.serializeElements($('text_field_one', 'text_field_two', ...))"

where 'text_field_one' and 'text_field_two' are html form element ids.

I like this because if my text fields are rendered through form_for or fields_for (e.g. form_for :product), and my action does not want to explicitly specify what the posted parameters are upon processing (e.g. Product.create params[:product]), then by using Form.serializeElements I can hide all the params inside the key params[:product] still.

Parsing time in custom format in Ruby

2007-11-27T23:50:00.000-06:00

I am sure you have used Time#strftime (or Date, DateTime) to format your date/time into your own custom format for pretty time display. This is fine and great. But how do you parse a date or time in your own specific format in Ruby?

There is a little method called DateTime#strptime. Its second parameter takes a string, containing basically all the options available to you in Time#strftime (eg. %m-%d-%Y) and return you a DateTime object. This method is strangely not available in Time class. Fortunately from there you can use the Rails methods #to_time to convert to the Time object you want.

Usage:

  DateTime.strptime("12/25/2007 01:00 AM EST", "%m/%d/%Y %I:%M %p %Z")

Stop tangling those iPod earbuds!

2007-11-21T01:35:00.000-06:00

You have an iPod? If you ever use their white trademarked earbuds, chances are every time you pull them out of your pockets you spend 40 seconds of your life untangling those stupid wire. It adds up.

Ever notice there is a little sliding thing at the Y-split of your earbuds on the wire? I assume it is used to reduce the split between the two ear pieces when you are wearing them. If they are for that purpose I have never used them. But I did find a better use of it...

Next time when you are done using them, slide that thing all the way to the ear pieces. This will make your Y-splitted earbuds into essentially one straight wire, and thus reduce a lot of unnecessary tangling. Hopefully this saves you some annoying faces in untangling the thing (maybe that's why those Apple billboards have no face on them...)

Numbers don't lie, but your test coverage numbers might

2007-11-18T15:30:00.000-06:00

We have learned all though our education that our decisions should base off of numbers. Can you support your family given however much you are making? What is the velocity of gravity? How many roses do you give your girlfriend on Valentine's Day (no satisfying answers ever...)

The same happens in software. On all software projects, various flavors of metrics are gathered and read by various types of people on the team. Code coverage being one of them that is most commonly mentioned.

But there is a misconception about code coverage: When the percentage is high, it is good; otherwise, it is not good.

Then people try to find definition of "high", some say 80% is a good number, others say 90%. Some even strive for 100%.

But let's decode this message a little more thoroughly. If your number is low, this means your code is not very well tested. Clearly this is not desirable in a code base that you have to go in every day and make changes here and there - good luck in not breaking stuff. So, a low coverage number is bad.

Now, a high coverage number means your code is well tested. But this number does not tell you a few things:

1) The quality of your code. It does not tell you whether your objects are coupled like spaghetti; it does not tell you whether your code is doing crazily unreadable nested iterations mixing with multiple levels of recursions; it does not tell you whether you are violating encapsulation and separate of concerns; it does not tell you whether you are copy & pasting code everywhere. If your code exhibits all of these symptoms, a 99.9% code coverage still means future changes to the code base is going to be a nightmare.

2) The quality of your tests. Tests are also code that needs to be maintained. If it takes someone 30 seconds to make a single line of code change, but it breaks hundreds of tests and takes that person an additional hour to duplicate that fix in all failed tests, then the burden of maintaining those tests outweights the feedback of whether the system still works or not. Further, if tests are hard to read as they are long and mocks/stubs everywhere, and the test method names do not reflect what the test method body is doing, then having your tests unmaintained just adds test maintenance time.

So how should you read the coverage number? You should read this number in combination with other metrics to gauge the health of the code base. For example, do defects constantly come from a feature area of the code when changes are being made? Are story estimates higher in certain areas of the code base but not others? Are stories in certain functional areas always being under-estimated during Planning Meeting?

Code coverage is just a number. It does not tell you whether your code's flexibility to reflect the pace of your business requirements change. True productivity comes in with good code solving a particular problem well. Well tested code can still be bad.