a.blog.about.code

Overwriting Console Output Using Curses in Ruby

2013-04-01T00:00:00-07:00

Καλό μήνα! A while back I wrote about how to overwrite console output in Ruby. Originally the project I used this technique for only required overwriting a single line of output. That project has now grown and I need to be able to maintain a mini-dashboard on the console.

Fortunately, Ruby rocks and has a standard library wrapper for Curses. I would encourage you to read through the documentation. I've written a simple script to show-off the most basic use-case. In this example I initialize ten workers. Each worker is responsible for keeping track of the work she has completed, @percent, and then reporting that progress on her own line, @index, of the console.

#!/usr/bin/env ruby

require 'curses'

Curses.noecho
Curses.init_screen

class Worker
  def initialize(index)
    @index = index
    @percent = 0
  end

  def run
    (1..10).each do
      work
      report
      sleep(rand())
    end
  end

  def to_s
    "Worker ##{'%2d' % @index} is #{'%3d' % @percent}% complete"
  end

  private

  def work
    @percent += 10
  end

  def report
    Curses.setpos(@index, 0)
    Curses.addstr(to_s)
    Curses.refresh
  end
end

workers = (1..10).map{ |index| Worker.new(index) }

at_exit do
  workers.each{ |worker| puts worker }
end

workers.map{ |worker| Thread.new{ worker.run } }.each(&:join)

Curses.close_screen

Because Curses clears the screen once it is complete, the at_exit block ensures that the last known state of the workers is echoed to the screen one last time.

Another command I found useful was the Curses.clear command. This function clears the entire screen before rewriting output. This is particularly helpful when you want to overwrite a long line of text with a shorter one.

I hope this was helpful! Have fun making some cool console applications. If you build something worth sharing, please leave a comment that links to your code.

Context Driven Refactoring

2012-12-16T00:00:00-08:00

I got excited about this refactoring website called refactr.it. Users post refactoring questions and vote on the best answers. The website is kind of a simplified version of the Programmers Stack Exchange. But as I explored the ruby section, which is definitely still in it's infancy, I was a little disappointed.

Here is an example of a question and solution that was titled "Simplifying Nested If Statements".

Question Code

@problems =
    if @tags.blank?
        unless @language.nil?
            Problem.where(language: @language).desc(@sort).page(@page)
        else
            Problem.desc(@sort).page(@page)
        end
    else
        unless @language.nil?
            Problem.in(tags: @tags).where(language: @language).desc(@sort).page(@page)
        else
            Problem.in(tags: @tags).desc(@sort).page(@page)
        end
    end

Solution Code

@problems = Problem
@problems = @problems.in(tags: @tags) unless @tags.blank?
@problems = @problems.where(language: @language) unless @language.nil?
@problems = @problems.desc(@sort).page(@page)

The solution is definitely a start. In fact it "simplifies the nested if statement" perfectly. And I would be a much happier programmer if I happened upon the solution code than I would be if I came across the question code. My disappointment is that the refactoring lacks context. It's just an example of a refactoring formula:

# Nested

if A
  if B
    Z.X(A).Y(B)
  else
    Z.X(A)
  end
else
  if B
    Z.Y(B)
  else
    Z
  end
end

# Not Nested

chain = Z
chain = chain.X(A) if A
chain = chain.Y(B) if B

The solution is "correct", and it's a valuable technique, but it misses the mark. Refactoring often falls short because people blindly apply a selection of these transformations and then call it a day. Refactoring isn't just about rewriting code. Let's take a look at a couple of questions that might help us devise a better strategy for approaching this problem.

Question One: Why should I refactor this code?

Refactoring is fun, but if the code is a mess, and you don't need to touch it, leave it alone. As Sandi Metz explains, if you can't see the mess, it doesn't exist. Refactoring should always happen for a reason. In this case there are multiple potential reasons we might need to refactor. But leaving that reasoning out of the discussion can lead to a misguided solution.

Do I need to add a new search parameter? Then I'm probably in need of a solution that reduces complexity.

Am I replicating this code somewhere else? Then I'm probably in need of a solution that DRYs up my existing implementation.

Step One: Establish a context for the refactoring.

Let's start with, "I need to add a new search parameter." This helps us establish that the code is too complex, and that we need to address this issue because we need to change the code.

Question Two: How am I going to test this code?

Imagine, first without using a database, how you would go about testing the question code, or even the solution code for that matter. You would definitely have to provide stubs for #in, #where, #sort and #page. You would have to test conditionals for @language, @tags, and your new search parameter. And all those tests have to combined with the rest of the controller tests for the search action.

Even if you allowed yourself use of the database, you would still have to construct objects for each of the test scenarios and verify that the conditionals were working properly.

That's a lot of stuff (otherwise know as responsibilities). The "always write tests" mantra isn't just meant to increase code confidence, it helps illuminate design flaws and unwanted complexity.

Step Two: Read/Write Tests.

Assuming the tests don't already exist, write them. Even though they are brutal to write after the fact, you need to have test coverage before you start moving code around.

Question Three: What did the tests teach us?

Probably that we have a "Fat Controller". You've probably heard of "Fat Models, Skinny Controllers", but this heuristic does not really mean Fat ActiveRecord::Base classes and Skinny ApplicationController classes. It means that objects responsible for connecting business logic and views, should do only that. It's the "Single Responsibility Principle" in Rails terms.

The reason we should strive for single responsibility objects is that they are easier to extend and test. In this case we've seen how difficult it is to test the code (in both the question and the solution) which indicates that the proposed solution should probably not be our first refactoring step.

Step Three: Refactor Your Code

Because the tests indicated an unnecessary complexity, we know that we should probably introduce a new object this is responsible for handling some of the logic, without involving the controller.

@problems = ProblemSearch.new(@language, @tags, @sort, @page).find

So what goes into ProblemSearch#find? Probably the original solution, but at this point it really doesn't matter because it's completely isolated. The original non-nested code is definitely simpler and easier to extend than the nested code, but that's the transformation part of refactoring, not the reason we are refactoring in the first place.

You should always simplify your programming constructs so that they are more intention revealing, but the meat of this refactoring is really in the domain modelling and the tests. The original code had too many responsibilities, it was hard to test, and it was hard to extend.

When we needed to add a new search parameter and tried writing tests, we realized how complex that was going to be and identified the need for a refactoring. Allowing that context to guide us, resulted in an intention revealing, testable and extensible solution.

Actually Always Refactoring

2012-09-26T00:00:00-07:00

I apologize, it's been a while since my last blog post. After being sick, doing a lot of climbing, and some traveling, I attended GoGaRuCo 2012. It was my first Ruby conference and it really opened my eyes to the active and friendly community of Ruby. It gave me the boost I needed to sit down and write another blog post.

While on my hiatus, some comments trickled in on old blog posts. I do want to go back and write updated versions of a few of my older articles. But today I want to talk about Refactoring. I watched Therapeutic Refactoring by Katrina Owen the other day and it got me thinking. While I was at GoGaRuCo I made a point of talking to people, and I noticed the sly smile on their faces when I told them that, "I love refactoring." They loved it too. So why don't we do more of it? If it's therapeutic, if it's exciting and makes our code better, why aren't we always refactoring?

The reason, I think, is that along with that feeling of making our code awesome, and coding purely to write code - no feature requests, there is a very real feeling of impending doom. What if I pull on a thread that unravels the system? What if I find out that refactoring the mailer actually means I need to refactor the entire user object?

We've all experienced that elated feeling after watching a great presentation about refactoring. We head back to our keyboards, determined to make our code better. But in reality the systems we encounter are never as simple as the examples we've just seen. The same tricks don't apply. Our objects aren't easy to isolate, they aren't easy to test and are often so complex that it hurts to even think about them.

So what are we to do? Here are my thoughts.

Change your mindset. The goal is not to make the system perfect and easy to understand. The goal is to make the system better and easier to understand.
It's okay to stop. Start small - even if it's not the final solution. Pick one function, a few lines or even one line of code, and work on that first. Think about creating a pull request with the changes you've just made. Would your peers be able to reason about your changes?
Don't be afraid to replace two messes with one mess (of equal and approximate size). It's an evolving process and each piece of code does not need to be final or pretty, it just needs to make your program better in one way. Take another stab at refactoring duplication or complexity once the first step is done. Too many times have I been sucked down the rat hole of trying to fix all the messy parts at once.
Find someone to share the experience with. Pair refactoring is a great way to tackle the larger messes.
Refactor all the time. When a new feature request comes along, see if you can refactor first, in a separate commit. Then implement the feature. The goal is to make the feature commit simpler.
Demand refactoring. If you don't have the support to refactor then get that support or find a company/client that values it. Refactoring makes code better. Better code makes better products and better jobs.

I hope that helps. And keep your spirits up. Refactoring isn't easy to do the first time. Soon enough though, you'll begin to view it as "your feature request." And it will be a regular part of your awesome day.

Fixtures and Factories

2012-06-21T00:00:00-07:00

I recently converted a test suite that only used factories to a hybrid that uses a combination of factories and fixtures. While I really like the simplicity and cleanliness of factories, fixtures are simply unbeatable in some cases. I had never really used both together in the same project, so I wanted to outline the pros and cons of each, as well as how they work together.

Loading and Creating Data (Performance)

If you're loading all of your fixtures at the beginning of every test you're missing out on one of their major benefits - loading a lot of data only once. Instead, switch to loading all of your fixtures at the beginning of the entire test suite, and then using transactions around each test for cleanup. That way the base fixture data hangs around and each test takes care of cleaning up its own changes. Since generating data can be more expensive than saving it to the database, this simple change can result in dramatic performance gains.

When using factories you have the benefit of creating only the data you want. You may face some degraded performance caused by create operations, since you'll have to process all those pesky callbacks, but you will benefit from having more isolated tests and data.

Sharing and Customizing Data (Maintainability)

Fixtures often make customizations to data more complex. It's certainly a lot harder to change fixtures because other tests depend on that very same data. Certainly a slew of update statements is going to offset some of that performance gain of having data pre-loaded as well.

I often find that fixture based tests end up doing a lot of pre-assertions to ensure that data is in the correct state. If that's the case in your tests it's probably worth asking yourself if you could be using factories instead.

One way to combat the fixture modification data is to combine fixtures and factories. For instance, if you need to parse a large XML file and then create a model from that data, do the parsing ahead of time, store it as a fixture, and then use that model as a base for your factories.

Factories should generally be an MVO, or minimally-viable-object. Data sharing becomes a non-issue because each object adds only what it needs without worrying about conflicting with other objects or tests.

Proximity of Data and Assertions (Ease of Use)

In the fixture approach it's often harder to find the data related to a test because it's usually contained in another file. Sometimes this is unavoidable, even with factories, but it's more often the case that fixture data lives in some other file. If you are making an assertion about data, it's best to have that data near the assertion.

In the factory approach the relevant data declarations and modifications are all contained in the same test file which makes the test cases easier to read and follow.

Rule of Thumb

No approach is ever 100% better. In fact you'll probably see the greatest performance gains and have the most fun by using both factories and fixtures. It feels great to use the right tool for the job. If you use transactions to clean your tests then the two approaches work very well together. Fixtures will load data before the entire suite, and then each test will automatically rollback any changes to fixture data, and remove any factories it created.

As a rule of thumb, if data is expensive to create, it's probably better to use fixtures. If data requires many small customizations, it's probably better to use factories.

Benchmark Your Bundle

2012-05-03T00:00:00-07:00

Your Rails app takes anywhere from 20 seconds to a minute to boot. Maybe you've even tried to optimize your gem loading. But which gems are actually causing you all that trouble? How can you pinpoint the gems to delay, replace or optimize?

Here's a simple script I wrote to benchmark the loading of each of the gems in your Gemfile. A gist can be found here.

#!/usr/bin/env ruby

require 'benchmark'

REGEXPS = [
  /^no such file to load -- (.+)$/i,
  /^Missing \w+ (?:file\s*)?([^\s]+.rb)$/i,
  /^Missing API definition file in (.+)$/i,
  /^cannot load such file -- (.+)$/i,
]

def pull(dep)
  required_file = nil

  begin
    Array(dep.autorequire || dep.name).each do |file|
      required_file = file
      Kernel.require file
    end
  rescue LoadError => e
    if dep.autorequire.nil? && dep.name.include?('-')
      begin
        namespaced_file = dep.name.gsub('-', '/')
        Kernel.require namespaced_file
      rescue LoadError
        REGEXPS.find { |r| r =~ e.message }
        raise if dep.autorequire || $1.gsub('-', '/') != namespaced_file
      end
    else
      REGEXPS.find { |r| r =~ e.message }
      raise if dep.autorequire || $1 != required_file
    end
  end
end

require 'rails/all'

# If you would prefer gems to incur the cost of autoloading
# Rails frameworks, then comment out this next line.
ActiveSupport::Autoload.eager_autoload!

$VERBOSE = nil

Benchmark.bm do |x|
  Bundler.setup.dependencies.each do |dependency|
    x.report(dependency.name[0..20].ljust(21)) do
      pull(dependency)
    end
  end
end

All of the code in #pull has been plucked right out of the Bundler require function. It's responsible for loading a gem depending on its Gemfile configuration. It considers custom require paths, and generally converts dashes into slashes if it can't load the file.

The script then loads rails (require 'rails/all') - something you won't be able to avoid either way. It also turns warnings off ($VERBOSE = nil) so that the benchmarking output looks a little nicer.

Once that setup is complete, it loops through each of the gem dependencies and requires them. Your output will probably look something like this:

rails                  0.000000   0.000000   0.000000 (  0.000215)
aws-s3                 0.160000   0.030000   0.190000 (  0.182991)
declarative_authoriza  0.850000   0.120000   0.970000 (  0.970152)
fog                    0.530000   0.060000   0.590000 (  0.589278)
mysql2                 0.010000   0.000000   0.010000 (  0.012908)
nokogiri               0.000000   0.000000   0.000000 (  0.000285)
oauth2                 0.140000   0.010000   0.150000 (  0.156330)
rack                   0.000000   0.000000   0.000000 (  0.000232)
rails_autolink         0.000000   0.000000   0.000000 (  0.000854)
rpm_contrib            0.450000   0.030000   0.480000 (  0.482697)
timezone               0.010000   0.000000   0.010000 (  0.007698)
jquery-rails           0.010000   0.000000   0.010000 (  0.008963)
hike                   0.000000   0.000000   0.000000 (  0.001602)
sass-rails             0.370000   0.020000   0.390000 (  0.400510)
coffee-rails           0.120000   0.020000   0.140000 (  0.136651)
uglifier               0.000000   0.000000   0.000000 (  0.002760)
capybara               0.010000   0.010000   0.020000 (  0.006783)
capybara-webkit        0.050000   0.000000   0.050000 (  0.053145)
database_cleaner       0.000000   0.000000   0.000000 (  0.007077)
gherkin                0.070000   0.010000   0.080000 (  0.068361)
jasmine                0.020000   0.000000   0.020000 (  0.025752)
jasmine-headless-webk  0.000000   0.000000   0.000000 (  0.010595)
launchy                0.060000   0.010000   0.070000 (  0.060992)
rspec                  0.350000   0.030000   0.380000 (  0.385097)
rspec-rails            0.000000   0.000000   0.000000 (  0.000903)
sham                   0.030000   0.000000   0.030000 (  0.025905)
spork                  0.000000   0.000000   0.000000 (  0.003553)
timecop                0.040000   0.000000   0.040000 (  0.041003)
guard                  0.000000   0.000000   0.000000 (  0.003354)
rb-fsevent             0.010000   0.000000   0.010000 (  0.004126)
guard-spork            0.000000   0.000000   0.000000 (  0.008679)
test-unit              0.130000   0.010000   0.140000 (  0.132788)
webmock                0.150000   0.020000   0.170000 (  0.167813)
vcr                    0.060000   0.000000   0.060000 (  0.068362)

I've condensed the output, but the total load time was 5 seconds. You can verify that by benchmarking Bundler.require in your own application. You'll also notice that the declarative authorization gem took up approximately 1 second (20% of the total time) to load. Maybe it's a good candidate for an upgrade, replacement or optimization.

Overwriting Console Output in Ruby

2012-04-16T00:00:00-07:00

I've always liked the way some long running console utilities display their status without having to print a new line. For example the little spinner that alternates displaying |, /, and \. Or various internet utilities that display the percentage of the file that has been downloaded.

I recently wrote a utility to kick off parallel RSpec tasks on remote instances and wanted to report the results back to the console after pinging the utility every few seconds. I accomplished this with a cool little trick - the \r.

(1..10).each do |percent|
  print "\r#{percent*10}% complete"
  sleep(0.5)
end

This little scripts begins by printing 10% complete and works up to 100% complete in 10 percent increments. It's a cool way to notify the user of your Ruby utility that the script is still progressing and that it hasn't simply stopped dead in its tracks.

One thing to keep in mind is that you are overwriting the previous line, so if your previous line is of a longer length than the next one, you might need to pad it with whitespace to cover up the last message.

Respect the Active Record

2012-03-22T00:00:00-07:00

Here's some bad Rails code I often come across:

Author.where(public: true).where('score > ?', 50)

The problem isn't that we're chaining. Because we're only handling ActiveRecord::Relation objects, we aren't increasing the complexity or responsibility of our code by the mere act of chaining. Chaining is a useful programming pattern. However, we are increasing the complexity of our code by making assumptions about the internal workings of Author - we're breaking encapsulation. Even if you feel that ActiveRecord makes this easy to do, it doesn't mean you should do it.

We shouldn't know that authors have internal score and public attributes. Keep in mind that this is different from being able to call Author#score and Author#public, because in the former case, we actually "know" a lot more about Author and we make some unhealthy assumptions.

In particular we make the assumption that these two statements are logically equivalent:

Author.all.select{ |u| u.score == 50 }

Author.where(score: 50).all

In short, this code is asking Author to perform an operation based on an assumed internal state. We are asking - not telling. That's not good.

Telling Author to give us this data looks very different:

Author.with_public_account.with_score_above(50)

Having this kind of interface for our objects benefits us in a couple of really awesome ways.

Readability

Using intentionally crafted public interfaces for your objects makes them easier to read and more descriptive. This is usually a by-product of the simple fact that you are naming your methods instead of using what is already there. Keep in mind that ORMs were created as a general infrastructure for many different kinds of applications. There is no need to expose that infrastructure to your own application. It makes more sense to write Author.with_public_account than it does to write Author.where(public: true). The intent is clearly communicated.

Searchability

Searching for where.*score in a code base can be tedious and might not result in catching all the possible use cases. Code structured like this can easily be missed:

conditions = {score: 50}
Author.where(conditions)

By contrast, searching for with_score_above is much more likely to yield usable and accurate results. Obviously having a full test suite helps alleviate this kind of a refactoring, but that doesn't mean you should make it difficult for others to track down your handiwork.

Changeability

Have a more explicit interface makes it easier to change our ORM from ActiveRecord if we ever choose to. It makes it easier to make small attribute level changes, like renaming score. That's because we now only need to modify the Author object, not every piece of code that filters authors on score. You could imagine a future implementation that looks something like this:

module AuthorRelation
  def initialize(authors)
    @authors = authors
  end

  def with_public_account
    @authors = @authors.select{ |a| a.public? && a.paid? }
    self
  end

  def with_score_above(score)
    @authors = @authors.select{ |a| a.cached_score > score }
    self
  end
end

If you're thinking - why would I ever want to change something like that? Then you've missed the point. This isn't a pre-optimization, this is a philosophy for creating persistence objects that behave like simple Ruby objects, without exception. It's a slippery slope to assume that some objects just aren't going to change and that kind of thinking causes headaches for the rest of your code and for other developers working with code you've written.

What if we didn't change the entire object; what if we simply denormalized and indexed a flag that combined score and public? Our carefully crafted public interface would allow us to make use of this new column, whereas being tied to the where way of doing things would mean having to modify a lot more code.

Testability

It's easier to begin building an application without being tied to the ActiveRecord way of doing things. This frees us up to write isolated tests and hide ActiveRecord as a true implementation detail, rather than making it a central part of our application. No object or collection of objects should dominate your code. Some objects might be more or less connected, but that doesn't mean they should be required to test one another. Isolation is a great thing.

Summary

Respect the ActiveRecord. Let it do what it does best, as a piece of infrastructure and as a 3rd party library. Its way of doing things and its method signatures should not infect the rest of your application. It does what it does best when it's limited to and hidden inside of your ActiveRecord::Base objects.

Reducing Metadata Queries in Resque

2012-03-08T00:00:00-08:00

If you're using Resque with your Rails application you may have noticed a large number of SHOW TABLES, SHOW CREATE TABLE and SHOW FIELDS FROM queries in your log while a job is processing. The most frustrating thing about this is that they probably don't appear in your web requests.

The solution for this is quite simple. Because Resque forks its processes before each job is executed it only maintains - in memory - the cached metadata that was loaded before the fork. To resolve this issue we need to load all this metadata before the Resque job forks. Fortunately Resque provides a before_first_fork hook which allows us to execute code before the first job, hence allowing us to pre-load a lot of metadata into memory.

Add the following to an initializer file, like config/initializers/resque.rb.

# Pre-load all the ActiveRecord column metadata.
Resque.before_first_fork do
  ActiveRecord::Base.send(:subclasses).each do |klass|
    unless klass.abstract_class?
      klass.primary_key
      klass.columns
    end
  end
end

This code loads all the ActiveRecord subclasses and then, if they have a table associated with them, loads their column and primary key information. The column information is associated with the SHOW FIELDS FROM queries and the primary key information is associated with the SHOW TABLES query.

A note of caution. If you happen to use a gem that also hooks into the Resque pre-fork process you may run into issues. Currently Resque only supports a single block/code snippet per hook. Hopefully this issue will be resolved in this pull request that I created yesterday.

But if you are itching to get this working right away you can use my branch that contains the updates by adding the following to your Gemfile:

gem 'resque', git: 'git://github.com/panthomakos/resque', branch: 'hooks'

This code allows Resque to keep track of, and register, multiple fork hooks, so gems like rpm contrib, which also hook into the Resque boot process, can co-exist with your hooks.

Enjoy the faster Resque processing!

Understanding The Law of Demeter

2012-02-27T00:00:00-08:00

The Law of Demeter is often stated as, "only talk to your immediate friends." In Object Oriented languages that use a dot as the field identifier this is often simplified to, "only use one dot." While this is an interesting heuristic of sorts, it's a very poor rule to follow because it's almost never that simple. This is often hard to describe to new programmers, who tend to seek a set of rules to help them advance their skills. The reality is that there exists no simple rule you can follow, it's really more of a sense. The best way I've been able to come up with to summarize the Law of Demeter is, "don't operate on objects that you might not understand."

To illustrate this, let's consider things from the perspective of the calling object, self in Ruby. Say we have an expression self.x.y. Then we should be less likely to tell the receiver, x, to do y, as it becomes harder to understand x or as it becomes more likely for x to change.

In the simplest case the receiver is the caller. This case is equivalent to the original and stricter Law of Demeter. So in the expression self.x, self is the caller and the receiver. And since we have a high level of confidence in our own methods, we can call x.

In the more general case, given self.x.y.z, if our level of confidence in x is low, we should not be telling x to do y. And this carries forward - if, from the perspective of self, we do not have high confidence in y, then we should not be telling it to do z.

A Ruby Example

[1,2,3,4].map{ |k| k*2 }.join(',')

Using the original definition this code violates the law since the call to Array#join is a case of talking to a non-immediate friend. [1,2,3,4] is the immediate friend of self in this case. The call to Array#map is fine, but the result of that operation is not an immediate friend, so we can't tell it to do anything.

Fixing this code leads us down a rat-hole and is highly non-productive. It most likely results in greatly increasing the complexity of an otherwise pretty simple piece of code. In fact this simple example demonstrates that under a strict version of the Law of Demeter we can't chain methods at all.

To solve this chaining problem, most people tend to introduce this caveat: 'your friend's friend is also your friend if she is of the same type as your your friend'. In our previous example Array#map returns an array and we started with an array, so we can act on the result of that operation. But we can't act on the string that Array#join returns because we didn't start with a string.

This caveat is not enough though. In some cases it's too restrictive and in others it simply is not restrictive enough.

An Example of Complex Code

User
  .where(active: true)
  .map(&:id)
  .map{ |id| Group.where(owner_id: id) }
  .map(&:name)
  .grep('Rails')

This is only slightly a violation of the Law of Demeter, but it is definitely a violation of the fuzzier version I introduced earlier. The issue here is really the complexity of the code. There is a good chance that the code related to either User or Group will change and it's not feasible to expect that we will know when that happens. Additionally, this code assumes that we have confidence in each operation in the chain, and that's not something I feel comfortable with. Not because any one call in the chain is complex, but because by the time we get to the call that deals with grepping for 'Rails', we have already lost track of what we were doing. Take a look at a simplified version of this code that addresses these issues.

users = User.active

Group.owned_by_any_of(users).with_name_matching('Rails')

That's the beauty of the Law of Demeter. It advocates writing code like this over the first example. It's not really about how many dots you are using, it's about the responsibility of the objects in your code. How many objects is the caller responsible for understanding and keeping track of? How much logic is the caller encompassing? And is the logical path of the chain easy to follow?

An Example of Unknown Output

user.group.try(:name)

If you find yourself writing code like this, then you're in violation of the Law of Demeter. Both versions. And this code should be simplified.

user.group_name

Let's take a look at a slight variation to the first example.

user.group.name

In this code there's a good chance that you're in the clear. In fact this code is quite easy to understand, and if we actually do have confidence that group exists and that we'll understand the output, it's perfectly fine to use this expression.

Summary

The Law of Demeter is not a law, and it's not as simple as counting the number of dots in your statements. It's really about developing a sense for constructing your code so that each object is confident in what it is telling other objects to do, and what it is responsible for understanding. Hopefully this post clarified the Law of Demeter for you.

From WordPress to GitHub Pages

2012-02-21T00:00:00-08:00

A little while ago I migrated ablogaboutcode.com from a wordpress.com hosted site to a jekyll-generated and github-pages-hosted site. This is a move I've been wanting to make for quite some time now. My reasons were primarily due to control over content and style combined with wanting to own the code behind it all.

Wodpress served me well when I was first getting started - and trying to decide if I wanted to commit to blogging and producing content on a regular basis. It was also a great way to familiarize myself with a full-featured blogging platform and what that could potentially provide. But wordpress.com imposes some constraints around commenting and formatting that I wasn't going to be happy with in the longer term.

I could have used my own wordpress installation, but I think that as a blogging platform it's overly complex and slow. And I like coding. Moving to jekyll means that I can write my blog posts in vim, using markdown, and then easily check the results on a local server - no internet required. I'm actually writing this particular blog post on a bus with non-functioning internet.

The migration process was relatively painless. Disqus provides a great import tool for migrating comments from wordpress. Transferring my domain registration from wordpress.com to GoDaddy was also a simple process.

The hardest part of the move was converting my posts into markdown. There are lots of tools to do this but in the end I still had to go through and clean up each post individually. This consisted primarily of converting code and highlight blocks to pygments and making sure that the formatting of my content was to my liking - 80 character line lengths and pretty. Vim macros really made this part of the process fast.

I started with the Tom Preston css template and added some of my own customizations for Disqus and a more fluid layout. I've also added a floating share bar which I plan on expanding in the future.

Enjoy!

The Ruby Global Interpreter Lock

2012-02-06T00:00:00-08:00

All of the code in this post was executed on Ruby 1.9.3 in February 2012. Ruby MRI refers to the C implementation of Ruby.

The Ruby MRI Global Interpreter Lock (GIL) is often misunderstood. Programmers hear "global" and "lock" and assume that threaded code won’t execute with any concurrency. While I'm hopeful that it will eventually be removed (jRuby and Rubinius Hydra) the GIL is not all that bad, and the state of Ruby MRI threading can only improve.

The reason I say it's not that bad is that the GIL only applies to Ruby operations. That means it really only applies to the work that Ruby is doing. Which makes sense, because it exists to protect the integrity of the Ruby Virtual Machine.

But besides Ruby operations, what kind of other work is there? In a typical web application – a lot. When you're querying the database, disk or memcached Ruby isn't doing work, which means you aren't invoking the GIL. Let's take a look at a simple example.

I/O Operations

require 'benchmark'
require 'mysql2'

x = Mysql2::Client.new
y = Mysql2::Client.new

Benchmark.bm do |b|
  b.report('w/o') do
    x.query("SELECT SLEEP(1)")
    y.query("SELECT SLEEP(1)")
  end

  b.report('with') do
    a = Thread.new{ x.query("SELECT SLEEP(1)") }
    b = Thread.new{ y.query("SELECT SLEEP(1)") }
    a.join
    b.join
  end
end

       user     system      total        real
w/o  0.000000   0.000000   0.000000 (  2.002874)
with  0.000000   0.000000   0.000000 (  1.001658)

In this example Ruby passed the SQL query into a C program that waited on the database to respond. During that time Ruby released the GIL because it was not performing any work. Querying the database is an I/O operation and the GIL does not factor in to I/O operations. That means that the Ruby interpreter had the freedom to process more than one thread at a time. To understand the difference, let’s take a look at this next example.

Ruby Operations

require 'benchmark'

Benchmark.bm do |x|
  x.report('w/o') do
    10_000_000.times{ 2+2 }
  end

  x.report('with') do
    a = Thread.new{ 5_000_000.times{ 2+2 } }
    b = Thread.new{ 5_000_000.times{ 2+2 } }
    a.join
    b.join
  end
end

       user     system      total        real
w/o  2.300000   0.000000   2.300000 (  2.312023)
with  2.330000   0.000000   2.330000 (  2.338784)

In this example Ruby had to perform computations and modify data, so threading provided no benefit since the GIL was active. In fact attempting to parallelize these computations incurred the overhead of creating two threads.

Summary

The Ruby GIL only applies to non I/O operations. While removal of the GIL would allow my last example to take full advantage of native threads, Ruby threading is not in a state of non-existence. Ruby threading does work, it's just hasn't evolved to the same level as other languages yet.

Hopefully this clarified the GIL and threading. At this point in time event-driven ruby, threads and fibers provide a great deal of concurrency in Ruby despite the GIL. I believe in the future this will only improve – particularly for threads. The GitHub Gist of this code also includes another example using Ruby’s sleep function.

A Simple Rails Boot Time Improvement

2012-01-12T00:00:00-08:00

This simple improvement can have varying effects on the application depending on how many Gems you require. Before you actually jump into this I would definitely recommend having a full unit-test suite. Anytime you're refactoring code or changing how dependencies are loaded you want to be sure you aren't messing things up.

The examples from this post were performed on a Rails 3.1.3 application running on Ruby 1.9.3 on a Mac OSX PowerBook. The Gemfile contained about 65 gems in development mode.

The first step to any performance improvement endeavor is to benchmark. It's also important that you perform this step more than once - this gives you a more consistent reading as the first run can often be inflated above the average run-time.

(master) % time bundle exec rake environment
bundle exec rake environment  29.09s user 2.44s system 61% cpu 51.570 total

(master) % time bundle exec rake environment
bundle exec rake environment  28.02s user 1.77s system 98% cpu 30.184 total

(master) % time bundle exec rake environment
bundle exec rake environment  28.24s user 1.84s system 98% cpu 30.632 total

Now let's take a look at your application.rb and measure how much time we're spending loading gems.

# application.rb
require 'benchmark'

Benchmark.bm do |x|
  x.report do
    if defined?(Bundler)
      Bundler.require(*Rails.groups(:assets => %w(development test)))
    end
  end
end

When we run this benchmark we get an interesting result:

(master) % time bundle exec rake environment
       user     system      total        real
  10.910000   0.900000  11.810000 ( 11.969414)
bundle exec rake environment  27.99s user 1.76s system 98% cpu 30.199 total

(master) % time bundle exec rake environment
       user     system      total        real
  10.770000   0.900000  11.670000 ( 11.721163)
bundle exec rake environment  27.71s user 1.78s system 99% cpu 29.721 total

We're spending about 1/3 of our time in this Bundler.require statement. The reason for this is that Bundler is loading and executing every gem in your Gemfile, and with 65 gems that's a hefty portion of time spent executing code. We can confirm this by benchmarking the two parts of Bundler.require - Bundler::Runtime#setup and Bundler::Runtime#require. I'm going to leave out the details, but require is the offender here, and that makes sense. Bundler isn't really doing anything fancy, but it is responsible for executing all those gems you've collected in your Gemfile.

Here's a simple way to improve that.

Begin with a minor gem. By that I mean a gem that is only used in one or two files or that has minimal impact on your application. That’s usually a good starting point because the effects are minor. Also, don’t pick a gem that rails automatically includes anyways – that won’t work.
Check if the gem has Railties. One way to do this is to navigate to its source code directory and search for uses of ‘Railtie’.
```
cd `bundle list flickr_fu`
find . | xargs grep Railtie
```
If you see an output then that gem uses Railties and you need to move on to another gem for this optimization.
If you don’t see an output then the gem does not use Railties. That means it doesn’t hook into the Rails initialization process, which in turn means that it does not need to be required during the application boot. So, let’s take care of that – tell bundler to not require that gem.
```
# Gemfile
gem 'flickr_fu', require: false
```

Find uses of that gem, and manually require the gem in those files.

# lib/flickr_decorator.rb
require 'flickr_fu'

module FlickrDecorator
  ...
end

Run your tests.
Run your benchmarks.

You probably won't notice a difference form a few measly gems, but once you're done with the entire process you'll be looking at a significant time improvement. In this particular application I was able to shave 10s off each application boot. There's an added benefit that you now know exactly which files are using which gems. This makes it easier to test these files in isolation and measure the impact/use of a gem. Enjoy!

The & Operator in Ruby

2012-01-04T00:00:00-08:00

I'm going to assume that you are already familiar with the double ampersand operator in Ruby - the logical AND. This post is going to focus on all uses of the single ampersand operator. & can be quite confusing because it has a different meaning depending on the context in which it's used. In fact, both unary (&object) and binary (object & object) operations have meaning in Ruby. To understand these let's look at the uses of & in core Ruby.

The Binary Ampersand

In Ruby 1.9.3 there are three uses for the binary ampersand operator.

Bitwise AND

Bitwise AND is the binary bit-by-bit equivalent of boolean AND. So binary 101 & 100 = 100 and binary 101 & 001 = 001. & is defined as a bitwise AND operator for Bignum, Fixnum and Process::Status and simply converts the integer into a binary representation and performs a bitwise AND on that representation. Process::Status converts the process status number to a Fixnum and uses that to perform the operation.

irb(main):001:0> 14 & 13
=> 12

The result of this operation can be clearly seen by converting the numbers to their binary representations.

irb(main):001:0> "#{14.to_s(2)} & #{13.to_s(2)} = #{12.to_s(2)}"
=> "1110 & 1101 = 1100"

Set Intersection

Possibly the simplest use of the binary & operator is in the Array class. & is the set-intersection operator, which means the result is a collection of the common elements in both arrays.

irb(main):001:0> [1,2,3] & [1,2,5,6]
=> [1, 2]

Boolean AND

On the FalseClass, NilClass and TrueClass binary & is the equivalent of the boolean AND. Keep in mind this does not work like the && operator, since it is only defined on these three classes.

irb(main):001:0> false & true
=> false
irb(main):002:0> nil & true
=> false
irb(main):003:0> true & Object.new
=> true
irb(main):004:0> Object.new & true
=> NoMethodError: undefined method `&' for #<Object:0x007f9e7ac96420>

Custom Definitions

When the binary ampersand operator is invoked (First & Second) Ruby executes the definition from the first object (First#&). You can write your own binary ampersand method easily. When I do, I try to keep the first two core Ruby uses in mind - bitwise AND and set intersection. I don't like using logical AND because it is already covered by the && operator and is only defined on true, false and nil, which are special classes. Here is a custom example that works like set intersection.

DNA = Struct.new(:chain) do
  def triples
    chain.split(//).each_slice(3).to_a
  end

  def &(dna)
    (triples & dna.triples).map{ |triple| triple.join }
  end
end

irb(main):001:0> DNA.new('AGGTTACCA') & DNA.new('TTAAGGCCC')
=> ["AGG", "TTA"]

The Unary &

The Unary & is a little more complex. It is almost the equivalent of calling #to_proc on the object, but not quite. To understand it let's go over some background first. In Ruby you have two kinds of code blocks, Blocks and Procs. The two are very closely related but have some important differences. You can define and reference Procs and assign them to variables. Blocks are always related to a method call and can't be defined outside of that context. The way you tell them apart is that Procs are always preceded by Proc.new, proc, lambda or ->() when they are defined.

# A block that is passed to the each function on the [1,2,3] Array.
[1,2,3].each do |x|
  puts x
end

# A proc assigned to a variable.
k = Proc.new{ |x| puts x }

All methods have one and only one implicit Block argument, whether you use it or not. You can access it by calling yield in the method body.

def two
  yield(2)
end

irb(main):001:0> two{ |x| x*2 }
=> 4

Blocks are pretty useless outside of function calls though, for instance you can't just define one.

irb(main):001:0> { |x| x*2 }
=> SyntaxError: syntax error, unexpected '|', expecting '}'

But you can define and reference Procs.

irb(main):001:0> Proc.new{ |x| x*2 }
=> #<Proc:0x007f9e7ab766a8@(pry):30>

Procs fall into two categories. Procs that are lambda?, lambda procs, and Procs that aren't, simple procs. Lambdas are defined using lambda or ->() and whereas simple procs are defined using Proc.new or proc.

irb(main):001:0> lambda{ |x| x*2 }
=> #<Proc:0x007f9e8a8c3f40@(pry):34 (lambda)>
irb(main):002:0> ->(x){ x*2 }
=> #<Proc:0x007f9e7a896ed8@(pry):35 (lambda)>
irb(main):003:0> Proc.new{ |x| x*2 }
=> #<Proc:0x007f9e7a8f95d8@(pry):36>
irb(main):004:0> proc{ |x| x*2 }
=> #<Proc:0x007f9e7a950e28@(pry):37>

I'm not going to delve into the details of lambdas and simple procs, but there are two basic differences and it's important to know that they exist. The first is that lambdas are strict argument checkers, like methods, they can throw an ArgumentError exception. Simple procs will just ignore incorrect, extra or fewer argument combinations. The second is that lambdas act like methods regarding their return status - they can return values just like methods. When you try to return a value from a simple proc you end up with a LocalJumpError.

Now back to our unary ampersand operator. Since both Blocks and Procs are useful, it's convenient to be able to switch between them - enter &.

&object is evaluated in the following way:

if object is a block, it converts the block into a simple proc.
if object is a Proc, it converts the object into a block while preserving the lambda? status of the object.
if object is not a Proc, it first calls #to_proc on the object and then converts it into a block.

Let's examine each of these steps individually.

If object is a block, it converts the block to a simple proc.

The simplest example of this is when we want to have access to the block we pass to a method, instead of just calling yield. To do this we need to convert the block into a proc.

def describe(&block)
  "The block that was passed has parameters: #{block.parameters}"
end

irb(main):001:0> describe{ |a,b| }
=> "The block that was passed has parameters: [[:opt, :a], [:opt, :b]]"
irb(main):002:0> describe do |*args|
irb(main):003:0> end
=> "The block that was passed has parameters: [[:rest, :args]]"

If object is a Proc, it converts the object into a block while preserving

the lambda? status of the object.

This is an extremely useful case of the & operator. For instance, we know that Array#map takes a block, but say we have a proc that we want to re-use in multiple map calls.

irb(main):001:0> multiply = lambda{ |x| x*2 }

irb(main):002:0> [1,2,3].map(&multiply)
=> [2, 4, 6]
irb(main):003:0> [4,5,6].map(&multiply)
=> [8, 10, 12]

Keep in mind that the operator also preserves the lambda? status of the original block. That's kind of neat because it means we are able to pass lambdas (not simple procs) as blocks. That means we can impose strict argument checking in our blocks and we can have them return values using the return keyword. The only exception to this preservation is methods, which are always lambdas regardless of how they are defined.

def describe(&block)
  "Calling lambda? on the block results in #{block.lambda?}."
end

irb(main):001:0> describe(&lambda{})
=> "Calling lambda? on the block results in true."
irb(main):002:0> describe(&proc{})
=> "Calling lambda? on the block results in false."

class Container
  define_method(:m, &proc{})
end

irb(main):001:0> describe(&Container.new.method(:m).to_proc)
=> "Calling lambda? on the block results in true."

If object is not a Proc, it first calls `#to_proc` on the object and then converts it into a block.

This is where the magic really happens because it makes passing objects to functions in the place of blocks very simple. The most common case of this is probably calling into Array#map with a symbol.

irb(main):001:0> ["1", "2", "3"].map(&:to_i)
=> [1, 2, 3]

This works because calling Symbol#to_proc returns a proc that responds to the symbol's method. So the :to_i symbol is first converted to a proc, and then to a block. This is kind of cool because we can create our own to_proc methods.

class Display
  def self.to_proc
    lambda{ |x| puts(x) }
  end
end

class FancyDisplay
  def self.to_proc
    lambda{ |x| puts("** #{x} **") }
  end
end

irb(main):001:0> greetings = ["Hi", "Hello", "Welcome"]

irb(main):002:0> greetings.map(&Display)
Hi
Hello
Welcome
=> [nil, nil, nil]

irb(main):003:0> greetings.map(&FancyDisplay)
** Hi **
** Hello **
** Welcome **
=> [nil, nil, nil]

Hopefully this post has clarified the binary and unary ampersand operator in Ruby. It's a fun operator. In the binary form it can provide us with a shorthand way of doing bitwise AND and set intersection like operations. In the unary form it provides us with some powerful functionality for converting blocks and procs.

Courteous Meta-Programming in Ruby

2011-12-29T00:00:00-08:00

Meta-programming is often kind of mean. It has a tendency to be hard to understand and it can be quite destructive - causing methods to be modified or overwritten in non-obvious ways. I decided to write a post about a more courteous form of meta-programming. I picked up the technique from a blog post about method_missing and respond_to?. I realized that although I am comfortable with meta-programming, other developers might not feel the same way. So, here's a simple pattern that can make your meta-programming more readable and less destructive.

I want to create a module named Greetings so that when I extend my classes with it I can declaratively add greetings to them.

class FrenchAndEnglish
  extend Greetings
  greeting "Bonjour!"
  greeting "Hello!"
end

irb(main):001:0> FrenchAndEnglish.new.greet
Bonjour!
Hello!
=> nil

There are a couple of ways to accomplish this with Ruby. But before we jump into code, let's add these constraints to the system. After all, we are trying to write courteous code.

Readability - the code should be easy to read and understand with minimal knowledge of meta-programming.
Super Support - the greet method should support the use of super to delegate to parent definitions.
Non Destructive - the extending class should be able to define it's own greet method.

For instance, consider the following use of the Greetings module. It supports the use of super to delegate to previous implementations of greet and it allows the base class to define it's own greet method.

class TriLingualWithEnglish
  extend Greetings
  greeting 'Hello!'
  greeting 'Howdy!'

  def initialize(message01, message02)
    @message01 = message01
    @message02 = message02
  end

  def greet
    super rescue nil
    puts "#{@message01} and #{@message02}!"
  end
end

irb(main):001:0> TriLingualWithEnglish.new('Bonjour', 'Kalimera').greet
Hello!
Howdy!
Bonjour and Kalimera!
=> nil

Let's look at the most common meta-programming approaches to solving this problem.

eval - using instance_eval or class_eval might be acceptable, but these options are generally destructive. They overwrite methods on the base class and this breaks our non-destructive requirement. If we were able to solve this problem by using alias method chains then we would definitely be entering the territory of un-readable code. Also method chains would not allow us to use super to call previous definitions of greet.
Tracking Class Variable - a variable defined on the class level that is responsible for keeping a list of greetings. This has lots of problems, for one it does not support super, but it also greatly restricts our flexibility. We can no longer define an alternate greet method on the base class.

Since these two approaches don't fit our courteous meta-programming requirements, here's the cool alternative that does.

module Greetings
  def greeting(phrase)
    m = Module.new do
      define_method(:greet) do
        super rescue nil
        puts(phrase)
      end
    end

    include m
  end
end

By using the anonymous module m, and including it we are actually creating a storage object in the class hierarcy for the method, instead of writing it on the original class. This approach is non-destructive and allows us to maintain a clean method chain that supports the use of super. The super rescue nil line in our TriLingualWithEnglish class now calls into the anonymous module and allows both greeting implementations to co-exist. The code is also very readable since the programmer does not need to understand class_eval and instance_eval. I would advocate this approach whenever possible since it preserves the class hierarchy and avoids some of the less readable meta-programming techniques.

Object Oriented Ruby and Rails in the Complex World of Gems

2011-11-30T00:00:00-08:00

Good Object-Oriented Programmers think of Objects and Responsibilities in terms of a one-to-one relationship. One object, one responsibility. This concept implies that applications should have a dichotomy between objects that are responsible for business logic and objects that are responsible for integrating with Gems. Another way to think of this is that your business logic should not also be responsible for understanding a 3rd Party API. The knowledge of how to interact with a gem shouldn't be duplicated in two different areas of your application. Ideally, if you upgrade or replace a gem you should only have to change one piece of code to make your application fully compatible with the new gem version. That way your code is less brittle and easier to maintain.

A Simple Example

Say you have a pagination gem that works by monkey-patching the Array object.

Array#paginate(per_page, page)

To wrap this functionality you could create the following object.

class Paginator
  def self.paginate(array, per_page, page)
    array.paginate(per_page, page)
  end
end

When you upgrade that particular gem you only have one piece of code that you need to change. And when an exception is raised, it is raised from within the Paginator which is a clean and obvious way of notifying you that the API has probably changed. If the author of that gem chooses to use named parameters in a hash in the next gem version, you can simply modify your Paginator object.

class Paginator
  def self.paginate(array, per_page, page)
    array.paginate(per_page: per_page, page: page)
  end
end

The Two Solutions

At some point my objects need to interact with external libraries directly, but I like to minimize where that happens.

The first way to achieve this is to use the Facade Pattern and have your classes interact with the facade object rather than the gem. The facade object is responsible for abstracting the gem API and is customized to fit the needs of your application.

The second way to achieve this is to use what I like to call the Isolation Pattern. In the Isolation Pattern the objects that interact with the external library meet two requirements. First, they are relegated to the single responsibility of interacting with the gem. Second, they are kept in their own module or directory so as to prevent any kind of mingling with your other objects.

The two approaches achieve a similar end result - only one area of your code is responsible for interaction with gem X. And that's the important take-away. Let's take a look at three examples.

RSpec

RSpec clearly falls into the Isolation Pattern solution. It is setup to operate in the spec/ directory with files named *_spec.rb. It should not creep into any other application files. It's also a very well known API. Other developers will find it easier to interact with RSpec directly than with a facade that you have created.

Facebook and Twitter

Facebook and Twitter authentication gems fall into the Facade Pattern solution. They are highly specialized and don't have a common or popular interface. Most developers won't need to interact with these gems on a daily basis, so a facade makes sense because it simplifies an often complex API.

ActiveRecord

ActiveRecord - now this one may seem complex because Rails developers tend to put everything under the sun into their ActiveRecord classes, but it really falls into the Isolation Pattern solution. I like to think of ActiveRecord objects as Objects that have the Responsibility of persisting your domain data. That's it. They sit in the app/models directory. They aren't responsible for authenticating my Users or suggesting other Users for them to follow. It's actually quite simple, here are my two ActiveRecord rules.

Don't call ActiveRecord finders directly. Finders, like `where`, are specific to ActiveRecord. They should not be called outside of the ActiveRecord classes.
```
class PostsController
  def index
    @posts = Post.where(:created_at.gte => Date.today-1.month)
  end
end
```
Should actually be
```
class PostsController
  def index
    @posts = Post.in_the_last_month
  end
end
```
The reason for this is that if ActiveRecord changes its query interface as it did between Rails 2 and Rails 3 then we shouldn't have to change our controllers and application logic.

Don't put domain logic in your ActiveRecord classes.

class User < ActiveRecord::Base
  def self.authenticate(username, password)
    user = where(username: username).first
    ...
  end
end

Should actually be

class UserAuthenticator
  def self.authenticate(username, password)
    user = User.by_username(username)
    ...
  end
end

The reason for this is that user authentication is application logic. It should not be dependent upon our persistence layer.

In a small application this may seem cumbersome. Why create so many extra objects to handle such simple functionality? As your application grows and more developers need to understand it and test it, using good Object-Oriented-Programming patterns makes sense. It makes your life easier and it makes coding more fun. There are few things worse than large-multi-responsible-classes. Massive classes take too long to understand and force longer and more complex function names. By following One Object, One Responsibility you can separate implementation logic from business logic and make your system less brittle, easier to test and easier to scale.

Ruby 1.9.3-p0 Released

2011-10-31T00:00:00-07:00

Among the smattering of new methods and bug fixes the most important improvement in Ruby 1.9.3 release, for Rails users, is going to be in application boot time. I've seen at least 50% performance boosts in load time simply by upgrading from Ruby 1.9.2. The installation was painless although I had some issues compiling the ffi gem extensions. I can't wait to migrate production code over to this new version and development is going to be so much more of a pleasure.

To upgrade with RVM:

rvm get head
rvm reload
rvm install 1.9.3

You can read the news post or changelog for additional details.

In particular I found the following of interest.

Random.rand now accepts a range as an argument Random.rand(10..20).
IO.write and IO.binwrite for writing to files in normal and binary mode.
New Encodings including UTF-16 and UTF-32.
ARGF new methods like ARGF.write and ARGF.puts.

Enjoy!

Using Rails Observers to write Faster Tests and Simpler Models

2011-10-24T00:00:00-07:00

Rails observers provide a great way to organize your code. If you follow the skinny-controller-fat-model practice in Rails then you may have already crossed the line from comfortably-plump models into massively-obese models. In fact it's a common misconception that your "models" need to map directly to database tables and it's quite easy to get carried away adding methods, associations, scopes, validations and callbacks to them. Do not fear - one of the benefits of writing Agile Object-Oriented Ruby is that you can easily simplify this mess with a little refactoring. I'm only going to touch on refactoring callbacks today, but a lot of what I cover here can be implemented for other methods as well.

What's Wrong with This Callback?

Let's consider the following seemingly innocent code. Assume for the time being that in production our emails are actually delivered in the background.

class User
  after_create :send_welcome_email

  protected

  def send_welcome_email
    if purchased_membership?
      GreetingMailer.welcome_and_thanks_email(self).deliver
    else
      GreetingMailer.welcome_email(self).deliver
    end
  end
end

Does this callback really belong on the User class? In some ways it does, it's definitely entirely dependent on the user's state and does a pretty good job of delegating to the GreetingMailer instead of trying to handle the mail implementation itself. But it does kind of break the "each object should only deal with one or a few things" OOP pattern. On a more academic level this code doesn't really have much to do with the user domain model. It's only really responsible for sending a welcome email when something is created.

Let's Refactor It

Let's rid the User model of this coupling by refactoring the code into an observer. Our WelcomeEmailObserver will have a single responsibility, namely to "send the appropriate welcome email once something is created".

# app/[models|observers]/welcome_email_observer.rb
class WelcomeEmailObserver < ActiveRecord::Observer
  observe :user

  def after_create(user)
    if user.purchased_membership?
      GreetingMailer.welcome_and_thanks_email(user).deliver
    else
      GreetingMailer.welcome_email(user).deliver
    end
  end
end

# app/models/user.rb
class User
end

# config/application.rb
class Application < Rails::Application
  config.active_record.observers = :welcome_email_observer
end

Looks good. A couple of things to notice. First, the User model is simpler. It is no longer coupled to the GreetingMailer. In fact, removing or modifying the GreetingMailer does not require us to touch the User model. A side-effect of this is also that the WelcomeEmailObserver is reusable. It is no longer strictly coupled to the User. Secondly, we aren't messing with The Law of Demeter, the WelcomeEmailObserver is only referencing its own parameters and first level attributes of those parameters.

Fixing Our Tests

Now that we've moved things around, let's take a look at our tests. First we'll ensure that they pass and that we didn't break anything.

describe User do
  context 'welcome email' do
    let(:mail){ stub(:deliver) }

    it 'should send the simple welcome email if not purchased' do
      WelcomeMailer
        .should_receive(:welcome_email)
        .and_return(mail)
      User.any_instance.stub(:purchased_membership?).and_return(false)
      User.sham!
    end

    it 'should send the welcome and thanks email if purchased' do
      WelcomeMailer
        .should_receive(:welcome_and_thanks_email)
        .and_return(mail)
      User.any_instance.stub(:purchased_membership?).and_return(true)
      User.sham!
    end
  end
end

Once we've verified that the code works, we'll move the tests from our user_spec into our welcome_email_observer_spec. After all we're no longer really testing the User.

describe WelcomeEmailObserver do
  let(:mail){ stub(:deliver) }

  it 'should send the simple welcome email if not purchased' do
    WelcomeMailer
      .should_receive(:welcome_email)
      .and_return(mail)
    WelcomeEmailObserver.new
      .after_create(stub(:purchased_membership? => false))
  end

  it 'should send the welcome and thanks email if purchased' do
    WelcomeMailer
      .should_receive(:welcome_and_thanks_email)
      .and_return(mail)
    WelcomeEmailObserver.new
      .after_create(stub(:purchased_membership? => true))
  end
end

That's a great first step, we've definitely simplified our test and are no longer coupled to the User class.

Reducing Test Dependencies

While it's not immediately obvious, our first implementation actually strongly coupled all of our tests that create a User to the GreetingMailer. Any time we created a user we inadvertently also delivered a welcome email. Aside from creating a dependency between our tests and the success of GreetingMailer methods we've also incurred a great performance penalty, namely the code related to constructing and rendering the email.

While the rendering part may not apply to all of our tests it's easy to deduce that almost any callback will impose a similar and undesirable overhead. This simple performance penalty is really the main reason that so many Rails test suites take so long to run.

Let's take a look at a solution to this overhead and how we can simplify our test.

Improving Test Performance

First, we'll install the no_peeping_toms gem. This gem allows us to turn off ActiveRecord observers on a case-by-case basis. I prefer to implement it by adding the following to my spec helper file.

RSpec.configure do |config|
  config.before(:suite) do
    ActiveRecord::Observer.disable_observers
  end
end

If you already have observers that are being tested, this change will most likely causes a lot of your tests to fail until you enable the related observers on individual tests. But alone this change can result in a very noticeable performance boost.

The one thing we might be missing from this refactoring is a good integration test. While it can be argued that testing observer integration is really just testing Rails, we can make an argument that an integration test is actually simply testing that our observer is in fact at least observing the User model. In other words that our observer contains the line observer :user. So, if we are compelled to do so, we can also add an integration test to ensure this.

describe User do
  it 'should send a welcome email on create' do
    WelcomeEmailObserver.any_instance.should_receive(:after_create).once
    ActiveRecord::Observer.with_observers(:welcome_email_observer) do
      User.sham!
    end
  end
end

Hopefully this post has enlightened or convinced you of the use of observers and given you some insight into testing them and improving the performance and reliability of the remainder of your tests. I would like to continue writing some more posts about refactoring and testing improvements, so if you have any suggestions please email me or leave a comment.

Dynamic Rescue Clause in Ruby

2011-09-21T00:00:00-07:00

I recently finished Avdi Grimm's Exceptional Ruby. It's definitely a great introduction to Ruby exceptions and how you can begin using and handling them in your own projects and libraries.

While working through an example in his book I realized that Ruby 1.9.2 actually greatly improves the rescue clause functionality. For instance, in Ruby 1.8.7 you can not use a proc to handle exceptions.

#!/usr/bin/env ruby

def match_message(regexp)
  lambda{ |error| regexp === error.message }
end

begin
  raise StandardError, "Error message about a socket."
rescue match_message(/socket/) => error
  puts "Error #{error} matches /socket/; ignored."
end

Running this code in Ruby 1.8.7 produces a TypeError.

:9: class or module required for rescue clause (TypeError)

Running this code in Ruby 1.9.2 results in a properly handled exception.

Error StandardError matches /socket/; ignored.

To get this code to work in Ruby 1.8.7 you have to create a new module and modify the === function on that module to handle the exception.

#!/usr/bin/env ruby

def match_message(regexp)
  m = Module.new
  (class << m; self; end).instance_eval do
    define_method(:===) do |error|
      regexp === error.message
    end
  end
  m
end

begin
  raise StandardError, "Error message about a socket."
rescue match_message(/socket/) => error
  puts "Error #{error} matches /socket/; ignored."
end

Fortunately Ruby 1.9.2 entirely removes the module/class requirement for rescue clauses which results in a 27 times faster alternative when using a proc over the module metaprogramming approach. I personally have not used this style of dynamic error handler in my code, I prefer to subclass my exceptions, but if you are using another developer's library this might be your only option to safely catch multiple exceptions.

For the jRuby fans out there, this is a syntax which jRuby does not support. Even if you are running jRuby with the JRUBY_OPTS="--1.9" you'll have to use the 1.8.7 code.

You can view a gist of the examples in this post here.

Sham 1.0.0 Released!

2011-09-13T00:00:00-07:00

Today I released a new version of the Sham gem (version 1.0.0). This new version is not backwards compatible, hence the major version release. I highly recommend upgrading - this new version has some cool goodies included in it.

Sham::Config.activate! is faster - much faster.
Shams can be included individually. You no longer need to activate all shams, you can simply include a file with a Sham definition in it and that Sham will be available to you. You can also define Shams inline in your specs and tests.
The Sham definition syntax has changed, you can read more about this in the documentation.
There are a few more helper functions, namely the #empty function which allows you to create a simple empty Sham.

Here's a quick sample using the new definition syntax:

Sham.config(Item){ |c| c.empty }

Sham.config(Item, :small) do |c|
  c.attributes do
    { :weight => 10.0 }
  end
end

Sham.config(Item, :large) do |c|
  c.attributes do
    { :weight => 100.0 }
  end
end

Item.sham!
Item.sham!(:small, :quantity => 100)
Item.sham!(:large, :build, :quantity => 0)

Enjoy!

after_create bug in Rails 3

2011-08-17T00:00:00-07:00

I ran into this callback ordering issue while working on a Rails project recently and decided to investigate. After writing some test rails apps I am pretty confident this is a bug in Rails 3.0.x. If you believe otherwise, please leave a comment or explanation.

Taking a step back from the actual code, in an ORM (Object Relational Model) I would expect that callbacks from objects are consistent in their ordering. That is, if I have a callback that fires when an object has been saved to the database, that callback will fire at the same time (after the object has been saved to the database) regardless of any dependent objects or where the callback has been declared. Here's some code to demonstrate this:

class User
  after_create { puts 'after create user' }
  has_many :posts
end

class Post
  after_create { puts 'after create post' }
  belongs_to :user
end

user = User.new
user.posts.build
user.save
=> 'after commit user'
=> 'after commit post'

Simple, right? Almost. The problem in Rails is that the after_create callback doesn't exactly work that way. When it fires is actually dependent on the order in which it was defined. What's more - this isn't actually the case with the after destroy callback. Here's some code to demonstrate the issue:

class User
  after_create { puts 'User - first after create' }
  after_destroy { puts 'User - first after destroy' }

  has_many :posts, :dependent => :destroy

  after_create { puts 'User - second after create' }
  after_destroy { puts 'User - second after destroy' }
end

class Post
  after_create { puts 'Post - first after create' }
  after_destroy { puts 'Post - first after destroy' }

  belongs_to :user

  after_create { puts 'Post - second after create' }
  after_destroy { puts 'Post - second after destroy' }
end

user = User.new
user.posts.build
user.save
=> User - first after create
=> Post - first after create
=> Post - second after create
=> User - second after create

user.destroy
=> Post - first after destroy
=> Post - second after destroy
=> User - first after destroy
=> User - second after destroy

Take a second and look at that - do you see what's wrong? The User#after_create callbacks are called at opposite ends of the Post#after create callbacks, but the User#after_destroy callbacks are called at the end. What does this mean? Well basically that after_create is not actually called after the record is created (and saved to the database) - it's called "after the record is created and any dependent relations defined before it are also created". Now that just seems wrong to me. Especially when the after_destroy callback behaves in a different way.

It seems that the flow should work like this:

The user is saved.
The User#after_create callbacks are executed.
The post is saved
The Post#after_create callbacks are executed.

Especially when the opposite is true for after_destroy:

The post is destroyed.
The Post#after_destroy callbacks are executed.
The user is destroyed.
The User#after_destroy callbacks are executed.

The problem right now is that the User#after_create callbacks are being called in two places - once after the user is created (if they are defined before the has_many :posts declaration) and once after the Post#after_create callbacks are executed (if they are defined after the has_many :posts declaration).

If you're into TDD, you would check that the first case actually produces this result:

=> User - first after create
=> User - second after create
=> Post - first after create
=> Post - second after create

So why is this a problem? Why not just define all your callbacks before your relations? The first problem with this is that it is inconsistent - after destroy and after_create should behave in the same manner. The second problem is that your code might fail and it's not obvious why.

class User
  include Poster

  after_create :do_something
end

module Poster
  def self.included klass
    klass.class_eval do
      has_many :posts, :dependent => :destroy
    end
  end
end

While this example is trivial it illustrates how easy it is to mix up the order or definitions in Ruby. Unbeknownst to us, simply looking at the User class, this would actually cause our :do_something callback to be executed in the wrong order.

Avoiding Nested Callbacks in JavaScript

2011-06-30T00:00:00-07:00

I was reading through the Mastering Node E-Book, which is a great resource for getting started with node.js by the way, and I came across this code example.

var fs = require('fs');

fs.mkdir('./hello',0777,function(err){
  if (err) throw err;

  fs.writeFile('./hello/world.txt', 'Hello!', function(err){
    if (err) throw err;
    console.log('File created with contents: ');

    fs.readFile('./hello/world.txt', 'UTF-8', function(err, data){
      if (err) throw err;
      console.log(data);
    });
  });
});

This code creates a directory ./hello, then writes a file ./hello/world.txt, and then reads the contents of that file. All the filesystem operations are handled through callbacks - that's the way we do things in JavaScript. The example is quite simple and while it's specific to node, it illustrates a pretty common case of deeply nested callbacks in JavaScript. This set of instructions might be easy enough to read here, but it doesn't take a genius to figure out that these nested callbacks will eventually become a bit of a headache.

Fortunately there is a better way - named callbacks. Here is how the latter code can be transformed into a less-nested program.

var fs = require('fs');

fs.mkdir('./hello',0777,makeDirectory);

function makeDirectory(err){
  if (err) throw err;
  fs.writeFile('./hello/world.txt', 'Hello!', writeFile);
}

function writeFile(err){
  if (err) throw err;
  console.log('File created with contents: ');
  fs.readFile('./hello/world.txt', 'UTF-8', readFile);
}

function readFile(err, data){
  if (err) throw err;
  console.log(data);
}

Pretty cool, huh? Well, until you have two or three callbacks that respond differently to fs.readFile and you start running out of function names, right? Not to worry, you can create closures for your callbacks as well. For example, let's say we want to return the result of fs.writeFile, but we want to write two different files, namely ./hello.txt and ./world.txt. While this example is a little inane it demonstrates how easily the closures help to partition your code without having to rely solely on unnamed callbacks.

var fs = require('fs');

function writeHello(){
  return fs.writeFile('./hello.txt', 'Hello!', writeFile);

  function writeFile(err){
    if (err) throw err;
    console.log('Wrote hello.txt');
  }
}

function writeWorld(){
  return fs.writeFile('./world.txt', 'World!', writeFile);

  function writeFile(err){
    if (err) throw err;
    console.log('Wrote world.txt');
  }
}

writeHello();
writeWorld();

Wait, how did that work, didn't we return before defining that function? Yes, but not quite, in JavaScript any functions defined using function name(){} as opposed to using var are available immediately upon entering the function or program body respectively. That means that we can define named callbacks after our return statements and still have a linear, readable and less-nested program. You can find a gist of this code on github, enjoy!

A Faster YARD Doc Server with Guard-Yard

2011-06-21T00:00:00-07:00

I recently started using YARD to view my documented source code in an HTML format. I really like the syntax and navigation that the web interface provides, and I feel that having this nicely formatted display of documentation really pushes me to think more about commenting my code while developing. Starting the YARD server locally is as easy as installing the gem, generating the docs and starting the server.

But I found that it's a bit of a pain in the ass to actually view documentation while it's undergoing changes. While you can run the server in "reload" mode, this requires your entire documentation to be re-generated on every request, and I feel that it's wildly inefficient and slow. To remedy this problem I released a gem that integrates with guard to bring you a speedy and up-to-date YARD Documentation Server.

There are two major benefits to the guard-yard gem. The first is that it provides a consolidation of all file-monitoring processes to guard. This means you only need to run guard to keep your documentation server up-to-date and running. There is no need to configure the YARD server or re-processes your documentation on every request. The second benefit is that the docs are updated quickly. The guard-yard gem only modifies the documentation for files that have actually changed. This means that viewing your changes happens sooner!

Staring the YARD server via guard-yard is very easy:

Install guard, guard-yard, and yard gems.
Update your Guardfile using guard init yard.
Start the Guard server using guard start.

For additional details check out the gem page.

How Javascript Loading Works - DOMContentLoaded and OnLoad

2011-06-14T00:00:00-07:00

There are actually three different windows of time during a web request in which we can load and execute javascript. These windows are delineated by the DOMContentLoaded event and the OnLoad event. We can load our scripts before DOMContentLoaded, after DOMContentLoad, and after OnLoad. But before I dig into the details of how this is done, what the heck are these two events and when are they actually triggered?

The Events

The DOMContentLoaded event is triggered when the page's Document Object Model (DOM) is ready. What this means is that the API for interacting with the content, style and structure of a page is ready to receive requests from your code. In general this means that the text, css and html are ready - but it's not quite that simple. Depending on the browser, HTML version, and where the stylesheet tags are relative to the script tags, the CSS might not be done rendering by the time this event fires. Fortunately there is a safe and performant way to ensure that the CSS is loaded first. It's as simple as placing your external stylesheets in the header and your external javascripts in the footer. This approach forces the javascript to execute only after the external stylesheets and HTML body have been loaded. Basically, this means that if you are placing stylesheets in the header and javascripts in the footer, your DOM will load all content, structure, and style before the DOMContentLoaded event is fired. That's a good thing!

The OnLoad event is triggered when the entire page has loaded. This is moment when the globe or loader in your browser's title bar stops spinning. At this point all scripts and stylesheets have finished loading and have been executed, and all images have been downloaded and displayed.

Loading Javascript

So how do we hook into each of these events; and more importantly, when should we use each? To examine the different hooks, let's have a look at this simple HTML page and the output that it produces.

<!DOCTYPE html>
<body>
  <script src="jquery.min.js"></script>
  <script src="external.js"></script>
  <script type='text/javascript'>
    //<![CDATA[
      $.getScript('ajax.js');
      $(document).ready(function(){
        console.log('DOM Loaded (DOMContentLoaded)');
        $('head').append('<script src="dom.js"></script>');
        $.getScript('dom_ajax.js');
      });
      $(window).load(function(){
        console.log('Page Loaded (OnLoad)');
        $.getScript('page_ajax.js');
      });
    //]]>
  </script>
</body>

If we examine the javascript console we can see the list of events play out like this:

external.js Loaded and Executed
DOM Loaded (DOMContentLoaded)
dom.js Loaded and Executed
Page Loaded (OnLoad)
ajax.js Loaded and Executed
dom_ajax.js Loaded and Executed
page_ajax.js Loaded and Executed

This console output implicates the three windows of javascript loading:

External javascript (external.js) is loaded before DOMContentLoaded.
External javascript inserted into HTML during the DOMContentLoaded callback (dom.js) is loaded before OnLoad.
AJAX requests do not delay DOMContentLoaded or OnLoad, regardless of when they are initiated (ajax.js, dom_ajax.js and page_ajax.js).

Let's take a look at each of these windows and see which one best suits our needs.

Before DOMContentLoaded

First, notice that both inline javascript and external javascript (external.js) is loaded and executed immediately, regardless of the DOM being ready or the page having been loaded. This may seem obvious, but it is necessary to execute this code before DOMContentLoaded so that we can actually hook into the event. The best way to think of this javascript is as a setup. You shouldn't be manipulating the DOM yet, since it isn't ready - you should be setting up your callbacks and then getting out of the way so the rest of the page can load quickly.

After DOMContentLoaded

These scritpts (dom.js) are executed once the DOM is ready to be manipulated. This where you want to put any code that actually changes the page visually as this is the earliest point at which DOM manipulation is possible, and it will be executed before the page is displayed. This is a good thing because we don't want the page elements jumping around and changing style after they are visible to the user.

After OnLoad

These scripts (ajax requests and anything in the window.load callback) are executed after the page has completed loading and the user can view the entire document. This is a great place to load long-running scripts and scripts that don't visually alter the page. Some examples of this include analytics and reporting libraries and code or data that might be used later and can be pre-fetched for the user. Google maps loads all of its map tiles using AJAX so that the page load is not halted.

Takeaway

Try to keep this page load process in mind when writing your javascript code. Simple placement of your code can lead to large gains in overall perceived page load time. I'm hoping this blog post clarifies the basic page load events and allows you to beter determine how and where javascript should be loaded in your own application. If you have any questions or examples of these methods in use, please leave a comment.

Recap and Thoughts on the Facebook Deploy and Push Process

2011-05-27T00:00:00-07:00

Yesterday, Facebook gave a presentation on their front-end push/deploy process called Pushing Millions of Lines of Code Five Days a Week. Facebook releases code once a week, but they push new code daily. While few companies operate on this scale there is a lot we can learn from their "push culture" and the tools they use to promote a bug free deploy process.

Culture

There was a large focus on push culture. While tools are great and can help, they don't mean a thing if there isn't a company culture to support a bug-free and painless deploy process. Creating this culture early on is important, that includes ensuring that developers are on the hook for the changes they make. The further the developer is from the end product and release date, the less likely they are to be held accountable for their code. Pushers at Facebook ensure that a developer is responsible for their code that day, that week and even if they move to another group within Facebook.

Tools

For the geekier viewer the tools part is always fun - we love gadgets! Facebook has built a suite of monitoring tools they use internally to ease the push process. For the most part there are free, open-source and DIY alternatives to a lot of these tools. Facebook is at a much larger scale than many of us, so in my Take Away section I am going to focus more on what you can do to implement these tools and the culture they promote.

IRC Bots are used because all internal communication is done through IRC, so it's important to provide automated answers to some of the most commonly asked questions and avoid overwhelming the push engineers. For instance, if a developer wants to know, "will my revision be in this push?" they need only ask an IRC bot.
Automated Tests are critical because engineers are responsible for writing unit and selenium tests to support their code. These tests are automated and they are used to determine if a revision is ready for release.
Shadow Branches are pre-live versions of Facebook. Internally facebook.com points to latest.facebook.com - a version of facebook that is ready for release this week. These shadow branches ensure the "you are always testing" mentality and help ferret out bugs before the push begins and during the first phases of the deploy process.
Error Tracking is controlled via a data-centric internal dashboard that includes information on where the error happened and which developer is to blame for that line of code.
GateKeeper is a fancy tool that developers use to roll-out features to subsets of users. Subsets can include percentages of the population, males or females only, people with or without a certain affiliation or group membership and more.
Perflab is a tool for tracking performance changes on code branches and revisions and the impact your change has on the site.
HipHop is a compiled version of PHP that Facebook has built internally. It decreases server load by about 50% and turns facebook.com into a 1GB binary file rather than a collection of PHP files. Facebook compiles in about 10 minutes.
BitTorrent has been modified to distribute the HipHop binary to all clusters and within clusters of machines. It is used to roll out changes quickly and efficiently. A new Facebook build can be rolled out in about 15 minutes on all machines.
Push Karma is a tool that is used to privately track how reliable an individual developer is. It was emphasized that this is not a public shaming tool, it is simply used to determine the chance that an individual developer's changes might mess things up. If someone is more prone to bugs then a last minute change is less likely to be accepted into the release.
SVN/Git are the SCMs of choice. The central trunk repository is SVN, developers use Git for daily work and feature development.

Take Away

The main take away is that if you want to focus on a bug-free push culture it's important to get developers on the hook for their changes and bring them closer to the final product. The more QA and administration hurdles a piece of code has to go over, the less likely this is to happen. If a developer can see and use his or her changes they are more likely to feel responsible for them and they are more likely to catch bugs. The most important tools that promote this mentality are the shadow branch, error tracking and push karma. The other three tools that I believe are of great importance to a web-application company are TDD/BDD, GateKeeper and Performance Monitoring.

The Shadow-Branch

The easiest way to implement shadow-branch is to have a staging server. If you don't already do this, you should. Staging servers are a great way to ensure that the code you are releasing works in an environment that mimics production as closely as possible. This usually also means using a live or replicated version of a live dataset, an external url (even if it's only internally available), and replicating things like content-delivery networks and user access patterns.

Error Tracking

An error tracking tool is also critical, and it's important to dedicate someone on your team to track these errors. If you can't automate notifications so that individual developers are notified of a bug, it's important to designate one person the task of monitoring errors on your site. If you don't know there are errors, you aren't going to fix bugs.

Push Karma

Push Karma is not as critical and I wouldn't even recommend building a tool to automate this process, but it's important that someone is aware of who is introducing bugs and who is responsible for them. Some adult person needs to be responsible for determining if a developer has a higher chance of creating bugs, and if they have gone through the appropriate code-review processes and test driven development processes. This ensures that last minute changes and large releases are smoother. It's important to not publicly or privately shame people. I think creating a stressful environment around bugs is not healthy. You want developers that are creative, happy and not stressed about introducing bugs. But it is important to know who needs a little extra code review and a little bit more time to release a feature.

Test/Behavior Driven Development

I think the test automation speaks for itself. If you are developing a large web application you cannot manually test everything and you cannot determine all of the side-effects of your code. Test/Behavior Driven Development means you are ensuring your code will work now and later down the road when someone else makes a change. It's just common sense.

GateKeeper

I believe it's good to build features with an off-switch, if something goes wrong you want to be able to turn things off. Especially with the prevalence of companies hosting their applications in the cloud, it's important to think about how you can keep your site running when code or an external service doesn't work. Every feature should have an off switch that doesn't require bringing down your website and re-booting.

Performance Monitoring

Studies show that small (under one second) degrade in page performance can result in users walking away. If a user is on your website for entertainment, in other words for something other than checking their bank account, you need to ensure that your site is performant. Make sure you either test performance or monitor it with tools like NewRelic.

Spork, RSpec, Sham and Caching Classes

2011-05-18T00:00:00-07:00

This is my favorite testing trifecta:

RSpec as the testing framework.
Spork as the Distributed Ruby environment.
Sham to create model factories.

The problem I ran into recently though was that when running RSpec tests using Spork, changes to my models were not causing the files to re-load in the DRB environment. This was a big problem - it meant that the benefits provided by Spork were virtually non-existent because every file change forced me to re-boot my Spork server. To solve this problem I released a new version of Sham (0.5.0) and made some changes to my spec_helper.rb and test.rb files. Here is what I ended up with:

Ensure your Gemfile is up to date, and update your bundle.

group :test do
  gem 'rspec-rails'
  gem 'spork'
  gem 'sham'
end

Update test.rb to ensure that you aren't caching classes when using Spork.
```
config.cache_classes = !(ENV['DRB'] == 'true')
```

Modify your spec_helper.rb file, adding the Spork.each_run block.

require 'spork'

Spork.prefork do
  ...
end

Spork.each_run do
  ActiveSupport::Dependencies.clear
  ActiveRecord::Base.instantiate_observers
  Sham::Config.activate!
end if Spork.using_spork?

You're done! Here's a complete gist of the files.

The Spork.each_run block only gets run when you are actually in the Spork DRB environment, this ensures that you aren't unnecessarily re-loading your classes outside of Spork. The ActiveSupport::Dependencies.clear line re-loads all your models and controllers. The next line re-instantiates your observers. Finally the Sham::Config.activate! line reloads Sham so that you can easily create valid model objects. TDD/BDD in real-time is possible again!

ActiveRecord 3.1 - Mass-Assignment Roles

2011-05-12T00:00:00-07:00

Rails 3.1 introduces the concept of mass-assignment roles which allow you to specify accessible attributes for a specific permission or role. For instance, if you wanted your users to be able to edit their names, but not their permissions you could limit their accessible attributes but allow administrators to edit permissions:

class User < ActiveRecord::Base
  attr_accessible :name
  attr_accessible :name, :permission, :as => :admin
end

While this syntax is awesome the AR#new, AR#create and AR#update_attributes syntax is horrible:

User.create(params[:user], :as => :admin)

It seems like the same primitive syntax that AR#find used in Rails 2.x:

User.find(:all, :conditions => ['name = ?', params[:name]])

That was replaced with elegant chaining in Rails 3.0:

User.where('name = ?', name).all

Why not do the same with mass-assignment roles?

User.as(:admin).create(params[:user])

User.as(:admin).attributes(params[:user]).create

I can see the conflict this might cause with scopes, but maybe there's an elegant balance so that we aren't passing multiple hash parameters to AR#new, AR#create and AR#update_attributes.

Spork Testing Tip - Caching Classes

2011-05-09T00:00:00-07:00

Spork is a great gem that runs a distributed Ruby server which you can run your tests against. The benefit of using a DRb server is that you don't have to wait for Rails to boot up every time you want to run your RSpec and Cucumber tests, which means doing test or behavior driven development with these testing frameworks in Rails is actually possible because you can avoid the 30+ second boot times that often accompany large Rails applications. Generally to make Spork useful you need to not cache your classes in your testing environment by adding the following line to environments/test.rb:

config.cache_classes = false

The reason for this is that you want spork to re-load the related classes one every test-run, so that changes that you make to your classes are re-loaded rather than using a cached version from when Spork first booted. The downside of this approach is that you lose the cached class performance boost to your tests when you are running outside of the DRb server, for example in your continuous integration environment. To remedy this simply change the configuration line to:

config.cache_classes = ENV['DRB'] == 'true' ? false : true

Spork sets the ENV['DRB'] variable to true when it is booted, so this line lets you conditionally cache classes depending on if you are running in the spork distributed ruby environment. Happy testing!

UPDATE (June 1, 2011): I've written a more complete post on cached-classes, spork, sham and Rspec, which includes a gist of the complete spec_helper.rb file, test.rb file and Gemfile.

Proper Git Commit Messages and an Elegant Git History

2011-03-23T00:00:00-07:00

Have you ever run a git log command, looked at the output, and though, "WTF happened here?". I ran git log on a project yesterday and found myself staring at this git commit history:

commit 26ae8c3f0cf2a3d0b4aa52570711b2047b939592
Merge: 02f1f9a a73ecc6
Author: Programmer One 
Date:   Tue Mar 22 12:03:59 2011 -0700

    Merge branch 'master' of github.com:exampe/app

commit 02f1f9a146936373fcf8918036684ea209782a4b
Author: Programmer One 
Date:   Tue Mar 22 12:01:21 2011 -0700

    fix test

commit a73ecc678165284627f67eeede0eb43512422e71
Merge: f8cbf68 62cab2b
Author: Programmer Two 
Date:   Tue Mar 22 10:17:59 2011 -0700

    Merge branch 'master' of github.com:example/app

commit f8cbf680943e81aa57c5e09226f63817c09a13b5
Author: Programmer Two 
Date:   Tue Mar 22 10:16:53 2011 -0700

    added logged out user page

What am I supposed to do with that? This tells me nothing about what's happening on in the project. If I ran the same command six months from now I would have no idea what test "fix test" was referring to or what user related page "user page" was referring to. In fact I have no idea what any of this means now. Let's take a look at the issues with these commits and explore how we can improve them.

What's the big deal with git log anyways?

Joel Spolsky put it best when he said, "The difference between a tolerable programmer and a great programmer is not how many programming languages they know, and it’s not whether they prefer Python or Java. It’s whether they can communicate their ideas... By writing clear comments and technical specs, they let other programmers understand their code, which means other programmers can use and work with their code instead of rewriting it. Absent this, their code is worthless. By writing clear technical documentation for end users, they allow people to figure out what their code is supposed to do, which is the only way those users can see the value in their code."

This same concept applies to git commit messages and git history. The end user in this case is your fellow programmers and you a couple months later when you don't remember what you did to fix that bug. If you want to be a good programmer you'll write good commit messages that others can understand, that properly documents what you have done. One of the best skills you can learn as a programmer is to write about your code and to explain what you are doing. Git commit messages and code comments are the best place to start.

Merge and Rebase Commits

When your git log has as many merge commits as it does normal commits it means you need to start rebasing some of your smaller commits. If your project doesn't have any merge commits, it means you need to start using branches and merging your larger feature branch. Every project should have it's own guidelines around merging and rebasing.

Why should you merge and rebase? Because using both methods provides a clean history and it helps differentiate a large branch merge from a simple fix or patch. Merge commits should have their own comments and they should provide a history of milestones.

The difference between merging branches and rebasing branches really comes down to how you want to format your history. In my opinion when you have a small atomic fix that is a single commit it's always best to rebase. When you have a large feature history in a separate branch that you've been working on for a while it's always best to merge. Anything between these two is really up for grabs, but think about what you want to convey when you merge your code into another branch. Is the merge commit really as significant as your original commit?

What should I include in my commit message?

The three basic things to include are a summary or title, a detailed description, and a tracking or ticket number, if it's applicable. Here is a sample git commit with all that information:

Fixed bug where user can't signup.

Users were unable to register if they hadn't visited the plans
and pricing page because we expected that tracking
information to exist for the logs we create after a user
signs up.  I fixed this by adding a check to the logger
to ensure that if that information was not available
we weren't trying to write it.

[Fixes #2873942]

This is usually followed by the change information for this commit that is automatically generated by git. Because of the way git logging functions format commit messages and summaries, it's best to not exceed 50 characters for the title or summary line. This makes it easier to create summary logs and perform interactive merging and rebasing. It's also best to use about 72 characters for each line of the detailed description. The reason for a seemingly arbitrary 72 characters is that a typical terminal window is 80 characters and the commit message is indented about 4 characters. If you leave an additional 4 characters on the other side for nice formatting this leaves you with 72 characters per line.

The tracking or ticket number is optional, but most projects use some kind of tracking or bug management tool so it's nice to be able to keep track of projects and bugs with a ticket number. It's often possible to integrate git with your tracking utility as well, so that your commits appear in your project tickets.

Conclusion

As always, this is all up for some customization. There are no hard-and-fast rules about rebasing and merging, but be consistent. You and your fellow developers should agree on a set of rules for when to rebase and merge. There is no 100% correct git commit message template, but use something that makes sense for your project. The guidelines you develop should help produce both a good concise history and a good expanded history. The main thing to take away is that you should actively be thinking about your git history and how you can use it as a tool to understand a project's history. Treat it as an exercise in making yourself a better programmer.

Rails 3 Patch: Encoding Bug while Action Caching with MemCacheStore

2011-03-08T00:00:00-08:00

While upgrading an application from Rails 2.3 and Ruby 1.8.7 to Rails 3.0 and Ruby 1.9.2 I ran into the following error message:

Encoding::CompatibilityError: incompatible encoding regexp match
(ASCII-8BIT regexp with UTF-8 string)

After some searching around I was able to determine that this was a MemCacheStore issue. In particular it was caused when trying to perform action caching on a URL that contained special UTF-8 encoded characters, like the umlaut. As luck would have it someone else was experiencing this same issue, but their lighthouse ticket had been around for a good two months without any activity. I guess not too many Ruby on Rails developers are performing action caching on pages with UTF-8 encoded parameters. Anyways, I took it upon myself to create a patch to solve this issue.

The bug was located in ActiveSupport's mem_cache_store.rb on line 33, where the following regular expression was defined:

ESCAPE_KEY_CHARS = /[\x00-\x20%\x7F-\xFF]/

By itself this doesn't seem like much of an issue, but what you might not realize is that the Rails source code is encoded in US-ASCII, so when your UTF-8 encoded string is gsubbed with this regular expression, you get an Encoding::CompatibilityError on the first line of the ActiveSupport::Cache::MemCacheStrore#escape_key function:

def escape_key(key)
  key = key.to_s.gsub(ESCAPE_KEY_CHARS){|match| "%#{match.getbyte(0).to_s(16).upcase}"}
  key = "#{key[0, 213]}:md5:#{Digest::MD5.hexdigest(key)}" if key.size > 250
  key
end

My solution to this problem was to change the escape_key function so that it was compatible with any encoded string. I decided that the simplest solution would be to force the memcached key into the same encoding as the regular expression, but to do that I would need to duplicate the key string, because performing a force_encoding on a string actually changes the string in memory, it doesn't create a new string. If the string is not duplicated, the original key will get sent back to the Rails application with a different encoding and this can cause issues if you perform any concatenation with that string, since most of the strings in your app will probably be encoded in UTF-8. Try this out for yourself in you console:

irb(main):001:0> foo = "bar"
=> "bar"
irb(main):001:0> foo.encoding
=> #<Encoding:UTF-8>
irb(main):001:0> foo2 = foo.force_encoding(Encoding::ASCII_8BIT)
=> "bar"
irb(main):001:0> foo.object_id == foo2.object_id
=> true
irb(main):001:0> foo.encoding
=> #<Encoding:ASCII-8BIT>

Once the string was encoded in ASCII-8BIT, it was compatible with the regular expression and no error would be raised. Here's the final function I came up with:

def escape_key(key)
  # Fix for EncodedKeyCacheBehavior failing tests in caching_test.rb.
  key = key.to_s.dup
  if key.respond_to?(:encoding) && key.encoding != ESCAPE_KEY_CHARS.encoding
    key = key.force_encoding(ESCAPE_KEY_CHARS.encoding)
  end
  key = key.gsub(ESCAPE_KEY_CHARS){|match| "%#{match.getbyte(0).to_s(16).upcase}"}
  key = "#{key[0, 213]}:md5:#{Digest::MD5.hexdigest(key)}" if key.size > 250
  key
end

Please give my patch a try, it includes tests for all encoding types and special characters. If you get your tests passing please leave a comment on the lighthouse ticket that you were able to verify the fix.

Update - This patch has been merged into the rails branch as of April 28, 2011.

Hoptoad Error Tracking in Ruby on Rails

2011-02-25T00:00:00-08:00

What do you do with error messages from your production servers? I used to have them all sent to my inbox. I would then have to filter them based on if they were coming from delayed jobs or what part of the site they were relevant to. I would then have to sift through this massive email to find the stack-trace of the environment variables, it was just a pain in the ass. I recently started using Hoptoad to track my error messages. Life has been much easier since that decision. Hoptoad is an online application that acts as an API for receiving your error messages, as well as website for tracking, grouping and resolving those error messages.

The Hoptoad interface is quite simple: related errors are grouped together and you can 'resolve' them with a simple switch to indicate that you have taken care of the issue. This is awesome because you don't have to resolve 10 error messages when there's really only one thing that's going wrong. Hoptoad also tracks deployments and errors/minute for some additional fun stats. For organization it splits your error summary, stack trace, environment variables, request parameters, session variables and similar errors into separate tabs for easy navigation. Hoptoad is a paid service but they offer a free plan for a single project, single user and five error messages per minute so that you can try the service out, and their 'tadpole', or smallest paid, plan comes in at $5 per month.

My favorite gem to get started using Hoptoad is hoptoad_notifier. You can get detailed installation instructions here. My setup goes something like this:

Add the gem to my Gemfile.
```
gem 'hoptoad_notifier'
```
Install my bundle.
```
bundle install
```
Generate my api key yaml file.
```
rails generate hoptoad --api-key
```

Add a deploy notification to my after_restart script.

run "cd #{release_path} && rake hoptoad:deploy
TO=#{@configuration[:environment]}
REVISION=#{@configuration[:revision]}
REPO=#{@configuration[:repository]}"

Add a notification handler to my delayed jobs.

# in config/initializers/delayed_job_with_hoptoad.rb
class Delayed::Worker
  alias :handle_failed_job_without_hoptoad :handle_failed_job

  def handle_failed_job(job, error)
    HoptoadNotifier.notify(error)
    handle_failed_job_without_hoptoad(job, error)
  end
end

You're set to go. All errors in your app and in your delayed job processes will get posted to Hoptoad, and Hoptoad will track your deployments.

Scopes in Rails 3

2011-02-19T00:00:00-08:00

Scopes have become much more useful in Ruby on Rails 3 with the adoption of Arel into ActiveRecord. The first and most obvious benefit related to scopes is that everything is scoped! That means you no longer have to use find(...) on an ActiveRecord model and have a query immediately be executed. You can actually build queries up and only execute them when you use them. For example:

In Rails 2.x the find method worked like this:

User.find(
  :all,
  :joins => :profile,
  :conditions => ['profile.age = ?', 33])

This would then execute the query and find all users with age 33. In Rails 3.x this is much simpler:

User.joins(:profile).where('profile.age = ?', 33)

Not only is this more readable since you don't have to put your entire query into a single function call, but the main difference between these two commands is that the second doesn't execute a query. You actually have to call .all, .count, .each or .first to get the query to execute. What's great about that? Well, it means you can chain conditions together before you execute them:

query = User.joins(:profile).where('profile.age = ?', 33)
query.where('users.name = ?', name) unless name.nil?
query.where('profile.email = ?', email) unless email.nil?
query.all

This really shines when you combine it with scopes. For instance, if we want to simplify the above code, we can do the following:

class User
  scope :by_age, lambda do |age|
    joins(:profile).where('profile.age = ?', age) unless age.nil?
  end
  scope :by_name, lambda{ |name| where(name: name) unless name.nil? }
  scope :by_email, lambda do |email|
    joins(:profile).where('profile.email = ?', email) unless email.nil?
  end
end

User.by_age(33).by_name(params[:name]).by_email(params[:email]).all

Isn't that just awesome!? Not only is the code easier to read and understand, but it's re-usable and scoped! Here's one more example:

class Tag
  belongs_to :post
end

class Post
  has_many :tags
  belongs_to :user

  scope :tagged, lambda do |tag|
    joins(:tags).where('tags.name = ?', tag).group('posts.id')
  end
end

class User
  has_many :posts
end

# How many of my posts are tagged with 'ruby-on-rails'?
User.where('users.id = ?', 232423).posts.tagged('ruby-on-rails').count

Ruby Autoloading Explained

2011-01-17T00:00:00-08:00

In the ruby programming language a distinction is made between loading and autoloading. For those of you familiar with Rails, this will shed some light on the change from config.load_paths to config.autoload_paths in Rails 3. The difference lies in when the class is actually called/initialized.

Let's say I have defined a SpecialLibrary class:

# In special_library.rb
class SpecialLibrary
  puts "Loading Special Library"
  ...
end

When I load or require the class, the entire class code is run:

irb(main):001:0> require 'special_library'
Loading Special Library
=> true

When I autoload the class, the class code is only run when I actually use the class:

irb(main):001:0> autoload :SpecialLibrary, 'special_library'
=> nil
irb(main):002:0> SpecialLibrary.new
Loading Special Library
=> #

The reason config.load_paths was changed to config.autoload_paths in Rails 3 is because the paths defined here were being auto-loaded and not simply required, so config.autoload_paths is more descriptive and accurately describes the behavior.

Using Custom Error Messages for Cleaner Code

2011-01-03T00:00:00-08:00

Either because you dread 500 Error Messages or you're on some kind of JSON kick you find yourself writing this style of ruby code:

class Group
  def join user
    if !self.permitted?(user)
      {
        :success => false,
        :message => "You don't have the proper permissions to join this group."
      }
    elsif self.member?(user)
      {
        :success => false,
        :message => "You are already a member of this group."
      }
    else
      self.members.create :user => user
      { :success => true, :message => "Welcome to the Group!" }
    end
  end
end

class GroupsController < ApplicationController
  def join
    @group = Group.find(params[:id])
    response = @group.join(current_user)
    if response[:success]
      flash[:notice] = response[:message]
      redirect_to @group
    else
      flash[:error] = response[:message]
      render :action => 'request'
    end
  end
end

Why? Well because you don't want to risk throwing a Ruby Exception when it could cause a 500 Error Message for a user, and you want to be able to control error messages from your Model. For example, the join method could return any one of the following responses:

{ :success => false, :message => "You are already a member of this group." }
{
  :success => false,
  :message => "You don't have the proper permissions to join this group."
}
{ :success => true, :message => "Welcome to the Group!." }

So what's wrong with that? Well, there are a couple of things.

You introduced conditionals into your model and controller that are entirely unnecessary. Conditionals are not the worst thing in the world, but they create divergent code paths, they make your code harder to understand and harder to test. If you can avoid them, it's easier for the next programmer to figure out what you had in mind.

You created an additional dependency between your model and controller code. Dependencies lead to bad things. The more modular your code is the easier it is to test and the more versatile it is. It's easier to use modular code in background jobs and other objects and functions.

Thirdly, and most importantly, you are replicating a perfectly good programming concept known as Exceptions. Exceptions are a program's way of saying: something went wrong - something is not as it should be. Any they are great to use because they are easy to read and they break the normal code flow, which is what you would expect if something went wrong.

How do we fix this code? Simple: we use Exceptions.

Create the custom exceptions.

class Group
  module Error
    class Standard < StandardError; end

    class AlreadyAMember < Standard
      def message
        "You are already a member of this group."
      end
    end

    class NotPermittedToJoin < Standard
      def message
        "You don't have the proper permissions to join this group."
      end
    end
  end
end

Raise the exceptions.

class Group
  def join user
    raise Error::NotPermittedToJoin unless self.permitted?(user)
    raise Error::AlreadyAMember if self.member?(user)
    self.members.create :user => user
  end
end

Catch the exceptions.

class GroupsController < ApplicationController
  def join
    @group = Group.find(params[:id])
    @group.join current_user
    flash[:notice] = "Welcome to the Group!"
    redirect_to @group
  rescue Group::Error::Standard => exception
    flash[:error] = exception.message
    render :action => 'request'
  end
end

A gist of this code can be found here.

A Simple Ruby Script to Gracefully Terminate System Processes

2010-12-18T00:00:00-08:00

To gracefully terminate a system processes you issue the kill -15 command. You then usually monitor the process to ensure that it terminates. If it does not, you issue the kill -9 command to ensure that it does. Since this is a pretty common task, I've created a simple Ruby script to automate it.

#!/usr/bin/env ruby

require "timeout"

ARGV.each do |k|
    begin
        Process.kill("TERM", k.to_i)
        Timeout::timeout(30) do
            begin
              sleep 1
            end while !!(`ps -p #{k}`.match k)
        end
    rescue Timeout::Error
        Process.kill("KILL", k.to_i)
    end
end

This script will issue the kill -15 command, monitor the process for up to 30 seconds to make sure it terminates, and if it still exists, issue the kill -9 command.

gracefully-kill 1234

You can also call this script for multiple process ids:

gracefully-kill 1234 5678 91011

Sham - Lightweight Factories for Ruby on Rails Testing

2010-12-06T00:00:00-08:00

In general test-driven development in Rails couldn't be easier, but while writing your tests you'll eventually run into the case where you need to create a valid model so that you can test some extended functionality. Take a look at this RSpec test.

describe Cart do
  it "should be able to calculate the total price" do
    user = User.create \
      :username => "username@example.com",
      :password => "password", 
      :password_confirmation => "password"
    one = Item.create :name => "Item One", :price => 10.00, :weight => 1.0
    two = Item.create :name => "Item Two", :price => 20.00, :weight => 2.0
    cart = Cart.create :user => user
    li1 = LineItem.create :cart => cart, :item => one, :quantity => 1
    li2 = LineItem.create :cart => cart, :item => two, :quantity => 1
    cart.price.should == li1.price+li2.price
  end
end

The problem with this approach is that if you modify the validations on Item, Cart or User you'll end up going back and adjusting your test. For example, if you required that items had a quantity that was positive, you would have to go back and add a quantity parameter every time you create an item - major pain in the ass. Not to mention your tests start to become quite wordy.

one = Item.create :name => "Item One",
        :price => 10.00,
        :weight => 1.0,
        :quantity => 10

Fortunately there are a couple of gems that are available for creating factories. Factories are "minimally valid models" that can be reused to create valid models. I have experimented with a couple of options, like factory girl or machinist. But I ran into issues with sequences (sequentially generated attributes that are used to avoid validation errors) or I disliked the way the gems and factories were setup and used in my tests. This led me to create my own lightweight factory gem called sham.

Here's how you can get started using Sham.

1. Install the Sham Gem.

gem install sham

Or add it to your Gemfile:

gem "sham"

And re-install your Bundle:

bundle install

2a. If you are using RSpec or Test::Unit, enable Sham in your test.rb file.

config.after_initialize do
  Sham::Config.activate!
end

2b. If you are using Cucumber, enable Sham in your features/support/env.rb file.

require 'sham'
Sham::Config.activate!

3. Create factories with your default attributes.

# in sham/item_sham.rb
class Item::Sham
  def self.options
    { :quantity => 10, :weight => 1.0, :price => 10.0, :name => Sham.string! }
  end
end

# in sham/line_item_sham.rb
class LineItem::Sham
  def self.options
    { :quantity => 1, :item => Sham::Base.new(Item) }
  end
end

# in sham/cart_sham.rb
class Cart::Sham
  def self.options
    { :user => Sham::Base.new(User) }
  end
end

# in sham/user_sham.rb
class user::Sham
  def self.options
    {
      :username => "#{Sham.string!}@example.com",
      :password => "password",
      :password_confirmation => "password"
    }
  end
end

4. Write Your Tests.

describe Cart do
  it "should be able to calculate the total price" do
    cart = Cart.sham!
    li1 = LineItem.sham! :cart => cart
    li2 = LineItem.sham! :cart => cart
    cart.price.should == li1.price+li2.price
  end
end

Much Simpler! Here are a couple things to keep in mind:

You can override attributes.

You can override any attributes that have been defined in a sham. For example, either of the following is perfectly acceptable:

Item.sham! :quantity => 100
Item.sham!

You can create random strings.

Sham.string! will generates a random string that can be used as a filler or to avoid validation issues. For example if there is a requirement that your usernames are unique, you can use Sham.string! to avoid users having the same username.

Normally, this would fail because the two users would have the same username:

User.create :username => "username@example.com"
User.create :username => "username@example.com"

But, this will not fail because each username will be globally unique:

User.sham! :username => "#{Sham.string!}@example.com"
User.sham! :username => "#{Sham.string!}@example.com"

You can nest shams.

Sham::Base.new(User) tells Sham that when it's creating a model it should create a User sham if one is not specified. That means that either of these two calls is valid:

Cart.sham! :user => User.sham!(:username => "person@example.com")
Cart.sham!

Check out the documentation for additional examples and features. Feel free to file an issue if something is not working as you would expect. There's much more you can do with this gem and while I do plan on continuing to develop features, they will remain lightweight and simple.

To be or not to be... RESTful - Ruby on Rails Best Practices

2010-11-22T00:00:00-08:00

Ruby on Rails assumes you will be developing your controllers and views with a RESTful architecture in mind. What this means is that if you have a table named posts, a model named Post and a controller named PostsController you have some URLs automatically available to you.

HTTP GET /posts: List all the posts in the system.
HTTP GET /posts/[id]: Show me a specific post.
HTTP POST /posts: Create a new post.
HTTP PUT /posts/[id]: Update a specific post.
HTTP DELETE /posts/[id]: Remove a post.

In fact, creating the associated migration, model, controller and view is only a couple of scaffold generation commands away. Once you have this all setup though, you begin to ask yourself "What's so great about this?" Besides being easy to create all of these files and hook them up, what's the actual added benefit of structuring my code this way? This only becomes more apparent when you start adding additional functionality, for example you want an ArchivesController that lists your blog entries by date.

HTTP GET /archives: Show me the archived years.
HTTP GET /archives/[year]: Show me the archived months for a year.
HTTP GET /archives/[year]/[month]: Show me the posts for a specific month and year.

You realize that you don't want all the fancy PUT, POST, DELETE actions, and your URL structure is actually entirely different from the RESTful architecture. Do you still have to keep using REST? Should you restructure your ArchivesController to fit the RESTful architecture?

The short answer is no, you don't need to use REST. REST is great for some projects and some entities, but not for others. Generally it's also a good starting point, and more importantly it creates a basic protocol that you or other developers, in the case of an API, can depend on. It doesn't require you to remember what you need to do to create a new entity, you simply make a POST request. That said it's not the be-all-and-end-all of software architecture. For example, a web-site's marketing pages (/about, /tour, /pricing, /special-offer-december) usually do not fit well into a RESTful architecture, and additional functionality like search (HTTP Get /posts/search) or stats (HTTP Get /posts/stats) is often added in parallel to the RESTful architecture.

Before you get started, here are some questions you might want to ask yourself:

Does this controller/view deal primarily with an object/entity like a Post, Blog, User, Comment?
Are create, update, delete, edit and new actions all going to be available on the web?

As a guideline, if you answer YES to these two questions, then it's probably best to start with REST and expect that you will eventually use the architecture as a building block for additional actions and views you might want to perform. Otherwise, pick a URL that best represents what the action will show or do (/archives, /tour, /december-offer) and make sure you use the proper HTTP Protocols (GET for display, PUT for update, DELETE for removing and POST for creating).

FULLTEXT Index and SPATIAL Index - MySQL Compatible Rake Tasks

2010-11-21T00:00:00-08:00

Have you ever run a rake task such as db:test:prepare or db:schema:load only to be presented with an error about key length?

Mysql::Error: BLOB/TEXT column 'column-name' used in key specification
  without a key length:
CREATE INDEX index-name ON table-name (column-name)

Here are some examples I've encountered:

Mysql::Error: BLOB/TEXT column 'keywords' used in key specification
  without a key length:
CREATE INDEX fulltext_keywords ON article_search (keywords)

Mysql::Error: BLOB/TEXT column 'mbr' used in key specification
  without a key length:
CREATE INDEX spatial_mbr ON client_locations (mbr)

These errors are the result of the ActiveRecord::SchemaDumper class not being able to generate a schema.rb file with proper commands for creating FULLTEXT and SPATIAL MySQL indices. Fortunately Ruby is a dynamic language, and it's easy to adjust the way the schema.rb file is generated.

NOTE: This code assumes your FULLTEXT indices have "fulltext" in their name and that your SPATIAL indices have "spatial" in their name. If you don't follow this convention then you'll need to write your own regular expressions into the code.

Create a file called `sd_indexes.rb` in your `config/initializers` directory, and add the following code.

module ActiveRecord
  class SchemaDumper
    def indexes(table, stream)
      if (indexes = @connection.indexes(table)).any?
        add_index_statements = indexes.map do |index|

          if index.name =~ <strong>/fulltext/i</strong>
            "  execute \"CREATE FULLTEXT INDEX #{index.name} ON #{index.table} (#{index.columns.join(',')})\""
          elsif index.name =~ <strong>/spatial/i</strong>
            "  execute \"CREATE SPATIAL INDEX #{index.name} ON #{index.table} (#{index.columns.join(',')})\""
          else
            statment_parts = [('add_index ' + index.table.inspect)]
            statment_parts << index.columns.inspect
            statment_parts < ' + index.name.inspect)
            statment_parts < true' if index.unique

            '  ' + statment_parts.join(', ')
          end
        end

        stream.puts add_index_statements.sort.join("\n")
        stream.puts
      end
    end
  end
end

This code overrides the basic index creation of the `ActiveRecord::SchemaDumper` class. When the schema dumper encounters an index, before issuing the standard `add_index` command, it checks if the index name matches **/fulltext/i** or **/spatial/i**. If there is a match, the schema dumper issues a MySQL [CREATE INDEX](http://dev.mysql.com/doc/refman/5.5/en/create-index.html) command to create the FULLTEXT or SPATIAL index manually.

Generate the schema.rb file so that your changes take effect.
```
rake db:schema:dump
```
Create your test database from the newly generated schema.rb file.
```
rake db:test:prepare
```

The added benefit of overriding the ActiveRecord::SchemaDumper#indexes function is that the schema.rb file will be properly generated from now on. That means that the next time you run a migration you won't have to make any adjustments to the schema.rb file, and you won't have to re-run the db:schema:dump rake task.

Serving Static Files with Passenger and Nginx

2010-11-19T00:00:00-08:00

The X-Accel-Redirect HTTP header is used on Nginx servers to allow applications to serve static files directly from Nginx. For those familiar with X-Sendfile, X-Accel-Redirect is the Nginx implementation. Using this header to serve static files from a Rails application has some great benefits, like not requiring a file to be streamed through the Passenger and Rails stacks - which can be costly. The setup is quite simple.

Create a controller to serve the file.

class BlogsController < ApplicaitonController
  def show
    @blog = Blog.find(params[:id])
    response.headers["X-Accel-Redirect"] = @blog.path
    response.headers["Content-Type"] = "application/json"
    render :nothing => true
  end
end

Write an nginx location block into nginx.conf. server { ... location /blogs/ { internal; root /data; } }

Serve the file.

blog = Blog.create(:path => "/blogs/my_first_blog.json")
uri = URI.parse("http://localhost:3000/blogs/#{blog.id}.json")
Net::HTTP.get uri

The request goes something like this:

The HTTP GET request goes to Nginx, Passenger, and finally to your Rails BlogsConroller#show action.
The BlogsController#show action responds with two HTTP headers (X-Accel-Redirect and Content-Type).
The X-Accel-Redirect header is trapped on the return trip by Nginx.
Nginx matches the X-Accel-Redirect header with the location /blogs/ block.
Nginx serves the file directly from /data/blogs/my_first_blog.json.
Nginx also persists the Content-Type HTTP header and passes it to the client.

An additional benefit of this method is that you can use the controller to check file permissions. For example:

class BlogsController < ApplicaitonController
  def show
    @blog = Blog.find(params[:id])
    if current_user.can_access?(@blog)
      response.headers["X-Accel-Redirect"] = @blog.path
      response.headers["Content-Type"] = "application/json"
      render :nothing => true
    else
      redirect_to permission_denied
    end
  end
end

a.blog.about.code

Overwriting Console Output Using Curses in Ruby

Context Driven Refactoring

Question Code

Solution Code

Question One: Why should I refactor this code?

Step One: Establish a context for the refactoring.

Question Two: How am I going to test this code?

Step Two: Read/Write Tests.

Question Three: What did the tests teach us?

Step Three: Refactor Your Code

Actually Always Refactoring

Fixtures and Factories

Loading and Creating Data (Performance)

Sharing and Customizing Data (Maintainability)

Proximity of Data and Assertions (Ease of Use)

Rule of Thumb

Benchmark Your Bundle

Overwriting Console Output in Ruby

Respect the Active Record

Readability

Searchability

Changeability

Testability

Summary

Reducing Metadata Queries in Resque

Understanding The Law of Demeter

A Ruby Example

An Example of Complex Code

An Example of Unknown Output

Summary

From WordPress to GitHub Pages

The Ruby Global Interpreter Lock

I/O Operations

Ruby Operations

Summary

A Simple Rails Boot Time Improvement

The & Operator in Ruby

The Binary Ampersand

Bitwise AND

Set Intersection

Boolean AND

Custom Definitions

The Unary &

If object is a block, it converts the block to a simple proc.

If object is a Proc, it converts the object into a block while preserving

If object is not a Proc, it first calls #to_proc on the object and then converts it into a block.

Courteous Meta-Programming in Ruby

Object Oriented Ruby and Rails in the Complex World of Gems

A Simple Example

The Two Solutions

RSpec

Facebook and Twitter

ActiveRecord

Ruby 1.9.3-p0 Released

Using Rails Observers to write Faster Tests and Simpler Models

What's Wrong with This Callback?

Let's Refactor It

Fixing Our Tests

Reducing Test Dependencies

Improving Test Performance

Dynamic Rescue Clause in Ruby

Sham 1.0.0 Released!

after_create bug in Rails 3

Avoiding Nested Callbacks in JavaScript

A Faster YARD Doc Server with Guard-Yard

How Javascript Loading Works - DOMContentLoaded and OnLoad

The Events

Loading Javascript

Before DOMContentLoaded

After DOMContentLoaded

After OnLoad

Takeaway

Recap and Thoughts on the Facebook Deploy and Push Process

Culture

Tools

Take Away

The Shadow-Branch

Error Tracking

Push Karma

If object is not a Proc, it first calls `#to_proc` on the object and then converts it into a block.