<3! — Chris Kampmeier on Ruby, Rails, and other delights

How to install MongoDB on OS X

2009-08-10T06:05:00Z

I started playing with MongoDB this weekend. It’s a cool little database, and John Nunemaker’s MongoMapper gem is a treat. Mongo’s maintainers are nice enough to provide pre-compiled binaries for OS X, but you still have to do a little setup and configuration. (There’s actually a portfile on MacPorts, but it wasn’t up-to-date with the latest version when I found it).

Here’s how I got the server installed and running as a daemon in OS X, for local development.

Download, unpack, and install the pre-compiled 64-bit binaries:

curl -O http://downloads.mongodb.org/osx/mongodb-osx-x86_64-1.0.1.tgz
tar xzf mongodb-osx-x86_64-1.0.1.tgz
sudo mv mongodb-osx-x86_64-1.0.1 /usr/local/mongodb
sudo mkdir /usr/local/mongodb_data /var/log/mongodb
sudo chown -R root /usr/local/mongodb

(If you’re on a 32-bit machine, substitute in i386 for each x86_64 above.)

Next, you’ll want to make a config file so you can change the server’s options without fiddling with command-line arguments.

Save as: /usr/local/mongodb/mongod.conf

# Store data alongside MongoDB instead of the default, /data/db/
dbpath = /usr/local/mongodb_data

# Only accept local connections
bind_ip = 127.0.0.1

Now, we’ll make a launchd job to register the server as an OS X daemon. launchd will start the server at startup, stop it before shutdown, make sure it stays up, and redirect its output to a nice log file.

Save as: /Library/LaunchDaemons/org.mongodb.mongod.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>org.mongodb.mongod</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/mongodb/bin/mongod</string>
    <string>run</string>
    <string>--config</string>
    <string>/usr/local/mongodb/mongod.conf</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
  <key>WorkingDirectory</key>
  <string>/usr/local/mongodb</string>
  <key>StandardErrorPath</key>
  <string>/var/log/mongodb/output.log</string>
  <key>StandardOutPath</key>
  <string>/var/log/mongodb/output.log</string>
</dict>
</plist>

Now we just need to load the launchd job:

sudo launchctl load /Library/LaunchDaemons/org.mongodb.mongod.plist

And that should do it! Try visiting http://localhost:28017 to see the status console for your database.

One last thing: you should probably stick /usr/local/mongodb/bin in your $PATH. That way you can use the other binaries that ship with MongoDB, like the mongo console, mongoexport, and so on.

Use a 503 for your Rails maintenance page

2009-07-26T06:38:00Z

Most of the docs out there about how to configure Apache for Capistrano’s deploy:web:disable task use a quick RewriteRule to the maintenance page.

That looks OK, but it doesn’t change the status code of the response: clients will usually get a 200 OK, indicating that your server is fine and that you’re sending the normal response. The correct status code to respond with is 503 Service Unavailable. With a 503, you’ll hopefully prevent search engines from indexing your maintenance page, make life easier on API consumers, enable AJAX requests to correctly deal with the site being down, and so on.

Here’s the configuration I’m using now:

# Show maintenance page if it exists
ErrorDocument 503 /system/maintenance.html
RewriteEngine On
RewriteCond %{REQUEST_URI} !\.(css|gif|jpg|png)$
RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$  -  [redirect=503,last]

About that suspicious looking redirect=503 flag, here’s an excerpt from the flags section of the mod_rewrite docs:

While this is typically used for redirects, any valid status code can be given here. If the status code is outside the redirect range (300-399), then the Substitution string is dropped and rewriting is stopped as if the L flag was used.

So technically speaking, you can give RewriteRule anything for that second argument, and the last flag doesn’t need to be specified because it’s automatically applied. I just find it more intention-revealing to use the path “-” and stick the last flag on there anyway.

How to prevent database contention in continuous integration

2009-05-10T02:49:00Z

We’ve used a few different continuous integration stacks for Rails over the last year at work—first CruiseControl.rb, which we found a little too complex to administer, then a custom bash script (which worked well, but took a lot of tweaking to get just right). When we eventually switched to git last year, we took the opportunity to try Integrity, a cute lil’ Sinatra app.

Integrity mostly “just works,” and it’s been a happy switch. One thing we lost in the move, though, was code that protected against resource contention when two builds are running at once. This is definitely a problem with Rails, since a typical database.yml tells Active Record to use the same database for all test runs. So you’ve got multiple builds hitting the database at once, dropping tables, creating records, and so on. Yikes.

Our old bash script used a filesystem lock and a queue to only run one build at a time, in order. In theory, this is the most sound approach, but hey—our build server has 8 cores and 16 GB of RAM, plenty of room for parallelism. During a pair-programming session this week with Jared Grippe, we decided that the best approach is to solve the contention issues and allow multiple simultaneous builds. We figured that’d keep the rapid feedback up-to-speed when the commits are flying in.

Stop putting code in that little “build script” box in Integrity’s configuration page. Instead, drop it in a rake task, so it’s versioned and kept safe.
In your build script, set a unique database name for the current build, and use ERB in the server’s database.yml to interpolate it in.
Have the build script run rake db:create and rake db:drop, so that your databases are created and cleaned-up automatically.

Here’s an example script:

desc "Run continuous integration suite"
task :build do
  ENV["RAILS_ENV"] = RAILS_ENV = "test"

  # use ERB in config/database.yml to make this the database name:
  # database: <%= ENV["DB_NAME"] %>
  now = Time.now.utc
  identifier = "#{Process.pid}#{now.to_i}#{now.usec}"
  ENV["DB_NAME"] = "myapp_test_#{identifier}"

  begin
    Rake::Task["db:create"].invoke
    Rake::Task["db:test:load"].invoke
    Rake::Task["default"].invoke
  ensure
    Rake::Task["db:drop"].invoke
  end
end

Appending the process ID and time in microseconds to the database name is about as unique as you can get, without generating a UUID or something. Note how we wrap the build in a block, and perform db:drop in the ensure section: that way, the database is removed even if the build fails (which would normally abort your rake task).

Keep in mind that the database might not be the only shared resource used by your build—watch out for filesystem use, in particular. You can probably use a similar strategy to solve that problem.

The steamroller

2009-02-19T07:24:00Z

One thing I do here and there at Rupture is interview Rubyists who have applied to work with us. The Mythical Man and his Month notwithstanding, we could use a few more pairs of able hands. You know, so I can work less and blog more.

Our interview process has two parts: pair programming, and then some mostly-unstructured talk about the non-code aspects of software projects. Today I had the pleasure of talking with Josh Ferguson, and we ended up in a conversation about Rails’ new direction. Josh was concerned that as Rails matures, it becomes less opinionated, and therefore less attractive to the people who identified with the movement in the first place.

I’m still thinking about that, but the whole thing reminded me of a note I submitted to Obie Fernandez for inclusion in The Rails Way. It’s been more than a year since his book was published, and I never got around to cross-posting here… Obie had asked for contributors to answer the question: “what does The Rails Way mean to you?”

There’s a line at the beginning of the SICP lectures—a famous course on functional programming—where Harold Abelson is defining “computer science” for his class:

“Computer science” is a terrible name for this business. First of all, it’s not a science; it might be engineering, or it might be art… It’s also not very much about computers, in the same sense that physics is not really about particle accelerators, and biology is not really about microscopes and Petri dishes, and geometry is not really about using surveying instruments.

The reason that we think that computer science is about computers is pretty much the same reason that the Egyptians thought that measuring their plots after the flooding of the Nile was about surveying instruments, and not geometry: when some field is just getting started, and you don’t really understand it very well, it’s very easy to confuse the essence of what you’re doing with the tools that you use.

In the same way, my sense of “the Rails way” is not really about Rails, Ruby, or any of these new tools we’re using. Its “essence” is an uncompromising drive to optimize for productivity and happiness. It’s a hard-learned pragmatism: processes for people who refuse to solve the same problem twice, who are annoyed enough by the speed bumps their tools sometimes introduce that they happily gas up the steamroller.

A necessary corollary to the idea that Rails’ secret sauce is distinct from the code frozen to our vendor directories is that one day, a better instantiation of these practices will come along. I love Ruby, Rails, and their communities, but I know that we’ll all move on at some point. When that day comes, and some new 10-minute screencast makes us squeal like kids, we should have the sense to jump into it head-first, with the same abandon with which we dropped all that stuff we used to do for a living.

Anyway, if you’re interested in the SICP lectures, they’re pretty much canonical. (Nathan Sobo introduced me to them, and made me think about functional programming in general, when we were living in Venice. I’m a lucky guy.) A good way to get started is the set of lecture videos recorded at HP in 1986 by Abelson and Sussman, who started the course. Their SICP book is available online for free, too.

irb tip: get last returned value

2008-06-02T19:44:00Z

Just use underscore.

1234	>> "Chunky bacon"=> "Chunky bacon">> _=> "Chunky bacon"

The fruits of an investigation I started when Jon Baudanza at Rupture offered a brownie bite for the first person to figure it out. I’ll do pretty much anything for a brownie bite…

Update: you can also use conf.last_value, as well as its aliases context.last_value and irb_context.last_value.

Testing your dependencies with RSpec

2008-02-04T04:30:00Z

I’m finding that managing my projects’ code dependencies is smelling worse and worse as time goes on. Code bases get bigger and acquire libraries as they grow; a part of your project sits untouched for a few months and its particulars leave your medium-term memory, and so on.

In Rails, we can freeze lots of stuff to our vendor directories. I do that as much as possible—gems that I only use for Rails apps get frozen to vendor/gems and then uninstalled system-wide; I use the gemsonrails plugin for this. If the little gem bits aren’t necessary, you can just pistonize a repository. Old news.

That’s not going to fly for platform-compiled gems, or even compiled libraries that aren’t gems at all (since you’re possibly running several different platforms between development and production). So I’ve been cooking up ways to keep myself sane:

The Simplest Thing That Could Possibly Work, I think, is just a quick test failure when a dependency is missing. If you’re already autotesting locally, and automatically running your test suite on each production machine as part of your deployment recipe, a quick, obvious exception could save you a little misery.

What I mean by “obvious”

This all came about because I went through two development platform switches recently: first, a clean install of Leopard, and just last week, a move to Intel from my old PowerBook. Both of those hosed my gems, and although I got test failures for each “broken” part of the app, certain libraries’ lazy/quiet-loading techniques don’t raise exceptions in a way that’s obvious.

For example, Rick Olson’s fantastic attachment_fu plugin is meant to work just fine for non-image files, so if you don’t have a compatible image processing library installed, it’ll just skip the thumbnailing for images and move right along. So my image-uploading tests failed on not creating the right number of files and records. It took me way too long to figure out what was going on, so I think it’d be better if I was checking for known dependencies directly.

First try

describe User do

  it "depends on one of three image processing libraries" do
    processors = %w(image_science RMagick mini_magick)
    lambda {
      begin
        require processors.shift
      rescue LoadError, MissingSourceFile => e
        retry if processors.any? or raise e, "Make sure an image processing library is available"
      end
    }.should_not raise_error
  end

end

Pretty good, although so much space between it and end makes me sad. Also, attachment_fu’s requirements are kind of an edge-case; I want to be able to spec a requirement for only one library, or several all at once.

Less sadness with matchers

Read up: if you aren’t using matchers, you aren’t using RSpec.

describe User do
  it "depends on an image processing library for attachment_fu" do
    one_of(:image_science, :RMagick, :mini_magick).should be_loadable
  end

  it "depends on SHA libraries for password hashing" do
    both_of('digest/sha1', 'digest/sha2').should be_loadable
  end
end

describe Event do
  it "depends on chronic for date/time string processing" do
    :chronic.should be_loadable
  end
end

describe Post do
  it "depends on a text processing library for Markdown support" do
    either_of(:maruku, :RedCloth).should be_loadable
  end

  it "depends on some XML libraries" do
    all_of(:hpricot, :builder, :haml).should be_loadable
  end
end

The matcher I wrote to do this is a little beefy, around 60 lines. To check it out, you can grab it from svn (or in the <3 warehouse), or from pastie.

I’m now using this all over the place, and it’s saved me at least a couple headaches. It’s really helpful for making sure your CI and deployment environments are up to spec, as well.

I’m sure there’s more to do—like checking gem versions. How are you checking your dependencies from platform to platform?

Update on Nov. 30, 2008: well, this was a useful experiment in writing matchers, but these days I’m just using Rails’ built-in config.gem tool. Highly recommended.

Operating Ruby-style

2007-08-06T04:00:00Z

At work I sometimes have occasion to write number-crunching C code, so I’m used to the & and | operators standing for bitwise AND and OR. I’m also a reformed PHP hacker (aren’t we all?), so I’m used to === meaning “equality without type coercion”. These definitions are only occasionally true in Ruby, because meaning depends on context: one thing I’ve come to appreciate is how many Ruby “operators” are just methods, and so they often have different implementations depending on the object receiving the method call.

Thing is, you gotta learn how to operate, Ruby-style.

Unusual operator expressions

& — In an integer context (Fixnum and Bignum), this is indeed a bitwise AND. But in a logical context, it’s just like && (logical AND), but without short-circuiting when the receiver (that is, the left operand) evaluates to false or nil. Both operands are always evaluated. This is because & is implemented as an instance method for the singletons TrueClass, FalseClass, and NilClass, and arguments to a method are evaluated when it’s called. (&&, on the other hand, is not a method. Like all the non-method operators and other punctuation, it’s picked up by the parser and handled in C. && is logically implemented in Ruby’s expression-evaluation function.)

Why would you want a logical AND without short-circuiting? Got me. In fact, I have a theory that the lack of short-circuiting is just a side-effect of the method-based implementation, and that the real reason for this operator to be defined for logical evaluation is to have a stricter AND that doesn’t allow duck typing in certain scenarios, since the receiver has to be true, false, or nil:
12
5 & true # => TypeError: can't convert true into Integer5 && true # => true
I pretty much just made that up, though, and I’m all ears if you have other ideas. It strikes me that all the obvious reasons to not want short-circuiting don’t cut it in this situation: wanting to ensure that the side-effect of the right operand happens seems very un-Ruby (say, opening a file or inline assignment—why force it into a logical expression?), and wanting to have arbitrary return values for the expression doesn’t actually work because & expressions always return true or false. It works for &&, though:
1234
# returns false or integer representation of privileges for userdef administrative_privilege_level is_admin? && privileges.to_iend
| — Same as &, but for OR.
^ — Exclusive OR: evaluates to true if exactly one of the operands is non-false and non-nil.

Since these are just methods, you can call them using the normal syntax. Take ==, another operator-as-method: evaluating Class == Class is the same as calling Class.==(Class), that is, passing the argument Class to the instance method == on the receiver, Class, which itself is an instance of type… Class. Amen.

A few more

=== — Used for checking, um, equaly-ness in case expressions. === is a synonym of == for many objects, but some will override === to create cool case idioms. For example, Range#=== evaluates to true if the argument is a member of the range, so you can do things like this:
1234567
def election_table last_initial = @last_name[0].chr.downcase case last_initial when 'a'..'m' then 1 when 'n'..'z' then 2 endend
Note that === is called on the when expressions, passing the case object as the argument. This is backwards from what you might assume at first glance, but it’s actually much more useful this way.

Take Module#===: it evaluates to true if the operand is an instance of the module or one of its descendants—essentially a backwards is_a?, something like is_class_of?. So you could use it in a case statement to classify objects by class. (You should have a good reason to do this, though. The benefits of duck-typing come from letting go of this explicit type-checking style.) Another is Regexp, where === evaluates to true when the operand matches. So we can classify strings by regular expression.
Regexp: ~ — a unary operator that is just like the more-common =~, except it always matches against $_. (The Pickaxe says: $_, a String, is “the last line read by Kernel#gets or Kernel#readline. Many string-related functions in the Kernel module operate on $_ by default. The variable is local to the current scope.”) Anyway, you can do this:
12
$_ = 'rebarbative barbarism'~ /barb/ # => 2
This leaves a bad taste in my mouth, though, like something shiny you find inside a mollusk. Add a few more layers of nacre, and skip the punctuation entirely using a Regex literal as a condition:
12
gets # reads until \n and stores input in $_return if /q(uit)?/ # operates on $_ magically
But that’s even worse. It’s deprecated now and you’ll get a warning if you try it. In my opinion, the only implicit receiver you should roll with is self.

String: % — Used for sprintfing with a single argument or Array of arguments, like this:

12	"%04x" % 61453 # => "f00d""%s%.2f" % ['$', 821.3333] # => "$821.33"

and, or, not — Not redefinable, but worth mentioning: these function identically to &&, ||, and !, but with super-low precedence. not has the next-higher precedence from and and or, which for some reason share the same precedence (&& beats out || usually). You can get into trouble with these:
123456
b = 3a = b or 5 # => 3, and a is set to 3b = falsea = b || 5 # => 5, and a is set to 5a = b or 5 # => 5, but a is set to false (!)
That happened because Ruby evaluated the expression as (a = b) or 5, since = has a higher precedence than or. I have to guess that this is just another feature imported from Perl that isn’t as idiomatically useful in Ruby.

Rolling your own

Now, just because many Ruby operators are implemented as methods (and can thus be overridden) doesn’t mean you can define new operators using whatever characters you want. The ability to call methods using operator syntax is hardcoded into the parser for the operators that are already defined (they’re tokens in the parser), so receiver operator argument and the unary version operator receiver won’t work. You can’t even use receiver.operator or receiver::operator, because the parser only recognizes that syntax with a name that follows the method naming conventions: a lowercase letter or underscore followed by upper- or lower-case letters, digits, or underscores, and optionally ending in !, ?, or =.

So, you can go and define all the new operators you want, but you’ll be using send to call them, since the parser won’t recognize the syntax.

class Hash
  define_method(:"<--") do
    inspect
  end
end

gadget = {:watch => :laser, :hat => :helicopter}
gadget <--               # => parse error
gadget.<--               # => parse error
gadget.send(:"<--")      # => "{:watch=>:laser, :hat=>:helicopter}"

Not much point to that. However, feel free to redefine the existing operator methods in your classes. Everything will just work like you’d expect, and your objects will be good Ruby citizens.

Update: check out Jay Phillips’ Superators gem, an interesting hack to add certain new operators to Ruby DSLs by overriding the existing binary and unary operators in sequence.

References and further reading

Table 18.4 from Programming Ruby — a nice summary of which operators are implemented as methods, sorted by precedence
How to Duck Type? — A very readable paper on approaches to duck-typing in Ruby
Logical operators in Perl and Ruby — All about and, or, and not
Input/output variable listing from Programming Ruby — $_ and friends

Thanks to Azita Mirzaian for the illustration.

Templated attributes in ActiveRecord

2007-07-30T17:33:00Z

On a recent project, we decided it would be nice to have “templated attributes” for certain fields in a form. A templated attribute has a helpful initial value—kind of like a default value—except that these aren’t valid data or saved in the database. They’re suggestions to the user about the expected formatting or content of a field.

So the game is:

keep these values out of the database
specify these values once in the model, super-duper-DRY
create a user experience that clearly implies that these values are just templates for valid data

Case one: a `website` attribute

We wanted a website attribute to be prefilled with http:// as a suggestion to the user about proper URL formatting—most users will just enter “example.com” when prompted for a URL. This is to avoid XSS problems with javascript: URIs and the like, and to keep browsers from resolving short URLs to be internal—nothing quite like being sent to http://yoursite.com/www.theirsite.com.

(Aside: there are better ways to handle this specific situation: a :before_validation callback to fix invalid URLs and check for XSS attacks, or the fantastic white_list plugin. But for now I’m interested in the general case.)

This attribute needed certain behavior:

When the user hits the form, the field should be pre-filled with http:// if the real value is empty or nil.
If the field is left as http://, we should convert it to nil before validation.
Client-side: to imply that the initial value is a suggestion, we’ll make the text color gray until the user makes a change. If the user’s only change is to empty the field, we should reset it to http:// and gray again on blur.

Case two: label attributes

There’s another use case, which you’ve seen before: a text field’s initial value is used as a replacement for its label. When the user clicks in the field, the “label” disappears. My example of this is a phone attribute, where we’d like to suggest a standard US area code format. Something like (123) 555-1234.

We don’t want the user to have to delete our dummy numbers and put in their own; it’s too much work. Instead we think that the reminder will help coax the right format out of the user by itself—so this field gets blanked on focus, unlike the website attribute.

You also see this pattern used for content suggestions instead of formatting hints: for example, search fields and login forms which are space-constrained, like the built-in search in Firefox and Safari.

Get on with the plugin, already

OK, OK. So we have two kinds of templated attributes: those with starting values, which are potentially the start of valid data, and labels, which are just helpful, ephemeral reminders.

Check out the goods:

1234	class User < ActiveRecord::Base templated_attribute :website, :starting_value => 'http://' templated_attribute :phone, :label => '(123) 555-1234'end

Validations work as expected, since unchanged template values get removed in a :before_validation callback. So you can sprinkle on a little :validates_presence_of and :validates_format_of for a really good time.

There’s also some nice, unobtrusive Javascript you can generate to get the behavior I mentioned above. If you’re using form_for, it’s totally automatic. It gets installed when you install the plugin, or you can install and remove manually with these rake tasks:

12	rake templated_attribute:installrake templated_attribute:remove

To turn off the Javascript for a given templated_attribute—say, because the generated stuff doesn’t jibe with your fancy-pantsy, AJAX-validating, Grey Poupon of a form—just throw :templated_javascript => false in the options hash for text_field or text_area. You’ll have to do any styling and event handling by yourself.

I’d like to make this work for fields other than text_field and text_area; the other contenders were file_field, which we can’t do because the Javascript security model doesn’t let us touch its value at runtime, and password_field, which I haven’t done because showing the template value would require dynamically switching the element to a text_field and back (to avoid all those asterisks). That one’s on the list, though.

Plugin resources

Browse the code:
http://railsplugins.code.shiftcommathree.com/browser/templated_attribute
Generated rdocs (with installation instructions):
http://docs.shiftcommathree.com/rails_plugins/templated_attribute
Raw svn action:
https://svn.shiftcommathree.com/rails_plugins/templated_attribute
Feed for svn changesets:
http://railsplugins.code.shiftcommathree.com/changesets.atom