Arto Bendiken on Rails

Opinionated Best Practices Elucidated

Arto — Thu, 26 Jul 2007 12:00:00 -0700

Railroad lines are, at any given moment, unidirectional. You can’t veer off the track that has been laid out in front of you, and you can’t make unscheduled stops unless you want the overnight express to run into you. These can be problems if you’re not altogether happy with the direction the track is taking you. Equally so if you would like to make any additional visits alongside the plotted course.

On the other hand, if you just keep your limbs inside the vehicle as advised, you can safely traverse the track at a very high velocity to arrive at your destination faster than with any other means of surface transport. You can watch the boring, mundane wasteland outside go by without needing to interact with it in any way.

So, riding the metaphor for all it is worth, when David Heinemeier Hansson took Ruby and created an opinionated, high-productivity web development framework, is it any surprise he named it Ruby on Rails? The name lays out the deal up front: our way, or the highway.

Could he possibly have been any more explicit? As I’m sure most readers know, DHH is anything but subtle. Yet people seem to keep missing the point.

A premise of Rails, and perhaps eventually its enduring legacy, is that constraints can be liberating, that a straightjacket for your mind can actually increase your productivity. As Hari Seldon might’ve put it: your freedom of action has been curtailed in order to guide you out of the Dark Ages along a well-planned, accelerated trajectory.

Instead of aiming to please everybody equally badly and making all things equally hard to do, Rails plays favorites. It’s software with an attitude, it’s politically incorrect, and as a result it is, of course, controversial. The goals themselves are simple, however: developer productivity and happiness (yes, happiness; motivation being a non-trivial component of productivity, if you can imagine such a state of affairs).

Take the little things: how should you organize and structure your project? How should you name your source files, class hierarchy, or your database tables and columns? What should your URL structure be like? What naming and coding conventions should you use in the code itself? If you’re honest with yourself, the answer to all the preceding is: who cares, as long as there is some sensible convention in the team. Rails gives you — or, rams down your throat, actually — sensible conventions that apply not only within your team, but across all Rails projects and the core Rails code itself.

What about documenting your code? What about writing unit and integration tests; should you do it, and if so, how? You betcha you should, and Rails makes sure you know how, and frowns on you if you don’t fulfill your end of the bargain. (Who knew writing tests could be so much fun? I guess when you take the suck out of something, the fun shines through.)

Think about it: how often have you spent minutes agonizing over how to name something in your project? From my own experience I’d venture that a programmer will need to make up to several hundred microdecisions like this during the course of a typical workday. These are precisely the kinds of trivial, yet time-wasting, aspects of development largely eliminated with Rails, if you follow its conventions. Add to that an iterative development approach using the very capable and dynamic programming language underpinning Rails, and the cost of changing your mind about anything — anything over which you still have veto power, of course — is so low as to hardly matter.

You may not agree with all of Rails’s conventions (I certainly don’t), but there are consequences to violating them. You’ll discover it is generally worthwhile to stick to them, or else to seek some other framework. These are not incidental matters as with most software, but key aspects of successful Rails usage. Remember, it’s the Rails way, or the highway (and the highway can get you there eventually, too).

Now, capable programmers tend to be control freaks, and geeks in general don’t like to have external authority enforced upon them, be it in politics or in technology. “It’s like herding wild cats,” confounded managers claim. This can mean that unless you happen to agree with most of the innumerable decisions that Rails does make for you and does hold you to, adopting the framework wholesale can be a bitter pill to swallow, at least initially. It truly puts to the test whether you care more about your ingrained habits and dearly-held opinions, or whether you can be railroaded into actually delivering the damn project (and perhaps learning something new while at it).

There’s good news, though. You may indeed have to let go of a little ego and micromanagement, but what you’ll gain will probably be worth it: there’s gold at the end of the rails. Provisionally accept the Rails way and gain some real proficiency with it, and you will complete the project faster than you thought.

Stick to the recommended practices and well-thought-out conventions, whether you agree with them or not, and your code will be well-organized and maintainable by other Rails programmers; as for being readily comprehensible or elegant, well, that’ll still be entirely up to you, of course. (Another aspect of this is that for a junior developer, learning and following Rails best practices and reading existing well-written Rails code can conceivably partly make up for the lack of some years of industry experience.)

There’s more. You won’t have to work in some painful legacy programming language that makes you dumber. In fact, you may actually start to enjoy programming again, perhaps even begin to indeed care about the work you do; rest assured this is not uncommon, and you don’t need to see the doctor about it. (You might ask your manager for a raise, though.)

If you’re currently working mostly with obsolete technologies and there’s no chance to use Rails there, you may be able to get back on the job market using Rails – there certainly never seem to be enough Rails programmers to satisfy demand.

You’ll probably eventually agree that a lot of the stuff you at first felt Rails shanghaied you into (say, test-driven development) are simply reasonable best practices you’ll want to adopt elsewhere, too, as appropriate.

For those of us who got onto Rails at the ground floor or close to it, all the aforementioned was admittedly a tad less painful. The adjustment process was gradual and addictive instead of abrupt and in-your-face. (I’ve written before about Ruby having been my gateway drug, getting me started on Norvig’s ten-year plan.) But there’s still a ways to go before the top floor, and plenty of room on board.

I think that in retrospect it is clear that on the development side of things, the emergence of Rails is the waterline that came to define Web 2.0. No post-Rails web development framework or toolkit can help but have been influenced by it; witness the explosion of Rails clones in programming languages left and right, and even spot on.

By now, most of the technical aspects of Rails have been disseminated, some even surpassed, to various degrees in the industry, and the playing field as a whole is better because of it. But perhaps more folks will yet realize that the “opinionated” part of the equation should be thought of not merely in terms of DHH’s demeanor, but as directly relevant to aesthetics and productivity.

To be soon continued with some thoughts regarding deficiencies in Rails, and what may lie beyond Rails or the next paradigm shift. And no, it won’t be airplanes. Perhaps seaside rockets will figure in there somewhere, though.

Spam Filters, Alien Technology and Ruby on Rails

Arto — Wed, 05 Jul 2006 14:19:00 -0700

When Paul Graham’s A Plan for Spam made its dramatic entrance into the anti-spam battle four years ago, it heralded the beginning of the end for spam — as we knew spam back then, anyway. Applying a simple statistical approach, based on word frequency analysis with a naive Bayesian classifier, Graham described how to create a spam filter accurate enough (99+%) that false positives effectively ceased to be an issue.

The central idea of the Graham Algorithm was quickly adopted en masse by spam filters, and as a result, the spam arms race has in the past few years tipped in favor of the good guys. “Successful” spam has devolved into exactly what Graham predicted it would: “some completely neutral text followed by a URL.” For me personally, the combination of good server- and client-side filters has made spam yesterday’s problem. (Well, that, and using Gmail as a front-end for lower-priority e-mail; spam all you want, it’s Google’s problem and they’re up to the task.)

Recently at $WORK, we’ve succeeded in applying similar text classification principles to another unrelated problem area, with the intent of forcing the computer to do the tedious job it was invented for, allowing us super-apes, in turn, to spend more time under a palm tree on the nearby beach, sipping tinto de verano and working out answers to deep existential questions, or whatever else it is that one does on the beach (note to self: need more practice).

The exact details of this covert project will have to await its escape into the wild, should it ever evolve the capability for that. For the time being, some of the technically more interesting tidbits will have to do as fodder for my ramblings.

First, as with most of our internal development, and an ever-increasing percentage of our client projects, this system was developed using the high-productivity Ruby on Rails framework, and reached the magical 0.1.0 mark (i.e., pre-alpha, but usable enough to solve many of the developers’ own needs) in less than a man-week of intensive coding (not to forget the skimming of a good number of research papers related to the subject).

However, to ensure a permanent gap on the competition (after all, it sometimes seems like half the world has already jumped onto Rails), we also pulled out the big guns: the top-secret alien technology known as Lisp.

(Don’t be fooled by the devious code name, intended to confound us earthlings with spurious ideas relating to speech defects — this is seriously powerful stuff: exposure is guaranteed to subtly but permanently alter your brain structure in ways not yet fully understood. In fact, the aliens have theorized that the Universe may actually be one giant Quantum Lisp machine, explaining how it is possible that lots of irritating, seemingly superfluous parentheses can act as magic incantations conveying an apparently inexhaustible power as per the principle of Clarke’s Third Law. But that’s neither here nor there for our present purposes.)

In our case, we simply integrated into the Rails system an interpreter for a subset of Scheme, a Lisp derivative; thus no doubt confirming Greenspun’s Tenth Rule once again. (Well, to be fair, the Lisp interpreter in question is only some few hundred lines of fairly elegant Ruby code.)

The system’s top-level classification and scoring algorithms are implemented in this Lisp subset, allowing us to easily fine-tune and try out new tweaks at runtime, and perhaps in the future letting us semi-automatically pit various competing implementations against each other in a manner not dissimilar to a genetic algorithm.

Due to the combined RAD-factors of Rails and Lisp, we quickly proceeded through a number of intermediate prototypes along the way, starting from a short-lived Ferret-based implementation, evolving to a hand-rolled tokenizer and SQL-backed corpora storage, and eventually ending up with the current version that delegates the content classification to a Dr. Strangelove-inspired piece of excellent software called the CRM114 Controllable Regex Mutilator.

(Speaking of CRM114, I was surprised to not find any existing Ruby bindings for it, and thus took the time to transcribe a previous Python wrapper into a Ruby version, to be released shortly.)

Anyway, the system appears to work more or less according to spec, but definitely still needs some more tweaking before embarking on world domination. For one thing, all this number crunching is, well, rather heavy (let’s just say it’s a good thing we have A/C in the server room where the development box is located). Although CRM114 itself is pretty light on its feet, we’re dealing with an exponentially growing data set, and the next challenge will be to put some checks on resource consumption.

So, for the time being, we’re not going to let the development box interact with our space systems department, to prevent any non-regulated growth or inadvertent contact with the aliens. More updates to follow as they happen.

Note to self: too many tinto veranos can make you forget there’s a fine line between tongue-in-cheek surrealism and plain-bad geek humor.

Update 2006/11/06: I’ve released the Ruby interface to CRM114 on RubyForge.