No, I don’t work in NYC, DC, or the valley, and I’m cool with that.

Like many (if not most) of you, I’m regularly contacted by recruiters. Unsurprisingly, they generally haven’t learned anything specific about me, aside from what they find on my LinkedIn or GitHub profile. If they have visited this site, they certainly haven’t read the tagline. I’m getting tired of sending out what amounts to the same e-mail (though still probably more customized than the typical recruiter e-mail) over and over again, so I hope those of you who primarily read this blog for useful bits of Ruby info will forgive me this brief and selfish digression.

I am a software developer in Louisville, Kentucky, and I am really freaking happy here.

Optimizing for Happiness

A common theme in the development community I most readily identify with (the Ruby community) is one of optimizing for happiness. I’d like to think this is largely because the language itself makes developer happiness a design goal. Whatever the reason, in the same way that GitHub has optimized their company for happiness, and Ruby has optimized itself for happiness, I strive to optimize my daily life for happiness.

Duh. Everyone does that, you might think.

Maybe. As I’ve talked to other programmers, I get the distinct impression that if they are optimizing for happiness, they’re being a good deal less intentional about it.

Let’s get this out of the way: Being a programmer is one of the most singularly satifying careers in the world. We get to wake up every morning and create things, solve interesting problems, and, if we’re doing our jobs well, make people happy. AND WE CAN DO THIS FROM ANYWHERE IN THE WORLD. This means that we get to shape our lives and careers in ways that are impossible in many professions. If that doesn’t make you smile, please see your doctor (who, by the way, doesn’t get to have as much fun on his job as you do).

Anyway, back to shaping our lives. Since we can do our job from anywhere in the world, and only a company that is either ignorant or supremely distrustful of its employees would insist otherwise (Right? RIGHT?), we can live anywhere, and that choice impacts so many other areas of our lives that we should pick a place that works for us, and then sort out the rest.

So, why did I pick Louisville?

Louisville is more than horse racing and fried chicken.

We also have bourbon. ;)

But seriously, I first chose to relocate to Louisville when I was 24. I remember joking when considering the move: “I’m not sure why I’m doing this, all they’ve got there is horse racing and fried chicken.” I was incredibly wrong, and it’s only gotten better since then. By the way, you should totally come visit: among a number of other rankings and recognitions, Louisville was named the #1 US Travel Destination of 2013 by Lonely Planet.

Aside from the obvious horse racing and tons of great dining options, it’s got a number of other things you’d expect from a “respectable” city:

• A rich history.
• A great skyline
• Enjoy live performances? The Kentucky Center for the Performing Arts has you covered.
• Are museums your thing? There are plenty of them here. My personal favorite is the Frazier History Museum (formerly the Frazier Historical Arms Museum) which regularly has exhibits on loan from The Royal Armouries, Leeds.
• Into the night life? We’ve got lots of bars and clubs downtown.
• Hipster type? There’s a definite hipster/college-town vibe going on in the Highlands.

There’s more, but the general point I’m trying to make is that Louisville has what you’d expect from any great city, the difference being that you can drive about 15 minutes in almost any direction and be in a more rural setting. This is a huge pro in my book, because…

Louisville Helps Me Manage Complexity

Ask any competent programmer his or her thoughts on managing complexity in code, and you’re likely to receive a lengthy response. We have an instinctive and universally negative reaction to the introduction of unnecessary complexity in our code, but somehow fail to maintain those defenses in our day-to-day lives.

I grew up in a very rural town in Pennsylvania. I enjoy the quiet, and, though you wouldn’t know it if you met me on the street or at a conference, I’m an introvert. Louisville gives me the option of plenty of stuff to do, but plenty of peace and quiet when I want to recharge. I work remotely, but having experienced what we call “rush hour” here, compared to other cities, even if I did have to work in a local office, I wouldn’t lose much of my life to the commute (more on commutes in a second).

I think I’m one of the only people in the country who doesn’t hate his cable company. They provide me with really reliable and surprisingly-high-speed cable Internet access (50/5 asymmetrical) for what I consider a reasonable price. This makes working remotely a breeze. This is important, because working remotely is a key part of my personal optimization for happiness.

Time is Not Money…

It’s often said that “time is money,” but nothing could be more misleading. Time is most definitely not money. Some of us have much more of the latter than others, but everyone has the same 24 hours in their day. Therefore, I value my time, and I protect it. When I do decide to “waste” time, I try very hard to do so on my terms. Here, again, Louisville helps me.

I already mentioned that I crave peace and quiet to recharge. If having that place to recharge meant dealing with an hour-long commute each day, or long drives every time I wanted to do anything even remotely entertaining, I don’t think I’d make the trade.

Because Louisville is basically a small town with big city perks, I can engineer a simplicity in my daily life that lets me conserve my time and energy for things I find important. It all ties together nicely:

• Availability of extremely reliable, very fast Internet access makes working remotely a breeze, which eliminates a commute, giving me more time in my day, and allowing me to pick the location that makes my life simplest, instead of paying a mortgage to live in a location based on where my employer decided to do business.
• I bought a house near my stepdaughter’s school, so she can walk to and from school, and transportation for extracurriculars is less of a problem.
• The same house is 2 minutes away from my Koko FitClub. Since their program fits both weight and interval-based cardio training into 45 minutes, I can make a trip to the gym over lunch, and grab a protein smoothie before coming home. Because I’m working from home, until Google Hangouts support the olfactory senses, my coworkers needn’t suffer as a result of my choice. This single thing has been so key to increasing my quality of life over the past 4 months it can’t be overstated. Seriously. Find a way to make workouts work into your schedule, and if you have a Koko FitClub near you, check them out. Their system is exceptionally appealing to nerds like me who like seeing numbers. :)
• I’m about 10 minutes from my place of worship, which is also a very important factor in my overall quality of life, and certainly my peace.
• Louisville is the worldwide air hub for UPS. This means that everything I ship (or receive) is super-speedy for no extra cost, and makes online ordering even more of a no-brainer (which makes for less errands to run)

…But Money is Nice, Too

The really awesome thing about being a remote worker is that your cost of living is dependent on where you live, but your value to the company that’s hiring you isn’t dependent on that, at all. Living in Louisville lets my dollar go farther than any of the “usual places” recruiters are trying to convince me to relocate.

To be honest, it leaves me scratching my head at interactions like this one, an actual exchange with a recruiter from a well-known NYC-based firm:

Hi Ernie,

I work at **** in R&D.

I came across your profile today while searching for people with Ruby experience.

**** is always interested in speaking with top notch technologists.

I would like to schedule a quick call to discuss current and future opportunities here at ****. Can you provide some times and a phone number?

First, holy form-letter, Batman! But still, in the interest of education, I responded:

Would this position require relocation to NYC?

…and received this response:

Yes, we do provide relo package.

Right. Because the cost of moving would be what prevents me from taking a job in NYC.

Let’s assume for a moment here that I was anything less than completely satisfied with working for LivingSocial (I’m completely satisfied).

The cost of living in Louisville is 7.6% below national average. The cost of living in NYC is 123.8% above national average. In other words, I’d need to earn over twice as much money to maintain the same quality of life in NYC.

“So live in Jersey, and commute,” you say? See previous discussion about exercising control over how I spend my time. Also, Jersey is still significantly more expensive than Louisville.

Similar arguments against San Francisco, DC, and pretty much anywhere else that would require a lengthy commute to make the cost of living increase even bearable. Life’s too short to spend so much of it in between the places you truly want to be.

Intangibles

So, time and money are easy to quantify. Here are some other reasons I love working in Louisville:

• We have generally mild winters without completely missing out on the change of seasons. I grew up with snow and wouldn’t enjoy living somewhere it never snowed (my wife will disagree with me on this one).
• People are just nicer here. It’s good to live in a town where people still randomly say hello on the street. I don’t realize it’s odd until I travel and get strange “do I know you?” looks when I say “good morning” to someone.
• I used to be depressed by the lack of a strong Ruby community here, but with recent developments (OpenHack Louisville, Code Retreat Louisville, and even Code Louisville) I’ve started to see this as a chance to help grow a community from the start, and especially to melt the brains of some of the Java/.Net-centric folks around here. I think good things are afoot in the Louisville tech scene, and I’m psyched to play a part in them.

Be Intentional – and Be Your Own Product Manager

If you aren’t a recruiter (many of whom I will no doubt be sending to this page) but a programmer, and somehow got to this point without getting bored:

1. Congratulations!
2. Since you’re here, let me talk to you for a second.

Are you happy where you are right now? Or, do you want to move to one of those “it” places when a recruiter asks you to? Great! I’m not going to try to convince you that Louisville’s a better choice for you (but it totally is). I just hope you’re taking a hard look at the choices you’ve made and the way you’ve always looked at your career. Be intentional about your choices.

I’ll leave you with this:

I was simultaneously encouraged and frustrated when I read Chad Fowler’s The Passionate Programmer. Encouraged, because he was eloquently and succinctly summarizing some of the things I’d come to realize over 15 years in software development. Frustrated, because I wished that I had been able to give that book to my 20-year-old self and prevent him from spending a decade in the cable telecommunications industry, being the guy doing things in a decidedly non-enterprise way, and being seen as the enemy for embracing “agile” techniques before I had ever even heard the term.

In the book, Chad dedicates an entire section to investing in your “product,” that product being yourself and your career. For many of us who grew up with parents who encouraged us to work hard and keep our heads down, that’s a foreign concept. These are the lessons I wish I had taught my 20 year old self, so he could be a better Product Manager:

1. “Job security” is a myth created by big companies to trap you in a boring and unchallenging job. You will sign an employment-at-will agreement at every job you take.
2. Being good at what you do does ensure that when that day comes that you’re looking for work, you’ll find (or create) it in short order. That is job security.
3. Strive to feel stupid. If you aren’t regularly finding yourself uncomfortable with what you don’t know, then you aren’t pushing yourself hard enough. Don’t get comfortable. See #1.
4. Don’t take (or keep) a job because you like the people. If you’re a decent person, you’ll find people you like (and who like you) at any job you take. More importantly, if you really like them, you will want them to invest in their “product,” even if that means they won’t be working with you any more. Take the job that makes you excited to get out of bed in the morning. If it doesn’t exist, create it.

Whew! That was a long rant.

Yeah, longer than I thought it would be, actually. Thanks for sticking with me. We now return you to your regularly scheduled Ruby ramblings.

Modules are Inheritance

First, a quick reminder. Module inclusion in Ruby is inheritance. To demonstrate:

module Foo
  @@foo = 'foo'
end
 
class Bar
  include Foo
end
 
Bar.ancestors.include?(Foo) # => true

Module extension adds the module’s methods to the extending object’s singleton class. So, as you might expect:

class Baz
  extend Foo
end
 
Baz.singleton_class.ancestors.include?(Foo) # => true

This is because extending the Foo module on Baz is the same as:

Baz.singleton_class.send :include, Foo

When I say “the same,” I mean it. Here’s the relevant code from MRI (eval.c):

void
rb_extend_object(VALUE obj, VALUE module)
{
    rb_include_module(rb_singleton_class(obj), module);
}

So?

Yeah, you know all this. OK, fine. Did you expect this? I didn’t.

Bar.class_variable_get :@@foo # => "foo"
Baz.singleton_class.class_variable_get :@@foo
# => NameError: uninitialized class variable @@foo in Class

I did a bit more digging, and it would appear that rb_include_module (which, by the way, is known as Module#append_features in Ruby-land, in a clear attempt to screw with my brain) doesn’t actually append constants and class variables from a module to a singleton, but does add the module’s instance methods. This is from experimentation only — mentally parsing through the many #defines invoked in rb_include_module and include_class_new (which does the actual appending) was giving me a headache, and I suspect digging deeper would turn this from a Tidbit post into something more substantial, which I don’t have time to write up, anyway. I’d welcome explanation from someone with deeper insight into MRI internals, though! :)

Anyway, that’s it for now — hope you found this little tidbit as interesting as I did!

[Update] BUT WAIT, THERE’S MORE!!

So, I had a few minutes to play around this evening, and I decided to see how JRuby and Rubinius handled this situation. I’m glad I did! As it turns out, both JRuby and Rubinius do what I initially expected to see:

Baz.singleton_class.class_variable_get :@@foo # => "foo"

So, now I guess the question (in absence of an official Ruby spec) is whether or not this means the behavior in MRI is a bug.

Seriously. I’ll wait.

Anyway, at the very top of this list is a recommendation to extract value objects, and it got me thinking about a pattern I really like, which I wanted to share with you today.

In my day to day work, my value objects most often represent some special kind of string. A “slug” for a URL, a tag name, a specially formatted record name… whatever.

When that formatting logic needs to be reused, I’ve often seen code that places it into a mixin. Instead, I’ve taken to creating a value object by subclassing string, allowing simple reuse of the formatting logic anywhere in the codebase, and the ability to check the class of the object, to see if it’s been formatted. A couple of examples:

Aformentioned “slug”:

class Slug < String
 
  def initialize(object)
   object = object.to_s.downcase.gsub(/[^0-9a-z]+/, '-').gsub(/^-|-$/, '')
   super
  end
 
end

A formatted “record name” (name attribute on an ActiveRecord object) — I use a unique index on the column, so I want to make sure the records are formatted consistently to avoid differences in spacing (that people might not immediately notice) from causing success/failure:

class RecordName < String
 
  def initialize(object)
   object = object.to_s.strip.gsub(/\s+/, ' ')
   super
  end
 
end

Ruby treats String subclasses just like strings. Unlike the kinds of equality tests you might implement in your own objects, no check is first made that both classes are the same, and it doesn’t matter which side of the operator you put them on. So:

Slug.new("Unformatted Text!") == 'unformatted-text' # => true
'unformatted-text' == Slug.new("Unformatted Text!") # => true

Now, you can check that the string you’re receiving has been formatted elsewhere in the codebase without a regexp, and format it if needed:

str = Slug.new(str) unless str.is_a? Slug

If you’re concerned about mutability, you can always freeze in the initializer — I prefer to adhere to a contract that I won’t mutate these objects. Or, with some additional work, the mutator methods could be updated to support the formatting requirements, too.

Now, a final example that yields somewhat mixed results: a refactoring of the Rating class in the article at Code Climate.

A case could be made that a Rating is, in fact, a special kind of string. It has a value from A-F, allows comparison against other things in string form, hashes like its string value, etc. The comparison order being reversed, however, removes much of the benefit of “being a string,” since we can’t use strings and ratings on either side of all operators, as above.

class Rating < String
 
  def initialize(object)
    object = cost_to_rating(object) if Numeric === object
    object = object.to_s.upcase
    raise ArgumentError, "Ratings are A-F!" unless ('A'..'F').include? object
    super
  end
 
  def <=>(other)
    val = super and -val
  end
  alias better_than? >
  alias worse_than? <
 
  private
 
  def cost_to_rating(cost)
    case cost
    when 0..2
      'A'
    when 2..4
      'B'
    when 4..8
      'C'
    when 8..16
      'D'
    else
      'F'
    end
  end
 
end

To be clear, if your object differs significantly enough from a string (as above) then you probably want to reconsider using this pattern. For quick and easily reusable formatting consistency that fits your domain model, however? The original value object, String, is hard to beat.

[Update] To be even clearer, this pattern is for value objects — that is, you shouldn’t be changing them. As nicholaides correctly points out in the comments below, Ruby does some voodoo to optimize certain string operations. Don’t expect your custom initializer to be called on objects instantiated as a result of things like String#gsub.

Let’s say you want to enforce some kind of parameter formatting on an inherited method. You probably already know that you can write something like:

def my_nifty_method(value)
  super(nifty_parameter_formatter(value))
end

But, did you know that you can also do this?

def my_nifty_method(value)
  value = nifty_parameter_formatter(value)
  super
end

I didn’t, until I just gave it a try this evening, for giggles. I figured it might work for methods that mutate an object, like Array#unshift, but since variable assignment is creating a new reference, I just assumed I’d need to explicitly pass in my modified value. Chalk me up as pleasantly surprised! Since I don’t ever remember seeing this information before, and a few quick Google searches turned up empty, I thought I’d share.

[Update: I've been informed this specific behavior was one of the topics covered in James Edward Gray II's (aptly named) talk, "10 Things You Didn't Know Ruby Could Do". Worth a look for some other nifty tricks!]

Code: here and here
Slides: here

QUICK! What’s the output of the following code?

1
2
3
4
5
6
7
8
9
10
11
12
13
14

#!/usr/bin/env ruby
 
class LineDemo
  def self.process(text, lineno)
    # do important processing on text
    puts lineno
  end
end
 
LineDemo.process <<TEXT, __LINE__
Here is some text on which we will do some important processing.
Here, also, is some important text to process.
TEXT
# All done!

The answer is obvious. Of course, it prints 13.

Wait, what?

Yes, it prints 13. Why not 10? Because Ruby does interesting things when parsing heredocs.

When you pass a heredoc to a method as a parameter, Ruby will, in general, stop what it’s doing, read through the file until it encounters the token that terminates the heredoc (TEXT, in our case), and then pick up where it left off.

Sort of.

You see, by passing __LINE__ to our method, we asked Ruby to give us the value of ruby_sourceline (see parse.y in the Ruby source tree). It just so happens that this value is equal to the line that contains the token which ends the heredoc. Note that I added a comment at the end of the script — interestingly, if Ruby encounters the end of the file on the same line that terminates the heredoc, __LINE__ returns 10, as you might have expected, and of course if the heredoc came after the __LINE__ keyword, then the line number wouldn’t be incorrect, either. This only occurs if the heredoc comes before __LINE__.

However, that problematic ordering of parameters is necessary in a very common use case for __LINE__, namely, evals, such as instance_eval:

instance_eval(string [, filename [, lineno]] ) → obj
instance_eval {| | block } → obj
Evaluates a string containing Ruby source code, or the given block, within the context of the receiver (obj). In order to set the context, the variable self is set to obj while the code is executing, giving the code access to obj’s instance variables. In the version of instance_eval that takes a String, the optional second and third parameters supply a filename and starting line number that are used when reporting compilation errors.

What’s a Rubyist to do? Trick Ruby’s parser into evaluating __LINE__ before things get all wacky-like. It just so happens that doing some math against __LINE__ will do so, since the computation will need to take place before the method is dispatched. This is why you will commonly see code like this in Rails:

activemodel/lib/active_model/callbacks.rb:

def _define_before_model_callback(klass, callback) #:nodoc:
  klass.class_eval <<-CALLBACK, __FILE__, __LINE__ + 1
    def self.before_#{callback}(*args, &block)
      set_callback(:#{callback}, :before, *args, &block)
    end
  CALLBACK
end

The operation forces the line number to be computed first, and the + 1 tells class_eval that while evaluating, line numbering starts at the line just below the method call, which is the first line of heredoc code. You could call any method against __LINE__, such as __LINE__.to_i, and get the first behavior. It just so happens that in the typical eval use case, we’ll want the addition anyway, so that’s what we’ll normally use.

Tying it all together

To summarize: here’s an example script that demonstrates the different behaviors.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

#!/usr/bin/env ruby
 
class LineNumber
  def self.before lineno, param
    puts "line number is #{lineno}"
  end
 
  def self.after param, lineno
    puts "line number is #{lineno}"
  end
end
 
print "before with __LINE__: "
LineNumber.before __LINE__, <<TEXT
This is some text.
As is this.
This, also, is some text.
TEXT
 
print "after with __LINE__: "
LineNumber.after <<TEXT, __LINE__
This is some text.
As is this.
This, also, is some text.
TEXT
 
print "after with __LINE__ + 0: "
LineNumber.after <<TEXT, __LINE__ + 0
This is some text.
As is this.
This, also, is some text.
TEXT
# fin

The output of this script:

before with __LINE__: line number is 14
after with __LINE__: line number is 25
after with __LINE__ + 0: line number is 28

That concludes this look at an interesting and odd little corner of Ruby. Now, the next time you tack on “+ 1″ to an eval’s __LINE__ parameter, you’ll know exactly why you’re doing so.

1.0.3 – 1.0.9, summarized

Bug fixes. That’s been just about it, really. Many of them are related to an ongoing battle I’ve had with ActiveRecord::Relation#merge. Oh man, how I hate that method. I’ve had a draft queued up for a while about that, actually. I’ll get around to polishing it up one day.

Anyway, I believe I’ve currently arrived at a solution that’s working as well as can be hoped for. With that out of the way, what else is new?

1.0.11

Here’s what’s new as of 1.0.11:

• Resolve issues when joining the same table twice in two different scopes (#157)
• Allow predicates in order and select clauses. (#128)
• Support Squeel syntax in Relation#from (#155)

There’s a bit of behind the scenes work that’s been done to support some of these changes, that should allow for easier support of one-off behavior in specific clauses in the future. Namely, each clause now has its own Visitor class. This was really an oversight in the way I modeled things initially. I made a naive assumption that there wer clauses that would contain predicates, and clauses that would contain attributes (or attribute-likes), with no overlap. That’s just not the case. As a side effect of this work, I finally had the impetus I needed to clean house in the visitors, and removed a bunch of duplicated code. The result is a cleaner and easier to understand codebase.

What’s in store for the future?

Squeel 2.0 will have some changes in store. Here are a couple, with more to come later.

1. Sifters will better coexist with scopes. Currently, I’m planning to have a sifter definition create a scope of the same name (either automatically or with an option, undecided as of yet). The sifter will be prefixed with “sift_” in its method name, and the sift syntax within the DSL will take this into account when referencing sifters.In current versions of Squeel, a sifter and a scope with the same name cannot coexist.
2. Core extensions will be removed. The symbol and hash extensions in Squeel 1.0 were provided as a convenience for those migrating from MetaWhere. A quick check of RubyGems.org shows that Squeel has surpassed MetaWhere in current usage, which means that the time is right to remove support for this stopgap measure. As of 1.1.0, loading these extensions will trigger a deprecation notice.

As both of the changes outlined above will result in modifications to the public-facing API of Squeel, they’ll only happen in version 2.0. You can safely continue to use Squeel 1.x without being concerned about a backport of these changes causing breakage.

Egalitarianism (from French égal, meaning “equal”) is a trend of thought that favors equality among living entities.
–Wikipedia

So, I may have taken a bit of poetic licence with this post’s title. A completely egalitarian ObjectSpace would be insane. We don’t want all of our objects to be equal, but we do generally want the ability to recognize when two objects are equal. Today, I’d like to share a story about an ActiveRecord refactoring I worked on over the weekend, and how it relates to the topic.

Relation merges and equality conditions

When you merge two ActiveRecord::Relation objects, ActiveRecord makes a few decisions on your behalf. One of them involves what to do if both relations have equality conditions against the same attribute. When you pass a hash of column/value pairs to Relation#where, ActiveRecord’s PredicateBuilder creates ARel Equality nodes (or In nodes, in the case of an array or range value). Here’s an example (altered for legibility/brevity):

relation = Person.where(:name => 'Bob').where_values
# => [
#      #<Arel::Nodes::Equality:0x1075fbfb0
#      @left=#<struct Arel::Attributes::String relation=#<Arel::Table (...)>,
#      @right="Bob", (...)
#    ]

When you merge two relations, ActiveRecord normally just appends the where_values of the second relation to those of the first relation, which results in all of the conditions being joined by AND in the resulting SQL. But what happens if we next write:

relation = relation.merge(Person.where(:name => 'Jim'))

Well, generating a query that selects all people whose name is Bob and whose name is Jim makes no sense, since both conditions can’t be true. As you probably already know, ActiveRecord uses the last equality node. In this case, that means we get all people whose name is Jim.

Here’s the code, straight from ActiveRecord that made this work:

lib/active_record/relation/merger.rb (from 8f58d6e5 in master):

unless relation.where_values.empty?
  # Remove duplicates, last one wins.
  seen = Hash.new { |h,table| h[table] = {} }
  merged_wheres = merged_wheres.reverse.reject { |w|
    nuke = false
    if w.respond_to?(:operator) && w.operator == :==
      name              = w.left.name
      table             = w.left.relation.name
      nuke              = seen[table][name]
      seen[table][name] = true
    end
    nuke
  }.reverse
end

This code reverses the order of the combined where_values, and if any of the objects quack like an Equality node, it notes the ARel Attribute on the left hand side has been seen, so it can discard any later Equality nodes. When the array is again reversed, this has the effect of keeping only the last occurrence of each Equality condition against a specific attribute.

Problem?

Yes, there’s a problem here. It makes sense to only do this for Equality nodes, but what happens if the left hand side of the Equality node isn’t an attribute? What if it’s a function, instead? This happens:

undefined method `relation’ for #<Arel::Nodes::NamedFunction>

This is because once we’ve detected an Equality node, we make an assumption that the left hand of that node is an ARel Attribute. That’s unfortunate, as it’s unnecessarily limiting. As a quick fix, we can tweak things like so, to prevent an exception being raised:

# We might have non-attributes on the left side of equality nodes,
# so we need to make sure they quack like an attribute.
if w.respond_to?(:operator) && w.operator == :== &&
  w.left.respond_to?(:relation)
  name              = w.left.name
  table             = w.left.relation.name
  nuke              = seen[table][name]
  seen[table][name] = true
end

But, as you can probably imagine, that’s only a partial fix. Now, we’ll ignore that Equality node altogether, meaning that we could end up with two conflicting equality conditions against identical functions. Far from ideal.

Refactoring for fun and profit

So, it’d be really great if we could just satisfy the obvious intent of the code in a more straightforward manner: Given an array of where conditions, we want to keep only the last equality condition for each left-hand value. Wouldn’t it be nice if we didn’t need to know anything at all about what that left-hand object was, and could just write:

# Remove equalities with duplicated left-hand. Last one wins.
seen = {}
merged_wheres = merged_wheres.reverse.reject { |w|
  nuke = false
  if w.respond_to?(:operator) && w.operator == :==
    nuke         = seen[w.left]
    seen[w.left] = true
  end
  nuke
}.reverse

Good news: We can! But first, we have to do some work. :(

What constitutes equality?

The answer to that question depends on who’s asking. And, for that matter, how they ask it. Oh, and also, what your object considers an equal.

The default Ruby implementation of object equality is extremely narrow. To demonstrate:

class MyNiftyObject # < Object is implied
  def initialize(value)
    @value = value
  end
end
 
MyNiftyObject.new('hi') == MyNiftyObject.new('hi') # => false

What? This is because the Object class has one concept of equality: two Objects are equal if and only if they refer to the exact same instance in memory. We created two instances of MyNiftyObject, and just because they happen to have the same values doesn’t mean they’re “equal” to Ruby.

Think about it: how else could you really compare two instances of the Object class, which has no attributes? Ruby leaves it up to the subclasses of Object to implement the various equality checks in ways that make sense for them, but it provides some guidelines. Here’s a quick rundown, by decreasing strictness:

• equal? – Shouldn’t be overridden. Returns true if both objects are literally the same object (the same in-memory instance)
• eql? – Should return true if both objects have the same value. This is normally synonymous with ==, but there can be subtle differences. Could be more accurately described by saying “objects have equal values, and are also the same class”. For instance, Numerics like Float and Fixnum return false when compared with 1.eql?(1.0)
• == – Should return true if the objects have the same values. Again, except in certain odd cases, synonymous with eql?
• ===? – More commonly referred to as “case equality”. Normally the same as == but often overridden to provide useful behavior in case statements (each when value will call its === method to compare with the value being checked). For instance, Class#===(object) behaves like object.is_a?(Class). Note that other methods, like Enumerable#grep may call this as well.

So, with these considerations in mind, your first instinct might reasonably be to try defining the ==(other) method on your objects. HAH! Your first instinct would be WRONG! But, for giggles, let’s try it.

class MyNiftyObject # < Object is implied
  attr_reader :value
 
  def initialize(value)
    @value = value
  end
 
  def ==(other)
    self.class == other.class && # Not just any object with a "value" attr
      self.value == other.value
  end
end
 
MyNiftyObject.new('hi') == MyNiftyObject.new('hi') # => true

“I thought you said my instinct was wrong, Mr. Smarty Q. Pantsington, III,” I hear you saying. Not so fast! Remember, we wanted to implement object equality in a way that would work with our “seen” hash, in ActiveRecord. Let’s try it.

hi1 = MyNiftyObject.new 'hi'
hi2 = MyNiftyObject.new 'hi'
hi1 == hi2 # => true
hi_hash = {}
hi_hash[hi1] = "hello, good sir"
hi_hash[hi2] = "i hope this missive finds you well"
hi_hash.size   # => 2
hi_hash.values # => ["hello, good sir", "i hope this missive finds you well"]

Well, that sucked.

The oddly-yet-aptly-named Object#hash

Why didn’t it work? Because there’s another Object method we need to implement, Object#hash. What’s that do? Let’s consult the docs:

Generates a Fixnum hash value for this object. This function must have the property that a.eql?(b) implies a.hash == b.hash. The hash value is used by class Hash. Any hash value that exceeds the capacity of a Fixnum will be truncated before being used.

The first time I read this in the docs, I’ll admit that I still had no idea what the method needed to do, or how, exactly, the Hash class used this method. In English, here’s how this all fits together:

The hash method is sort of an optimization trick used by Ruby. Certain Ruby classes (Hash and Array, notably) make use of a hash table (see st.c in the Ruby source) which “bins” data based on the Fixnums returned by their hash methods. As more values are stored in the hash, the number of bins can be increased. Since it’s super-efficient to look up entries in this table, it can be used to quickly determine whether there’s a chance the object might exist, and roughly where to find it, before bothering to actually retrieve or set it.

Using this trick, a Hash knows very quickly whether it can just stow your data in the given key, because there’s no chance of needing to overwrite an existing object, or whether it needs to do a bit more legwork. Likewise, an Array knows whether it might need to remove an object when you call uniq. There’s a bit more to it than that, but this post is already too long, so let’s skip to the chase, shall we? The short version is that unless your hash method returns the same value as another object’s hash method, Ruby won’t even bother calling your object’s eql? method, which is why our little experiment failed.

Implementing a custom hash method

How should we write a custom hash method? If you only have one instance variable in your class, it’s easy. Just have your hash method call its hash method, and you’ll be set. If you have more than one, there are a couple of ways to do it. The general idea is that whatever method you choose should return a Fixnum, and not a Bignum, so it’s probably not best to just add all of your instance variables’ hashes together. That being said, Ruby is kind enough to truncate the value for you if you violate this “rule”, or even coerce your returned value to a Fixnum if you return something really crazy (as long as whatever you return implements to_int).

My personal favorite way to handle hashing an object with multiple instance variables is to let Array do the work for me:

def hash
  [@var1, @var2, @var3].hash
end

Back to the problem at hand

Remember our friend ActiveRecord::Relation#merge? If we want to use objects as keys in that “seen” hash, we have some work to do.

At a minimum, we need to implement the hash and eql? methods on any objects we think might end up on the left side of the Equality nodes. But taking it one step further, it’s always a good idea to implement the == method, as well. Thankfully, that’s as easy an aliasing it to eql?.

For a moment, pretending our little sample class above was going to be in the left-hand of an equality node, this is our minimal implementation:

class MyNiftyObject
  attr_reader :value
 
  def initialize(value)
    @value = value
  end
 
  def hash
    @value.hash
  end
 
  def eql?(other)
    self.class == other.class &&
      self.value == other.value
  end
  alias :== :eql?
end

With ARel, it’s a bit trickier, because most nodes have more than one instance variable (in some cases many more), and even Arel::Attributes::Attribute, which inherits from a Struct (which already has a sane default implementation of the methods shown above), contains an Arel::Table, which itself must implement equality tests, and so on… Equality all the way down.

What’s that end up looking like? A whole lot of typing. But in the end, we get what we wanted!

Great. So when should I actually go through this hassle?

It really depends. Do you have complete control over how your objects are used? Then it’s probably up to you. If you’re writing code intended for reuse (and aren’t you always?) then you should consider whether it might be the case that a user of your class (even the future you) might want to compare instances for equality, or use Array#uniq on an array containing your objects, or… My personal rule is this: If you’re wondering whether you should write equality methods for your class, you probably should.

As for the hassle? Well, it’s really not much hassle at all, in most cases. But after typing all that code for ARel, I decided to make it easier, anyway.

Equivalence

Equivalence is a tiny gem that implements the equality code discussed in this post via a macro. Check the README for more info. It’s easy enough for the most common use cases that you have no excuse, now.

Go forth, and write egalitarian code!

@jeffreyiacono I’ll see what I can do. It’ll probably end up being a separate post if/when I get the time to write it :)

— Ernie Miller (@erniemiller) July 19, 2012

Anyway, better late than never, right? Here goes. Bear with me, this post is going to be heavy on philosophy and light on code.

Confession

The previous post’s title was more than a little hyperbolic. With that out of the way…

Why is “Persistence is a Solved Problem” a useful viewpoint to hold?

Imagine that you’re writing a new, non-trivial application. You haven’t written a line of code, yet. You’re just thinking through how the application might be structured, essentially modeling the domain of the application in your head. When an application is in this stage, you start out with tremendous flexibility in how you can choose to think about the problem you’re addressing.

Now, your thought process is going to fall somewhere on a continuum between “mental database schema” and “there is no spoon.”

The “Greenfield App Continuum”.

I’ve found that my tendencies naturally lie toward the blue-shaded area of this continuum. The result is that I voluntarily and artificially constrain my thought process to the concepts easily described in terms of my chosen persistence layer. This, of course, means that I’m already deciding on implementation details (which persistence layer will I use as scaffolding for my thought process) before I’ve written a line of code, which narrows my thinking considerably.

Based on my review of many applications, especially Rails applications, I don’t believe I’m alone in this tendency. My taking the (admittedly extreme) view that persistence is a solved problem is an attempt to push that decision forward to some future point, removing friction while thinking about how best to model the domain.

Why do we (as Rails developers) gravitate toward the “mental database schema” way of thought?

First, because the RDBMS and its associated concepts have been so thoroughly ingrained in us. They’re like a trusty hammer. Until recent history, in fact, if I sat a group of developers down in a room, we’d possibly end up in a debate about which RDBMS we should use for the new application, but certainly not if we’d use one.

Second, because of the Active Record pattern (particularly ActiveRecord, as implemented in Rails), which presents a ridiculously low barrier to entry. I love ActiveRecord for lots of reasons, and a large portion of my open source work relates to ActiveRecord. It makes basic CRUD operations almost effortless. But let’s remember that the Active Record pattern existed before Rails was a sparkle in DHH’s eye, and Martin Fowler, who named the pattern, had this to say, in 2002, in Patterns of Enterprise Application Architecture (emphasis mine):

Active Record is a good choice for domain logic that isn’t too complex, such as creates, reads, updates, and deletes. Derivations and validations based on a single record work well in this structure.
[...]
Their primary problem is that they work well only if the Active Record objects correspond directly to the database tables: an isomorphic schema.
[...]
Another argument against Active Record is the fact that it couples the object design to the database design. This makes it more difficult to refactor either design as the project goes forward.

(Incidentally, Martin recommends the Data Mapper pattern as an alternative that addresses these concerns. Thankfully, we have a partial — thanks for the correction, @lsegal — implementation of this pattern in the Ruby world, too.)

So, the Active Record pattern excels for simple domain logic such as CRUD operations. It should come as no surprise, then, that following this particular well-worn path as we think about modeling our non-trivial application’s domain may well end with us looking up to see a dark forest surrounds us, and the path has all but disappeared.

This happens because we allow our persistence model to drive our domain model. The consequences to the resulting code will be that our system is harder to maintain.

Achieving freedom through ignorance

What I’ve come to believe is that this problem, allowing our persistence model to drive our domain model, doesn’t manifest itself only in our tightly-coupled code. It infects our thought process.

If you’ve built at least a few web applications, consider this: what is the most significant challenge you typically face when reasoning about a new application? With exceptions for applications requiring massive scalability on day one (hint: your application doesn’t), I’m going to guess that “how to read and write all those ones and zeroes” wasn’t your answer. It’s probably not even in your top three, if you really think about it. Sure, when to persist your data, or where to persist it, or what to do once it’s persisted (or read). But not how.

Maybe this seems like an oversimplification.

Have you ever inherited responsibility for a large application? Was the application a joy to maintain, or a nightmarish mudball of code with files you were frightened to even open, much less modify?

I’ve seen my fair share of both kinds of applications, and I’ve come to the conclusion that the most difficult part of reasoning about a new system is, in fact, figuring out how to model a system that is easy to reason about.

If success is achieved in this area, other questions will almost seem to answer themselves. Here are a few:

• What should this internal API look like?
• What tests should I write?
• How would I extend this object’s behavior?

Of course, now that we’ve decided the ideal way in which we’d like to interact with our objects, there’s one more big question that we can begin to make an informed decision about:

• What is the most effective way to implement object persistence, while best enabling our ideal method of reasoning about the problem domain?

At this point, it is always possible that we might find ourselves without a “perfect” solution to our persistence problem, and we are free to make some concessions in our domain model for the sake of performance or pragmatism. But now, we find ourselves at a vantage point from which we recognize the sacrifices we are making. More importantly, because our application will be easier to reason about, we’re in a stronger position to recover the ground lost by those concessions when a better option presents itself.

Jeff (and anyone else who’s still reading), I hope this better explains where I was coming from in the previous post.

Further reading:

• Making ActiveRecord Models Thin
• Objects on Rails
• Building a Rails App Without Rails

True, it is easy. But, you know what? Persistence is a solved problem. Granted, solved in any one of about a hundred different ways, depending on what you’re up to, but solved.

Recently, I’ve been finding it incredibly freeing to model a domain as if persistence didn’t matter. That’s right. Ignore its very existence. Put it out of my mind, completely. It helps me think more clearly about the problem I’m really trying to solve. Once that part’s figured out, I’m in a better position to make a reasoned decision about which persistence solution fits, and what concessions I’m willing to make, if any.