It was then that I pulled up WIkipedia’s list of HTTP status codes and started reading through them. Taken out of context, some of them could be downright amusing, and I commented that many of them sounded like the kinds of answers you get from a Magic 8-ball. That got some chuckles, so I spent 10 minutes hacking a quick app together that would return a random status code.

Then it got out of hand. That night I started working on twitter oath integration, the next night, some basic layout, and the next, comments. Anyway, the completely useless end result is at http://h8ball.com for your mild amusement. At least I can say I got to play a bit with Twitter integration, which I’d been putting off until now.

The bug hunt

If you’re a typical MetaWhere user, you’ve gotten into the habit of specifying your conditions with the improved syntax pretty quickly, and they are almost second nature to you now. They certainly were to me, at least.

So much so, in fact, that in my last project, without thinking, I had declared this in an association:

has_many :optional_photos, :class_name => "Photo",
                           :conditions => {:subtype.matches => 'optional%'}

This was a silly mistake on my part, but I’d gotten so used to using MetaWhere it didn’t hit me until later, when none of my optional photos were showing up. I was confused for a moment. I checked the log to see this query:

SELECT COUNT(*) AS count_id
FROM (SELECT 1 FROM `assets` WHERE (`assets`.`type` = 'Photo')
  AND (`assets`.project_id = 2 AND (`assets`.`subtype` = 'optional%')))
  AS subquery

Did you spot the problem? My intended LIKE query was being converted to an = query. This happens in activerecord/lib/active_record/reflection.rb, in Association#dependent_conditions:

def dependent_conditions(record, base_class, extra_conditions)
  dependent_conditions = []
  dependent_conditions << "#{primary_key_name} = #{record.send(name).send(:owner_quoted_id)}"
  dependent_conditions << "#{options[:as]}_type = '#{base_class.name}'" if options[:as]
  dependent_conditions << klass.send(:sanitize_sql, options[:conditions]) if options[:conditions]
  dependent_conditions << extra_conditions if extra_conditions
  dependent_conditions = dependent_conditions.collect {|where| "(#{where})" }.join(" AND ")
  dependent_conditions = dependent_conditions.gsub('@', '\@')
  dependent_conditions
end

The important part is the line that calls sanitize_sql on options[:conditions]. That method converts hashes to standard equality conditions.

Why this matters

“So, why not make sanitize_sql understand MetaWhere conditions? Isn’t that the solution?” I’m glad you (okay, I) asked. Because it’s completely reasonable for the existing behavior to occur in associations.

One of the key reasons you would choose to use an association over a plain instance method to access associated records is that they bake in some special magic that allows you to create new instances of the associated model. For instance, to build a new instance of a comment associated with the first article:

ruby-1.9.2-p0 > Article.first.comments.build(:body => 'hey!')
 => #<Comment id: nil, article_id: 1, body: "hey!", created_at: nil,
      updated_at: nil>

Let’s assume, for a moment, a has_many association that looks like this, though (note: Don’t do this. MetaWhere won’t let you now, anyway):

has_many :old_comments,
         :class_name => "Comment",
         :conditions => {:created_at.lt => "2009-01-01".to_time}

What would constitute the “correct” way to create a new record, now? To set the created_at value to 1 second before New Year’s Day, 2009? The beginning of time? It doesn’t really make any sense.

This is why MetaWhere will now check whether you are trying to use MetaWhere conditions in an association macro, and raise the error. Better to stop in your tracks before you go too far down that road. That road leads to madness.

If not an association, then what?

In my case, since I was only using that association to load optional photos, and not create them, I went with an instance method:

def optional_photos
  assets.where(:type => 'Photo', :subtype.matches => 'optional%')
end

If you weren’t intending to find only those objects associated with an instance, then you weren’t looking for an association in the first place, anyway — use a scope instead.

Of course, the error message will tell you as much:

MetaWhere::MetaWhereInAssociationError: The :lame_comments association has a MetaWhere::Column in its :conditions. If you actually needed to access conditions other than equality, then you most likely meant to set up a scope or method, instead. Associations only work with standard equality conditions, since they can be used to create records as well.

Why?

Line Workers build widgets. Someone hands them a blueprint¹ for a widget. The blueprint tells them how tall and how wide the widget should be. It tells them that the widget should rotate in a counter-clockwise direction and be made of a material strong enough to withstand 25 pounds of pressure. The Line Worker diligently creates the widget that is requested of him, applying design patterns as appropriate (if his employer is lucky). Enterprisey companies looove Line Workers, because they churn out lines of code like nobody’s business.

Artists find widget building tedious and repetitive. They push back when the blueprint they receive makes no sense, and they recognize the similarities between that blueprint and 10 others just like it in the past 2 months. Instead of building widgets, they build a widget builder, so that they never have to build another widget. As a result, they churn out less lines of code. The enterprisey company sees them as difficult, unproductive primadonnas. They fail to recognize the value proposition of a coder who wants to code himself out of a boring job.

The result? Companies that hire Line Workers assume creativity is the realm of someone in another department, someone who often doesn’t even know what might be possible via code, unless it’s shown to them. They’re building a house, and they hire an architect who doesn’t understand structural integrity and a color-blind interior designer.

So, how does a company attract and nurture Artists?

Getting Started

For starters, Artists need room for creative expression. If the only place your Artist can get creative is in his code comments, because everything else about the application has already been rigidly defined, then prepare yourself for some scathing comments. Bring him in early, while the ideas for your application are still being fleshed out. Besides showing the Artist that you value what he brings to the table, this will also benefit the project in tangible ways:

1. The Artist will give your client more bang for his project’s budget. He’ll identify when something that seems simple is anything but, and he’ll identify when something adds a lot of value for very little effort. He will educate your clients and your sales team while simultaneously showcasing your in-house talent.
2. Letting your Artist in on these early meetings will help him better understand what the project pipeline looks like. This will allow him to more effectively prioritize tasks, and identify upcoming cases for code reuse, letting him know when it will pay off to create those widget-builders he’s known for. You are hiring managers of one, aren’t you?

Ongoing Process

The term “Agile Development” has been overused and abused. Many companies are “agile” in word but not deed, and still others embrace the term as synonymous with “our developers do code sprints” but still go about defining project requirements the same old waterfall way they always have. I prefer to describe the ideal development process as “Common Sense.”

Example: If you’ve spent 40 hours (remember, when meetings are involved, attendees x duration = hours) describing functionality without at least having some wireframes and some proof-of-concept functionality, you’re doing it wrong. Don’t keep your Artist in the dark. In particular, as your designers (if the Artist isn’t also your designer) build out those wireframes, they should work closely with the Artist. He will quickly recognize the patterns they’re repeating over and over, and save them hours of effort by writing code to do it for them.

Tools

Choose a language and framework which gives your Artist the most joy. An Artist will put in extra hours working when work feels more like play.

I’ve tried my hand at at many languages over the past 26 years, and a nice helping of frameworks as well. I’ve never enjoyed development more than since I started working with Ruby and, by extension, Ruby on Rails. When possible, I approach projects using these tools because I find they bring me the most joy. Why? Because Ruby’s flexible metaprogramming capabilities allow me to make boring projects interesting. If it’s boring and repetitive to write certain code, there’s a high probability that writing code to write that code is a more interesting (and reusable) problem to solve.

Now, Ruby might not be a good fit for your typical project, or your Artists. I’m not suggesting that you use a screwdriver to drive in nails. Just find a screwdriver that makes your Artist happy, and when someone asks you to hang a picture on the wall, prefer screws to nails if given the choice.

One final note: choose a language that gives your Artist enough rope to hang himself, and trust your Artist not to end up a piñata. Make sure your tool of choice provides enough flexibility for your Artist to really express himself, and you (and your Artist) will be glad you did.

1 In our industry they’re called “functional specifications.” Why, I don’t know, because they’re about as dysfunctional a concept as I’ve ever seen.

Smarter Order Clauses

This one is pretty simple. ActiveRecord 3.0 query syntax normally dictates that you supply a valid SQL string to the order method, like so:

Article.joins(:comments).order('comments.created_at desc')
=> "SELECT "articles".* FROM "articles"
   INNER JOIN "comments" ON "comments"."article_id" = "articles"."id"
   ORDER BY comments.created_at desc"

With MetaWhere, you can supply your order clauses in Ruby, instead of SQL. Since you’re supplying hashes, the same inference about table names is in effect here as in your where clauses, meaning that multiple joins of the same table will reference the proper table alias automatically, and (of course) you can use autojoin.

Article.order(:comments => :created_at.desc).autojoin.to_sql
=> "SELECT "articles".* FROM "articles"
   INNER JOIN "comments" ON "comments"."article_id" = "articles"."id"
   ORDER BY "comments"."created_at" DESC"

Speaking of table aliasing…

Multiple Self-referencing Join Support

Let’s say you have a model called Person. A person belongs_to a parent and has_many children, both of which are also instances of Person (unless biology has changed significantly since when I learned it). Now, you want to search your people for people whose grandchildren are named “Jacob” (this is yanked straight from the MetaWhere test suite by the way):

Person.where(:children => {:children => {:name => 'Jacob'}}).autojoin
=> "SELECT "people".* FROM "people"
   INNER JOIN "people" "children_people"
     ON "children_people"."parent_id" = "people"."id"
   INNER JOIN "people" "children_people_2"
     ON "children_people_2"."parent_id" = "children_people"."id"
   WHERE ("children_people_2"."name" = 'Jacob')"

As you can see, MetaWhere takes care of figuring out which table alias you want your condition to apply to. This works in both directions, so, for example, assuming a Person belongs_to both a father and mother, this will let you get a person whose father is named Abraham and mother is named Sarah:

Person.where(
  :father => {:name => 'Abraham'},
  :mother => {:name => 'Sarah'}
)

Very readable.

Opt-in Operators

Overriding operators is a very polarizing thing. Some people get really bent out of shape over it, claiming it’s a surefire way to earn yourself a one-way ticket to coder hell. Personally, I think that if the overrides suit your problem domain, then go for it. Still, I can respect the concern that an override on Symbol operators stands a good chance of conflicting with someone else’s code, so they are off by default in MetaWhere. If you want to use operators in your conditions (such as :created_at >= 1.week.ago instead of :created_at.gteq => 1.week.ago you will need to add MetaWhere.operator_overload! to an initializer in your application.

That’s it for now! I hope you’re enjoying Rails 3 as much as I am, and I hope that MetaWhere proves to be a useful tool in your Rails 3 development!

In your view

First up, as before, you’ll need to tell Uploadify about the name of your application’s session key. The important change here is where the session key name can now be found: Rails.application.config.session_options[:key].

<% session_key_name = Rails.application.config.session_options[:key] %>
  $('#<%= "#{asset_name}_upload" %>').uploadify({
      uploader        : '/uploadify/uploadify.swf',
      script          : '<%= url_for @project %>',
      fileDataName    : 'project[<%= "#{asset_name}_attributes" %>][data]',
      cancelImg       : '/uploadify/cancel.png',
      fileDesc        : 'Photo or Video',
      fileExt         : '*.mov;*.mp4;*.avi;*.wmv;*.png;*.jpg;*.gif',
      auto            : true,
      sizeLimit       : <%= 100.megabytes %>,
      width           : 150,
      height          : 25,
      hideButton      : true,
      wmode           : 'transparent',
      buttonText      : 'Upload',
      onComplete      : function(a, b, c, response){ eval(response); $('#<%= "#{asset_name}_queue" %>').html(''); },
      queueID         : 'noQueueForMePlease',
      onProgress      : function(event, queueID, fileObj, data){ $('#<%= "#{asset_name}_queue" %>').html(data.percentage + '%') },
      scriptData      : {
        '_http_accept': 'application/javascript',
        '_method': 'put',
        '<%= session_key_name %>' : encodeURIComponent('<%= u cookies[session_key_name] %>'),
        'authenticity_token': encodeURIComponent('<%= u form_authenticity_token %>')
      }
  });

A brief aside about my application

The preceding snippet was taken from an actual application which is a work-in-progress. This application has several file upload “slots” that are handled by accepts_nested_attributes_for on my Project model, and because the request is always an “update” of the parent Project model, it’s safe for me to hard-code the _method as put. That may not be the case for you. You will want to examine the following parameters from this example and change them as needed for your application’s requirements:

• script: the URL you want Uploadify to post to. Remember that this can differ depending on whether you want to create or update a record. Thankfully, url_for takes this into account when generating a URL from an AR object by checking whether it’s been persisted or not.
• fileDataName: this should be the same as the file field name generated for your file attribute by the FormBuilder file_field helper.
• fileDesc and FileExt: modify to suit the file extensions you would like to accept. You should still validate at the model level, but this will prevent the file select dialog from allowing selection of other file types.
• sizeLimit: this one should be fairly obvious.
• hideButton and wmode: leave these out if you want the default button to be visible. I’m replacing it with some text placed behind the (invisible) Flash button.
• _http_accept in scriptData and onComplete: I’m opting to send along a request for a .js.erb by passing _http_accept, and eval the response in onComplete in much the same way as jQuery does when you pass ‘script’ to the dataType parameter of jQuery.ajax.
• queueID and onProgress: Uploadify will create a queue with a progress bar below the button by default. I don’t like this behavior as the queue is too large for the slide-out drawer I’m using for files. I pass a bogus queueID and set up the onProgress callback to instead replace a percentage number in a div of my choosing.
• _method in scriptData: Since I’m always posting an update to an existing project (because i’m using accepts_nested_attributes_for) I can safely hardcode put here. If you may be creating a new record, you will want to set this conditionally to post, based the result of @myobject.persisted?

The middleware

Now, you can see we’re passing the session key and form authenticity token in the above code. The trick is that the session key is stored in a cookie, and Flash doesn’t know about it. Enter a slightly modified version of John’s FlashSessionMiddleware.

app/middleware/flash_session_cookie_middleware.rb:

require 'rack/utils'
 
class FlashSessionCookieMiddleware
  def initialize(app, session_key = '_session_id')
    @app = app
    @session_key = session_key
  end
 
  def call(env)
    if env['HTTP_USER_AGENT'] =~ /^(Adobe|Shockwave) Flash/
      req = Rack::Request.new(env)
      env['HTTP_COOKIE'] = [ @session_key,
                             req.params[@session_key] ]
                           .join('=').freeze unless req.params[@session_key].nil?
      env['HTTP_ACCEPT'] = "#{req.params['_http_accept']}"
                           .freeze unless req.params['_http_accept'].nil?
    end
 
    @app.call(env)
  end
end

Note that aside from setting the HTTP_COOKIE header to include the session, we’re also setting the HTTP_ACCEPT header (to allow our Rails app to serve up the .js.erb in a respond_to block). Place this file somewhere handy (I followed his suggestion of app/middleware) and make sure it’s loaded at Rails startup by adding the following to config/application.rb inside the class Application block :

    # Add additional load paths for your own custom dirs
    %w(observers mailers middleware).each do |dir|
      config.autoload_paths << "#{config.root}/app/#{dir}"
    end

Inserting our middleware

This was the trickiest part. A lot has changed about the Rails boot process between 2.3 and 3.0. While searching for help on getting this set up, I found a bunch of information, most of it bad, on what would work, from different locations for the code to different code altogether. In the end, as it turns out, you can still place the middleware loading code in config/initializers/session_store.rb, but it’s changed quite a bit from Rails 2.3:

Rails.application.config.middleware.insert_before(
  ActionDispatch::Session::CookieStore,
  FlashSessionCookieMiddleware,
  Rails.application.config.session_options[:key]
)

That’s it! Hope this helps you avoid the hassles I dealt with in getting Uploadify up and running with Rails 3.

Why scope searches?

While the simplest (and probably most common) use case for MetaSearch is a simple Model.search(params[:search]) in an admin form, there are plenty of cases in which your general user base should be able to search your data, too. The difference between Joe User and an admin, however, is that Joe probably shouldn’t be able to search every record in your database.

Well, duh. So how do I do it?

I’m glad you asked! :)

So, let’s say you have a bunch of Projects in your database, and you’ve already set up a scope to limit visibility. Maybe your scope looks something like this, assuming you’re using MetaWhere — you are using MetaWhere, right?

scope :visible_to_user,
      lambda {|user| where(:user_id.eq % user.id | :public.eq % true)}

You’re also probably using something like José Valim’s excellent Devise for authentication, so you’ll have a current_user method. Now it’s just a matter of filtering your results to projects visible to that user when they do a search.

Prior to 0.5.0, MetaSearch always started searches with a blank slate, which meant that if you wanted to further scope a search, you had to do so by accessing the underlying ActiveRecord::Relation object, like so:

Prior to 0.5.0 (don’t use now!):

@search = Project.search(params[:search])
@projects = @search.relation.visible_to_user(current_user)

That’s not so bad, but it’s not very intuitive, either. We can do better. And so we shall. As of MetaSearch 0.5.0:

The new hotness:

@search = Project.visible_to_user(current_user).search(params[:search])
@projects = @search # Or maybe @search.paginate(...)

Now, we’re doing what should come naturally — scoping the search before we ever hand it off to MetaSearch to begin with. It’s that simple!

I’m really excited about these releases, as they’ve been a long time coming. The last MetaSearch gem release was way back in March, while the last MetaWhere gem was released in April — this felt like an eternity in the fast-moving edge Rails world. :) I hope you enjoy the new release, and thanks in advance for submitting any bugs you find!

Now, on to what’s new since the last gem releases…

MetaSearch 0.5.0

Changes since 0.3.0 (released March 16th, 2010):

• Refactoring to make use of Arel predicates in query creation
• Addition of validators — customizable code blocks to determine whether a supplied value is sufficient to use as a search term
• Improvements to check_boxes and collection_check_boxes to make them easier to style
• Attribute and association exclusion syntax has been updated, and a whitelist version has been added as well (attr_searchable/attr_unsearchable and assoc_searchable/assoc_unsearchable)
• Support for custom search methods — any class method which returns a relation can be used. This, of course, includes scopes
• sort_link helper added. Supply a MetaSearch::Builder instance and the name of the attribute to sort on to generate sort links (similar to Searchlogic in Rails 2.3). This works across associations as well.
• Multi-level association support. Can go up to 5 levels deep, including the search base.
• Searches can be built against a relation with pre-existing where conditions to search across a narrowed scope

MetaWhere 0.5.0

Changes since 0.3.3 (released April 30th, 2010):

• Smarter order clauses: Company.joins(:developers).order(:developers => :salary.desc)
• Operator overloads on symbols are now opt-in, via MetaWhere.operator_overload!
• Multiple self-referencing joins are supported
• Relations scope new records properly when equality conditions exist (same as stock AR but supporting all equality predicate syntax)
• Support for merging relations with a different base