techfreak.net

Simple CSS Preprocessing with cpp

2014-05-17T00:00:00+00:00

Sometimes when prototyping a quick, small app, I find myself wanting a subset of Sass’s features without everything. Specifically, I love having variables available, and the option to split into multiple files for organization.

Then I read this post and wondered how I could use cpp towards such a goal. The answer is pretty simple. An example app.css file:

#define $text_color #333
#define $border_color #eee

#import "comments.css"
#import "posts.css"

And in posts.css:

.posts .post {
  border: 1px solid $border_color;
}

Running cpp -P app.css outputs the following:

.posts .post {
  border: 1px solid #eee;
}

This is no “replacement” for sprockets or sass, but a nice trick to have up your sleeve!

A CouchDB view example

2013-11-14T00:00:00+00:00

Newcomers to CouchDB offerings often fall into two categories: people that use it purely as a key-value store, and people that are stuck wondering how to query non-primary-keyed data.

One answer built in to CouchDB is “map-reduce”.

Let’s dive in on a simple example. We’ll model a recipe book of bartending drinks. This example provides two interesting points. First, modeling recipes in relational databases provides for strong integrity, however, it is not very intuitive modeling. I once tried to explain to a Rails newcomer how to create join models between a recipe and ingredients, and they found it fairly confusing. Secondly, drink recipes provide a clear use case for a secondary index search by ingredient.

First, we’ll store some simple single document drink recipes that look like this:

{
  "type": "DrinkRecipe",
  "name": "Angler's Cocktail",
  "ingredients": [
    { "name": "Gin", "amount": "2 oz" },
    { "name": "Angostura Bitters", "amount": "3 dashes" },
    { "name": "Orange Bitters", "amount": "3 dashes" },
    { "name": "Grenadine", "amount": "splash" },
    { "name": "Red Maraschino Cherry", "amount": "1"}
  ],
  "note": "Serve in an old fashioned glass over cracked ice"
}

Briefly, if you’re not familiar with CouchDB, take note of the type property. That’s a CouchDB convention of marking the document to distinguish them from each other. Unlike collections in MongoDB, or tables in SQL stores, your docs for an app typically exist in a single database. The rest of the properties are pretty self explanatory, but take note that it models the recipe as you might write it on a bit of scrap paper (minus the braces and keys).

Now let’s talk about two simple views. First, it’s pretty likely you’ll want to be able to list all your recipes by name. Here’s the one possible view for that:

function(doc) {
  if (doc.type === 'DrinkRecipe') {
    emit(doc.name.toLowerCase(), doc.name);
  }
}

We’ll store that in the ‘drinks’ design doc as ‘byName’. We’re emitting the document name in lowercase as the key (the first argument to emit), and the name as the value so we can preserve the case. Let’s grab at that view with curl:

$ curl http://127.0.0.1:5984/drinks/_design/drinks/_view/byName

{"total_rows":4,"offset":0,"rows":[
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":"angler's cocktail","value":"Angler's Cocktail"},
{"id":"3b510371b46c2f20cd7d72a52700af45","key":"manhattan","value":"Manhattan"},
{"id":"3b510371b46c2f20cd7d72a527008dd5","key":"martini","value":"Martini"},
{"id":"3b510371b46c2f20cd7d72a527009c27","key":"old fashioned","value":"Old Fashioned"}
]}

The most important thing to note here is that the keys come back in sorted order, in this case alphabetical.

Now, say we wanted drinks starting with ‘m’. We can leverage the start and endkey params as such:

$ curl 'http://127.0.0.1:5984/drinks/_design/drinks/_view/byName?startkey="m"&endkey="n"&inclusive_end=false'

{"total_rows":4,"offset":1,"rows":[
{"id":"3b510371b46c2f20cd7d72a52700af45","key":"manhattan","value":"Manhattan"},
{"id":"3b510371b46c2f20cd7d72a527008dd5","key":"martini","value":"Martini"}
]}

Note that total_rows is 4, that’s because the view has 4 rows in total, but, our startkey “m” and endkey “n” only return two rows. Kind of strange but just be aware of it. Also, we’ve got back just our rows that start with “m” but not “n” because we disabled the inclusive_end.

OK, let’s introduce a second view to enable searching by ingredient:

// map
function(doc) {
  if (doc.type === "DrinkRecipe") {
    doc.ingredients.forEach(function(i) {
      emit([i.name.toLowerCase(), doc.name.toLowerCase()], null);
    });
  }
}

// reduce
_count // uses the built in _count implementation

We’ve included the ingredient name as the first element of the key array (recall we had a string as key on the previous view), and the drink name as the second. This gives us already sorted results, as well as some search and aggregation ability as we’ll shortly see.

Let’s query it, we’ll get a row for every ingredient in every drink. We’ll pass it reduce=false to tell it to only run the map phase:

curl 'http://127.0.0.1:5984/drinks/_design/drinks/_view/byIngredient?reduce=false'

{"total_rows":17,"offset":0,"rows":[
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":["angostura bitters","angler's cocktail"],"value":null},
{"id":"3b510371b46c2f20cd7d72a52700af45","key":["angostura bitters","manhattan"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527009c27","key":["angostura bitters","old fashioned"],"value":null},
{"id":"3b510371b46c2f20cd7d72a52700af45","key":["bourbon","manhattan"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527009c27","key":["bourbon","old fashioned"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527008dd5","key":["dry vermouth","martini"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":["gin","angler's cocktail"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527008dd5","key":["gin","martini"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":["grenadine","angler's cocktail"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":["orange bitters","angler's cocktail"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527008dd5","key":["orange bitters","martini"],"value":null},
{"id":"3b510371b46c2f20cd7d72a52700af45","key":["orange peel","manhattan"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527009c27","key":["orange wedge","old fashioned"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":["red maraschino cherry","angler's cocktail"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527009c27","key":["sugar","old fashioned"],"value":null},
{"id":"3b510371b46c2f20cd7d72a52700af45","key":["sweet vermouth","manhattan"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527009c27","key":["water","old fashioned"],"value":null}
]}

Cool, let’s try some queries with different options.

Find any drink recipe that uses gin:

curl 'http://127.0.0.1:5984/drinks/_design/drinks/_view/byIngredient?reduce=false&startkey=\["gin"\]&endkey=\["gin",\[\]\]'

{"total_rows":17,"offset":6,"rows":[
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":["gin","angler's cocktail"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527008dd5","key":["gin","martini"],"value":null}
]}

The startkey is [“gin”] and the endkey is [“gin”,[]]. The endkey contains a bit of a conventional trick, it is the start key with [] appended on to it. Since we have a 2 element array key, this is a bit like a wildcard search on the last element, and matches on the first. Because CouchDB will keysort [] after a string like ‘zzzzzzz’ we are sure to get all our gin drinks back.

Now let’s get a little trickier and use the reduce phase as well as a group_level option.

curl 'http://127.0.0.1:5984/drinks/_design/drinks/_view/byIngredient?group_level=1'

{"rows":[
{"key":["angostura bitters"],"value":3},
{"key":["bourbon"],"value":2},
{"key":["dry vermouth"],"value":1},
{"key":["gin"],"value":2},
{"key":["grenadine"],"value":1},
{"key":["orange bitters"],"value":2},
{"key":["orange peel"],"value":1},
{"key":["orange wedge"],"value":1},
{"key":["red maraschino cherry"],"value":1},
{"key":["sugar"],"value":1},
{"key":["sweet vermouth"],"value":1},
{"key":["water"],"value":1}
]}

By default, a view with a reduce specified runs, so we just remove that param from our query. Also we’ve added group_level as 1. This means that we will be adding up the number of rows using the first key element only for our reduce. The value above is the sum of rows for each key, so there are 3 drinks that use “angostura bitters” in them. Had we used group_level 2, in this case everything would come back with value “1” because all our keys in full are unique.

Now, a natural reaction to searching by ingredient leads to a question: what if you want to search by two ingredients? Well, at this point, there are at least two obvious solutions. First, you could perform 2 queries of the view, 1 for each ingredient, and then do an in app intersection. A second would be to use couchdb-lucene to enable full text search on your ingredients. Cloudant also bakes this straight into their hosted BigCouch offering, you can read their docs here about it if you want to get up and running quickly.

At the outset I said we’d make two views. Let’s add a bonus one. Imagine you’ve got some bizarro recipe that calls for an ingredient of the same name twice (this happens much more in baking than mixology). An observant eye of our byIngredients view would spot that, we’d output two rows for the same ingredient, thereby possibly offsetting our results (think pagination) when duplicates are involved.

We could add another (or tweak our original code()byIngredient) view to include the doc._id. Let’s call it byDedupedIngredient:

// map
function(doc) {
  if (doc.type === "DrinkRecipe") {
    doc.ingredients.forEach(function(i) {
      emit([i.name.toLowerCase(), doc.name.toLowerCase(), doc._id], null);
    });
  }
}

// reduce
_count

Now let’s add a rather contrived “Extra Gin Martini” to our book:

{
   "type": "DrinkRecipe",
   "name": "Extra Gin Martini",
   "ingredients": [
       {
           "name": "Gin",
           "amount": "2 oz"
       },
       {
           "name": "Gin",
           "amount": "1 oz"
       },
       {
           "name": "Dry Vermouth",
           "amount": "1/4 oz"
       },
       {
           "name": "Orange Bitters",
           "amount": "A dash"
       }
   ],
   "note": "Shake with ice and serve in a martini glass"
}

Let’s query it twice, first without a reduce, and secondly with a reduce phase

curl 'http://127.0.0.1:5984/drinks/_design/drinks/_view/byDedupedIngredient?reduce=false&startkey=\["gin"\]&endkey=\["gin",\[\]\]'

{"total_rows":21,"offset":7,"rows":[
{"id":"3b510371b46c2f20cd7d72a527007e2a","key":["gin","angler's cocktail","3b510371b46c2f20cd7d72a527007e2a"],"value":null},
{"id":"3b510371b46c2f20cd7d72a52700c0fe","key":["gin","extra gin martini","3b510371b46c2f20cd7d72a52700c0fe"],"value":null},
{"id":"3b510371b46c2f20cd7d72a52700c0fe","key":["gin","extra gin martini","3b510371b46c2f20cd7d72a52700c0fe"],"value":null},
{"id":"3b510371b46c2f20cd7d72a527008dd5","key":["gin","martini","3b510371b46c2f20cd7d72a527008dd5"],"value":null}
]}

curl 'http://127.0.0.1:5984/drinks/_design/drinks/_view/byDedupedIngredient?reduce=true&group_level=3&startkey=\["gin"\]&endkey=\["gin",\[\]\]'

{"rows":[
{"key":["gin","angler's cocktail","3b510371b46c2f20cd7d72a527007e2a"],"value":1},
{"key":["gin","extra gin martini","3b510371b46c2f20cd7d72a52700c0fe"],"value":2},
{"key":["gin","martini","3b510371b46c2f20cd7d72a527008dd5"],"value":1}
]}

Can you spot the difference? Look closely for the “extra gin martini”. In the first query, it shows up twice. In the second, because we are running a reduce and group on the exact key at a group level of 3, it dedupes and returns with a value of 2.

Hopefully this has been of some help to folks interested in CouchDB, but not familiar with it, especially its map-reduce side. I should hatch a post in the near future to show you how to make more complicated collated views for returning richer, associated data in a single query.

Redis, It's Great for Small Stuff, Too

2013-06-07T00:00:00+00:00

Unless you live under a rock, you’ve heard of the key-value store Redis. It’s possible though, you’ve never used it. Maybe you don’t have giant, loaded site needs like Imgur, so you haven’t added it to your stack. That’s cool. In fact, I’m a proponent of not adding an additional cog to a production stack unless it either greatly reduces load on an existing part or makes the code easier to understand. That said, Redis has a reputation for helping out when your site is under giant load. But there’s something I think is great about Redis that makes it useful for small tasks: its simplicity.

Redis has a few different structure types, take Hashes and Lists. For example, using the redis-rb client, you can set values in a Redis hash:

redis.hset "user:1", :name, "Brendon"
redis.hset "user:1", :gender, "male"
redis.hget "user:1", "name" #=> "Brendon"
redis.hgetall "user:1" # => {"gender"=>"male", "name"=>"Brendon"}
redis.hvals "user:1" # => ["male", "Brendon"]

This feels like a simple, hash like primative to work with. Though it is syntatically different than working with a plain old ruby Hash, the feel is similar. I think this is a big win; it’s easy to understand and feels familiar, plus, it’s network distributed.

Check out some List basics:

redis.rpush "comments", "Awesome"
redis.rpush "comments", "Super"
redis.llen "comments" # => 2
redis.lpop "comments" # => "Awesome"

Again, syntatically a little different than using an Array, but still familiar.

Redis itself is very easy to install. You can install it from homebrew, or apt-get or just compile from source. It’s small, installs quickly, and unless you fill it with gigabytes of data, it’s likely you will forget it’s there. I install it by default on most my systems, even little underpowered ones. The reason being, I love Redis for my little one off scripting tasks because of a few features:

It’s liteweight
Data has optional expiration so it will clean up after itself for free
Its commands for get/set, lists, and pub/sub are great for basic IPC.

Here’s some examples. I’m excluding exception handling and Redis watches/multi/transaction concepts to focus on the core idea.

Redis is great at deduping. Say you want to write a script that pushes campfire notifications to gently remind the team to keep commit messages under 50 characters, but not resend reminders in the case of forced pushes, rebases, merges, etc:

# Generate your own digest to avoid git hash changing
id = Digest::MD5.hexdigest("#{commit.author}:#{commit.message}")
# unseen will only be true if the id is not already in Redis
unseen = redis.setnx(id, Time.now.to_i)
# expire after 90 days to save on memory
redis.expire(id, 7776000) if unseen
# run notification if unseen

Or imagine you have a system that is applying regex to RSS feeds, and you want to be alerted at most once a day about matches:

id = Digest::MD5.hexdigest(regex_pat.to_s)
# Exclusively set the key, returning false if it already exists
unseen = redis.setnx(id, Time.now.to_i)
redis.expire(id, 86_400) if unseen

I do this in small scripts all the time. Now, I could write to sqlite, use the ruby PStore, or maybe a tempfile as lock, etc, but this just feels so dang simple, and I don’t worry about file locking, sql queries, weird commit code, etc.

Maybe you have scripts you want to “shut up” for a few hours. You could write a file and stat it, or you could just set a value to read:

# Don't bug for 5 minutes
redis.setex("shutup", 300, "true")

Often I have little tasks or cronjob on my home server that I want notifications about. Currently I use Pushover for all those notifications. Someday I might change my mind and switch to email or SMS. So what I’ve done is treat Redis like a good old Unix named pipe. But now I don’t have to worry about creating the pipe, or permissions, and I seem to remember Redis commands much easier (pop quiz…how do you call mkfifo in Ruby without shelling out?).

If you want to make a FIFO for notifications, write something like this:

# Producer script, cron job, etc
redis.rpush("notifications", "You've got mail!")

# Notification daemon
loop do
  # blpop is a blocking read left pop
  key, message = redis.blpop("notifications")
  # send message to SMS, email, Pushover, etc
end

Sometimes, your notification daemon might die and not restart properly. Maybe you want to “catch up” on messages when you restart it, then the above list use is good. If you don’t want to get flooded by those missed notifications, simply switch to pub/sub:

# Producer
redis.publish("notifications", "A wild message arrives!")

# Notification daemon
Redis.new.subscribe("notifications") do |on|
  on.message do |channel, msg|
    send_notification(msg)
  end
end

Now for one last tip. Redis provides redis-cli, a command line client for interacting with Redis. If you fire it up with no commands, you’ll get an interactive prompt. However, you can also invoke it with a command to run. This is great for shell scripts:

redis-cli publish notifications 'John sent you a private message on irc'

If you haven’t tried Redis yet and program small tasks, you can get started today!

Simple Two-Factor Auth with Shield

2013-06-05T00:00:00+00:00

Shield is a simple authentication gem I tend to reach for first when developing small Sinatra or Cuba apps. The code is short, reliable, and easy to understand. It’s not a kitchen sink solution, it does one thing well: lets users log in with a password.

As an experiment, I wondered what it might look like to layer a simple two-factor authentication scheme on top of Shield. If you are unfamiliar with two-factor authentication, it typically follows that a user will first log in to a site with their password as usual, and if successful, must pass a second step of entering a PIN from a SecureID type token, or out-of-band delivery message such as SMS. Popular sites implementing two-factor auth include PayPal, Twitter, and Google.

Let’s break down some of the components of what a simple system might look like.

First, we’ll create a basic User model with Ohm, a persistance library backed by Redis.

class User < Ohm::Model
  include Shield::Model

  attribute :email
  attribute :crypted_password
  index :email

  def self.fetch(email)
    User.find(email: email).first
  end
end

This is pretty straightforward. We’ve established a user with an email, crypted password, and the fetch method Shield expects per its implementation. Moving on, we’ll tweak the Shield helpers a little and setup some of our own for handling the second factor.

helpers do
  include Shield::Helpers

  alias_method :initially_authenticated, :authenticated

  def authenticated(model)
    if user = initially_authenticated(model)
      user.id.to_s == session["#{model}_secondary_auth"].to_s && user
    end
  end

  def challenge_authentication(model)
    ChallengeAuthentication.new(initially_authenticated(model))
  end

  def send_challenge(model)
    challenge_authentication(model).push
  end

  def challenge_accepted(model, challenge)
    if challenge_authentication(model).check!(challenge)
      user = initially_authenticated(model)
      session["#{model}_secondary_auth"] = user.id.to_s
    end
  end
end

There’s a few interesting things to note here. First, we alias Shield’s authenticated method out to initially_authenticate. We’ll use this to check if a user passes the initial password authentication step. Next, we define our new authenticated method, which will rely on password authentication, and a second check against the session to see if the user has passed the secondary authentication step. Sprinkle in some methods around checking our challenge authentication (more details on that in a minute) and our helpers are good to go.

Now let’s move on to some simple Sinatra app and routing setup. First, we’ll handle 401 Unauthorized errors by redirecting to the /login path:

error 401 do
  redirect '/login'
end

Next, add in the /login routes Shield typically expects, however, instead of redirecting on success to the main page, we’ll redirect the browser on to a verification step:

get '/login' do
  erb :login
end

post '/login' do
  if login(User, params[:login], params[:password])
    remember(initially_authenticated(User)) if params[:remember_me]
    send_challenge(User)
    redirect '/login_verification'
  else
    redirect '/login'
  end
end

Note that when a user passes the first login stage, we’ll send them the challenge for the verification step.

Now we’ll get into the meat of the routing and diverge from the vanilla Shield login flow a bit. Setup the verification handling:

get '/login_verification' do
  error(401) unless initially_authenticated(User)

  erb :login_verification
end

post '/login_verification' do
  error(401) unless initially_authenticated(User)

  if challenge_accepted(User, params[:challenge])
    redirect '/'
  else
    redirect '/login_verification'
  end
end

For either action to succeed, first the user must be initially authenticated. If so, we’ll verify the challenge presented, in this case a randomly assigned 6 digit PIN, matches up. Then we’ll let the user proceed on to the app.

The core of the challenge authentication is handled in a ChallengeAuthentication class:

class ChallengeAuthentication
  EXPIRE_TIME = 300

  attr_reader :user

  def initialize(user)
    @user = user
  end

  def redis
    Ohm.redis
  end

  def push
    challenge = generate_challenge
    redis.setex key, EXPIRE_TIME, challenge
    deliver_challenge(challenge)
  end

  def check(challenge)
    return false if user.nil? || challenge.to_s.empty?

    # Should be a secure compare to prevent timing attacks
    redis.get(key) == challenge
  end

  def check!(challenge)
    !! ( check(challenge) && redis.del(key) )
  end

  def key
    [user.class.name, user.id, 'challenge'].join(':')
  end

  # Returns a 6 digit challenge phrase
  def generate_challenge
    (SecureRandom.random_number * 1_000_000).to_i
  end

  def deliver_challenge(challenge)
    # send an out of band challenge like SMS or Pushover here
  end
end

The flow of the challenge check is fairly simple. First, we can push a new challenge by setting a random 6 digit pin for the user, and deliver that out of band. This can be easily accomplished via SMS with a provider such as Twilio, or my favorite push notification service Pushover. Checkout my Rushover gem as a simple client for sending to Pushover. We’ll toss that PIN into Redis with a 5 minute expiration; this provides a simple limited window for which the PIN is valid. Likewise, if you want to implement this on top of a SQL ORM, you could add challenge and expiration timestamps on to your User model.

For checking that the user has provided a valid challenge PIN, we can compare against the value in Redis if it exists. Upon match, we’ll delete the PIN from redis to invalidate it and confirm the challenge is accepted.

You can find the complete code for this example as a gist. Hopefully it’s easy enough to follow, and now you can provide an extra level of security for your social cat-video apps!

Until next time…

On Authorization Failures

2013-04-24T00:00:00+00:00

As a slight extension to the previous post, I wanted to make a quick point about authorization failures.

Given you’ve raised SomeAuthorizationFailure exception in a controller action, you might have a general rescue handling it:

rescue_from 'SomeAuthorizationFailure' do
  render :text => "Bad user!", :status => 403
end

The key here is the 403 status, Forbidden. This is a pretty natural, and technically correct status to feed the client.

Cool, let’s wrap that up, it’s done! Hold on, not so fast.

If you use Github (if?), you may have noticed something that struck you as curious the first time it happened. Say you’re hanging in the dev campfire room, and somebody pastes a link to a line of code for you to checkout (like /foocorp/awesomeproject/config/application.rb#L7). You clicked on the link but forgot you’ve logged out. Boom….403 Forbidden.

Wait..no that’s not a 403, it’s a 404. What the heck?

The answer is pretty simple. On your little todo app you run for friends and family, it’s probably not a huge deal for somebody to hit /todo_lists/42/item/5 and get a 403. Wow, somebody now knows you have a todo list 42 and item 5. Probably not a big deal.

But on a site like Github, let’s change that application.rb link to say, /foocorp/awesomeproject/config/initializers/devise.rb. 403? Oh look, that project is using Devise!

The moral of the story: best to give a 404 status on authorization failures if you don’t want to cater to mining and leaking of sensitive info.

On Authorization Patterns

2013-04-23T00:00:00+00:00

Once upon a time, I’d heavily lean upon scoped finds for cheap authorization in Rails controller actions. For instance, a system might have many users, for which each has many projects they can manage. In order to find a project that a user can administer, an action may include the following:

@project = current_user.projects.find(params[:id])

This can work. If a user tries to hit project id 42, for which they aren’t associated with, the execution short circuits at that point. The security on that project has been maintained.

I think most people know at this point, this is a poor general authorization scheme, because for one, it spreads your authorization logic, no matter how simple, around the application. With a few controllers in a small system, this probably isn’t a big deal.

Enter an authorization scheme, you might write:

@project = current_user.projects.find(params[:id])
authorize @project

The authorize method here will typically take the current user, lookup some policy object, and run a check. If a user can be associated to a project, but not be able to edit it, this will probably pan out as you expect.

Sometimes though, you could simply write such code to be:

@project = Project.find(params[:id])
authorize @project

This can be a subtle, but I believe powerful, difference. First, your finder usage is simplified. But second, and I believe more importantly, the code becomes more straightforward and your exceptions more accurate. Take another look:

# If no project is found, raise ActiveRecord::RecordNotFound
@project = Project.find(params[:id])
# If the user is not authorized, raise SomeAuthorizationException
authorize @project

This is a worthwhile difference. Want to keep metrics or get alerted on security violation attempts? Now it can be clearly split. Or perhaps, you take different action or set different flash messages; this can be handled more cleanly now.

When it comes to patterns, remember, it’s never one-size-fits-all. What’s good to realize is that sometimes you can write your code in simpler fashions, and more importantly, think about the explicit exceptions your system should be throwing, if any.

A note about "friendly" passwords

2013-01-26T00:00:00+00:00

Often in a web application, the time will come where you opt to generate temporary passwords for users. One common approach to this is to use a helper that combines a small about of random data (such as 4 random numbers) with a word randomly selected from a pared down dictionary list.

Please don’t do this. There are at least two reasons:

Using a word list radically reduces password entropy
Eventually a password will unintentionally perplex, or worse, offend someone

If the day comes and the system generates the password “cigar1984” for somebody trying to quit smoking, that could be awkward.

Now, I’m no crypto expert, but I’m going to assume that the ruby SecureRandom library will do a better job than me at outputting random strings. So it’s of use here. For example:

require "securerandom"

# outputs something like "Ht25IeNqIBUp"
SecureRandom.urlsafe_base64.gsub(/[^a-z0-9]+/i, '')[0,12]

This strips non-alphanumeric characters such as ‘_’ and ‘-’ out. This is useful because people are less used to typing them, and also, certain mouseclick-to-copy behaviors will split on those characters depending on the environment.

It’s worth noting that certain characters often will confuse users if they are manually entering a password as viewed from the screen. Fonts can make characters including the following difficult to distinguish

0 (zero)
1 (the number one)
I (the uppercase i)
O (the uppercase OH)
l (lowercase L)

You can String#tr these out for substitutions, or strip them alltogether. This will slightly reduce entropy, but by keeping a longer password you compensate somewhat.

An Example of Wrapping

2013-01-15T00:00:00+00:00

You’ve been tasked with adding comments to some internal system at work. You throw together some new controllers and views into your app, and churn out the feature quickly and efficiently.

A few days pass, and a peer comes and informs you, “Hey, have you seen the comments? Some people are swearing up a storm and Bob is irritated!” You are left wondering, what to do. You quickly discover there’s an Obscenity gem for Ruby, and get cracking. At stage one, you’re just going to output sanitized versions of comments, rather than resort to draconian measures.

Let’s assume an overly simple, Comment model with one property, content, that looks like this:

Comment = Struct.new(:content) do
  #...
end

Dont’ worry about database, etc, it’s beyond the point right now. Dropping in a #clean_content method is quick:

Comment = Struct.new(:content) do
  def clean_content
    content && Obscenity.sanitize(content.dup)
  end
end

Now off to update the views and change the references to @comment.content to @comment.clean_content and you’re done. Wait, not so fast, that’s only one option, with others to consider. Possible options include:

Changing the view references, as mentioned
Using a helper method like sanitized_comment(@comment) to return the clean content
Opening your model back up and changing the content to return sanitized content, and storing the original content in #unsanitized_content
Wrapping your @comment instance and taking advantage of Ruby duck typing.

Here’s a quick example of accomplishing the last. The presenter/exhibit/delegate pattern in Ruby are often presented as a way to decorate new methods onto an instance, such as taking an underlying object with an #amount_in_cents attribute and adding a new method for outputing it as readable currency. Another way to leverage this is to intercept calls to an existing method, like #content, and change its behavior. Let me show you what I mean.

First, SimpleDelegator can provide an easy wrapping for instances:

class CleanComment < SimpleDelegator
  # In case you want to get back at the original
  def unsanitized_content
    __getobj__.content
  end

  # Ensure clean content
  def content
    clean_content
  end
end

When you want to sanitize the comment, say after finding it via a controller, wrap it:

@comment = CleanComment.new(comment)

Now your views can keep rolling on with a calls to @comment.content and be none the wiser. Remember, Ruby’s duck typing is powerful; rely on what instances respond to, as opposed to what they are instances of.

This is partially a matter of taste, remember there’s often not a “right way”. What’s important is to have options, and leverage the option that feels right given the situation at hand. Different approaches have different pros & cons. With wrapping, for instance, you have to remember to wrap! And if it’s a collection, you must wrap them all. There’s gems like draper or display-case that can help you on your way.

Two Support Objects You May Have Missed

2011-10-16T00:00:00+00:00

If you spend time daily in a large ruby project (such as a Rails app) that has ActiveSupport pulled in, you are likely relying on its string, time, hash, and other extensions. I’ve found two objects it provides prove useful, and having found them lesser known amongst my coding friends, figured they are worth sharing.

The first useful tidbit is the ActiveSupport::StringInquirer class. It’s a simple method missing call that lets you do prettier equality tests on strings. If you’ve ever done a Rails.env.development? check, it uses this implementation. Let’s go to the code:

Rails.env.development? # => true
Rails.env              # => "development"
Rails.env.class        # => ActiveSupport::StringInquirer

si = ActiveSupport::StringInquirer.new("foo")
si.foo? # => true
si.bar? # => false

I think this is great for two reasons. First, it’s a more expressive use of code, and secondly, it implies less coupling to a string outside of an object. Let’s take a simple example: a role for a User object. Imagine you start simple, where role is just a string. Now let’s say we’re using CanCan to add simple authorization to our app, with an ability class that looks like this:

class Ability
  include CanCan::Ability

  def initialize(user)
    user = user || User.new

    if user.role == "admin"
      can :manage, :all
    else
      can :read, :all
    end
  end
end

Note we’re doing an #== for comparison on that role. This is a bit ugly and not as expressive as I’d like. Let’s get rid of ugly with the help of the StringInquirer.

class User < ActiveRecord::Base
  def role
    role = read_attribute(:role).to_s
    ActiveSupport::StringInquirer.new(role)
  end
end

class Ability
  include CanCan::Ability

  def initialize(user)
    @user = user || User.new

    if role.admin?
      can :manage, :all
    else
      can :read, :all
    end
  end

  private

  def role
    @user.role
  end
end

I’ve redefined User#role to wrap the attribute in a StringInquirer, and updated the Ability class to call the role with the predicate #admin? method. Our end behavior for ability checking is the same, but I think we’ve got more readable code. There’s another win on this: we’ve decoupled from treating our role like a string which can pay out nicely in the future. Imagine for instance the day arrives when a user no longer has one role, but many. A simple user may be able to function as both a forum moderator or a comment moderator. You can shift to supporting many roles per user with a bitmask method and leave your external calls untouched. A simple application of define_method or method_missing on your role attribute wrapper is all you need to keep rolling. Now, you could also define #== on your role object for such string comparisons, but comparing to a string reads more like the caller knows too much of an implementation detail. I haven’t touched on the User#role= setter here; you may need some sanitizing and cleanup on it if you were assigning it the results from the getter method anywhere (and, ahem, possibly breaking encapsulation with your own string assignments, too). I’ll leave that as an exercise for the reader.

Our second friend is the ActiveSupport::SecureRandom interface. Actually, saying this is from ActiveSupport is a little misleading. If you are working on an older Rails 2 project, you’ll probably be using this by way of ActiveSupport. However for modern and future use, this is deprecated and delegated to Ruby 1.9.x stdlib’s SecureRandom. SecureRandom is great for generating random character strings on the fly that are useful as API keys, temporary passwords, tokens, etc. It’s simple to use, and can replace those naive calls to rand() you’ve been making for generating random strings. Don’t reinvent the wheel! I’ll leave you with a few examples:

SecureRandom.hex    # => ace59c788b498fadcaa88216e45cf800
SecureRandom.base64 # => iJKR2NQ8Jk1wBdp0nU/fhA==

# Optionally pass 5 for 5 hex pairs
SecureRandom.hex(5) # => a5f8bf212f

Spec your codebase, not just your code

2011-10-01T00:00:00+00:00

Many rubyists are familiar with using Test::Unit, RSpec, or MiniTest for specifying the behavior of their application code. Over the past year, I’ve discovered a novel and less conventional use of testing tools I like to call “specifying the codebase”. Let me show you what I mean.

If you’ve ever worked on a sizeable Rails project, you’ve probably experienced some codebase quirks at some point. For example, I’ve got an application that had issues when the Liquid Templating engine was loaded as a gem. The fix was to include a specific version as a plugin. While it’s usually best to get to the root of the problem, sometimes it’s not feasible or expedient and such measures must suffice. Once Liquid was included as a gem, we noticed some failing tests, and then pieced together that, oops, Liquid had been reconfigured as a gem. A code review may have caught this, or perhaps not, since it’s a little quirky. Now the second time we made the mistake, it dawned on me, write a spec to send up a red flag and make it easier to spot the immediate problem when tests fail in the future:

describe "The codebase" do
  it "doesn't load the liquid gem" do
    Gem.loaded_specs.any? {|s| s.first == "liquid"}.should be_false
  end
end

It’s pretty easy to make the mistake of not properly compressing or optimizing assets, especially images, in your app’s public directory. Code review or possibly even your vcs can help prevent such problems, but you can also let your specifications alert you to the problem:

it "/public directory includes files less than 1MB" do
  files = `find #{File.join Rails.root, "public"} -type f -size +1M`
  files.split(/\n/).should == []
end

The pattern emerging here is pretty simple: if you or your teammates make any repeat mistakes with your codebase, write a spec to prevent further repeats. By all means, make sure to communicate and talk out application guidelines, yet, nothing says “hey, over here!” like a failing test.

The applications here are pretty endless. Maybe you’ve got a Sinatra app on Heroku where you precompile dynamic stylesheets, and have made the mistake of checking in a busted css one too many times; there’s a spec for that! Or perhaps you’ve got an overzealous teammate adding or removing crazy things from git file index. First, do better code review and, secondly, there’s a spec for that:

it "leaves config/database.yml out of the repository" do
  git = Git.open(Rails.root)
  git.ls_files("config/database.yml").should be_empty
end

Finally, there’s some important safety notes for techniques like this. First, you may find yourself using system commands like find or grep. If you’re confident you’ve got a fairly consistent environment your application is developed and run on (i.e. *nix boxes) this is probably OK. However, don’t trickle this practice over towards reusable code, like gems you write and share with the world. It’s not kind to Windows users out there. Secondly, tests like this are often going to integrate with a real filesystem, so for goodness sake stick to read only operations. Don’t touch, rm, cp, or mv files about. If you find yourself wanting to perform destructive operation, the rakefiles are that-a-way. And again, don’t share this behavior with your application or gem code, but rather, use gems like FakeFS for testing in isolation of the filesystem.