Thomas Mango

How I Keep Limited Run Running

2014-02-21T00:00:00+00:00

A little less than three years ago, I wrote a post that talked about how I kept our web services running, while being the only developer and operations person in our company. Since then, the amount of traffic we handle has increased more than 3000%, our revenue has grown over 5000%, and this topic is more important than ever.

And yes, I’m still our only developer and operations person.

Background

I’m Tom, and I’m half of Limited Run, Card Included, and a few other things. I also organize most of the Rails Rumble, but that’s a seasonal thing.

For our consumer facing services, I’m the only developer and the only operations person. Having an actual work-life balance while in this role has taken a serious amount of time to develop, but that’s for another post. This post is about how I manage to keep our services, specifically Limited Run, up and running. While some services’ availability is less critical than others, Limited Run is a self-serve store platform. We have customers all over the world. These store owners must have the ability to work on their store 24 hours a day, 7 days a week, 365 days a year. Additionally, every second one of their customers cannot visit their store or make a purchase, is money they’re losing. It doesn’t matter to our customers or our customers’ customers, that I’m out to dinner, or at a baseball game, or asleep at 3 am. If something should happen, it needs to be fixed as quickly as possible.

In companies with a larger staff than ours and similar availability requirements, on-call schedules help to ease the situation. Further, operations people spread out across the world help to keep everyone sane. We’re not in that situation, so we’ve built an enormous number of automated systems, checks, and fail-safes over the years to ensure we can keep everything operating as smoothly as possible. However, simply having more employees available to resolve issues, is no excuse for not doing everything possible to fortify your infrastructure.

There are three important areas I focus on to keep things running smoothly: monitoring, automation, and on-the-go preparedness. Here’s how I approach each of them.

Monitoring

Without thorough monitoring, you absolutely cannot expect to be able to keep your service operating. I use and seriously endorse the following services to help with monitoring:

Honeybadger - If your application isn’t configured to notify you of errors, just stop reading right now and go set something up. There is just no excuse for that. We used to rely on email notifications directly from our application for this, and that served us well for a while. That is, until we ran into an issue with some dkim signing logic that prevented email notifications - even those for errors - from being delivered. So, we went with Honeybadger to handle notifying us of errors, and haven’t looked back.
Clicky - This is a fantastic, realtime analytics service. Not only does it offer everything you’d expect from a Google Analytics type service, but statistics come in as they occur. Clicky will tell you approximately how many people are currently on your site and, with Spy, will even let you see every action as it happens. If you aren’t using a realtime analytics service like this, you’re flying blind. The benefit of knowing exactly what’s going on, at any given time, is immeasurable.
Pingdom - On it’s own, Pingdom is extremely useful. Get an email, SMS, or push notification if your site goes down. An uptime monitor like Pingdom is essential if you want to know when there is a service disruption.
Amazon CloudWatch - We are heavy users of and believers in Amazon Web Services. Their CloudWatch service is essential for not only automation (see below), but monitoring as well. We’ve configured various triggers, mostly related to anomalies, that will send us email notifications that can be acted upon in different ways.
PagerDuty - Email notifications are well and good, but at some point, they may become just noise. It’s important to separate the most important notifications, such as those from Pingdom, and bring them to your attention no matter where you are or what time of day it is. That’s where PagerDuty comes in. Additionally, I’m a heavy sleeper, so Pingdom downtime emails are not enough for me in the middle of the night. With just Pingdom, you may still find yourself worrying. However, if you couple Pingdom with PagerDuty, you will never worry again. PagerDuty will call your cellphone within seconds of receiving an email from an uptime monitor, like Pingdom. Knowing that you aren’t missing downtime notifications is extremely comforting. I recommend using this sparingly, for only the most important things, like downtime notifications. Also, as an iOS user that loves scheduled Do Not Disturb so my phone doesn’t constantly buzz from emails all night, adding PagerDuty’s alert phone number to my favorites and only allowing phone calls from those contacts while in Do Not Disturb, can’t be beat.
Twitter - A lesser known feature of Twitter is their SMS and Push Notification delivery of individual tweets (called “Mobile Notifications”). I subscribe to @PagerDutyOps and @Cloud_Status’s Mobile Notifications so that I know when PagerDuty and AWS are having their own problems. We rely on these services to ensure our services stay up, but sometimes shit hits the fan for them too. Stay informed.
New Relic - New Relic is like analytics for your application, rather than your traffic. Together with CloudWatch, I use New Relic to track down performance regressions and determine where the bottlenecks are. The faster your application is, especially in the places that are hit hardest under heavy load, the easier time you’ll have scaling and better handling traffic surges. New Relic was critical for us during a major refactoring of our entire cart and checkout process late last year, which both became magnitudes less resource intensive.
Sprint.ly - While not strictly related to keeping our services running, we rely heavily on Sprint.ly for organizing and planning our day-to-day development efforts.

In addition to paid services, we’ve found it necessary to build a number of custom services to help keep an eye on various things. Here are just a few services that get run via cronjobs:

Clickylert - This is a little script that polls Clicky for the number of people currently on our sites and detects significant, upward changes. When a large spike is detected, Clickylert emails me how many people are currently on the site along with how the traffic changed over the past hour. Additionally, Clickylert can be configured to send an email to PagerDuty, which would call my phone. If Pingdom is how I get notified when something very bad happens, Clickylert is what notifies me when something out of the ordinary is happening, which may or may not lead to something bad. Over the years, our scale has grown so much and our infrastructure has improved even more so, that Clickylert is no longer configured to call me. We have massive stores running on Limited Run sending waves and waves of traffic at all hours of the day and night. It simply became easier to handle huge influxes of traffic out of nowhere, than to worry about spikes in traffic.
EC2 Instance Watcher - A few years ago, we had an issue that resulted in runaway EC2 instance replacements. This was a costly mistake, as you pay for a minimum of one hour for each instance, even if that instance was only booted for a couple of minutes. This script will deliver notifications, first by email, then to PagerDuty if things get really out of hand, whenever our instance count (running + terminated) goes above the range we normally expect it to be in. Nowadays, you can configure EC2 auto scaling groups to automatically email you on every event (which I highly suggest), so this script isn’t quite as useful as it once was.
Backlog Job Watcher - A while back, we had an issue that prevented our background jobs from being properly run. Everything from email notifications to processing lossless audio albums halted for a few hours. This script checks our backlog of jobs every so often and ensures it doesn’t get too long. If it does, it contacts me.

One other important thing we do is use god (there are more modern alternatives, but this is simple and has worked well for us for many years) to ensure various services continue running on each of our servers. We use god to ensure HAProxy, Nginx, Apache, and our background workers continue running. We also have a cronjob that starts god every so often in case it dies, which does happen. Just like subscribing to push notifications about PagerDuty’s and Amazon’s status, layering your monitoring is important. Things may fail, but multiple failures at once are much less likely.

Architecture & Automation

Invest in your architecture. Not necessarily with money directly, but with time. The more of your architecture you can automate and depend on, the better you’ll sleep at night and the more quickly you’ll be able to recover if something goes wrong. It seems like an obvious point, but I think a lot of people don’t take this seriously enough at first. If monitoring will get you half the way, self configuring server instances and auto scaling will get you the rest of the way. Being a solo developer is not an excuse to not invest time in automation. In fact, being a solo developer is even more of a reason to invest the time.

I’m a big believer in Amazon Web Services. As someone who was never really great at server administration, AWS has really helped me wrap my head around a lot of concepts and has given me a great place to experiment with configuring and running various systems. If you’re the only developer of your web service and don’t want to or can’t pay for a fully managed service like Heroku, you’re going to eventually have to learn to configure and fix things. AWS is a great place to learn and practice because you can spin up as many fresh, clean installs of a server instance as you need to get things right and it won’t cost more than a few dollars.

Going further than just using AWS directly for testing, there have been a ton of recent advancements in automated server configuration. I personally use Ansible for our entire infrastructure, but there are other alternatives like Chef, Puppet, and Amazon OpsWorks. I cannot stress enough how important Vagrant and automation software like Ansible is to building out your infrastructure. Pick one, learn it, and use it.

I use Ansible to develop and configure our base machines, but I then turn each into a self configuring AMI. When an instance launches, it pulls down the latest configuration files for the role it’s going to play, checks out any code that needs to be run and connects itself to the rest of the infrastructure. When you combine self configuring EC2 instances with AWS’ auto scaling and load balancing (or with your own custom load balancers, like we do), you’ve already gone a long way in ensuring your service stays up and running. When most people hear “auto scaling”, they think about when you get a rush of traffic and you automatically fire up additional server instances to keep up. Although that’s a main benefit of auto scaling, another benefit is simply keeping your service online. If an instance dies for some reason, a new instance can automatically start up and take its place. Additionally, having all of your instances behind a load balancer (even just a single instance), means that application servers can come and go without affecting connectivity to your service.

To take the idea of auto scaling further, you have to design your systems to be loosely coupled. For most web services, if you remove data from the equation (into Amazon RDS, for example), you’re simply left with processing. For Limited Run, we use a combination of RDS, S3, background queues, and EC2 instances that can be turned on and off at any time, without affecting anything. When you can think of EC2 instances running your web application as disposable processing servers, auto scaling is no longer just about adjusting for traffic, it’s about keeping your service online, no matter what happens.

One final piece of the infrastructure puzzle, is redundancy. Being an AWS user, we leverage RDS’s Multi-AZ support, which places a slave of our main database in another availability zone. Additionally, we run EC2 instances across multiple availability zones, and have Route 53 configured with failovers to backup load balancers in various availability zones. Whatever your specific setup is, the main point is that taking out a single server, or every server in a single availability zone, shouldn’t cause you any problems. Okay, there may be a brief minute or so issue when things failover, but you aren’t left scrambling to fix things. As much as possible, things should be fixing themselves.

On-The-Go Preparedness

Over the years, as our monitoring, automation, and infrastructure has improved, I’ve become much less dependent on having to be prepared to deal with an issue while away from home (where my main office is). Additionally, while I may run web services, and work on software all day, I’m actually not that into technology when I leave the house. For me, as I get closer and closer to 30, part of that work-life balance I mentioned earlier, means being able to step away from technology, or at least, not have it in my face all the time. Having said that, I don’t go anywhere without, at the very least, my phone. But if I’m going out for the entire day, I will generally throw a bag in my car that also has an iPad and MacBook Air in it, just in case. Also worth noting, my iPhone is on AT&T and my iPad is on Verizon. Cell coverage isn’t amazing where I live, so being able to tether on either network, is nice.

On my iPhone and iPad, I have have access to the following apps:

Prompt - A really nice SSH application from Panic that works on iPhones and iPads.
Screens - A fantastic VNC application. Screens will get me into a machine in my home office, letting me do whatever I need to do from a development environment. This is more of a last resort.

In addition to these, I also have apps for the AWS Console, PagerDuty, New Relic, and Pingdom, but rarely use them. If I have my iPad with me and need access to the AWS Console, I end up just using the website, which works great.

Again, this is all just a final safety net in case automated systems don’t resolve something that arises while I’m out. The goal is to not need to rely on these things, but if something does come up, it can be resolved from my phone. Having an iPad or a laptop with me can sometimes speed things up in certain circumstances, but generally isn’t necessary.

Conclusion

Werner Vogel, Amazon’s CTO, famously said: “everything fails, all the time”. You need to design for failure and you need to be ready to react when something goes wrong. For me, as a single developer, that means thorough monitoring, being able to rely on an automated architecture, and always being connected.

Custom Error Pages, AWS's ELB, and Route 53 Failover

2013-08-28T00:00:00+00:00

There has been a thread on the AWS support forums since July 25th, 2011, calling for AWS to add customizable error pages to their ELB (Elastic Load Balancer) product.

AWS recently added health check and failover features to their Route 53 (DNS) product, and now some people are wondering whether or not this removes the need for custom error pages on Elastic Load Balancers:

Nalog wrote: What about using DNS Failover on ELB?

Short answer, no, it doesn't.

Long Answer:

Using Route 53's DNS failover, while similar, is unfortunately not the same thing. With a DNS failover setup, clients would wait the TTL (probably 60 seconds in this case) before re-requesting DNS entries for the request (thus getting a failover IP address). However, Route 53 won't failover until some amount of time passes and it believes that the ELB should be failed over from. So you may be in the 2 minute range at this point.

During this 2 minute or so range, blank pages are being served to clients from the ELB. Additionally, clients (browsers) aren't always great about handling transitions between the blank pages from ELB requests and some failover, say a static site being hosted out of S3. Even with no-cache headers, and updated DNS entries because of the failover, browsers will often cache the results and not show updated pages (due to a failover or recovery) by simply clicking the refresh button. Restarting the browser generally forces a refresh of the page, but end users aren't going to expect to have to do that, nor should they have to.

If ELB was able to serve a custom error page, rather than a blank response, when no instance responded to a request, this would solve all of the above issues. Clients wouldn't need to wait for new DNS records, nor would they be affected by cached pages. They would simply receive the error page as the response of their intended request. If they refresh and the ELB can return a real response, they'd display the real response.

If AWS ever adds the ability to have an ELB respond with a custom error page when no instance responds to a request, it will most likely work like a standard HAProxy setup. With HAProxy, you can configure a "backup" instance. For example, you can configure a backend like:

backend my-web-app
  server service-unavailable 127.0.0.1:8000 weight 1 backup
  server i-eb1c5083 10.154.148.222:80 weight 1 check
  server i-f41c531e 10.141.158.72:80 weight 1 check

listen my-web-http
  bind :80
  default_backend my-web-app

In this scenario, you configure HAProxy with a backend that has 2 real instances and then you also run Apache on port 8000 on the same instance running HAProxy. If neither real instance can respond to a request, HAProxy automatically routes the request to 127.0.0.1:8000 and you would configure Apache to rewrite every request and respond with your error page.


  ServerAlias *

  DocumentRoot /var/www/
  
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
  

  RewriteEngine On

  RewriteCond %{DOCUMENT_ROOT}/503.html -f
  RewriteCond %{SCRIPT_FILENAME} !503.html
  RewriteRule ^.*$ /503.html [L]

Route 53's DNS failover is useful for some things, such as your ELB completely going away, but it isn't a substitute for ELBs being able to actually serve error pages, rather than blank responses, when your instances don't respond.

Goals & Routines or: How I've Lost Almost 90 100 Pounds

2013-07-02T00:00:00+00:00

I can no longer count the number of times that I’ve started writing, deleted, and restarted this post. Although I am proud of myself for what I’ve accomplished over the past year or so, I’m embarrassed for even having a chance to accomplish it. But putting those feelings aside, maybe reading this will help someone else set a goal, and change their routine for the better.

2012

Last year, with my 28th birthday just a few months away, I realized it was time to lose weight. A lot of weight. I kept thinking back about something I read: “once you hit 30, it’s a lot more difficult to lose weight”. I’m embarrassed to admit that, at this point in my life, I had weighed 265 pounds or so for a while. Oh sure, from time to time, maybe I’d be motivated and lose 10 pounds or so, before snapping back to my usual ways and gaining it back, but for the most part, my weight stayed roughly the same for years.

If you’ve never been this overweight, it’s hard to explain. No clothing I owned fit me and I no longer remembered what it was like to be comfortable. Working from home for years, rarely going out and not really having a social life to speak of, made it easy to ignore this all. But after years of being like this and just concentrating on my work, I had gotten into this routine. And while I’m good at routines, this one just happened to be really bad for me.

So it was time for change. Having tried Atkins when I was much younger, losing weight for six months only to gain even more back than I had lost, I was opposed to trying another diet of this type. In fact, I didn’t want to “go on a diet”. I wanted to change the way I lived, and perhaps foolishly, I just wanted to do this on my own. I wanted to drink less coffee, eat healthier, and most of all, eat less.

In March of 2012, I put Weightbot on my first home screen and set a goal of 250 pounds. Instead of eating a bagel with cream cheese for breakfast, despite not being hungry (I soon realized that I was never actually hungry, I just ate to eat), I ate 2 Nutri-Grain cereal bars. Instead of going to Wendy’s and getting a bacon cheeseburger and fries, I went to Wendy’s and got a salad. Instead of getting a chicken cutlet, cheese, and bacon hero with a sweetened iced tea from my favorite deli, I got a tuna melt on a whole wheat wrap and a bottle of water. Instead of getting something delicious from the restaurant around the corner, I got a chef salad. Instead of having cookies at night, I had a yogurt.

I simply began substituting everything I ate with something that was healthier (even if it wasn’t that much healthier - Wendy’s salads, for example). But more than the substitution, I made a point to eat much less than I used to eat. I stopped eating everything on my plate just because it was there. I ate more slowly and tried to be more conscious about when I was actually hungry. I stopped grabbing a snack for no reason while pacing around the office thinking about how to implement a feature for Limited Run.

I was lucky in that I very quickly began to noticeably, at least to myself and Weightbot, lose weight. Seeing results is an incredible motivator. It shows that you’ve made a choice that actually made a difference. Obvious maybe, but important.

I continued updating my goal, first by ten pounds at a time 240, 230, 220, then by less, 215, 210, 205. I continued to try and eat healthier and eat less. Trying to find that point when I was just full. I cut out an afternoon snack. I found that only eating one Nutri-Grain cereal bar for breakfast was satisfying. I continued to iterate on my routine and my goal, getting better each week.

Oddly, I never really got into exercising. I tried, but it didn’t really stick. I didn’t feel like I was accomplishing anything and convinced myself I was still too overweight for it to matter. Deep down, I wanted to be one of those people that could run every day, but I had a goal already. I didn’t share my exact goal of 180 pounds with anyone, but that was secretly my goal and simply by getting into a new routine, I was able to make steady progress toward it. (Having an english bulldog who will only walk a third of a mile with you once every other day didn’t help either, but I guess I can’t blame him.)

After nine months, I hit 182 pounds and plateaued. I was a little disappointed that I didn’t actually get down to 180 pounds, but I decided that losing 83 pounds was an impressive feat and I no longer considered myself overweight (I’m just over 6 feet tall). And if I was being honest with myself, 180 was my really just my secret, far fetched goal, that I never believed I could get close to in the first place. I was damn impressed with 182 pounds.

So, it was the end of the year, I had new clothes that actually fit me (some were even too big) and most importantly, I finally felt comfortable. I felt comfortable with myself and I felt comfortable in public. I’m not ashamed to admit that it feels great when your family sees you and comments on how you lost so much weight. Or even when a complete stranger looks at the photo on your license as you’re buying something and asks if you are really the same person. It just flat out feels good. Physically and mentally, losing that weight was (and still is!) amazing.

2013

As I wasn’t losing any additional weight and it was winter, I decided to ease up. Once or twice a week, I’d eat a bit more than I had been eating previously, or perhaps get something that I really used to enjoy eating. I must stress that, unlike before my weight loss last year, I did this consciously. I continued keeping track of my weight, although not every day, and decided I would be okay so long that I didn’t go over 190 pounds.

Now, I should be clear that although I lost a lot of weight, I hadn’t been exercising and wasn’t in shape at all. Not even in the slightest. On top of that, I became (I suppose I always was?) a weakling. My 55 pound bulldog could now drag me anywhere at will. There’s no stopping him. But I did need some time to reset myself. I needed time to adjust to my new weight, and relax a bit, and enjoy being comfortable, and try and understand how to maintain my weight while keeping things balanced.

The months went on. Without much effort, I gained a few pounds, lost a few pounds, gained a few pounds. In May, however, I hit that magic 190 pounds. Maybe it seems obvious to you, but only then did I realize that the only reason I was able to lose the initial weight was because I had an actual goal. I was striving for something. Toward something. So in my mind, staying below 190 pounds was not the same kind of goal as getting down to 180 pounds. Staying below 190, was more like an alarm. If you hit 190, it’s time to take action. It wasn’t something I could work toward.

So, I hit 190 and realized staying below 190 wasn’t an actual goal. It was time to set one. Setting another weight goal wasn’t necessarily what I wanted, though. Last year I lost a lot of weight. But this year? This year I wanted to become a healthy person. I wanted to exercise. I want to turn myself into a runner. I wanted to become someone that had an ongoing routine of always being healthy, thinking about what I ate, and most importantly, about being active. I didn’t want to just sit in front of my desk all day and night writing code (although I bet our Limited Run customers would like that).

My new realistic goal was to be someone who keeps track of their health and who could go out and enjoy running every day. My secret fantasy goal is to run a 10k (was secret, now that I’ve told you), but in reality, I just want to keep running and be healthy. So, I bought a Fitbit Flex and a Couch to 10k app for my phone. I set goals in Fitbit and began tracking how many steps I took in an average day (not many), and what I ate throughout the day. I started walking and increasing my steps and started on the Couch to 10k program. So while I’m still going after a goal, I’m approaching losing weight differently now. Instead of just losing weight, I want to be healthier and keep track of what I’m eating, while I work on becoming a runner.

In truth, wearing an activity tracker isn’t something I ever thought I’d do, but now that I’ve started, there’s something fascinating about wearing a Fitbit on your wrist. It’s out of the way, but you’re still conscious of it. It’s there. You know it’s there and it’s a subtle reminder that you should eat healthier and that you should be more active. Instead of parking in the closest spot at the grocery store, you park further away. Instead of taking your dog outside to the curb, you trick that lazy dude into walking a loop around the block. And while, psychologically, using a Fitbit is an incredible thing that will help you make better decisions as you start using it, eventually it just helps you get into a better overall routine. Over time, it becomes less about making conscious choices at each turn and instead lets you train yourself to always be making better, healthier choices, with less effort. Plus, as a nerd, it’s also fun. It’s fun to keep track of things and chart your progress and look at stats.

After about six weeks or so of keeping track of what I eat and exercising almost every day, I’m already down below 178 pounds, and more importantly, I feel great. Running is actually fun. I look forward to it and am happy at night knowing that I went for a run that day. It’s also really rewarding being able to run longer and longer each time I go out.

I’m generally a really private person. I’m embarrassed I used to be so overweight and unhealthy, but I’m really happy with how far I’ve come in just over a year and I guess I don’t really mind boasting a bit now that I’m here. And so if you want to lose weight, start exercising, or just be healthier, you should know know that, even in your late 20’s, it’s actually possible. If I can lose nearly 90 pounds, there’s no telling what you can do if you try. It just takes making attainable goals, getting into a good routine, and constantly updating your goals as you hit them.

I’d love to hear from you, so please feel free to send an email to tsmango over at gmail.

Thanks for reading!

Update: Aug 31, 2013

I’ve continued my progress running and eating healthy and have now dropped 100 pounds in total since March of 2012. But much more important than that number (in fact I’m now working to maintain my weight, rather than lose more), is that my usual routine has me running 5km, 6 days a week. Also, last week, I ran my first 10km (that secret fantasy goal I had just a few months ago).

After all of these years, I finally feel like an active, healthy person.

When I started this a year and a half ago, I never expected to be able to come so far and change my life so thoroughly, but once you get into a good routine, have seen, and felt, real progress, it helps to further your motivation. If you’re thinking about making a change to be healthier, I really hope this post proves to you that it’s absolutely possible. Even if your work has you sitting behind a desk all day.

Getting to Know ActiveSupport::Callbacks

2011-09-02T00:00:00+00:00

While writing ActiveConfiguration, I needed a way to run a callback when certain methods were executed. Thankfully, ActiveSupport was already available, so I had access to ActiveSupport::Callbacks.

What is a Callback?

At this point, you may be wondering what a callback even is. Simply put, a callback allows you to run some code (usually a method) automatically when another piece of code is run. In Rails, you’ll commonly see callbacks that run before, after or even around other bits of code. In fact, if you’ve ever used before_create, after_save or any of the other ActiveRecord life cycle callbacks, you’ve already been using the ActiveSupport::Callbacks module, albeit indirectly and through a layer of abstraction.

While those life cycle callbacks are indispensable in an ActiveRecord model, they wouldn’t help me in ActiveConfiguration. So, what if like me you want a custom callback or need callbacks in a class that isn’t an ActiveRecord model?

Walking Through ActiveConfiguration::Option

Let’s start by looking at a simple class with three methods:

class Option
  def default(value)
    @default_value = (value.is_a?(Symbol) ? value.to_s : value)
  end

  def format(value)
    @allowed_format = (value.is_a?(Symbol) ? value.to_s : value)
  end

  def validate!
    ...
  end
end

Here, the plan is to run the #validate! method after the #default and #format methods are run. To do that, let’s first include ActiveSupport::Callbacks:

Next, we’ll define a callback named validate (not to be confused with the #validate! method defined above - more on this later):

After defining a new callback named validate, we’ll use set_callback to actually set a callback up to run:

When using ActiveSupport::Callbacks, I think #set_callback may be a source of confusion for some people. Unfortunately, by showing a complicated example with a subclass in addition to using the same name for both the callback itself as well as the method to be run, the documentation doesn’t help to make it any easier.

Admittedly, my validate example isn’t much better. Here, the callback is named validate and the method that I want to be run when the callback fires is validate! - a subtle, but important distinction.

So, what exactly are we passing to #set_callback?

The name of the callback we used in define_callbacks.
When the method should be run in relation to the callback being defined ¹.
The method to execute when the callback fires.

The way I remember what set_callback means is by reading it backwards:

“Execute the validate! method after the validate callback is fired.”

Putting It All Together

Up to this point, we’ve only defined what should happen when the validate callback fires. We’ve yet to specify when the validate callback actually should be fired.

To do that, we’ll use the #run_callbacks method:

class Option
  include ActiveSupport::Callbacks

  define_callbacks :validate

  set_callback :validate, :after, :validate!

  def default(value)
    run_callbacks :validate do
      @default_value = (value.is_a?(Symbol) ? value.to_s : value)
    end
  end

  def format(value)
    run_callbacks :validate do
      @allowed_format = (value.is_a?(Symbol) ? value.to_s : value)
    end
  end

  ...

  def validate!
    ...
  end
end

(For the unabridged version of ActiveConfiguration::Option, see the source on GitHub.)

That’s really all there is to it when looking to add custom callbacks to your own classes. By wrapping your code in #run_callbacks, any callbacks you’ve setup to run before, after or even around, will be executed at the correct time.

Life Cycle Callbacks in ActiveRecord

Fair warning, things start to get a bit more hairy from here on out, so if you’re just interested in using callbacks in your own class, feel free to skip the rest of this article. But if you’d like to dive into Rails a bit, keep reading.

As I mentioned earlier, ActiveRecord supports a number of life cycle callbacks that are extremely useful. These callbacks are actually defined using ActiveSupport::Callbacks under the hood and after looking at the example above, we should be able to figure out how that’s being done.

First, let’s look at the ActiveRecord::Callbacks module:

require 'active_support/core_ext/array/wrap'

module ActiveRecord

  module Callbacks
    extend ActiveSupport::Concern

    CALLBACKS = [
      :after_initialize, :after_find, :after_touch, :before_validation, :after_validation,
      :before_save, :around_save, :after_save, :before_create, :around_create,
      :after_create, :before_update, :around_update, :after_update,
      :before_destroy, :around_destroy, :after_destroy, :after_commit, :after_rollback
    ]

    included do
      extend ActiveModel::Callbacks
      include ActiveModel::Validations::Callbacks

      define_model_callbacks :initialize, :find, :touch, :only => :after
      define_model_callbacks :save, :create, :update, :destroy
    end

    def destroy #:nodoc:
      run_callbacks(:destroy) { super }
    end

    def touch(*) #:nodoc:
      run_callbacks(:touch) { super }
    end

  private

    def create_or_update #:nodoc:
      run_callbacks(:save) { super }
    end

    def create #:nodoc:
      run_callbacks(:create) { super }
    end

    def update(*) #:nodoc:
      run_callbacks(:update) { super }
    end
  end
end

Some of this should be familiar to you. First you’ll notice that ActiveModel::Callbacks is being extended, so we’ll want to look at that in a moment. Next, you’ll see two calls to define_model_callbacks and multiple mentions of run_callbacks, which we used earlier in our #default and #format methods.

Let’s pick out just one of the methods above to focus on, say create_or_update:

This method is overriding AR’s #create_or_update method and wrapping a call to the existing method (super) with run_callbacks(:save). From what we learned earlier, we know this means that any callbacks to save should be fired here.

But, this doesn’t tell the entire story. We haven’t seen a call to set_callback yet which we know is necessary to actually hook a method up to a callback.

To get to the bottom of this implementation, let’s venture into ActiveModel::Callbacks:

I admit, this may look a little scary, but let’s go through it a piece at a time.

First, our faithful ActiveSupport::Callbacks is included:

Next, we see define_model_callbacks, which was called in ActiveRecord::Callbacks:

def define_model_callbacks(*callbacks)
  options = callbacks.extract_options!
  options = {
     :terminator => "result == false",
     :scope => [:kind, :name],
     :only => [:before, :around, :after]
  }.merge(options)

  types   = Array.wrap(options.delete(:only))

  callbacks.each do |callback|
    define_callbacks(callback, options)

    types.each do |type|
      send("_define_#{type}_model_callback", self, callback)
    end
  end
end

If you’ve ever gone spelunking in the Rails source before, the beginning of that method where options are being extracted should look familiar, otherwise, let’s not worry about it.

The more relevant part to this discuss is when the array of callbacks is iterated over. For each callback, define_callbacks (the method we used earlier) is called. This means callbacks are defined from those specified back in ActiveRecord::Callbacks such as :initialize, :find, :touch and :save, :create, :update, :destroy.

Additionally, after specifying each callback, the allowed types (before, after, around) are iterated over and the actual callback methods are defined.

Let’s just take one example. One of the callbacks defined was save. For this callback, send("_define_#{type}_model_callback", self, callback) would be called three times (once for each of before, after and around). Looking further down the implementation, we’ll see the three methods being called:

def _define_before_model_callback(klass, callback) #:nodoc:
  klass.class_eval <<-CALLBACK, __FILE__, __LINE__ + 1
    def self.before_#{callback}(*args, &block)
      set_callback(:#{callback}, :before, *args, &block)
    end
  CALLBACK
end

def _define_around_model_callback(klass, callback) #:nodoc:
  klass.class_eval <<-CALLBACK, __FILE__, __LINE__ + 1
    def self.around_#{callback}(*args, &block)
      set_callback(:#{callback}, :around, *args, &block)
    end
  CALLBACK
end

def _define_after_model_callback(klass, callback) #:nodoc:
  klass.class_eval <<-CALLBACK, __FILE__, __LINE__ + 1
    def self.after_#{callback}(*args, &block)
      options = args.extract_options!
      options[:prepend] = true
      options[:if] = Array.wrap(options[:if]) << "!halted && value != false"
      set_callback(:#{callback}, :after, *(args << options), &block)
    end
  CALLBACK
end

If you look at the first method, _define_before_model_callback, you’ll see that a class method is being dynamically defined for the callback given (save). So, this call to _define_before_model_callback will define the method before_save and make it available for you to use in your ActiveRecord model, like so:

class Post < ActiveRecord::Base
  before_save :generate_slug

  def generate_slug
    self.title = self.title.mb_chars.normalize(:kd).gsub(/[^\x00-\x7F]/n, '').to_s.gsub(/[']+/, '').gsub(/\W+/, ' ').strip.downcase.gsub!(' ', '-')
  end
end

Now, if you look again at the the before_save method that was defined from within _define_before_model_callback, you’ll see a call to set_callback. This means that a method you pass to before_save will be passed into the set_callback method and will be executed before the save callback.

Looking back to ActiveRecord::Callback, you’ll remember the following method:

This means that the method passed to before_save in the Post class above is set as a callback method to the save callback and before the #create_or_update method is run, the callback method you set will be executed.

Final Thoughts

When used thoughtfully, ActiveSupport::Callbacks can be very helpful and I think this example of how Rails uses the underlying module to expose specific life cycle callback methods to ActiveRecord models is fascinating. And really, this module is just one of the many excellent things you’ll find in the Rails codebase. So while it may feel overwhelming to try and understand the entire Rails source all at once, taking an example like this where you look at a specific method and peel back the layers of how it’s implemented can be very approachable and can lead to understanding and reusing some great code that’s already available to you.

If you have any feedback, please feel free to send me an email. I’d love to hear from you.

This can either be before, after or around. ↩

ActiveConfiguration: A Generic Settings Store

2011-08-30T00:00:00+00:00

ActiveConfiguration is a Rails engine that exposes a generic settings store to ActiveRecord models. Made for very configurable applications, it allows you to avoid implementing specific ways to store settings for each model that needs such a configuration. If your application isn’t very configurable, ActiveConfiguration isn’t what you want.

For example, if your application had a Category model that only had a configurable sort attribute, ActiveConfiguration would be overkill. Rather, you would just read and write values using a sort column and restrict the allowed values using something like validates_inclusion_of.

However, if your Category model was more flexible in its configuration, you may want a sort setting, a limit setting and multiple price_filter settings that can be configured by your end user. Without ActiveConfiguration, you would have to develop a way to store and validate these settings for this specific scenario. The sort and limit settings are simple but because price_filter can accept multiple rules, you’d have to set up an additional model. Still, this isn’t really an issue when you’re dealing with just a single configurable model. When you’re dealing with many, things tend to get messy.

With ActiveConfiguration, all of your settings, even for price_filter, can be stored in a generic way. ActiveConfiguration provides a place to store settings for each of your models and even handles validation when you restrict the allowed values or format of an option.

Source

The source for ActiveConfiguration is available at GitHub.

Installation

Add the following to your Gemfile:

Generate the migration for the settings table:

Migrate your database:

Example Configuration

class Category < ActiveRecord::Base
  configure do
    option :sort do
      default  'alphabetical'
      restrict 'alphabetical', 'manual'
    end

    option :limit do
      format 'fixnum'
    end

    option :price_filter do
      format    'float'
      modifiers 'eq', 'lt', 'gt', 'lte', 'gte'
      multiple  true
    end
  end
end

After installing ActiveConfiguration, the #configure block is available to every ActiveRecord model. If the #configure block is defined with a valid configuration, additional methods are made available on the model.

Example Usage

>> category = Category.create(:name => 'Vinyl Records')
=> #<Category id: 1, name: "Vinyl Records", created_at: "2011-08-03 15:46:11", updated_at: "2011-08-03 15:46:11">

?> category.settings
=> #<ActiveConfiguration::SettingManager:0x10e7d1950 @configurable=#<Category id: 1, name: "Vinyl Records", created_at: "2011-08-03 15:46:11", updated_at: "2011-08-03 15:46:11">>

?> category.settings[:sort]
=> {:value=>"alphabetical", :modifier=>nil}

?> category.settings[:sort][:value]
=> "alphabetical"

?> category.settings[:sort][:value] = 'manual'
=> "manual"

?> category.settings[:price_filter]
=> []

?> category.settings[:price_filter] = [{:modifier => 'gt', :value => 10.00}, {:modifier => 'lte', :value => 25.00}]
=> [{:value=>10.0, :modifier=>"gt"}, {:value=>25.0, :modifier=>"lte"}]

?> category.save
=> true

?> category.settings[:sort][:value]
=> "manual"

?> category.settings[:price_filter]
=> [{:value=>10.0, :modifier=>"gt"}, {:value=>25.0, :modifier=>"lte"}]

Additional Notes

I accept that ActiveConfiguration’s concept is a bit out there, but I’ve found it very useful in certain circumstances and hope that others find it useful as well. I think the README provides all of the information necessary to get started using this gem, but there are additional details in the documentation.

If you have any questions or feedback, I’d love to hear from you. Please feel free to email me or open an issue at GitHub.

Generated by Jekyll & Open Source Plugins

2011-08-26T00:00:00+00:00

I’ve spent my free time over the past week replacing my existing site with this new, Jekyll generated site and I couldn’t be happier with how it came out. Jekyll, written by GitHub’s Tom Preston-Werner, is an extremely extendible static site generator. While developing my new site, which is hosted in an S3 bucket on Amazon, I ended up implementing a couple of simple plugins that I’ve opened sourced in hopes that others may find them useful. If you’d like to see the entire source of this site, check out the GitHub repository and if you’d like to learn more about my plugins, keep reading.

AliasGenerator

This generator looks through the YAML Front Matter of each post and generates redirect pages when an alias configuration is set. I found this plugin necessary when migrating my existing posts on tumblr to a more sane url structure.

Example Configuration:

---
  layout: post
  title: "How I Keep Limited Pressing Running"
  alias: /post/6301645915/how-i-keep-limited-pressing-running/index.html
---

This alias is set on my “How I Keep Limited Pressing Running” article which was originally published at tumblr and will redirect anyone going to the old address.

View the source at GitHub →

FlickrSetTag

This tag generates a simple image gallery from a set of photos on Flickr.

Example Usage:

This will result in a bit of generated html displaying thumbnails of each photo in the set. Check the repository’s README for configuration details.

View the source at GitHub →

More About Plugins

Although there are a decent amount of plugins already available for Jekyll, like me, you may not find the exact plugin to suit your needs. Luckily, Jekyll makes it pretty straightforward to write your own. I highly suggest picking out a few interesting plugins and reading through the source and understanding how they work.

How I Keep Limited Pressing Running

2011-06-07T00:00:00+00:00

Splitting a Large Rails Application in Three

2011-05-06T00:00:00+00:00

After weeks of planning and a month of development, we finally split Limited Pressing’s free auction, trading and collecting services out into a separate site last week. Although I have been involved in a couple of “big rewrites” before, this project was the most complicated I’ve ever done. It’s one thing to rewrite a large, internal application or an entertainment application, but Limited Pressing is a large e-commerce platform that people depend on. People are constantly buying and selling things, so downtime had to be minimized and everything really had to go just right.

A Brief History of Limited Pressing

Before going into details about splitting the application itself, I should back up and briefly explain how Limited Pressing evolved over time. LP, an e-commerce platform catering to the music industry, has been in development for about two years. The original idea was to be something of an eBay alternative for independent music. We had three major points we wanted to hit: auctions, trading and stores. We launched first with auctions and trading and immediately went to work on stores, the only feature we were going to charge for.

The Need to Separate Stores & Community

After releasing stores to the general public, we began to get a lot of interest in them. We had solved a lot of issues people had with other store platforms and provided a great set of features specific to labels, bands and musicians. Unfortunately, it became clear that our free auction and trading services were distracting from our stores service. A lot of people were confused as to if we were a store platform or a community marketplace. We tried to be both for a while, but these two aspects clashed more and more. Additionally, users of the free services didn’t really want to be bothered with stores and it was hard for people looking for a store to make it past our other services.

In the end, we decided that for both aspects to be as good as they could be, we had to move our free auction and trading services out into a separate site completely. This new site is called The Old LP.

Extracting Authentication & Being an OAuth Provider

Luckily, there was very little overlap between stores and community features. There were a few low level utilities related to currency and region support and then there was our user and authentication system, but otherwise, things could be clearly separated. Without this minimal overlap, we may never have be able to go down this road.

The first thing I did was duplicate the codebase into a new application. I then began to delete every controller, model and view that had nothing to do with that application. I removed auctions, deals, collections, messages, etc. from Limited Pressing and I removed stores and all store related features from the new application, The Old LP.

After removing features, I completely tore out the authentication systems, which was built on top of Authlogic. Like many Rails developers, I was lured into using a bloated authentication system. I’ve tried a number of them including Authlogic and Devise. Unfortunately in my experience, I feel like these authentication systems always end up either being overkill or requiring too much overriding and customization that it makes no sense to bother using them. Because one of our requirements was to have the same logins across both applications, I took this as a chance to get rid of Authlogic. I ripped it all out and started building another application, called Multipass.

Multipass is my favorite part of this project. Very simply, Multipass started with the Limited Pressing database. I removed every table, except the users table and then removed all application specific columns from the users table and boiled it down to just user accounts. My inspiration for Multipass was 37signal Accounts: a secure single sign on service that works across all of our applications.

I’m not sure how 37signals went about doing cross application authentication through their accounts system, but with Multipass, I decided to make it an OAuth provider. To make this happen, I started with oauth-plugin. This is a fantastic library, but there were a few things I wanted to avoid, which meant I had to make a number of changes to what oauth-plugin gives you by default.

For now, I disabled the ability for non-administrator’s to create client applications. Although we don’t currently want anyone to be able to create an OAuth Client Application and use our login system on their own, this is something we may allow in the future. We have a lot of Limited Pressing IDs and there may be other music related sites and services that would find it useful to piggy back on our authentication system at some point.

The bigger change was that we didn’t want users to have to allow access to their account when signing into Limited Pressing or The Old LP (or any future application of ours). Although the idea of granting access to your account has become more mainstream because of Twitter and Facebook, I don’t think everyone gets it yet. It’s confusing and scary for people who don’t understand what’s going on.

I decided to implement a similar authentication flow to Twitter’s “Sign in with Twitter” /oauth/authenticate flow. Normally, every time a user attempts to log into a client application, that user would have to allow the third party application to use your account - this is how oauth-plugin works by default. However, “Sign in with Twitter” only requires a user to allow the third party application once. All subsequent logins skip the allow / deny process and send the user right back to the third party client. I achieved this by first modifying the generated RequestToken model. Rather than always generating a new access token on every sign in (oauth-plugin’s default), I now first check if a valid token exists for that user and client application. If one does exist, I return that and skip right over the allow / deny request.

Although this helped a lot in that users wouldn’t have to re-allow every time they signed in, this wasn’t good enough. I already know that Limited Pressing, The Old LP and any other applications of ours are trustworthy, so why should I ask the user to grant access to the account? I modified my new /oauth/authenticate style flow further to detect whether or not the client application that’s requesting access was trusted or not (I define a trusted application as one that was created by and tied to an administrator). If the application is trusted, not only do I skip all subsequent allow / deny requests when a valid access token already exists, but I even skip the initial request when a new access token needs to be generated and I automatically allow access.

By implementing the concept of trusted applications in addition to the Twitter style “Sign in with Twitter” authentication flow, this allows Limited Pressing and The Old LP to send a user off to Multipass for authentication, but does not require that user to specifically grant access to our trusted applications. If we end up allowing third parties to create client applications in the future, they will not be inherently trusted and will require the user to allow the request for access.

On the client side of things, I wrote a custom OAuth based strategy for the excellent OmniAuth gem. This allowed me to quickly and easily tie into our Multipass authentication system. I then removed duplicate user fields from the client applications and wrote a plugin to automatically pull fields that exist in Multipass accounts from Multipass directly. That means if you load up an instance of a User model in Limited Pressing and call first_name, last_name, email_addreess, etc., those calls automatically pass through that User and hook back into Multipass for the current values. This means not only are we not duplicating data across applications, but all existing code in each application that called those values on user objects wouldn’t have to be changed. Additionally, all objects in each application still point to the local users table and use those ids for foreign keys, but every user also has a uid that represents the associated Multipass user.

By turning Multipass into an OAuth provider, not only do we now have a secure, single sign on system that can be used across all of our current and future related web applications, but we also now have a proper authentication system that can be used for a possible, future API.

Maintainability & Our Support System

Aside from the business related decisions in splitting Limited Pressing into two services, there were also benefits in terms of future maintainability. Limited Pressing’s codebase is roughly 2 years old and it had numerous customer facing features, in addition to authentication, administration tools and our custom help and support system.

With the launch of The Old LP, I removed authentication from the mix and built Multipass. However, our custom help and support system¹ is still baked right into Limited Pressing. I plan to continue down the path of reducing the complexity of each application by moving our support system into it’s own, separate application.

Looking past the obvious benefits of smaller, more focused codebases, splitting up Limited Pressing has other advantages. First, our services are now decoupled from each other. If our store platform does not depend on our auction system, something affecting our auction system will no longer affect our store platform. Next, breaking LP up makes it easier to upgrade everything to Rails 3 in pieces. Limited Pressing and The Old LP are still running on Rails 2.3.11, but Multipass is a Rails 3 application and when removed from LP, our support system will also be running on Rails 3. I will be able to upgrade LP and TOLP to Rails 3, individually, as time permits.

Final Thoughts

Although this was a big undertaking, a lot of the work was in planning, testing and deployment of the three resulting applications. I ran through tests in all three systems and audited every bit of code to ensure everything would continue working properly after being separated. I spent days developing and rehearsing a deployment strategy. I took snapshots from our production database and wrote a detailed plan for our period of scheduled maintenance. During maintenance the three new applications would be deployed with their databases starting as duplicates of the production LP database. Each database was then stripped down to the necessary tables and modified to support changes to that specific application. I even timed myself and figured out what operations could be done at the same time to shorten the maintenance window. For example, I ended up deploying each application while restoring and migrating the next, depending application’s database to minimize downtime.

But in the end, despite all of my planning, there were still some bugs that made it into production - something I knew could not be avoided no matter how much preparation was done. I spent the day of our deployment putting out minor fires here and there. And now that a week has passed and I can reflect on the entire project, I’m incredibly relieved it’s over, but also extremely excited about our newly de-coupled applications. It was absolutely worth the effort.

This is essentially a custom built support system, like Tender Support, only less polished. At some point we may just move to Tender, but for now, this works for us and only had an upfront cost of a weekend or so to build. ↩

Changing the Conversation

2011-05-06T00:00:00+00:00

We have a very easy to use digital sales system on Limited Pressing. You create a product in your store and if you want to attach a digital download to that product, you upload a file and then purchase credits (either in batches of 125, 400 or 1000). Whenever someone orders that product from you, they get to download the file.

Back when we rolled out our new support system, we thought it was a great opportunity to collect and review the most common questions we received. After a few weeks, we figured we could create help documents based on the questions we’d been asked. As it turns out, one of the biggest questions was whether or not digital credits could be used across multiple products. If you purchased 1000 credits on one product, could you use those credits on all of your products? To everyone’s disappointment, the answer is no.

We have a valid reason for doing things this way: it costs us money to store your files. We can’t let you buy 1000 credits and upload 1000 albums. We’d go broke, Limited Pressing would shut down and you’d be stuck selling your digital goods from somewhere else that’s way more expensive. Neither of us wants that.

My co-founder (and brother), Nick, suggested we make a single change to combat this issue: stop using the word “credit” and instead use the word “download”. The problem is that when people hear the word “credit” they think of a bank. They think of a big stockpile that they can grab from whenever they need it. We decided to make the change to our marketing pages, in our product editor where you purchase the “downloads” and when we speak to customers about setting up digital downloads. Incredibly, this simple change to how we talk about and describe our digital system, resulted in us never receiving another question about whether “downloads” could be used across multiple products.

Sometimes you need to change the way you do things in your application to make it more clear to your users what’s going on and other times you just need to change the conversation.

Resque and Very Fast Jobs [Spoiler: Avoid Them!]

2010-11-18T00:00:00+00:00

Although I suspect most people using Resque already know to avoid this, I thought I would write about an experience I had with Resque earlier this week in the hope that someone will benefit from what I learned.

Background

We recently launched version 2.0 of Firefly (my day job). With 2.0, came a completely re-written backend revolving around Twitter’s new Site Streams API. Firefly 2.0 is focused on seeing where your friends are. It does this by collecting tweets with location information (including geo tagged tweets, tweets with photos that are geo tagged and third party check-in service tweets like Foursquare, Gowalla and many others). Once you sign into Firefly’s iPhone app or website, you’re added to one of our available Site Streams and we start receiving and processing your home_timeline in addition to various other messages that come through a Site Stream connection.

Backlog

A very important thing to remember when working with any of Twitter’s streaming APIs is that you must read from the stream as quickly as possible. If you don’t read the stream fast enough, your connection will be terminated. If your connection keeps getting terminated because you aren’t reading fast enough, they may block your access. With this in mind, I decided to use our existing Resque background processing setup. I took every item that came through the site stream and queued it up to be processed by a Resque worker later. I figured, “most of these items we ignore so the job will just happen instantly and it won’t be a big deal, plus we get to read from the stream instantly because we aren’t doing any work inline”. If you’re shaking your head right now, you already know where this is going.

Here’s what I naively forgot to consider: the length of time it takes to process a job != the amount of time it takes to process the logic you wrote inside of that job. What’s missing from the equation? The overhead in processing the job! It takes time for a Resque worker to start and stop processing a job and as it turns out, it takes enough time that when we hit a certain threshold of active users on Firefly, our queue started growing in size at an insane rate.

We peaked at close to 60k jobs in the queue before I could shut our Site Stream connections down, start investigating the issue and work on a fix.

Solution

When I realized what was going on, it turned out that there was an obvious fix: reduce the number of unnecessary jobs being queued into Resque. Because of the nature of Firefly, we only end up processing a small subset of tweets (ones that we can collect location information on). Almost every other tweet and streaming message just gets passed over and these are the streaming items we don’t want to queue to be processed later. The key to doing this is to remember that these streaming items still need to be processed as fast as possible to avoid slowing down the reading of the Site Stream connections.

First, I ensured that we never went to the database when determining whether or not a streaming item was something we wanted to queue. Next, I wrote a set of rules, specific to Firefly, that would recognize and reject any streaming item we didn’t care about. And lastly, I ordered the rules so that the the fastest to process were first (each rule can reject a streaming item so if the most expensive rules were last, we always spend the least amount of time to reject an item).

Conclusion

If a job will run so fast that it can be executed inline, don’t queue it. It’s that simple. Just be conscious of every job you’re creating.

It’s embarrassing to admit that I let this through to production, but I did. The important thing is that I learned to be very aware of how long a job is really going to take and whether or not it’s worth it to queue it in the first place. I also learned that I need to start monitoring queue size and setup notifications if the queue continues to grow in size, but that’s a story for another day.

A Support System That Doesn't Hurt

2010-11-06T00:00:00+00:00

We opened Limited Pressing to the public a year ago and up until last week, we’ve been accepting support requests to an email address we provided on the site as well as a public forum. As our traffic grew, handling support requests this way became unbearable. Often, a request involves some discussion internally before responding. To do this, we’d forward the email to each other to discuss it and when we replied, we’d CC each other. It was tiresome and unwieldily. It was also easy to lose track of who we were talking to because we didn’t have any context, just an email address.

Well, that changed last weekend. We said enough was enough. We talked for a few minutes Sunday morning about what we wanted out of a support system and by 10pm the public forum was gone, the support email address had an auto responder and we had a brand new support system in place. And what I find most interesting about the whole thing is that we’ve had more support requests this week than any previous week in the past year. Thankfully, every one of those requests was a pleasure to deal with because of the new system.

Requirements

When designing the new support system, we had a few must haves:

The word “tickets” is too technical for most people.
You shouldn’t need an account to submit a support request.
You should receive email notifications to confirm we received your request and whenever there’s an update to your request.
Staff members should receive email notifications whenever someone submits a request and whenever a request is updated by someone else.
Staff members should be able to have an internal, private discussion right on that support request’s page. Context is everything.
Each support request should have as much information we know about the person asking the question so staff members don’t have to hunt around for it.

There were a lot of other ideas we threw around that would have been neat, but not worth the time. For example, it would be nice to know when other staff members were currently looking at (or typing a response to) the same support request you were looking at. It would be useful, but we wanted to get the new support system up and running as quickly as possible and this just didn’t make the cut.

Another feature that didn’t make it in was the ability for a staff member to assign a question to another staff member. A formal system would be nice, but since we could leave staff only notes on each support request, the same basic thing could be handled without a separate feature.

Frequently asked questions and response templates are a couple of other features we were thinking about implementing. We may still implement these, but were they necessary on the first iteration? Definitely not.

12 Hours Later

The system ended up being extremely simple and did exactly what we needed.

Users, whether they’re logged in or not, ask us questions.
Every question has a unique URL that isn’t guessable. The URLs aren’t completely secure, but they’re “secure enough”.
Staff members can have a conversation about the question right on the question page.
Everyone gets notified when there’s something new to see.
Questions automatically change state between Open, Pending and Solved depending on who answers what and when.

More Support Requests Than Ever Before

With our old setup, an open support email address and a public forum, we received a decent amount of support requests. When moving to the new question system, we setup an auto responder on the support email address that pointed people to the new system and we completely removed the forums. The result? Within 12 hours, people started using the new system. No instructions, no anything. We’ve had about 5x our usual load of support requests with the new setup this week and we’ve spent about 1/4 of the time handling them. The new setup is a complete success.

So why the sudden interest in Limited Pressing support? I have a theory. A lot of people feel like they’re being a bother emailing a support email address, so they just don’t. And the support forums? People don’t want to ask personal questions on a public forum and people don’t want to ask questions they think may make them look stupid on a public forum. People don’t want to be a bother and don’t want to look stupid. It’s that simple. Our private questions area solved both of these issues.

Additionally, we wanted our support system to feel friendly. Rather than “creating a support ticket”, you “ask a question”. Here’s what the ask a question form looks like:

It’s a minor difference that I think changes how people approach it. It’s easy to ask a question. It’s more of a commitment to “open a new support ticket”.

Also, unlike support emails and forum topics, questions don’t require subjects. Instead, you choose from a list of several general categories:

I have a question before getting a store.
I’d like to request a new feature.
I’m confused about how something works.
I think something is broken.
Other.

Conclusion

Without narrowing the scope of the project, I would have spent too long building the new support system and it would have included a ton of bells and whistles we didn’t need. It just wouldn’t have been worth it. With the narrowed scope, I only invested a single day and we still ended up with a vastly improved way to manage support requests.

Always reduce a project’s scope as much as you can and see how it works in production. You can add things later if you really need to, but if you’re lucky, you won’t.

Experimenting with #newtwitter

2010-11-02T00:00:00+00:00

When Twitter recently launched their new site, #newtwitter, I was delighted to learn it was actually an API client. As someone who builds web applications and works on APIs myself, it was inspirational to see this all done with HTML and Javascript. Needless to say, when my account gained access to #newtwitter, I fired up my Web Inspector and started digging around with the console – you can learn a lot with the console’s tab completion.

After experimenting for a bit, I whipped up some quick Javascript that appends a “?” link to each trend. When clicking the “?” link, a standard #newtwitter dialog opens up and an explanation of the trend, fetched from What The Trend, is displayed.

One interesting thing to remember when working with the trends list on #newtwitter is that trends update every once in a while. Simply executing this script once won’t do the trick. I decided to let jQuery’s live function handle that issue.

Check out the source below if you’re interested in how I did it:

jQuery(function(){
  var TrendExplanation = {
    dialog:null,
    trend:null,

    // Open a dialog with the explanation of the given trend.
    open:function(trend) {
      this.dialog = new twttr.widget.Dialog({draggable: true});
      this.trend = trend;

      this.setup().explain();
    },

    // Set the dialog's title and show it.
    setup:function() {
      this.dialog.$title.html('About &quot;' + TrendExplanation.trend + '&quot;');
      this.dialog.$find('.twttr-dialog-content').html('<p>Loading...</p>');
      this.dialog.show();

      return this;
    },

    // Fetch an explanation for this TrendExplanation's current trend.
    explain:function() {
      $.ajax({
        url: 'http://www.whatthetrend.com/api/trend/getByName/' + encodeURIComponent(TrendExplanation.trend) + '/jsonp?callback=',
        dataType: 'jsonp',
        success: function(data) {
          if(data['api']['trend'] && data['api']['trend']['blurb']['text'] != '') {
            TrendExplanation.update(data['api']['trend']['blurb']['text']);
          } else {
            TrendExplanation.update('Unable to find details about that trend.');
          }
        },
        error:function(data) {
          TrendExplanation.update('There was a problem attempting to find details about that trend.');
        }
      });

      return this;
    },

    // Update the dialog for this TrendExplanation with the given explanation.
    update:function(explanation) {
      this.dialog.$find('.twttr-dialog-content').html('<p style="margin-top:5px;">' + explanation + '</p>');
      this.dialog.$find('.twttr-dialog-content').append(footer);

      return this;
    },

    // Hide this TrendExplanation's dialog.
    hide:function() {
      this.dialog.hide();

      return this;
    }
  };

  // When moving over this trend-item, show the explain-trend link.
  // If a Trend doesn't have an explain-trend link, add one.
  $('.trend-item').live('mouseover', function(index) {
    if($(this).find('.explain-trend').length == 0) {
      $(this).append($("<a/>", {
        "href":   "http://whatthetrend.com/trend/" + encodeURIComponent($(this).find('.trend-link').text()),
        "class":  "explain-trend",
        "target": "_blank",
        "text": '?'
      }));
    }

    $(this).find('.explain-trend').show();
  });

  // When moving off of this trend-item, hide the explain-trend link.
  $('.trend-item').live('mouseout', function(index) {
    $(this).find('.explain-trend').hide();
  });

  // When clicking an explain-link, find and display this Trend's explanation.
  $('.explain-trend').live('click', function() {
    TrendExplanation.open($(this).parent().find('.trend-link').text());

    return false;
  });

  // Build a footer with details about this script that can be attached to the dialog showing a Trend's explanation.
  var footer = $("<p/>").css({'border-top': '1px solid #eeeeee', 'font-size': 'smaller', 'margin-top': '10px', 'padding-top': '5px'})
    .append($('<span>Explanation from @<a href="/#!/whatthetrend">whatthetrend</a>.</span>'));
});

Introducing My Rails Rumble 2010 Entry: do*connect

2010-10-17T00:00:00+00:00

My entry to Rails Rumble 2010 is a service called do*connect. If you don’t know what the Rails Rumble is, their site explains it as:

The Rails Rumble is a kickass 48 hour web application development competition. As a contestant, your team gets one caffeine-fueled weekend to design, develop, and deploy the best web property that you can, using the awesome power of Ruby and Rails.

My application, do*connect, gives you a simple way to build and grow a third party developer community around your service. I decided on this project because I feel that developer communities are always hobbled together. Google Groups are often used as a mailing list, sometimes Tender Support is used for documentation (but usually just non-developer support), but many times services have to roll their own.

Just look at SimpleGeo, Foursquare and Gowalla. Everyone does a great job hosting their documentation, but it shouldn’t be so time consuming to start a developer community around your web service. I hope that with do*connect, it won’t be.

Regardless of the outcome of Rails Rumble 2010, I plan to continue expanding on do*connect. I have a handful of additional features I’d like to implement, including some premium ones. I haven’t decided on a cost yet for the premium service, but it will definitely support custom domains and a customizable appearance.

The free version will continue to live on and I expect to add features like a status dashboard (another developer-facing resource that everyone builds themselves – see: Twitter, GitHub, 37signals and SimpleGeo), announcements and extending discussions to act like a mailing list.

One feature I’d like to highlight briefly, that isn’t shown in depth in the below screencast, is the permissions system. Communities can be open, closed or private. Open communities are publicly visible but require you to join to post. Closed communities aren’t publicly visible, but anyone can join. Private communities aren’t publicly visible and memberships must be approved by Community Managers. Additionally, Community Managers can promote any other member to be a Manager, giving them the ability to create Pages and Documentation as well as delete Pages, Documentation and Discussions. I’ve always been a fan of simple permissions and I think what do*connect supports is a nice balance between flexible and simple.

Here’s a quick screencast to explain do*connect:

Testing for Booleans

2010-07-27T00:00:00+00:00

I just released a new plugin called to_b. Just like my other public projects on github, this plugin is extremely simple and is meant to help with something I’ve found very annoying: the lack of a to_b method.

Installing it in your Rails project will add a to_b method to NilClass, TrueClass, FalseClass, Fixnum and String. I’ve found this most useful when providing optional boolean parameters in API methods. I explain this situation a bit more in the README, which I’ll embed below.

README

A simple plugin that adds a .to_b method to various classes to aide in testing for boolean values against variously typed objects.

Why nil, false and true

If you look at the source, you’ll note that nil.to_b returns nil, rather than false. Additionally, a string like ‘notaboolean’.to_b also returns nil. This was done on purpose. I always ensure all the boolean columns in my database are created with :null => false to avoid any issues relating to null values in boolean fields. However, to_b returning nil in certain circumstances allows me the flexibility when using this in conjunction with an API to accept nil, false and true values for specific parameters.

An example of why I want the ability to test for nil, false and true is that if a boolean parameter is left out I want my SQL to search WHERE true OR false, if the parameter is set to true I want it to search WHERE true and if the parameter is false, false. It is simply easier to have a scope like the following:

Rather than explicitly searching WHERE true OR false, I simply leave out the condition. When the value is true or false, I include the condition.

Examples

>> true.to_b
=> true

>> false.to_b
=> false

>> nil.to_b
=> nil

>> 'true'.to_b
=> true

>> 'TRUE '.to_b
=> true

>> 'false'.to_b
=> false

>> 'F'.to_b
=> false

>> 'random'.to_b
=> nil

>> 1.to_b
=> true

>> 0.to_b
=> false

>> 5.to_b
=> nil

Geico's Payment Form: Indentation Matters

2010-07-20T00:00:00+00:00

While paying my car insurance on Geico’s website this morning, I was faced with the same payment form that bothers me every month. Actually, it’s only one section of the form: the saved account section:

Every time I come to this form, I instinctively check the box above my saved bank account information. However, if you read the label attached the checkbox you’ll see that the checkbox actually lets you use a different account.

The fact that the checkbox is above your saved account details and your saved account details are indented, leads you to believe that the checkbox is tied to that account.

I mentioned this on Twitter and, within an hour, @geico_service replied and said they’d tell the website team. I have no idea if they’ll fix it or not, but I figured that was my chance. I edited the html of the form quickly and sent them a mockup of what I always expect to see in that area:

It’s a really minor change, but I think it helps a lot. The original form had the intention of allowing you to continue without doing anything. Your account information is saved so if you want to use it, don’t click anything. I think my mockup here drives that point home more.

You have two options, the default is to use your saved account (notice the radio button is above the indented account details) and the second option, that isn’t selected by default, is below the saved account details and allows you to use a different account.

Little things like this can go a long way when designing a form in your web application.

Thoughts on Draft

2010-06-26T00:00:00+00:00

Earlier this week, 37signals released their first iPad application. Their app, called Draft, is billed as a “straightforward sketch app for iPad” and is being sold for $10.

Reading on, I can see that:

“It only comes in black. You can draw in white or red.”

And:

“It automatically saves whatever you draw.”

At this point, I was baffled. I’m roughly three full screens down and three screenshots deep and it looks a lot like Adobe Ideas (which is free) and a host of other apps that are already available.

But further down the page, there was more:

“You can share sketches via email or Campfire.”

And there we go. There had to be a reason they were charging $10 for this app, we just had to find it.

Now, the problem I have is with the way they’re marketing it. It’s a “sketch app” but oh it also let’s you “share via Campfire”. And judging from the hundreds of comments on their announcement post, there were a lot of people calling them crazy for charging so much for an app that “doesn’t do anything more than Adobe Ideas”. Yet, it does do more and honestly, what are the chances that the minds behind Basecamp, Rework and SVN, aren’t damn sure of what they’re doing?

But I’m getting ahead of myself. I should explain why I’m even writing about this. Over the past few years, I’ve started thinking more about building products than building applications. Not just “can this be done” or “what can be added”, but what makes this a product and how do you sell it. And by sell it, I mean “sell it” for either money or just getting people to buy into the idea. A lot of things have contributed to my thinking more about whole products over the years, and reading what 37signals has had to say is certainly one of those things.

Back to Draft

Let’s first make something clear: the people behind 37signals are incredibly successful and know what they’re doing. They know how to give you what you need and they know how to make you want to take out your credit card. They do a lot of things well, and making money is seriously high on that list.

With that aside, I think they could have launched (and can market) Draft in a much more effective way: by changing the name to “Draft for Campfire”. The only people who are going to buy this app are people who use Campfire. There just simply isn’t any reason to buy this over cheaper sketch apps, unless you use Campfire. And 37signals is clearly directing this at paying, Campfire users. If you had the choice, why would you even want people who won’t spend money on apps, when you have existing customers that probably pay you upwards of $150/year? Existing Campfire users: that’s who it’s useful to and that’s who, I can only assume, they want using it.

So, Instead of taking the criticism, which 37signals does a great job at, by changing the name to “Draft for Campfire”, I think that most people bashing them on that blog post would realize right away the point of the app. It’s no longer a sketch app that also can share via Campfire, it’s a sketch app for sharing via Campfire.

But it’s not just about negative feedback, it’s also about their marketing page. Their current marketing page does an awful job at immediately conveying to me the reason as to why I should buy this app for ten dollars. The fact that it can share via Campfire (mentioned very low on the page), feels a lot different than if the product was billed as for sharing via Campfire right off the bat - even though it’s the exact same app.

It’s a small change, but I’d bet it makes a big difference to a lot of people landing on that page. Then again, I haven’t written two books or launched a host of insanely successful applications, so what the hell do I know?

Transferring a Private Repository on Github

2010-06-24T00:00:00+00:00

Important Update (08-21-2011)

GitHub now has a native support for transferring the ownership of a repository to another user. Just click into your repository’s admin area and scroll down to the Danger Zone™.

Original Post (06-24-2010)

I recently had to transfer a private git repository that was being hosted on Github. It turned out to be embarrassingly easy to transfer the repository while keeping the history intact, but I figured I’d document it in case others were wondering.

What Didn’t Really Work

The first thing I tried was a fork. I added the receiver’s Github account as a collaborator to the private repository. Then, I forked the project on the receiver’s Github account. Afterwards, I removed the receiver as a collaborator from the original project and removed the original account as a collaborator on the newly forked project (which is automatically added by Github).

More or less, this did the job. The newly forked project was it’s own repository and neither repository owner could see the other repository. The only issue was that the new repository still said “forked from {original account}”. The receiver wasn’t happy about this and to be honest, it was a little crude being that they purchased the codebase.

The Correct Way

The correct way to do this is basically just as easy.

Create a new repository on Github under the receiver’s account.
Add someone with access to the original repository as a collaborator to the newly created repository owned by the receiver.
Duplicate your project directory locally (as someone who has access to both repositories).
- cp ./projects/{project_name} ./projects/{project_name}_backup
Change into the project and remove the existing remote origin that points at the original repository.
- cd projects/{project_name}
- git remote rm origin
Add the new remote origin (see Github’s excellent first run screen) that points to the newly created repository owned by the receiver.
- git remote add origin git@github.com:{receiver}/{repository_name}.git
Push to the new repository.
- git push origin master
Delete your local copy pointing to the new repository.
- cd ..
- rm -rf ./projects/{project_name}
- mv ./projects/{prject_name}_backup ./projects/{project_name}

This isn’t rocket science, but I figured it would help someone.

Resque in Production

2010-05-27T00:00:00+00:00

Over at Limited Pressing, we’re using a wonderful background queue called Resque. Resque was built by Chris Wanstrath of Github fame and runs on top of Redis. It’s fantastic and I absolutely love it. If you haven’t had a chance to try it, go check it out.

When setting up Resque in production, I ran into a few issues that weren’t extremely obvious to me. I ended up working through them in the end and thought I’d write something up to help others out in the future.

Monitoring with God

Resque comes bundled with a config file for god, but I had an issue with the way it would start up a worker. My example is only using a single worker, but you can combine my method with the bundled one. Here’s how I did it:

# Redis
%w{6379}.each do |port|
  God.watch do |w|
    w.name          = "redis"
    w.interval      = 30.seconds
    w.start         = "/etc/init.d/redis start"
    w.stop          = "/etc/init.d/redis stop"
    w.restart       = "/etc/init.d/redis restart"
    w.start_grace   = 10.seconds
    w.restart_grace = 10.seconds

    w.start_if do |start|
      start.condition(:process_running) do |c|
          c.interval = 5.seconds
          c.running = false
      end
    end
  end
end

# Resque
God.watch do |w|
  w.name          = "resque-1.8.0"
  w.interval      = 30.seconds
  w.start         = "cd /var/www/apps/limitedpressing/current && rake environment RAILS_ENV=production resque:work QUEUE=high,medium,low"
  w.start_grace   = 10.seconds

  # retart if memory gets too high
  w.transition(:up, :restart) do |on|
    on.condition(:memory_usage) do |c|
      c.above = 350.megabytes
      c.times = 2
    end
  end

  # determine the state on startup
  w.transition(:init, { true => :up, false => :start }) do |on|
    on.condition(:process_running) do |c|
      c.running = true
    end
  end

  # determine when process has finished starting
  w.transition([:start, :restart], :up) do |on|
    on.condition(:process_running) do |c|
      c.running = true
      c.interval = 5.seconds
    end

    # failsafe
    on.condition(:tries) do |c|
      c.times = 5
      c.transition = :start
      c.interval = 5.seconds
    end
  end

  # start if process is not running
  w.transition(:up, :start) do |on|
    on.condition(:process_running) do |c|
      c.running = false
    end
  end
end

Restarting Workers on Deploy

The other issue I ran into was that I needed my workers to restart each time I deployed. Because Resque loads your application’s environment, a new deploy with changed models, would mean that your existing Resque workers are running outdated code. The way I handled this was to QUIT my workers and let god start them back up (using the updated codebase).

...

# Restart Passenger
deploy.task :restart, :roles => :app do
  ...

  # Restart the resque workers
  run "cd #{current_path} && rake queue:restart_workers RAILS_ENV=production"
end

namespace :queue do
  task :restart_workers => :environment do
    pids = Array.new

    Resque.workers.each do |worker|
      pids << worker.to_s.split(/:/).second
    end

    if pids.size > 0
      system("kill -QUIT #{pids.join(' ')}")
    end

    system("rm /var/run/god/resque-1.8.0*.pid")
  end
end

Conclusion

At first, I wasn’t really happy about quitting and restarting my workers this way, but after reviewing how Resque handled shutting down workers, it seemed like an okay approach. If you have any tips on how the environment can be reloaded without waiting for god to start new workers up, I’d like to hear them, so shoot me an email.

Update 02/03/2011

A special thanks to Nate Daiger of ChunkHost for pointing out a much better way of collecting the worker pids in my queue:restart_workers rake task. Essentially, I was iterating over Resque.workers and calling #worker_ids on each worker. Nate pointed out that #worker_ids actually returned all pids anyway so it was unnecessary to loop. Another thing he suggested was replacing the use of #worker_pids with splitting the output of #to_s. The idea here is that #worker_pids uses ps under the hood and could erroneously pick up similarly named processes, such as resque_web.

These improvements have already been applied to the above gists. Thanks, Nate!

Small Victories

2010-05-07T00:00:00+00:00

I often find myself at the start of very large projects that are scheduled to ship in way less time than is seemingly necessary to complete them. However, I don’t think this is such a bad thing.

The fastest way to get something out the door is to set an arbitrary ship date. I could iterate over the same algorithm or interface for weeks, perfecting it, if I had the time. But, if I did that, I’d never actually ship anything. And when it comes down to it, if you don’t ship software, you may as well not write software.

At the start of a big project, I lay out the major things that need to be done. I write them all down in a list on a big whiteboard next to my desk. I set a date that isn’t very far away, a date I’d love to get this stuff out the door by, not necessarily a date that would give me enough time to feel comfortable. That’s the key, you shouldn’t have so much time that you feel comfortable that everything will be perfect. I’m not sure where I read it, but someone once wrote, if you aren’t embarrassed even a little when you ship something, you didn’t ship it soon enough.

Once I get started on the project, I cross things off the list on my whiteboard, I don’t erase them. I make sure that the list is somewhat general at first. I try my best not to stay on one area of the project for too long. If I notice I’m spending too much time on one thing, it usually means I’m obsessing over some small detail. I step back, figure out what I need to do to round out the area and make it good enough to ship. Then, I take all of the small details that the perfectionist inside me would really like to finish, and I add those to a second list on my whiteboard.

I keep iterating like this, moving from large area to large area and then circling back and crossing off smaller details where time permits. The best part of this is that as you move from large area to large area and cross things off, you start to get on a roll. Looking up at a huge whiteboard with tons of things crossed off gives you the feeling of accomplishment. I’d rather cross off ten small to-do items in a day than one large to-do item in two days. And let me be clear, it isn’t starting with a large to-do list. Large to-do lists never get done. You start with a small to-do list and as you cross off things because they’re good enough, you add much smaller to-do items to another list. Don’t keep adding things unless you’re also crossing things off.

But, it isn’t just the feeling of accomplishment. Because you’re worrying about the details later, the entire project progresses forward faster. If you need to ship something large in two weeks, you can still ship it if the majority of things across the board are done without worrying about completing every little tiny thing in each section before moving on to the next section.

I usually end up being able to complete almost everything in my arbitrary timeframe, but by doing things this way, it lets me easily evaluate the necessity of specific items. I believe that constantly adjusting scope to what’s important now rather than blindly implementing some predetermined “feature list” is the only way to ship software on time. Who knows, by the time you ship, you may circle back to some little fringe feature that you realize isn’t even necessary anymore.

Set a date and ship the damn thing.

Six Weeks Later

2010-02-01T00:00:00+00:00

I spent the past six weeks working full time on Limited Pressing and I’m really happy with what was accomplished. We teamed up with some big names to host charity auctions (as of this writing we’ve raised almost $15,000 that will be donated to Doctors Without Borders) and rolled out a lot of new features. The biggest feature we launched was our brand new Stores infrastructure that I think makes it incredibly easy (not to mention inexpensive) to get a site and store up and running for your label or band.

Aside from software work, I was also able to get my feet wet over at Amazon with things like EC2 and RDS. Although there were a lot of new things I wanted to spend time learning that got overlooked, I hope to rectify that in the coming weeks.

Starting today, I begin a brand new project with Gary Culliss that I’m really looking forward to working on.

Why I Spent $60 On An Alarm Clock

2010-01-06T00:00:00+00:00

For years, I used one of the original iHome iPod dock alarm clocks. I only docked my iPhone a few times - when I felt like listening to an audiobook without my earphones. Otherwise, I used it as an alarm clock. At least, I attempted to use it as an alarm clock.

To set the time or set the alarm, you have to spin a wheel. A slow, loud, clicking wheel. Shortly after getting the clock, the wheel started to break. You would spin it one way and the time would start going in the opposite direction. But that wheel wasn’t even the worst part about it - nor the interference with my GSM phone. No, the worst part about it was that it wouldn’t keep time. It would lose a few minutes every month or so. I never knew what time it was and started looking at my phone to double check. I tried resetting it frequently, but gave up. Instead, I began to set my alarm for earlier and earlier to compensate.

I should have replaced it years ago.

What Do Clocks Even Cost?

Before I was given the iHome “clock” as a gift, I had a regular alarm clock. I don’t remember how or when it came into my possession. I would say a decade, at least, had gone by since I had actually purchased an alarm clock.

Not knowing what alarm clocks cost, I searched Amazon. Wow. There are clocks with big numbers, small numbers, weather reports and ceiling projectors. There are clocks that roll off your nightstand and hide. There are clocks with iPod docks. There are fold up clocks. There are wind up clocks. There are clocks with puzzles on top that, when the alarm goes off, forces you to find the pieces that were thrown into the air and put them back together on the top of the clock before the alarm shuts off. There are $5 clocks and there are $105 clocks.

Learning My Lesson

Page after page, I began to see the problem. Every clock has a gimmick. Every clock does something else. But I had that clock. I had the do something else clock and it was horrible. It was so bad, it couldn’t even keep time.

I tried to find the alarm clocks that were only alarm clocks. Weather reports? I don’t want you. Ceiling projector? Sorry, not for me. The clocks that were left ranged from $5 to $15. Each one had reviews that said something along the lines of “cheap, works well”.

Unfortunately, these all looked cheap. I started to think, what if these clocks can’t even keep time. How long would these cheap clocks last? Sure, they were just clocks which is what I wanted, but I don’t want to have to buy another clock next year.

Then I stumbled on the Neverlate Executive Alarm Clock. I am not an executive. Nor do I need 21 separate alarm “banks”. But, something seemed right about this ridiculously expensive alarm clock. It was expensive, but it didn’t do anything else. Sure, it has separate alarms for each day of the week in addition to seven daily alarms and a quick nap alarm. It also has a gradual wake setting (increasing alarm volume) and a decremental snooze interval (10 minutes first snooze, 5 minutes second snooze, etc).

It has a lot of features, but each feature is alarm clock related. If there is an alarm clock out there that is actually a good alarm clock, I thought, this must be it. Did I spend too much money on an alarm clock? Yeah, probably. But I will say that I was able to set the clock up in just a couple of minutes and I woke up on time this morning to a nice alarm sound with a gradually increasing volume.

It doesn’t matter if you’re writing a piece of software or making an alarm clock. Keep your product focused and you won’t end up with a wheel you need to spin 500 times to get from 12:00AM to 7:00AM or a clock that can’t even keep time.

One of Those End of The Year Posts

2009-12-31T00:00:00+00:00

I suppose waiting until 11PM on December 31st is as good a time as any for an end of the year wrap up post, right?

2009

The year really started early February for me with my Grandfather passing away. I’ve had relatives pass away before, but this was different. This was someone I was really close to and saw all the time. His death was sudden and was really hard to get past - not only for myself, but for the rest of my family. I don’t think I’m over it yet and I’m not sure I ever will be.

The rest of the year, I buried myself in work. I would say that my Grandfather’s death was the reason, but to be honest, I’ve been burying myself in work since as far back as I can remember, so maybe this was just an excuse.

Regardless, 2009 was a year of working hard - maybe too hard.

The year started wrapping up on December 18th with my last day of new development on gawkk. It’s been a long road. I was first hired by Gary to work on gawkk at the start of 2007. Working part time, I was able to lay the groundwork. By September of 2007, I had left a software engineering job I had been at for over six years to work full time on gawkk.

In the past two years, gawkk has grown and changed a lot. Towards the end, we were constantly trimming features and simplifying processes and user interaction. As it stands, I feel that gawkk is a superb video aggregator and discovery engine. Behind the scenes, it scours thousands of video hosting sites and automatically imports and categorizes everything it finds (millions of videos). For our end users, it provides a very simple way to share and discuss videos with their friends. Gawkk suppresses the noise from people you don’t care about, allowing the videos enjoyed by only your friends to surface just for you.

With that said, the time has come to let gawkk do its thing and focus attention elsewhere.

Other than working on gawkk full time through 2009, I’ve worked on a handful of very small freelance projects - all from clients I first did work for many years ago. I’ve released a few simple plugins and worked on a bunch of new side projects that have taken up most of my nights and weekends.

Twenty Ten

I’ll be starting the new year with an entire month of not doing any paid work. Instead, I will spend most days and nights slaving away on a couple of my side projects and reading every technical book I can get my hands on. Another one of my goals for this month is to completely refactor my mornings in an attempt to get a better work-at-home routine down. Lastly, I hope to squeeze in a couple of snowboarding day trips here and there, because there is nothing more relaxing to me than sitting at the top of a snow covered mountain.

I hope to have my new morning routine all straightened out when February comes, because I’m lucky enough to be starting a brand new project with Gary. We’re completely changing direction with this project and I’m really excited to get rolling. It’s going to be really fun to use some of the new things in the Rails ecosystem that I’ve had my eye on as well as just getting to start a new, fresh project (I’m looking at you, Rails 3!). In addition to new Rails specific things, we plan on using AWS for our entire infrastructure. Everything from EC2 and RDS to S3 and CloudFront. I honestly can’t wait to get my hands on that infrastructure. Everything Amazon has been doing with their Web Services continues to impress me. Just the other day they released object versioning for S3 - really neat stuff.

Aside from my plan, I hope that the Rails community continues to thrive and that, for everyone’s sake, this economy picks up in 2010.

Happy New Year!

A Look Back At Staples

2009-12-16T00:00:00+00:00

In early 2007, I was contracted to implement a document management web application for Staples Digital Copy Services. Customers can register and upload documents for very large jobs to Staples’ regional copy centers. The app handles documents for about ten locations. The only non-rails part was a multi-file uploader I wrote as a Java applet. Staples wanted drag and drop file uploading that also supported dropping multiple files at once, which I was able to do with Java.

Certain privileged Staples employees can log in and download documents, read descriptions and manage which jobs have been completed.

I was contacted a few days ago for a quote to work on a new feature for the application and was happy to see the app has been running smoothly these past few years. They keep adding locations to the system and it’s quickly closing in on managing 100,000 documents.

Automatically Resizing a Textarea's Height

2009-09-18T00:00:00+00:00

Here is a simple Javascript method that can be used to automatically resize the height of a textarea based on the amount of content inside.

Prototype is used in this example.

Click through if you don’t see the code snippet below.

<!-- Automatically resize a textarea's height (like Facebook) -->

<textarea id="comment" rows="1" cols="50"></textarea>

<script type="text/javascript" charset="utf-8">
  function autoResize(fieldId) {
    var length = $(fieldId).value.length;

    if(length > 0) {
      $(fieldId).rows = Math.floor(length / $(fieldId).cols) + 1;
    } else {
      $(fieldId).rows = 1
    }
  }

  $('comment').observe('keyup', function(){autoResize('comment');});
</script>

A Cacheable Hash That Stays Synced

2009-09-07T00:00:00+00:00

Rails.cache freezes values in hashes that are cached directly. CacheableHash is a wrapper for Hash objects that prevents that from happening. Changes to the Hash being wrapped are automatically persisted back to the cache store so that everything stays in sync.

You can clone (or fork) my new MIT licensed plugin over at github.

Example

An instance of CacheableHash is used like any other hash. A cache key is the only thing required.

>> fruit = CacheableHash.new('fruit_hash_key', :hash => {:apple => 'red', :banana => 'yellow'}, :expires_in => 1.week)
>> fruit[:banana]
=> "yellow"

Changes are automatically persisted.

>> fruit = Rails.cache.read('fruit_hash_key')
>> fruit[:apple]
=> "red"

>> fruit[:apple] = 'green'

>> Rails.cache.read('fruit_hash_key')[:apple]
=> "green"

About Synchronicity

Whenever a method is called on an instance of CacheableHash, that instance automatically writes itself back to the cache store. Often, this is unnecessary. If you call .keys, for example, there is no reason to write back to the cache store. Only when reading a value from the hash using [], is the copy in the cache store not specifically updated. But, rather than worrying about the specific methods that change the contents of the hash, it’s safer to just update the instance in the cache store whenever not specifically reading values.

Updated: Mobile Twitter Trends Application (2.1)

2009-09-06T00:00:00+00:00

I wasn’t able to add all of the polish and features to my Trends application that I really felt was necessary last night, so I’ve just rolled out a quick update. This is definitely the last update for a while on this app.

Version 2.1

The front page now lists not only the currently trending topics on Twitter, but it also shows topics that have trended in the past day and the past week.
Hashtag topics are now working (they were broken in v2).
The timestamp on the topic page now more accurately states when the topic was first seen trending.
Added some protection against strange (and actually kind of frequent) errors thrown by the Twitter API when requesting the list of trends.
If What the Trend doesn’t know about a particular topic, rather than throwing an error, you’re now redirected back to the trends list and a dialog is displayed telling you what happened.
Fixed a visual bug in TankEngine related to the top 1px border of the toolbar (it used to be a dark line, it now matches the actual iPhone toolbar element).
Updated the default loading image used for the fullscreen Trends application (accessed through the home screen shortcut) so that the toolbar matches the updated toolbar.

Mobile Twitter Trends Application

2009-09-05T00:00:00+00:00

Update: Version 2.1 has been released.

Just over a month ago, I released a mobile web application that used the API from whatthetrend.com to present a list of recently trending topics on Twitter, along with their explanations. This little application turned out to be pretty handy, but for a while I’ve wanted to change it a bit. Most importantly, I wanted to have the front page of the application be the list of currently trending topics on Twitter.

Trends: Version 2

The interface has been re-implemented from the ground up (actually the entire application was rewritten from scratch) and is now styled like a normal list-based iPhone application.
When you first load the application, it shows you a list of the topics that are currently trending on Twitter.
Clicking on a topic will bring you to that topic’s page. The explanation as to why that topic is trending is pulled in from What the Trend? and displayed here.

I will be adding lists for the daily trending topics as well as the trending topics for the week - for now, only the current trending topics are available.

Please visit Trends on your iPhone and add it to your home screen for the best experience.

Sweat the Details

For this project, I used Noel Rappin’s TankEngine plugin for the iPhone’s user interface. I ended up having to hack it a bit so that it would run in fullscreen mode without breaking out into Safari when clicking links, but overall it was very enjoyable to use. You can run the Trends application in fullscreen mode by tapping the + button in MobileSafari and then tapping ‘Add to Home Screen’. When you launch Trends from the new icon on your home screen, it will launch in fullscreen mode.

heroku

In addition to wanting to update this application a bit, I’ve been really wanting a reason to try heroku and I thought this was the perfect application to use as my test run. heroku is a service that allows you to deploy ruby applications quickly and easily. You know what? Saying it allows you to deploy ruby applications quickly and easily is an understatement – heroku is, by far, the most wonderful service I have used in recent memory. I set out tonight to rewrite my mobile Trends application from scratch. Once I had a working version, I went to heroku not knowing anything of how it worked, signed up and in less than two minutes my application was deployed to heroku and running perfectly.

If you have a simple application that you’re planning to host on your own server, I highly suggest taking a minute to deploy it over at heroku. It’s free for small applications and I guarantee you will immediately see what an incredible service it is. I honestly can’t say enough.

Conclusion

I don’t sleep enough and heroku is awesome.

Rails Rumble Reflections

2009-08-24T00:00:00+00:00

I competed in Rails Rumble 2009 over the weekend. It was my first time competing, so I wanted to share some of what I learned while working on a project from start to finish with other people in a 48 hour window.

Get a real designer

Our team consisted of three developers and no designers. We could all write HTML and CSS and develop front ends, but we weren’t designers. We didn’t possess the talent necessary to develop a really stunning interface, while actually developing the rest of the application in the time allotted.

Although we produced a full featured application that ended up looking halfway decent, there is no doubt in my mind that we would have benefited immensely if we had a real designer. And by designer, I mean someone who actually makes a living doing design work for web applications.

Be opinionated, but be flexible

Any project with multiple engineers is bound to bring about disagreements. People are going to discuss and argue their points. I believe that when it comes to software development, you must be opinionated to develop good software. Having strong opinions about something means you are passionate about it. When you’re constrained by a 48 hour window, being opinionated is even more important. You don’t have the luxury of having an entire discussion about every little thing that comes up.

The problem is that when opinionated people disagree, they spend a lot of time in heated discussion. Normally, these discussions will lead to a better overall implementation and everyone benefits, but when you’re constrained by time, you must not only be opinionated, you must also be flexible. You have to pick your battles and make concessions.

Working with people you trust

Being able to make concessions is related to another point I have. You should work with people you trust. If you don’t trust the people you are working with, you are going to have a hard time giving in when the other person yells louder. You aren’t going to be able to sit back and say:

“I’m working with this person for a reason. I have to trust them, they are very passionate about this.”

Being able to trust your team members is not only important during a competition like this one, it’s always important. The difference with a competition, though, is that you just don’t have the time to spend on every disagreement.

Don’t write tests

Nowadays a lot of people talk about test drive development. Write tests for code that doesn’t exist yet. When you write the code, you know you’re done because it passes the tests. The problem with TDD is that it can be time consuming. Additionally, because you only have a 48 hour window, the scope of your application must be small enough to complete. If it’s small enough to complete, it will probably be small enough to manually test throughout the competition.

I spoke to some other contestants who feel the same way. Some even went into the project doing tests and quickly abandoned them because they were too time consuming. Even Lowdown, a Rails Rumble entry specifically built to help developers, designers and managers collaborate on Cucumber feature stories said:

Ironically – especially considering the purpose of our application - we didn’t test, spec, or write features. There were times we could have benefitted from TDD, but many other times it would have just slowed us down. We considered the app to be a prototype/proof-of-concept anyway, not a polished and hardened work of craftsmanship.

I think this is the key. Testing is good, but testing during the competition could prove fatal. If you only finish a third of your application because you were busy writing tests, you don’t get more points - even if that third is really perfect. It’s better to get to a feature complete state as early as possible and then iterate and iterate and iterate until the last moments of the competition.

Conclusion

As you can probably tell, I am very opinionated when it comes to software development. Being opinionated can also leak into the rest of my life. This article, these reflections, are my reflections. I’m sure many other participants, possibly even my own teammates, don’t agree with me - I’m okay with that. The fact is, nothing I said here (other than having a real designer on your team) is necessarily “right”. It is, however, right for me and probably a lot of other people.

In the end, I was lucky enough to work with smart people and smart people can work through differences to deliver something they are proud of. And despite our differences, the three of us were really happy that we were able to deliver the application we set out to develop. Being proud of that accomplishment is something we could all agree on.

I’m really looking forward to Rails Rumble 2010.

Making Your Existing User Base Play Nice With Your Facebook Users

2009-08-20T00:00:00+00:00

This is the second article in a series dedicated to getting your existing rails application on Facebook without clawing your eyes out. You should read the first article if you haven’t already.

In the previous article, I talked about how to use facebooker to extend your existing rails application with Facebook only views. Requests that came into your application from Facebook, were authenticated by facebooker to be genuine Facebook requests, the Facebook user was required to add your application and all of your Facebook specific .fbml.erb views and layouts were automatically used instead of your standard .html.erb views. While all of this was happening for requests coming from Facebook, your regular application kept chugging along, doing its own thing, unaware you were cheating on it with that blue haired girl from your Calc class.

Chances are however, your application has users (at least you hope it does). What you really want, is for users on your existing Rails app and new users coming in through Facebook to be treated like equal citizens. Sure you may email one and you may notify another, but aside from how you communicate with them, the content they generate in your application should be shared by everyone. If data generated by regular users and Facebook users couldn’t be seen or accessed by one another, there would be no reason to having a single action with two different views (html, fbml).

There are a few things we need to get our different types of users to play nicely.

A place to store a user’s Facebook user id and last session key.</li>
A way to catch authenticated Facebook sessions for new users to our application.</li>
A way to automatically log in an existing user who has authenticated via a Facebook request.</li>

Tracking Facebook Account Details in Your Users Table

If you only plan on doing Facebook related stuff (you have no interest in having Twitter oauth support in your app in the future), the easiest thing to do is just add a couple of fields to your User model. We can just pull these migration lines from what facebooker does:

Tracking Facebook Account Details in a Separate Object

Alternatively, if you plan on having not only Facebook authentication, but Twitter oauth at some point, you may want to create a new model called FacebookAccount to store your user’s Facebook account details (as well as a foreign key back to the User it belongs to). Then you can add a TwitterAccount object as well and do what you have to do.

For this example, I’ll assume you just added some fields to your User.

Hey, do I know you?

Now that we have a place to store Facebook account details, we need a way to make a new User for a new Facebook user. Let me preface this by saying that there are probably a hundred different ways to do this. What I’m going to detail, however, is the approach that worked well for me.

Let’s go back to our application_controller and see how we left it:

before_filter :require_login_for_facebook

def require_login_for_facebook
  if params[:format] == 'fbml'
    ensure_authenticated_to_facebook

    if !session[:facebook_session].nil?
      # we'll do something with users
      # here in the next post about fb
    end
  end
end

If you follow a Facebook request coming into our application, you’ll see that it hits the require_login_for_facebook before filter, it should pass the (params[:format] == 'fbml') test because facebooker will have forced the format to be fbml because it already verified the request came from Facebook. Next, the famous ensure_authenticated_to_facebook method is called to make sure we get a Facebooker::Session. After that, I do one more test to ensure we really do have a Facebooker::Session sitting in session[:facebook_session] and we can be on our way.

Inside of the if statement checking if !session[:facebook_session].nil?, we’re going to want something along the lines of:

unless user_logged_in? or (controller_name == 'facebook' and action_name == 'connect')
  facebook_session = session[:facebook_session]

  if user = User.find(:first, :conditions => {:facebook_id => facebook_session.user.uid})
    session[:user_id] = user.id
  else
    redirect_to :controller => 'facebook', :action => 'connect'
  end
end

I should probably mention that I don’t use restful-authentication. I generally roll my own. Please, take a minute to yell and scream about what a horrible person I am. Blah blah blah. Okay, done? Despite not using the norm, you can see some of the same basic things here. First up, I have a helper method called user_logged_in?. Can you guess what that does? Good. I’m sure you have the same kind of method. Just make sure there isn’t already someone logged in. Don’t worry about that next part of the unless statement ensuring we aren’t in FacebookController#connect, you’ll understand that soon.

So, after we ensure we have a Facebooker::Session, we make sure there isn’t someone already logged in. Next, we try and locate the User in our users table that has a facebook_id that matches the user id of the Facebook authenticated user (session[:facebook_session].user.uid). If we find someone with that facebook_id, we can automatically log them in. For me, I set session[:user_id], for everyone else in the world, do what you normally do to log someone in.

The reason we can automatically log this person in is because that user already had to log into Facebook and when Facebook made a request to us, facebooker did the heavy lifting to ensure that this was a real Facebook request. We wouldn’t have gotten this far if it wasn’t a real Facebook request. And, since this is a real Facebook request, we can assume the uid of the user inside of the Facebooker::Session is really who they say they are. Thus, they are sufficiently authenticated.

But how do we actually get a user object with a facebook_id into the database so that we actually have someone to automatically log in? It’s actually pretty easy. First, generate a controller called FacebookController. Inside of this controller, create a method called connect. I know, I know, Facebook Connect! Although I won’t be talking about it in this article, this is actually in preparation for allowing users to add your Facebook app inside of Facebook, then go out to your existing external application and log in using Facebook Connect while still retaining the same user object they used from your Facebook applicaiton.

Inside of the FacebookController#connect method, you’re going to want to prepare a new User model. This method is going to act like a Users#new and Users#create method. The idea is that we need this user to essentially register for our site. User’s in our existing rails app register, so why not Facebook users? The difference, however, is that we want to ask for as little information as possible while still being able to have full fledged accounts for Facebook users that can be shown on our existing site. For me, that means I need a username. Just pick a username and I’ll be happy. Oh, and don’t forget to add an app/views/facebook/connect.fbml.erb view for this method (for Facebook Connect, we’d have a connect.html.erb - get it?).

def connect
  if request.get?
    # Lets collect some information about this user
    @facebook = Hash.new
    @facebook[:id] = facebook_session.user.uid
    @facebook[:name] = facebook_session.user.name
    @facebook[:description] = facebook_session.user.about_me
    @facebook[:image_small] = facebook_session.user.pic_square_with_logo
    @facebook[:image_large] = facebook_session.user.pic_big
    @facebook[:profile_url] = facebook_session.user.profile_url.gsub(/^http:\/\/www\.facebook\.com\//, '')

    # We're going to want this later, no point in hitting the API again
    session[:facebook_credentials] = @facebook

    # To make things easier, I like to suggest a username.
    # I won't do it here, but generally I run the suggested username
    # through a unique_username? check about 3 times, appending a
    # random integer on the end to try and make it unique.
    # If I can't find a unique username quickly, I let the user worry about it.

    @user = User.new
    if !@facebook[:profile_url].blank? and !@facebook[:profile_url][/^profile.php/]
      # This user has a new vanity url so that is a preferred username
      @user.username = @facebook[:profile_url].first(15).gsub(/\s/, '').gsub(/\./, '')
    elsif !@facebook[:name].blank?
      # This user doesn't have a vanity url, just use their name
      @user.username = @facebook[:name].first(15).gsub(/\s/, '').gsub(/\./, '')
    end
  else
    @facebook = session[:facebook_credentials]

    # Create the new User object from params[:user]
    # Make sure to store the @facebook[:id] and session key (see facebooker docs)
    # Fetch and store their avatar locally (Facebook likes you to use their (x)fbml
    # tags to display a user's current profile picture, you could do that instead.)

    # Log the user in
    session[:user_id] = @user.id

    redirect_to path_to_front_page
  end
end

I know it looks like there’s a lot going on in the connect method, but it’s pretty straight forward and the inline documentation should really explain everything. Obviously, you’ll want a form in your app/views/facebook/connect.fbml.erb view. You’ll need a username field and you’ll probably want to display the image in @facebook[:image_small] on the page somewhere.

Now, whenever an existing Facebook user comes back to your application, they will automatically be logged in with their full fledged account. They never have to worry about logging in or the fact that they “registered”. In fact, I tell my user’s they are just picking a username. That is technically all they’re doing. Having to “register” scares people off. Picking a username is easy.

I may write another article that explains how you can setup Facebook Connect to allow your existing Facebook users to log in directly at your main external rails app (outside of Facebook), using Facebok Connect. Doing this would actually allow you to get users to sign up using Facebook Connect directly from your main site as well. After the post connect callback from Facebook Connect, you’d just the same kind of the check in the application controller to see if you have the user with that facebook_id, either log them in automatically or send them off to your FacebookController#connect method (only it will be the .html.erb version this time) to finish the account sign up. It’s really very similar to what we do here.

Getting Your Existing Rails App on Facebook

2009-08-19T00:00:00+00:00

There’s a ton of information already out there about writing a new Facebook app with Ruby on Rails. You use facebooker, you call a method at the top of your application_controller and you’re done. It’s pretty easy. The problem is that when you have an existing Rails app and you want to extend it to support alternate fbml views for a Facebook application, things are a bit trickier.

Here’s how I did it:

You’ll have to start with the usual setup:

Create a Facebook app from the Developer Application
Install the plugin
Setup your ssh tunnel for development
Setup your config/facebook.yml

I won’t bother explaining how to do all of this stuff. There are tons of resources out there.

After you get that done, it’s time to start serving authenticated fbml views for requests coming from Facebook. In the facebooker world, you need to call ensure_authenticated_to_facebook. Unfortunately, if you do that in your application_controller, it will try and do a Facebook authentication for every request coming into your application, not just the ones actually coming from Facebook. Since you are trying to extend your existing application with Facebook views, this isn’t what you want.

It turns out that facebooker does two very important things for you by just having the plugin configured in your application. The first is that it forces the format of all of the Facebook requests to be fbml. Next, it adds an object to session[:facebook_session]. It’s safe to say that if your request has a format of fbml and there is something in session[:facebook_session], you’ve got yourself a Facebook request.

To catch and authenticate just the requests coming from Facebook, you’ll want to add the next bit of code to your application_controller:

before_filter :require_login_for_facebook

def require_login_for_facebook
  if params[:format] == 'fbml'
    ensure_authenticated_to_facebook

    if !session[:facebook_session].nil?
      # we'll do something with users
      # here in the next post about fb
    end
  end
end

Now, let’s say you have a movies_controller that uses an app/views/layouts/movies.html.erb layout. You’ll want to add an app/views/layouts/movies.fbml.erb layout for requests coming from Facebook. Next, you want a Facebook specific view for your index action. You already have an app/views/movies/index.html.erb right? Well, just add an app/views/movies/index.fbml.erb.

That’s it, you’re done! Now, when a request comes into your regular application for /movies, the regular index.html.erb view will be rendered inside of your regular movies.html.erb template. But, when a request comes in from your Facebook application to /fb-app-name/movies, it’s authenticated with facebooker, the facebook user will have to have added your application and your new index.fbml.erb view will automatically be rendered inside of your movies.fbml.erb layout.

I will post another article soon talking about how to approach integrating your existing user base with a Facebook user base.

Like this article? Be sure to read the second article in this series titled: “Making Your Existing User Base Play Nice With Your Facebook User Base”.

Keeping Text Fields and Text Areas Consistent With CSS

2009-08-08T00:00:00+00:00

Let me just start by saying that I don’t claim to be good at design at all, but as I don’t work with a designer, I have had to fend for myself over the years. I’d say I “get by” in the design department by sticking to things that aren’t overly complicated. It’s probably also worth noting that I can be a perfectionist at times. I care a lot about the code I produce - whether it’s 5 lines of code that can be written more simply or a few pixels lining up better (admittedly, I’m way better at refactoring code than I am at interfaces).

What I’d like to explain is a very simple thing I do to keep my text fields and text areas looking consistent with just a couple lines of CSS.

By default, most browsers like Safari and Firefox render text fields with a simple 3D type depth. It makes the field stand out. Text areas, on the other hand, are usually rendered without that 3D type effect. They are given a simple, flat border. This is what 2 text fields look like compared to a text area, by default, without any CSS:

The first step to keeping things consistent looking is to simply set the border of text fields and text areas with CSS. Now both widgets are at a level playing field.

Just setting the border is a nice first step, but that slight 3D depth given to text fields by default is actually pretty useful to indicate to the user that she can click and type into the area. To do this, you can simply set the border-top of the text fields and text areas to be slightly darker than the border you set in the last step.

I suspect that most users looking at this form wouldn’t even think twice about this subtle tweak, and that’s the beauty of it. It’s so simple yet still does such a great job of making your forms look way, way sexier.

Sample CSS:

Maybe the word for me is neurotic.

PGnJ 1.1 - Free For Good, QuickJoin, Crasher Fix

2009-08-07T00:00:00+00:00

After 8 months of being on sale, I would like to announce that PGnJ 1.1 is now available and that it is completely free (and that it will absolutely stay free).

It comes down to the fact that the vast majority of PGnJ’s user base decided to stay on 0.8 (the last free version). There wasn’t huge interest in a pay version of PGnJ and because of that, interaction with customers along with feedback totally dropped off the map. But I still had things I wanted to add to PGnJ. I still use PGnJ every day. I wanted people to get those new features. So, the PGnJ-for-pay experiment is officially over.

What’s new in 1.1? The changelog may look pretty slim but it comes with a couple of very major improvements as well as a fix for a crashing bug that came about in the past couple of weeks.

Changelog:

PGnJ is now free. Enjoy!
When using the DataBrowser (double click a table from the database tree in the sidebar), you can how double click any cell to edit the contents (PostgreSQL and MySQL only).
Developed a brand new, one of a kind feature that I call QuickJoin. When you open the DataBrowser, it will automatically add a QuickJoin bar to the top of the window with a list of all foreign keys for the current table. Clicking a foreign key will bring up a drop down list of all fields in the referenced table. Selecting a column (you can select as many as you want, across as many referenced tables as you want) will automatically generate JOINs under the hood and display the selected column right inline next to the fk column. See below for a quick demonstration.
Fixed a major crashing bug that came about with OS X Java Update 4. Essentially, the application must be run in 32 bit mode.

Using Partials as Reusable Inner Layouts

2009-08-05T00:00:00+00:00

You may find this pretty basic, but it’s useful nonetheless.

Using partials in Rails is a wonderful example of code reuse. For example, you may have comments that use polymorphism so that you can attach comments to different models in your system. If you always want your comments to display using the same UI (and why wouldn’t you) on not only Project Messages but also To Do items, you can create a /comments/_comment partial and render these partials from either the Project Message page or the To Do item page. But, we all knew that.

Sometimes, however, you may want to use a partial like an inner layout inside of your page’s layout.

<!-- /dashboard/_container - This partial is used as a template -->

<table id="dashboard">
  <tr>
    <td id="tabs">
      <!-- Links to different areas of the dashboard -->
    </td>
    <td id="main">
      <%= yield %>
    </td>
  </tr>
</table>


<!-- /messages/_inbox - This partial uses the dashboard _container as a wrapper -->

<% render(:layout => "/dashboard/container", :locals => {:selected => 'inbox'}) do -%>
  <div id="messages">
    <%= render @messages %>
  </div>
<% end -%>

Above is a really simple example, demonstrating how you can use a partial as an “inner layout”. Essentially this allows you to have a reusable partial with contextual, custom view code somewhere inside that reusable template - just like you’d use a regular page layout, except now you can have them scattered about your application.

Yes, this example in particular is really simple, but if you’ve never seen partials used this way, your brain is probably already cranking on how immensely useful this can be.

A better Array#rand

2009-07-02T00:00:00+00:00

I released a new plugin today, called rand. This plugin expands on the #rand method ActiveSupport adds to Array.

As background, the rand method that Rails gives you, returns a single random value back from the array it’s called on.

This rand plugin overrides the rand method so that you can pass in an integer value that corresponds to how many random values from the array you actually want back.

Additionally, the existing functionality is still in place so that if you don’t pass in a parameter, it still gives you back a single random value.

A better Rails.cache.increment

2009-06-25T00:00:00+00:00

I started working with Rails.cache.increment and Rails.cache.decrement today but quickly found that it didn’t work as I planned. For example, it seemed that when calling these methods they would return the previous value rather than the newly incremented value. Additionally, if you decided to read directly using Rails.cache.read and the same key, it would always return nil despite being able to call Rails.cache.increment again. Another annoying thing is that if you tried to increment a value that wasn’t in memcached, instead of defaulting to 1, it returned nil. Just all sorts of wacky stuff going on. I decided to just write my own.

# app/models/util/cache.rb

class Util::Cache
  def self.increment(key, amount = 1)
    if (value = Rails.cache.read(key)).nil?
      Rails.cache.write(key, (value = amount))
    else
      Rails.cache.write(key, (value = value + amount))
    end

    return value
  end

  def self.decrement(key, amount = 1)
    if (value = Rails.cache.read(key)).nil?
      value = 0
    else
      Rails.cache.write(key, (value = value - amount))
    end

    return value
  end
end

Generating RSS Feeds in Rails

2009-04-01T00:00:00+00:00

It’s very easy to generate RSS feeds in Rails. In fact, if there is any possible benefit to having RSS feeds in your rails application, there is absolutely no reason you shouldn’t take a few minutes and set them up.

These days, you just have to create a file like :action.rss.builder to go along with your :action.html.erb file. You don’t even need a respond_to block in your action. As long as you have a route setup that knows how to handle additional formats, rails will automatically render your .rss.builder file rather than your .html.erb file if an .rss extension is at the end of the url.

For the basics of generating RSS feeds in rails, check out this screencast.

If you’d like to include media in your RSS feeds you should use the Yahoo media namespace xml definitions. To include this namespace in your feed you can use:

You can then generate a namespaced XML element like this:

I Searched and Searched for searchd

2009-03-16T00:00:00+00:00

For a while now I’ve been working on a really long and detailed post comparing the different search systems available for rails projects. I just deleted the draft.

I deleted it because I realized there really isn’t anything to compare. If you don’t need live updated search results and you can live with an index that’s updated every 15 minutes or more, just use Sphinx. Sphinx with Thinking Sphinx by Pat Allan is simply the most elegant, full featured and most importantly, stable search system available for rails today. We’ve been using it over at gawkk for months now and haven’t had a single unhappy moment since we switched from Solr.

SQL searches are slow. Ferret is crap in production. Solr is almost as unstable as Ferret in production and will bring down your database with unnecessary connections every time a damn object is instantiated. Sphinx is rock solid and fast as hell.

I will only offer two lines of proof:

Over a million documents in 117 seconds. Try and do that with anything other than Sphinx. I dare you.

A Simple .union Extension For ActiveRecord

2009-02-27T00:00:00+00:00

I just wrote and released my first rails plugin, called union. It’s absurdly simple and fairly naive. I wrote this plugin so that I didn’t have to look at an ugly UNION I was running in a find_by_sql.

The first parameter, parts, is an array of hashes. Each hash is what you would normally send into a single find and represents each SELECT. All parts will be unioned together.

The second parameter, options, is a hash of remaining options to be applied to the UNION of the parts (ie: order, limit, offset).

A simple (and useless) example would be:

This example produces the following SQL:

Essentially you can do any union, but it’s up to you to make sure you don’t pass the wrong stuff in because it’s a pretty dumb implementation.

MySQL Isn't Very Sensitive

2009-02-11T00:00:00+00:00

When writing SQL conditions in MySQL, using an equal sign is case insensitive. For example:

Will return results where thread_id actually equals 2a and 2A. To make this case sensitive, you can use LIKE BINARY:

But what happens when you want to use IN? For example:

This would return where thread_id actually equals 2a, 2A, 2b, 2B, 2c and 2C. This is no good because I need it to be case sensitive. You could do something like:

But this is a problem when you want dynamically built this query using an ActiveRecord find method like:

I haven’t found a solution yet, but I hope to soon.

A Fresh Start

2009-01-14T00:00:00+00:00

I picked up a 320GB 7200RPM hard drive for my MacBook Pro and rather than cloning my current drive, I figured I’d take the opportunity to start fresh.

Starting fresh is kind of scary when you’re a software engineer because you have to get your environment setup just the way you like it before you can really get started on work again. While I’m waiting for the last of my files to back up, I thought it’d be nice to detail how I prepare for a fresh operating system install.

I have a Western Digital 1.5TB USB hard drive connected to my machine at all times running Time Machine backups. While I feel this keeps me safe for day to day operations, when I do a fresh install I always do a manual backup of the things I need. You can never have too many backups.

First I copy over my main directories in my home directory. OS X makes this easy. I backup:

~/Desktop
~/Documents
~/Downloads
~/Library
~/Movies
~/Music
~/Pictures

I then export important data from applications to make it easy to import in my fresh install. I export from:

Address Book
Camino (Bookmarks)

Next, I take various lists of things I have installed and screenshots of my current setup.

I run ls -la in my /Applications directory and copy the output to a text file
I run gem list and copy that to a text file
I run which on various binaries like gem, ruby, rails to ensure my paths are setup properly when I reinstall
I take a screenshot of my Dock
I take a screenshot of my Menubar
I take a screenshot of System Preferences

Stragglers:

I copy my ~/.ssh directory so that I have my ssh keys
I copy various config files such as ~/.gitconfig

I then copy my ~/Development directory which contains things like:

~/Development/assets
~/Development/cocoa
~/Development/iphone
~/Development/java
~/Development/rails

Each of my ~/Development/ directories has directories like ./documentation/ and ./projects/.

Lastly, I make backups of all of my development PostgreSQL and MySQL databases using pgAdmin and MySQL Administrator, respectively.

Working From Home

2008-12-12T00:00:00+00:00

It’s incredible how much you can get done when working from home. Before I went full time at Gawkk over a year ago, I had a software engineering job at a regular company. I was in an office. People would call all day. People would stop by. I had to physically go to meetings. It really affected productivity.

When I started working from home, I thought that it would be tough.

“I’ll really need to try and concentrate. There will just be so many distractions.”

I mean, it’s what everyone was telling me.

“You’re going to work from home? Wow, I could never do that. I’d never get anything done!”

But really, what distractions are there? You don’t have to stop what you’re working on to make it to a meeting. You don’t have to answer the phone calls from people who don’t want to wait for a response from an email. You also don’t have people dropping by to just shoot the breeze.

Once you realize that 9 to 5 isn’t when you have to be productive. Everything else falls into place. Sure, I like to generally be at my desk before 9 and I try and remember to pull myself away after 5 (although I can get sucked into my code), but if I was up late the night before and feel better starting at 10, it’s not that big of a deal. The key is working when you are productive. There is no sense at sitting at your desk doing nothing from 9 to 10 if on that particular morning you feel like crap. Work when you are productive and the distraction-less environment will help facilitate that.

It probably also helps if you actually enjoy doing what you do.

Customer Support

2008-12-10T00:00:00+00:00

A few days ago, I ordered a pair of snowboard pants from The House. When they showed up last night they were the wrong size. Further investigation revealed that the drop down containing sizes on their website listed them wrong. The size I selected was not the size they thought I ordered.

Luckily, The House has great customer support. I called the toll free number and a woman answered immediately. No automated menus. She answered, I said I was sent the wrong size, she transferred me to the right person.

The person who I was transferred to helped me find a different pair of pants. Discounted them. Ensured they would ship the same day and gave me free shipping on them. In the box will be the return shipping labels for the pants that were the wrong size so that I won’t have to pay shipping to send those back.

There was only one painful part of the experience. The on hold music was just a single song on loop. James Brown’s “I Feel Good”.

The Road to 1.0

2008-12-02T00:00:00+00:00

Almost four years in the making, I’m very pleased to announce version 1.0 of my Database Development Environment, PGnJ. Although it started as a side project, since February’s release of 0.8, I have kicked developed into high gear. My goal was to release version 1.0 by year’s end. A goal that I am very happy to have met.

When it came time to lock down what I felt would be worthy of the 1.0 badge, I found myself back at the roots of PGnJ’s design. Rather than simply packing PGnJ with the same old features that other database clients all have, I tried my best to stay true to the fact that PGnJ is meant to be a development aid. For example, I was less concerned with the standard table modification wizards that you see everywhere, than I was with a flexible system that could be extended in future releases. This is where SQL Templates were born. PGnJ 1.0 comes with a few stock DDL SQL Templates for easily creating and altering tables and columns. These templates include :variables that can be easily navigated using SHIFT+TAB and even special :type variables that show a tooltip with commonly used data types for the current database. In a future (already in the works) release of PGnJ, you will even be able to edit these templates as well as create your own.

Although there is already a long list of new features and refinements in 1.0, for me it still comes down to PGnJ staying out of your way and at the same time making it as easy as possible for you to get your database work done.

Thank you to everyone who has sent feedback and used PGnJ in the past. I hope that you continue to send any and all comments you may!