Viget.com Blogs

The Neo Fallacy: How Our Mental Model of Developers Leads to Bad Bug Reports

Josh Korr — Fri, 10 May 2013 14:28 GMT

In two previous posts about writing better bug reports, I've detailed the information developers need to be able to reproduce — and hence diagnose and fix — bugs. Since then, an unstated assumption in both posts has been nagging at me: that developers must be able to reproduce a bug in order to diagnose and fix it.

To a developer, this is not an assumption — it's truth, and obvious. But to non-developers, the necessity of reproducibility is not obvious. If it were, devs wouldn't see so many bug reports that say little more than "A vague thing vaguely happened — fix it!!!"

I think this disconnect is the primary cause of frustrating-to-devs bug reports. If we can better understand the reason for the disconnect, maybe it'll be easier to convince non-devs of the necessity of reproducibility.

So what causes this disconnect? Here's my theory:

I believe that most non-developers think developers are like Neo in The Matrix: that devs can see all parts of an app simultaneously at all times, like Neo's living-ASCII-art view of the Matrix itself. That devs are omniscient, omnipotent beings in regard to an app. I've come to think of this mental model as the Neo Fallacy.

If pressed, I don't think we could say how we think developers do this. (Optic fiber running from servers to devs' brain stems? Magic emanating from the bowels of 4chan? Google Glass?) But I can't think of a better explanation for why so many of us assume devs can figure out a bug from so little information.

Returning to theory No. 1, I asked some Viget developers why reproducibility is so important. Some of their answers:

"So we can hopefully write a failing test first, and then have one in place to prevent future regressions"
"To confirm it's an actual bug"
"To know when we've fixed it"
"To be able to investigate the issue deeply"
"So we don't waste our super valuable time"

All make sense. But those answers are beside the point to someone operating under the Neo Fallacy: "Fine, but why do you need to reproduce the bug to achieve all those things? Can't you just 'see' what's happening?"

Here's an answer that I hope helps reframe the way we think about developers:

Developers are not Neo. Yes, they have superpowers when it comes to writing code. But they're mere mortals when it comes to knowing what's happening in an app in real time. They're not omniscient.

Occasionally, they may be able to quickly ascertain a problem based on an intuitive sense of an app's behavior and an auto-generated error report from Airbrake. But most of the time, a context-free, vague bug report will mean no more to a developer than it would to you. Literally: without the details I outlined in my other posts — URLs, login creds, specific steps — you might as well submit the bug report to yourself.

And having to spend time trying to glean meaning from such a vague report? For devs, that's about as fun as hanging out with these guys.

Simplify your build-out process using Layouts with Views

Tommy Marshall — Fri, 03 May 2013 15:16 GMT

I recently found myself building out a comp which seemed to have only enough code repetition to include a header and footer. However, the further I got into build-out the more code repetition emerged. I saw an opportunity for a MVC-like views system to help, so I set out to build one to make my build-out process easier.

Code Examples

Let's say you have a basic layout with a reusable header and footer, like the one in the example below.

<!-- location: views/layouts/default.php -->
<html lang="en">
<head>
    <title>Example Layout</title>
</head>
<body>

    <header>
        <?php $this->render('shared/header'); ?>
    </header>

    <div class="content">
        <?php $this->getContent(); ?>
    </div>

    <footer>
        <?php $this->render('shared/footer'); ?>
    </footer>

</body>
</html>

This default layout is rendering a header and footer file located in the views/shared/ directory. Now, if we wanted to create an About page with some content that uses this default layout, we would do something like this:

<!-- location: views/about.php -->
<?php $this->layout('default'); ?>

<h2>About Page</h2>

<p>Lorem ipsum dolor copy.</p>

If we opened domain.com/about in the browser, we would see the About page content loaded within the default layout we created above. (Note: In the above example we could lose the layout declaration at the top since Layouts with Views expects to load default.php as the default layout. This setting can be changed in settings/config.php.)

The above example shows a basic application of Layouts with Views. Check out the Github page for examples of features like:

Multiple layouts
Nested views
Passing variables to views
Allow for common control structures (if, foreach, etc.)
Referencing page assets (images, stylesheets) using an absolute path
Simple routing (Create the file, done)

Download

git clone git@github.com:tommymarshall/layouts-with-views.git

.. or click here to download a zip.

In Conclusion

Layouts with Views sets out to solve a small problem and that's it. There's no customizable routing, no crazy templating features, no unusal syntax to learn. It’s just a simple way for me to build out comps. That's what I like about it.

JavaScript Execution Patterns for Non-Web Apps

Trevor Davis — Tue, 30 Apr 2013 13:29 GMT

A big challege with sites utilizing JavaScript is determining which JS should be executed on each page. Here are some approaches to doing this, along with my preferred method. While libraries like Backbone and Ember have ways of executing JavaScript, this article will mainly focus on executing JavaScript for web sites, not web apps. So I'm talking about more marketing style sites instead of applications. An example would be this very site versus something like Gmail.

For these examples, I'm assuming the following:

jQuery is being used on the site
All of the JS for the site is concatenated into a single file and loaded on every page.

Everything in document.ready

Before I go into preferred methods, I figured that it makes sense to talk about what you shouldn't do. This is how I used to do it when I first learned how to use jQuery, and I still see some people doing this today.

$(document).ready(function() {
	$('.tabs').tabs();
	
	$('.carousel').carousel();
	
	$('.slider').slider();
});

Just don't do this. Even though jQuery fails silently, there is no reason to just throw stuff at the DOM and hope it sticks. Come up with a plan to execute only the JS that you need for each page.

Dom Based Routing

Paul Irish initially talked about this technique in 2009. The idea is to use the id and class attributes on the body element to determine which JavaScript should be fired.

Let's pretend that your body element looks like this:

<body class="shopping" id="cart">

And you have some JS that looks like this:

SITE = {
	common: {
		init: function(){ ... },
		finalize: function(){ ... }
	},
	shopping: {
		init: function(){ ... },
		cart: function(){ ... },
		category: function(){ ... }
	}
};

The execution code is set up so that it calls the following methods:

SITE.common.init()
SITE.shopping.init()
SITE.shopping.cart()
SITE.common.finalize()

Garber-Irish Implementation

This technique came out of my coworker, Jason Garber, working on a Rails project and had a desire for a little more structure. The idea is to use the Rails controller and action to execute specific JavaScript.

For example, on the users show page, the body element would look like this:

<body data-controller="users" data-action="show">

So you create objects based on controllers with a method for the action. The JavaScript would look like this:

SITE = {
	common: {
		init: function() {}
	},
 
	users: {
		init: function() {},
		index: function() {},
		show: function() {}
	}
};

The execution code is set up to call the following methods:

SITE.common.init();
SITE.users.init();
SITE.users.show();

This pattern has worked great for a number of sites, and I've even used the pattern for non-Rails projects. But, as we have continued to build more responsive and modular sites that are more system-based instead of page-based, the pattern has become less and less ideal.

Feature-Based Execution

This technique takes a feature based approach instead of a page specific approach. The idea is to break each feature out into a separate object, and then add the features that you want to execute as an attribute on the body. Here is an example of what our body element would look like:

<body data-features="timeline tabs filters modal">

Here is how we have each of the features broken out into separate objects:

SITE.features = {
	filters: {
		init: function() {}
	},

	modal: {
		init: function() {}
	},

	tabs: {
		init: function() {}
	},

	timeline: {
		init: function() {}
	}	
};

Then, I typically add an init method onto our SITE.features object to initialize all of the feature based execution:

SITE.features = {
	init: function() {
		var features = $('body').data('features');
		var featuresArray = [];
 
		if(features) {
			featuresArray = features.split(' ');
 
			for(var x = 0, length = featuresArray.length; x < length; x++) {
				var func = featuresArray[x];
 
				if(this[func] && typeof this[func].init === 'function') {
					this[func].init();
				}
			}
		}
	}
};

View as Gist

So when we call SITE.features.init() on document load, we grab the data attribute off of the body, and execute each feature. It would execute the following methods:

SITE.features.timeline.init()
SITE.features.tabs.init()
SITE.features.filters.init()
SITE.features.modal.init()

Feature-Based Execution + Script Loader

An idea that I have been playing with, but haven't implemented yet on a site, is to use a script loader to load features for the page instead of including them in a single concatenated file. Here is what the modified SITE.features.init() method looks like using the yepnope script loader:

SITE.features = {
	init: function() {
		var features = $('body').data('features');
		var featuresArray = [];
 
		if (features) {
			featuresArray = features.split(' ');
 
			for(var x = 0, length = featuresArray.length; x < length; x++) {
				var func = featuresArray[x];
 
				yepnope([{
					load: 'scripts/features/' + func + '.js',
					complete: function () {
						if(this[func] && typeof this[func].init === 'function') {
							this[func].init();
						}
					}
				}]);
			}
		}
	}
};

View as Gist

jQuery Meetup

I recently had the opportunity to give a presentation on this topic, along with jQuery plugin development (go to slide 72 if you want to skip the jQuery plugin development piece) at the DC jQuery Meetup.

What Did We Learn?

Don’t just haphazardly fire a bunch of JS functions that may or may not be used. Come up with a pattern for triggering JS on each page that makes sense for your site.

So what methods have you used for executing JavaScript on sites?

Why I Chose to Learn C

Patrick Reagan — Fri, 29 Mar 2013 16:05 GMT

This past fall, a group of us set out to (re-)learn the C programming language using Zed's Learn C the Hard Way as our primary resource. In the age of high-level languages like Ruby and JavaScript, it may seem a bit strange to take such a "step back" when it's been 10+ years since I have had any significant experience using the language. So why do it?

I wanted a new challenge.

As an experienced programmer, the basics of the language were unsurprising (as expected). Variables, conditionals, and loops presented no surprises as I've seen these concepts time and time again in other languages. I will admit that understanding pointers was initially as confusing as when I first encountered them more than 15 years ago. This confusion reinforced the fact that C is not a language for the faint of heart -- there are a number of challenges that a programmer has to overcome when working in a lower-level language.

These challenges weren't without their benefits. During our course of study, we became proficient with a number of language features and related tools:

Memory allocation — Once I moved past writing trivial programs, heap allocation became a necessity. Not only did I learn how to use malloc and free appropriately, but also when to use realloc to create dynamic data structures and calloc to create robust allocations for storing string data.
Makefiles — Zed's introduction to make and subsequent discussion of its advanced features really helped to de-mystify the tool for me. The additional documentation and alternate applications helped me understand its utility for building C programs and other automated tasks.
Function pointers — It was interesting to me to see that such a low-level language contained functional programming concepts. I found function pointers a useful refactoring technique when dealing with different collections of structs. For example, when sorting an array of integers or strings, I was able to use functions with the same signature and behavior but with different implementations.
C pre-processor — From the beginning, Zed made heavy use of the C pre-processor to create macros for debugging, iterating, and otherwise simplifying the code that I needed to write. After gaining more experience with the language, I learned when to choose a pre-processor macro over a subroutine (or vice versa) to refactor and improve readability of the code.
Variable argument functions — Surprisingly, this language feature came in handy more often than I thought. Internally, this is how the printf family of functions is implemented. I used it to create functions like max_length(char *s1, char *s2, ...) to get the overall maximum length of n number of strings, and array_push(Array *array, ...)` to add elements to an array based on its underlying storage type.
Valgrind — I have no idea how you would create a stable program without this tool. Time and time again, I would write what I thought was robust code, but valgrind would call me out on my mistakes. Whether it was a simple memory leak or an off-by-one error when accessing a segment of memory, I was able to quickly diagnose those mistakes and correct them. Without valgrind, these errors could easily slip by undetected.

Overall, the challenges presented with C really forced me to think deeply about how I organized my code and interacted with the machine. Understanding that balance between bare-metal performance and human understandability definitely revealed the language's sweet spot. Even in an industry where older technologies are constantly rendered obsolete, that balance is the reason developers of major modern software projects continue to choose C for their implementation language.

If you're interested in (re-)learning the language yourself, I recommend LCTHW as a good starting resource. Working through the exercises as a group helps keep everyone motivated and generates some fun ideas for other programs to write.

Preventing Interface-Induced Backbone.sync Firestorms

Nate Hunzaker — Tue, 26 Feb 2013 14:19 GMT

Draggable controls raise an interesting challenge when Backbone.Views share a common model.

Such controls include colorpickers, slides, and positionable elements. When users move around a color wheel or acutely adjust the position of an element, change events should trigger to let other components know their portion of the interface needs updating.

For this to happen, the browser's data layer must update immediately. But how does the application know when to pass the torch to the database?

Requesting to persist data on every pixel deviation can be disastrous. When a user wants the perfect shade of yellow, there's nothing like a torrent of PUT requests to dampen their mood.

To illustrate:

Not exactly performant. Fortunately UnderscoreJS provides a great toolset for tackling exactly this problem.

Only you can prevent forest-fires, by debouncing

UnderscoreJS's debounce method is particularly well suited for our use case: it postpones invocation of a function until a specified expiration time that resets on every function call. This can be used to bundle together Backbone.sync executions from the same source, significantly lowering requests without nasty side effects.

Update: Originally I handled debouncing directly within the sync method of the Backbone.Model, however a sharp commenter has suggested a more optimal solution by moving this functionality into the Backbone.View that oversees the input.

To avoid directly manipulating the Backbone.sync method, an event must be bound in the view that controls such inputs. On change, the view should immediately provide actions required to update the user interface, requesting to sync in the background.

var View = Backbone.View.extend({

    events: {
        'change input.slider' : 'onSliderChange'
    },

    saveInBackground: _.debounce(function(params) {
        return this.model.save(params, { silent: true });
    }, 800),

    onSliderChange: function(e) {

        // Take immediate action, such as setting the values locally:
        this.model.set({
            value: e.target.value
        });
       
        // Then call a debounced save method, which will eventually
        // trigger Backbone.sync
        this.saveInBackground();
       
    }


});

Beautiful and Simple

This minor addition achieves exactly what is needed! No matter how furiously the user updates information, the ~~model~~ view is now smart enough to send an AJAX request once, a short period after the drag motion has completed. Minor tweaks may be required depending on the purposes of the application, but the resulting code lets the user interface immediately respond to the change, while not causing servers to catch on fire.

Arduino, Redis, Dropcam + Holiday Cheer = Jinglebots!

Eli Fatsi — Wed, 19 Dec 2012 21:33 GMT

Hopefully, you've been able to experience our Jinglebots -- the 2012 Pointless Corp holiday project -- for yourself. Klaus 5000, Cornelius, and Ru 2 Dee 2 are the Jinglebots -- “robots” which reside in each of our Viget offices. They “speak” tweets and messages for all of the world to hear through a Dropcam feed. As a user, you can check out the web site and either tweet with the #jinglebots hashtag or submit a message through a basic messenger on the site.

I've estimated the amount of coordinating technologies to be around a bazillion. For those who may be interested in how to use your computer to control Christmas lights, power electric toy trains, or do other cool things (using Ruby!), I’ll walk you through how to do it below.

Step 1 - Instant Communication

We knew that we'd be responding to messages and tweets in real time. For this, we made use of Redis pub/sub capabilities. Redis is super simple to get going locally and use of RedisToGo gives you access to a free online Redis instance you can use in production. When a message comes in, we publish the message data (sender name, message, timestamp, etc.) in JSON to a Redis channel.

REDIS.publish("holiday_messages", data.to_json)

Each of our Jinglebots is subscribed to the holiday_messages channel.

REDIS.subscribe('holiday_messages') do |on|
  on.message do |channel, raw_data|
    data = JSON.parse(raw_data)
    ... some awesome stuff using the data
  end
end

Instant communication has never been easier.

Step 2 - Receiving Messages/Tweets

On the web site, a form allows users to submit a message and a name. On submission success, the app then compiles the necessary data, converts it to JSON, and publishes it to the holiday_messages Redis channel.

To capture incoming tweets with the #jinglebots hashtag, we made use of EventMachine and the twitter-stream gem. This bit of code owes great credit to Twerrible Towel which has their code open sourced. The following code takes care of all the Twitter listening for us.

EventMachine::run do

  stream = Twitter::JSONStream.connect(
    :path    => '/1/statuses/filter.json',
    :auth    => "#{twitter_username}:#{twitter_password}",
    :method  => 'POST',
    :content => "track=#jinglebots"
  )

  stream.each_item do |item|
    tweet = JSON.parse(item)
    if tweet && tweet["user"]
      data = assemble_data(tweet)
      REDIS.publish("holiday_messages", data.to_json)
    end
  end
end

Step 3 - The Fun Part

So, with a stream of messages coming in, we had to figure out what to do with them. If you're on a Mac, open up the terminal and type say hello master. With the volume turned up, your computer has been given a voice! After some massaging of the incoming message (removing special characters, subbing out bad words for a system beeeeep), we now had robots which could speak any message we threw at them.

We weren't done though. What's a talking robot without a face? Enter the tag team work of designer Joseph Le and front-end developer Nate Hunzaker. A simple-yet-awesome Node.js app on Express meant that we had three different faces (one for each Jinglebot) which could be turned to "talking" and "not talking" mode with a curl command to the local server.

`curl --data "" http://localhost:3000/say?message=true`
`curl --data "" http://localhost:3000/shutup`

Open sourced code is available here.

Using the same Node.js/Express combination, a gif(t) displayer was coded up so the talking robots had something to display while they read your message aloud.

Step 3.5 - The Really Fun Part

Arduino time! An Arduino is a programmable microcontroller which allows you to develop in Processing and do cool things like turn on LEDs or make an Iron Man Costume. We're still pretty new at this stuff, so we went with the ability to control Christmas lights and an old electronic toy train set (maybe next year we'll go for the Iron Man Costume).

There's one more piece of this puzzle to introduce which made this process super easy -- dino. Dino is an awesome gem which lets you control your Arduino using Ruby. Simply load up the dino code to your Arduino, set up your board in Ruby, and start sending power where you want it.

require 'dino'

board = Dino::Board.new(Dino::TxRx.new)
pin   = Dino::Components::Led.new(pin: 13, board: board)

pin.on
sleep 4
pin.off

We made use of a relay and a power cord cut in half. When the relay switch is open, no power can pass from the wall to our appliance. When we give the command pin.on, we send 5V to our relay, closing the switch, powering our appliance with a full 120V from the wall. A basic tutorial of how to set this up is available here.

Note: To get the Christmas lights to flash, we actually set a Redis key named "flash" to true or false and had a Ruby script checking that variable every second and flashing the lights independent of the robot. This was done to keep blocking operations from interfering with each other.

Step 4 - The Web Site

Viget's home turf. Designer Mark Steinruck, UXers Jason Toth and Lance Gutin, and again the almighty front-end developer Nate came together to make a festive, easy, and fun-to-use site. We leveraged the Dropcam already set up at each office to let users watch each Jinglebot right on the sight, and displayed a running list of the most recent messages, as well as aggregate stats, on the bots and messages.

Messages and tweets that came in were not only published to the Redis channel, but were also saved to a PostgreSQL database with the help of DataMapper. In this way, we had a clean and organized record of all messages that came in to the Jinglebots.

Step 5 - The Jingle Report

As a little bonus, we wanted to give the user more than just a talking robot. A Jingle Report was created for every message or tweet that came in, and tweets were sent to users with the Report link to view. Here's a personal favorite - http://jinglebots.com/report/901.

Each report required a screenshot of a Jinglebot scene, as well as a voice recording of the message. LAME was used to convert the saved audio files (say -o file/path/to/save.aiff message) into web-friendly formats, some Dropcam craftiness was used to grab screenshots from a given time (thanks to a post in this forum), and Amazon S3 was used to store these assets. Boom: Jingle Reports.

Step 6 - The Auto Pilot

Cronjobs were a big win for this project. Having the code systematically restart every so often meant that if anything did go wrong (a lot of coordinated tech -> a lot of potential problems), it would take care of restarting itself.

This was a major component for things like the amount of sleep Ben and I could get without having to restart the Jinglebots after they mysteriously crashed in the middle of the night. The code behind this is here.

Summary

On Friday the 21st, the Jinglebots had been spreading holiday cheer for two whole weeks, and the project was a blast to work on. It was a great combination of Viget’s coding skills, as well as our discovered "building-real-things" skills. More on why the crew involved came together to build something like this can be read here.

Jinglebots was the brainchild of Ben Eckerson (without him the project would never have existed) and the collaborative effort of many - Becky Tornes, Khanh Stenberg, Anjali Mckenzie, Mike Ackerman, and the whole Viget dev team for their advice and mentorship.

Happy Jinglebottin!

Using Time Zones with Rails

Eli Fatsi — Wed, 19 Dec 2012 15:25 GMT

On a recent Rails application I built, I needed to determine which time zone a user was in to correctly display and use time-related information. I'll walk through two techniques I've found to get the job done.

The theory around working with time zones is pretty straightforward. Save all time-related data in Coordinated Universal Time (UTC), and display all time-related data in the time zone of any given user. If you're unfamiliar with ActiveSupport's TimeWithZone, quickly check out this blog post from David Eisinger and familiarize yourself with the in_time_zone method for display purposes.

The question then becomes, "How do we know what time zone a user is in?"

Method 1 - Quick, simple, 90% right

In search for the answer, I first discovered some simple JavaScript code:

var currentTime = new Date();
var utcOffset   = currentTime.getTimezoneOffset()
document.cookie = 'time_zone_offset='+utcOffset+';';

new Date() constructs a date object with the current time based on your system clock. getTimezoneOffset() returns the number of minutes from which you differ from UTC. Throw that in a cookie and you're done with the JavaScript.

On the Rails side of things, you can grab that offset from the cookies and run the following code to determine which time zone has that offset.

offset    = cookies["time_zone_offset"].to_i
time_zone = ActiveSupport::TimeZone[-offset.minutes]
current_user.update_attribute(:time_zone => time_zone)

However, This doesn't exactly work. There are multiple time zones that share a common offset. If you use this technique on the east coast of America, it's likely you'll get (GMT-05:00) Bogota, the capitol of Columbia, or (GMT-04:00) Atlantic Time (Canada), depending on if you've recently sprung forward or fallen back. This is due to the fact that these time zones come before (GMT-04/05:00) Eastern Time (US & Canada) alphabetically.

While this approach will still give you the accurate time for the present day, Daylight Savings standards of other regions could affect your code in the future. If your app does not care about the future, this solution is fine. My app does care about the future, though, and suddenly all my tests started failing when Daylight Savings Time came around.

Method 2 - Quick, a little less simple, 100% right

Enter JSTZ! This stands for jsTimezoneDetect and is a sweet bit of code which parses out the actual time zone from your computer's system time. Source code can be found here.

With the set of functions jstz offers, the following lines of code will correctly determine the user's system time zone and store it in a cookie called jstz_time_zone.

var timeZone = jstz.determine();
document.cookie = 'jstz_time_zone='+timeZone.name()+';';

From there, you can retrieve the cookie in any controller and update the time_zone attribute on the user.

current_user.update_attribute(:time_zone => cookies["jstz_time_zone"])

Excellent! Now we are correctly retrieving a user's time zone from their computer and saving it on the user.

Confirming

At this point you have retrieved the user's system time zone, however it's possible that this is not the correct time zone. While it's a good guess to have when asking the user for input on the matter, it's necessary that you ask the user to confirm or select a time zone to guarantee correctness.

<% unless cookies[:selected_a_time_zone] %>
  <div class="time-zone-confirmation">
    <%= "Are you currently in the following time zone: #{current_user.time_zone}?" %>
    <%= button_tag "Yes", :class => 'hide-time-zone-confirmation' %>
    <%= button_tag "No",  :class => 'show-time-zone-selection' %>
    <div class="time-zone-selection" style="display:none;">
      <%= simple_form_for @user, :url => "/time_zone", :method => :put do |f| %>
        <%= f.input :time_zone, :collection => ActiveSupport::TimeZone.us_zones.map(&:name) %>
        <%= f.submit "Submit" %>
      <% end %>
    </div>
  </div>
<% end %>

The following jQuery code accompanies the form to make for a nice user interaction.

$('button.hide-time-zone-confirmation').on('click', function() {
  document.cookie = 'selected_a_time_zone=true;';
  $('.time-zone-confirmation').hide();
});

$('button.show-time-zone-selection').on('click', function() {
  $('.time-zone-selection').show();
});

A route is added in routes.rb to direct the post to the correct controller.

put "/time_zone" => "time_zones#update"

And lastly, here is the controller code to save the decision.

def update
  current_user.update_attribute(:time_zone => params["user"]["time_zone"])
  cookies[:selected_a_time_zone] = true
  redirect_to :back
end

To sum up, we first make our best guess as to what time zone the user is in. We then ask the user to either confirm our guess, or select a time zone from a given list. In both cases, a cookie named selected_a_time_zone is set to true, which prevents the confirmation form from being displayed in the future. Updating a time_zone attribute on the current_user can then be used to correctly display all time-related info.

Rails Engine Testing with RSpec, Capybara, and FactoryGirl

Brian Landau — Thu, 06 Dec 2012 14:15 GMT

UPDATED: I've updated the instructions based on feedback in the comments suggesting the use of the --dummy-path=spec/dummy --skip-test-unit options on the rails plugin new command.

Recently, we've been doing a lot more with Rails engines. We've developed a few engines that we've released publicly, and even more that we use privately on applications. We've found it's a good way to organize and share reusable code across a number of applications.

We really like using RSpec, Capybara, and FactoryGirl to test our Rails applications, and we like to use them to test our engines too. Here's a few steps to get your new engine up and running with these gems in no time:

Run rails plugin new ENGINE_NAME --dummy-path=spec/dummy --skip-test-unit --full or rails plugin new ENGINE_NAME --dummy-path=spec/dummy --skip-test-unit --mountable. This is a good discussion of why you'd choose mountable or full engines.

Add these lines to the gemspec file:

s.add_development_dependency 'rspec-rails'
s.add_development_dependency 'capybara'
s.add_development_dependency 'factory_girl_rails'

Add this line to your gemspec file:
```
s.test_files = Dir["spec/**/*"]
```

Modify Rakefile to look like this:

#!/usr/bin/env rake
begin
  require 'bundler/setup'
rescue LoadError
  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
end

APP_RAKEFILE = File.expand_path("../spec/dummy/Rakefile", __FILE__)
load 'rails/tasks/engine.rake'

Bundler::GemHelper.install_tasks

Dir[File.join(File.dirname(__FILE__), 'tasks/**/*.rake')].each {|f| load f }

require 'rspec/core'
require 'rspec/core/rake_task'

desc "Run all specs in spec directory (excluding plugin specs)"
RSpec::Core::RakeTask.new(:spec => 'app:db:test:prepare')

task :default => :spec

This will setup your Rakefile to run your specs and make that the default task. It will also setup the specs to reload the test database just like how it works inside a Rails application.

Create a spec/spec_helper.rb file:

ENV['RAILS_ENV'] ||= 'test'

require File.expand_path("../dummy/config/environment.rb",  __FILE__)
require 'rspec/rails'
require 'rspec/autorun'
require 'factory_girl_rails'

Rails.backtrace_cleaner.remove_silencers!

# Load support files
Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each { |f| require f }

RSpec.configure do |config|
  config.mock_with :rspec
  config.use_transactional_fixtures = true
  config.infer_base_class_for_anonymous_controllers = false
  config.order = "random"
end

This code requires all the gems you need for writing your specs, loads the dummy application, and configures RSpec.

Finally, add this config to your engine file (lives at lib/my_engine/engine.rb):

module MyEngine
  class Engine < ::Rails::Engine

    config.generators do |g|
      g.test_framework      :rspec,        :fixture => false
      g.fixture_replacement :factory_girl, :dir => 'spec/factories'
      g.assets false
      g.helper false
    end

  end
end

Here, we're telling Rails when generating models, controllers, etc. for your engine to use RSpec and FactoryGirl, instead of the default of Test::Unit and fixtures

After this, you can start writing specs with FactoryGirl factories and integration specs (inside the spec/features directory) with Capybara. Have fun testing!

Rescuing exceptions without notifications or: How to fail without knowing

Brian Landau — Mon, 03 Dec 2012 13:30 GMT

Have you ever written code like this:

def my_method
  begin
    object.method_that_can_raise_exception
  rescue Exception
    # some exception handling
  end
end

Whenever we write code like this we should stop ourselves and ask: “Is this an exception I need to know about?” Sometimes the answer is legitimately “no”, but there are important cases where you need to know about these caught exceptions.

For instance, sometimes you’re just rescuing a ActiveRecord::RecordNotFound error, in these cases you often know exactly why the error happened, and you know exactly what needs to happen because of it. In these very specific cases you have excellent insight into what’s happening, and the error you’re rescuing is very specific, so silently swallowing these errors is fine.

However, there are some common situations where you need to be aware that exceptions are happening:

1. Rescuing a very generic exception class

If you’re going to be rescuing Exception, StandardError, TimeoutError, IOError, RuntimeError, a descendant of SignalException, or something equally generic, you should probably be notifying yourself of these. If you’re going to be rescuing something this generic, it’s a signal to yourself that you probably don’t know a lot about what can go wrong here or why. Capturing what went wrong is the first step to understanding and solving a problem.

2. Making a connection to an external service

If you’re putting a rescue around a network call to an external service (any service you don’t control, and perhaps maybe those you do control but are on separate servers), you should probably notify yourself that something went wrong. Similar to the situation above, you often have little insight (at the time you’re writing the code) into why it might fail. The external service may be down, or maybe your servers are having network issues. Whatever the reason, a remedy needs to be found, and the easiest way to start diagnosing the problem is with good insight into went wrong.

3. Mystery code

Perhaps you’re using someone else’s library, or maybe some code internal to your application was written long ago and is now legacy code. Whatever the reason, even if you’re using a more specific exception class, if you don’t know why you need to rescue the exception you should probably be notifying yourself that it’s happening.

How you notify yourself and track these errors is entirely up to you. We use Airbrake, but there are lots of options available including just using ActionMailer. Here’s an example of how I might use Airbrake to handle catching an error from an external service:

module APIClient

  private

  def api_call(method, options)
    # network code to external service
  rescue Exception => exception
    Airbrake.notify(exception, {
      :component  => self.class.to_s,
      :action     => method,
      :url        => url_of_api_we_are_using,
      :parameters => options,
      :cgi_data   => ENV
    })
  end

end

class MyAPI
  include APIClient

  def some_api_method(options)
    # some other code perhaps
    api_call('api_method', options)
  end

end

With this notification, I’m trying to capture the name of the API (self.class.to_s), the API endpoint/method called (method), the exact URL we are hitting (url_of_api_we_are_using), the options/parameters we’re passing to the API (options), and any ENV data that might be configured on the server. If you’re worried about any portion of this having sensitive data (the options or the ENV for instance), you should scrub that data first, or not include that data at all. If you’re constructing a large body to post (in an XML API for instance), it can be valuable to capture that too. With Airbrake, I’d probably put a large body like that in the hash I pass to :parameters.

For errors not connecting to an external service, there’s often a lot less data to report. It might be as simple as this:

Airbrake.notify(exception, {
  :component  => self.class.to_s,
  :cgi_data   => ENV
})

You might, however, have something relevant to put in the :parameters option.

Whatever service you choose and data you send though, it’s important that you get these notifications and investigate exactly what went wrong. You’ll thank yourself later.

Simple App Stats with StatBoard

David Eisinger — Wed, 28 Nov 2012 21:16 GMT

We build a lot of small apps here at Viget as part of Pointless Corp, like SpeakerRate, OfficeGames, and BabyBookie. It’s fun to track how many people are using them, and rather than write yet another Rakefile to generate reports, I decided to create a simple Rails Engine to display some basic stats. Announcing, then, StatBoard:

Installation is a cinch: add the gem to your Gemfile, mount the app in routes.rb, and set the models to query (full instructions available on the GitHub page). The code itself is embarrassingly simple, so if you have any ideas for improvements, or just want to see how a simple Rails Engine works, take a look.

Refactoring Patterns: The Rails Middleware Response Handler

Patrick Reagan — Mon, 26 Nov 2012 14:29 GMT

Recently, I have found myself writing more middlewares to handle responses that need to live outside of the main Rails application. The API that Rack provides works well for something trivial, maybe where you want to allow a load balancer to query the application's status:


class HealthCheck
  class Middleware

    def initialize(application)
      @application = application
    end

    def call(environment)
      if environment['PATH_INFO'] == '/health-check'
        if HealthCheck.healthy?
          [200, {}, ['OK']]
        else
          [500, {}, ['PROBLEM']]
        end
      else
        @application.call(environment)
      end
    end
  end
end

I'm sure you've seen something similar to the above code in some projects you've worked on — it's not particularly offensive, so there's no urgent need to refactor. But what about those times when your middleware is more complex than just a simple health check?

Consider the example of delivering a mobile-optimized version of your site to your clients:


module Mobile
  class Format

    def initialize(application)
      @application = application
    end

    def call(environment)
      request = Rack::Request.new(environment)

      user_agent = ::Mobile::UserAgent.new(request.env['HTTP_USER_AGENT'])

      if request.params['mobile'] == '1' || user_agent.mobile?
        request.env['HTTP_ACCEPT'] = 'text/html+mobile'
      end

      status, headers, body = @application.call(request.env)

      response = Rack::Response.new(body, status, headers)
      response.finish
    end
  end
end

Although this code functions as expected, it's not particularly clear what's going on to someone who will later need to maintain this codebase. In addition to ensuring that my code works, part of my job as a programmer is to create code that's easy to understand and, by extension, maintain. Working within the constraints of the middleware API, I might opt for some named abstractions to improve understandability:


module Mobile
  class Format

    def initialize(application)
      @application = application
    end

    def call(environment)
      request = Rack::Request.new(environment)

      set_mobile_accept_header(request) if serve_mobile?(request)
      response = response(request)
      response.finish
    end

    private

    def user_agent(request)
      Mobile::UserAgent.new(request.env['HTTP_USER_AGENT'])
    end

    def mobile_requested?(request)
      request.params['mobile'] == '1'
    end

    def serve_mobile?(request)
      mobile_requested?(request) || user_agent(request).mobile?
    end

    def set_mobile_accept_header(request)
      request.env['HTTP_ACCEPT'] = 'text/html+mobile'
    end

    def response(request)
      status, headers, body = @application.call(request.env)
      Rack::Response.new(body, status, headers)
    end

  end
end

This refactoring is a step toward the goals of clarity and maintainability, but this code could still be improved. The passing of request to supporting methods is a code smell. It suffers from two problems — it decreases the clarity of my named abstractions and it breaks encapsulation. To fix this problem, you might be tempted to store environment as an instance variable and instead lazily load request:


module Mobile
  class Format

    def initialize(application)
      @application = application
    end

    def call(environment)
      @environment = environment

      set_mobile_accept_header if serve_mobile?
      response.finish
    end

    private

    def request
      @request ||= Rack::Request.new(@environment)
    end

    def user_agent
      Mobile::UserAgent.new(request.env['HTTP_USER_AGENT'])
    end

    def mobile_requested?
      request.params['mobile'] == '1'
    end

    def serve_mobile?
      mobile_requested? || user_agent.mobile?
    end

    def set_mobile_accept_header
      request.env['HTTP_ACCEPT'] = 'text/html+mobile'
    end

    def response
      status, headers, body = @application.call(request.env)
      Rack::Response.new(body, status, headers)
    end

  end
end

While it looks like this solves our problems, lazily loading request actually introduces a subtle bug. Because the value is memoized across all requests, the first request to this middleware will determine the result for subsequent requests. This means if the first client is to be served the mobile version of the site, then all other clients will see that version as well.

Out of the options I've explored above, none are viable solutions for a production application. Instead, I opt for an approach that I've found successful on some of our recent projects that I have been calling the "middleware response handler":


module Mobile
  class Format

    def initialize(application)
      @application = application
    end

    def call(environment)
      responder = Mobile::Format::Responder.new(@application, environment)
      responder.respond
    end

    class Responder

      def initialize(application, environment)
        @application = application
        @environment = environment
      end

      def respond
        set_mobile_accept_header if serve_mobile?
        response.finish
      end

      private

      def request
        @request ||= Rack::Request.new(@environment)
      end

      def user_agent
        ::Mobile::UserAgent.new(request.env['HTTP_USER_AGENT'])
      end

      def mobile_requested?
        request.params['mobile'] == '1'
      end

      def serve_mobile?
        mobile_requested? || user_agent.mobile?
      end

      def set_mobile_accept_header
        request.env['HTTP_ACCEPT'] = 'text/html+mobile'
      end

      def response
        @response ||= begin
          status, headers, body = @application.call(request.env)
          Rack::Response.new(body, status, headers)
        end
      end
    end
  end
end

The key change here is that by introducing the Mobile::Format::Responder class that encapsulates environment, I can more easily perform an extract method refactoring and improve the readability of this code. Additionally, each time that this middleware is invoked, I'm guaranteed to have a new instance of the responder — I won't see state bleeding over from previous requests. In my experience on projects like Puma, TimeLife, and Shure, this strategy has been a great way to significantly improve the maintainability of our middleware code.

The above code is also available as a Gist for your forking pleasure.

Server Maintenance Mode for Rails, Capistrano and Apache2

Ryan Foster — Tue, 06 Nov 2012 03:09 GMT

It's a good idea to have a server maintenance strategy in place during the early phases of any project. Along with CI, provisioning multiple stages, and an efficient way to deploy new features to the client and eventually to production, a strategy for server maintenance from the outset of the project will save time as the application ages.

If your Rails app is already in the wild, no worries. Here are a few easy steps to modify Apache to detect a maintenance mode and a Capistrano task to turn it on and off.

First, you'll need to add these lines to you Apache virtual host configuration:

RewriteEngine On
ErrorDocument 503 /system/maintenance.html
RewriteCond %{REQUEST_URI} !.(css|gif|jpg|png)$
RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$ - [L,R=503]

This tells Apache to bypass the app when it detects the presence of a maintenance file. Instead of serving the request, it will return the maintenance page with the status code 503. If Apache is already running, you'll want to restart it before you try to roll it into maintenance mode.

Next, you'll want to create a maintenance page for your app. A good place to put the file is /public/maintenance.html. Remember, the page will bypass Rails completely, so make sure it's a fully valid HTML file (although the rewrite rules will still serve CSS and image files).

Last, add two Capistrano tasks to turn maintenance mode on and off. The first copies the maintenance page to the place where Apache will start serving it. The second will remove it. Add these lines to deploy.rb:

namespace :deploy
  namespace :web do
    desc "Enable maintenance mode for apache"
    task :disable, :roles => :web do
      on_rollback { run "rm -f #{shared_path}/system/maintenance.html" }
      page = File.read('public/maintenance.html')
      put page, "#{shared_path}/system/maintenance.html", :mode => 0644
    end

    desc "Disable maintenance mode for apache"
    task :enable, :roles => :web do
      run "rm -f #{shared_path}/system/maintenance.html"
    end
  end
end

Now when you want to set a stage into maintenance mode (integration in this example), just run the task:

cap integration deploy:web:disable

To disable maintenance mode, run:

cap integration deploy:web:enable

And there you have it: maintenance mode, easy mode.

A few caveats:

Make sure the shared path for Capistrano has a system directory. Otherwise, you'll get an error writing the maintenance file there.
Be careful if there is a load balancer in front of your app servers. It might be configured NOT to serve 503s, but instead start ignoring that app server.

A few links I used to write this post:

Stack Overflow post on the cap task for maintenance mode.
Gist for the suggested Capistrano maintenance check.
The Capistrano tasks used to exist but have since been removed. This gem will add them back.

Client-Side Separation of Concerns: Are We Doing It Wrong?

Doug Avery — Wed, 31 Oct 2012 13:40 GMT

Separation of Concerns is a programming principle that encourages grouping functionality in ways that reduce overlap. 'Concern' is loosely defined - Wikipedia describes it as 'any piece of interest', like a behavior or piece of functionality. The core concept is that by improving the separation of concerns, you improve a program's clarity and durability. Actual separation is done in three ways:

Encapsulation — bundling the concern as a single idea, in a single place
Modularity — making concerns portable and independent
Information Hiding — smoothing concern interaction by providing interfaces between them

Using these methods, you increase the separation of your concerns, and strengthen your system. Easy enough in programming, but how do we apply this principle when working on the client side — that is, CSS/HTML/JS?

I always thought I knew, but in the past few years, some new challenges caused me to rethink the approach. To explain, I'll start by examining the most common SoC model: Content, Presentation, Behavior.

The Classics: CPB

If you haven't heard of the SoC principle, you've probably heard of the three 'layers' of Content, Presentation, and Behavior. These are the three classic concerns that front-end developers are taught to keep separate. How to separate them is another matter, owing to some interpretation. Here's the version I learned:

The concerns are divided into their own files (and languages)
The content (markup) is dumb — that is, style- and behavior-agnostic. Usually, this means writing it first, and only minimally changing it to accommodate JS and CSS when necessary.
Classes are content-oriented, not presentational ('additional-info' vs. 'right-col')
Elements are styled using clever selectors and the cascade
JavaScript is unobtrusive — it selects, operates on, and injects markup to aid behaviors

This should look familiar; it's the classic 'best practice' for front-end work. Comparing it with the first three bullets about SoC, how does it stack up?

Encapsulation — check. Concerns are clearly encapsulated.
Modularity — arguable. While there's a degree of portability (see CSS Zen Garden), it depends on having a very consistent markup structure, which brings us to...
Information Hiding — the big missing piece with the classic, 'markup-first' separation of CPB. Dumb markup, markup that's written solely for content and not for presentation or behavior, doesn't provide an interface. By doing this, it simply offloads the complexity to CSS and JS, which have to parse and select it in clever ways. This leads to all the problems SoC is supposed to prevent — brittleness, overrides, excessive coupling.

This issue — exactly how to separate content, presentation, and behavior — is at the heart of new CSS strategies like OOCSS and SMACSS. The traditional wisdom that dumb markup is good markup is under seige.

So, maybe we don't know exactly how to separate our concerns. But let's step back — why are content, presentation, and behavior our primary concerns in the first place?

Are they the right ones?

CPB is certainly an appropriate concern set for documents, where the content is core and CSS/JS just 'theme' it. This problem domain is what HTML was designed for, and it's what most of us started our careers writing. The question, though: is CPB is right for a Rails site with 100+ views, an all-JS chat application, an online store, or a web-only game?

If not CPB, then what? To take a cue from programming, maybe the right concerns are simply the aspects of your buildout, and your task should be to determine those aspects and how best to separate them. Once you understand your concerns, the task is not how to separate them along the CSS/JS/HTML lines, but to separate them from eachother.

Easier said than done, right? How do we identify concerns, and how do we detangle them? The rest of this post is just that — some loose definitions of common concerns, and some very stripped-down examples of how a dev could improve their separation.

New concerns

Layout

Blueprint, now 5 years old, was one of the first examples of client-side devs addressing a new concern — Layout. It's still one of the easiest concerns to abstract out from your build, and Blueprint has numerous competing frameworks that accomplish the same goal. The concept is simple: using presentational, layout-only classes separates the problem of visually structuring your design from the task of visually styling it.

Unseparated:

In this example, the article type dictates a layout behavior for the additional-info box. In some cases, this kind of behavior needlessly entangles two concerns that aren't otherwise related.

  .news-article .additional-info,
  .blog-article .additional-info {
    background: url('more-news.png');
    float: right;
    width: 320px;
  }

Separated:

By definining a 'layout' to control this behavior, we can separate this responsibility more clearly. This makes the layout itself more portable, and cleans up the article-specific styles.

  .wide-layout .aside-col {
    float: right;
    width: 320px;
  }

Page

What page is a user on? Why does it matter? To separate the page as a concern, we have to consider how much control the current 'location' of a user has over our design, and whether that control is warranted. It's too easy let the Page micro-manage other concerns, which is unfortunate, because the Page is pretty bad at it — reduce Page responsibilities whenever you can.

Unseparated:

In this example, the about and contact pages have a special contact form that appears the footer. The design calls for some additional padding above the form. You might think to handle it this way:

  .about-page .footer,
  .contact-page .footer {
    padding-top: 30px;
  }

Separated:

...but really, it's the form, not the page, that makes the difference. Using the page class just because it's there might be convenient, but it couples two very different concerns badly. Instead, we can give the footer the ability to modify itself, using a smart name that tells us why the modification is necessary:

  .form-footer {
    padding-top: 30px;
  }

Component

A reusable, portable, piece of the interface (this one should look pretty familiar).

At the very least, you can give components a consistent interface in all three languages (by naming them), but you can go further by isolating their bits (CSS, JS, images, possibly markup as a template/partial) into a single directory. Give them subclasses for variations, instead of relying on the cascade or the DOM context. If you find that they interact with other components too much, give them a top-level object to talk to instead.

Each component is a concern, however minor. The key to durable components is separating them correctly from your other concerns.

Unseparated:

There are a few problems here. First, the component has mis-matching names throughout. Second, a supposedly self-contained component is scattered between several different, larger files. Third, the component itself reaches out to the Page.

  // show.html.erb
  <ul class="photos">
    <% @post.images.each do |img| %>
      <li><%= image_tag(img) %></li>
    <% end %>
  </ul>

  // application.js
  $.fn.carousel = function(methodName) {
    if ($(body).hasClass('gallery')) {
      addPagers(this);
    }
    ...
  };

  $('.photos').carousel();

  // application.css.scss
  .js .photos {...}

Separated:

This concern can be cleaned up with some renaming, some new files, and a small interface provided by a data-attribute. Now, it's much more neatly encapsulated.

  // _carousel.html.erb
  <ul class="carousel" data-pagers="true">
    <% images.each do |img| %>
      <li><%= image_tag(img) %></li>
    <% end %>
  </ul>

  // components/carousel/scripts.js
  $.fn.carousel = function(methodName) {
    var options = ({}, $(this).data());
    options.pagers && addPagers(this);
    ...
  };

  $('.photos').carousel();

  // components/carousel/styles.css.scss
  .carousel {...}

Content

When I say Content, I don't mean 'everything on the page' — there's a big distinction between blocks of big, readable, WYSIWYGable text and the other stuff. WYSWIGable areas are wildly unpredictable. They're constantly breaking out of their boxes, using incorrect heading tags, vomiting out iframes, and worse, they have tons of completely unclassed markup that you're suppose to anticipate and make pretty.

The traditional stylesheet wisdom is to style all your baseline elements first, and override those styles for other components, but this mixes the concerns (content, component) pretty badly. One way to detangle them is to apply Content styles/behaviors only in specific contexts:

Unseparated:

Setting the baseline styles up front means that later, we have to unset them. In this case, that means that every UL on the site is tied to this one — when this UL changes, they might all have to change with it.

  // show.html.erb:
  <%= @post.content %>

  // application.css.scss
  h2 {
    font-size: 20px;
    margin: 0 0 15px;
  }

  p {
    margin: 0.4em 0;
  }

  ul {
    list-style-type: disc;
    margin-left: 20px;
  }

  .sidebar p {
    margin: 0;
  }

  .sidebar ul {
    list-style-type: none;
    margin-left: 0;
  }

Separated:

Now that Content is treated like a separate concern, things clean up quite a bit. We can now safely work on styles inside .txt areas without worrying about how they affect other concerns in the interface.

  // show.html.erb:
  <div class="txt">
    <%= @post.content %>
  </div>

  // application.css.scss
  h2, 
  p, 
  ul {
    list-style-type: none;
    margin: 0;
  }

  .txt {
    h2 {
      font-size: 20px;
      margin: 0 0 15px;
    }

    p {
      margin: 0.4em 0;
    }

    ul {
      list-style-type: disc;
      margin-left: 20px;
    }
  }

Window

With responsive design, changes to the window become important information that informs both behavior and style. It's easy to start mixing in media queries and window.innerWidth checks to your code, but the further down this path you go, the more Window starts to feel like its own concern.

Unseparated:

This code doesn't look terrible, but it scales poorly. The window clearly has a special state (<320px), but there's no communication about what this state means, and the logic for it might need to be re-applied across many instances. The Gallery actually binds a window-size check to the window itself.

  @media screen and (max-width: 320px) {
    .content,
    .aside,
    .callouts .col,
    .nav {
      float: none;
    }
  }

  var Gallery = function() {
    $(window).resize(function() {
      if(this.enabled && window.innerWidth < 320) {
        this.disable();
      }
    }
    ...
  }

Separated:

The separated version moves responsibility to an 'appWindow' object that publishes events out to any listening components with some help from a super-dumb pub/sub-type pattern.

  @media screen and (max-width: 320px) {
    .stack-at-phone {
      float: none;
    }
  }

  ...

  appWindow = {
    subscribers: [],
    prevWidth: $(window).width(),

    init: function() {
      $(window).resize(this.checkWidth.bind(this));
      this.checkWidth();
    }, 

    checkWidth: function() {
      var width = $(window).width();
      var sizingDown = width < 320 && this.prevWidth > 320;
      var sizingUp   = width > 320 && this.prevWidth < 320;
      sizingUp   && this.publish('phone');
      sizingDown && this.publish('unphone');
      this.prevWidth = width;
    },

    publish: function(eventName) {
      $.each(appWindow.subscribers, function() {
        this();
      });
    },

    subscribe: function(method) {
      this.subscribers.push(method);
    },
    
    ...
  };

  var Gallery = function() {
    appWindow.subscribe('phone', this.disable);
    appWindow.subscribe('unphone', this.enable);
    ...
  };

Tracking

If you're serious about tracking users' web use with GA (we are!), you'll know that tracking often puts its stubby fingers into the many pies of your other concerns. Good analytics folk want to track all kinds of interactions — page resizes, carousel navigation, scroll distance, you name it — and it can lead to a lot of disparate JS sprinkled around your application.

Because so many concerns need tracking, it can be hard to completely separate them, but you can at least encapsulate everything you can into a single object. Using a plugin like Trackiffer, or the data-track-event approach, you can even wire your markup straight to your tracking with a thin interface.

Unseparated:

Finding and binding markup like this is unmaintainable at a certain scale. Calling _gaq.push directly also gets tricky past a point — it hamstrings you when you want to add transformations or filtering to your tracking (for example, escaping characters).

  // analytics.js
  $('.callouts a').click(function() {
    _gaq.push(['_trackEvent', 'Callout', $(this).text()]);
  });

   // gallery.js
  gallery.$pager.click(function() {
    _gaq.push(['_trackEvent', 'Gallery', 'Page']);
  });

Separated:

Separation here is a matter of creating interfaces. First, between the markup and the tracking concern; allowing the markup to explicity talk to the tracking. Second, between everything else and the tracking concern — creating a helpful object that can parse tracking data, watch for clicks, and take over the responsibility of tracking from other modules.

  // show.html.erb
  <a href="#" data-track-event="Callout|Rebate">Rebate</a>

  // gallery.js
  analytics.watch(this.$pager, 'Gallery', 'Page');

  // analytics.js
  $('body').on('click', 'a[data-track-event]', analytics.track);

Closing

I hope these points and examples, if nothing else, affected how you think about concerns in the context of front-end development. If you have any specifics you'd like to discuss, please add them to the comments.

Testing Google Analytics with PhantomJS

Nate Hunzaker — Wed, 17 Oct 2012 16:19 GMT

Google Analytics is a great tool for tracking events happening on your website. Yet ensuring that correct information is being recorded can be a tedious task. For my most recent engagement I needed to add analytics support to a large number of pages with challengingly different environments. There was a constant worry that some rules would collide with others or not fire correctly when juxtaposed with existing scripts.

Navigating page-by-page to ensure stability can provide solid verification that code is working properly. However, when working with a nontrivial amount of pages, manually conducting this work is time-consuming and daunting.

Cue PhantomJS, a headless WebKit browser designed for page automation and browser-based JavaScript testing. When coupled with SinonJS spies and the Mocha testing framework the difference between testing a single page and twenty at a time became marginal.

I'd like to go over these technologies in detail and provide a basic tutorial of our methodology.

What is a headless browser?

PhantomJS runs without a graphic interface. It can navigate as if it were Google Chrome or Safari without actually having to open a window. This is a great boon to JavaScript testing as it provides quick access to a browser-based environment with a simple API to automate page navigation and JavaScript interaction.

var page = require("webpage").create();

page.open("http://google.com");

page.onError = function(e) {
    console.log("Something went horribly wrong!", e);
};

SinonJS

SinonJS is a testing library that provides access to spies in JavaScript testing. Spies can track activity surrounding targeted bits of code. They help to verify that the correct methods are being called and if they behaved as expected.

When testing we'll trigger an event that activates Google Analytics and use a spy to see if the correct method was fired. We'll also use them to confirm that the correct parameters were pushed up to Google Analytics, which will provide some additional comfort that cross-pollution between events is not occuring.

var spy = sinon.spy(_gaq, "push");

_gaq.push(['_trackEvent', 'Header Links', 'Click']);

console.log(spy.called) // => true

Mocha

Mocha is a JavaScript testing framework authored by the ever prolific TJ Holowaychuck for Node.js and browser environments. Its syntax is similar to Jasmine and in some cases the two are mistaken for one another.

describe("Sample test", function() {

    it ("should perform a sanity check", function() {
        assert(true);
    });

});

Later in this article we'll delve deeper into Mocha, creating some more tests and integrate a custom test reporter that will help us to communicate the results back to PhantomJS.

Digging in to the code

I have hosted the final result of this article on Github. I am working with the following file structure:

Building the sample rules

First we need to add some events to the page that we want to track. We'll add a couple of rules to the homepage to track some of the various activities on the site. Remember we're navigating to viget.com which already loads jQuery:

src/rules.js

(function($) {
    
    // Track clicks on outbound links
    $("body").delegate("a", "click", function(e) {
        
        if (this.hostname === 'viget.com') return;

        e.preventDefault();
        
        _gaq.push([
            '_trackEvent', 'Outbound Links', 'Click', this.text
        ]);

        setTimeout(function(url) {
            document.location = url;
        }, 100, this.href);

    });

    // Track clicks on the news items section
    $(".grouping-homepage-news-items a").click(function(e) {

        e.preventDefault();
        
        var data = $(this).data("track-event").split(",");
        data.unshift('_trackEvent');
        
        _gaq.push(data);

        setTimeout(function(url) {
            document.location = url;
        }, 100, this.href);

    });

    // Tracks links to careers
    $("a[href*='about#careers']").click(function(e) {

        e.preventDefault();

        _gaq.push(['_trackEvent', 'Careers', 'Click Link']);

        setTimeout(function(url) {
            document.location = url;
        }, 100, this.href);

    });
    
}(window.jQuery));

Pretty straightforward. Remember that we need to delay the link so that ~~.push() has enough time to send information~~ Google Analytics has enough time to setup before the browser navigates to a new page.

Lines 3-18: Whenever an outbound link is clicked, track it.

Lines 20-34: When a news item is clicked, pull some out some data and track it.

Lines 36-47: When a link to our careers section is clicked, track it.

Creating the tests

Now that we know what we want to track, let's build a test that evaluates if we were successful:

tests/home-test.js:

describe("Viget Home Test", function() {

    var spy = sinon.spy(window._gaq, "push");
    var assert = sinon.assert;

    beforeEach(function() {
        spy && spy.restore();
        spy = sinon.spy(window._gaq, "push");
    });  
    
    it ("tracks outbound links", function() {

        var link = $("a[href='http://www.pointlesscorp.com/']");

        link.click();
        
        assert.called(spy);
        assert.calledWith(spy, [
            '_trackEvent', 'Outbound Links', 'Click', 
            'Pointless Corp.'
        ]);

    });

    it ("tracks clicks on the news items section", function() {
        
        var link = $(".grouping-homepage-news-items a").first();
        
        var data = link.data("track-event").split(",");
        data.unshift("_trackEvent");
        
        link.click();

        assert.called(spy);
        assert.calledWith(spy, data);

    });

    it ("tracks clicks on careers links", function() {
        
        var link = $("[a[href*='about#careers']").first();

        link.click();

        assert.called(spy);
        assert.calledWith(spy, [
            '_trackEvent', 'Careers', 'Click Link'
        ]);

    });
    
});

In the code above, we generate a suite of tests for the Viget homepage. Here's what's going on:

Line 3: An empty variable for our spy. At the beginning of each test this spy will be regenerated so we need to provide a variable with this level of scope here.

Line 4: The assertion library used for testing the results. This is a set of functions provided by SinonJS.

Lines 6-9: Before each test a spy is generated to monitor activity around _gaq.push(). This spy is reset for each test so that it has a fresh copy to work with.

Lines 11-50: The tests. JavaScript is used to replicate the interaction we wish to track and assertions are executed to determine the activity around the push method of _gaq. A check is also made to confirm it was called with the correct arguments.

The Custom Reporter

Now that tests have been created a method of communicating the results back to PhantomJS need to be added. With Mocha this is done through reporters, which listen to activities that occur during the testing process and provide feedback to the tester.

Reporters involve a lot of framework specific code that isn't completely relevent to this article so I've included this reporter in the following Github Gist. For those of you interested in learning more about Mocha, reporters are a really cool part of the framework and I would encourage you to dig deeper into them.

Building the PhantomJS test runner

If you don't have PhantomJS installed there's an excellent guide on their website. You'll want to make sure that you have the latest version (1.7 at the publication of this article) as more recent versions of PhantomJS have a new method of communicating between the page and PhantomJS that we use in our reporter.

Personally, I installed it using Homebrew:

brew install phantomjs

The following code will be used to run our tests:

./test-runner.js:

var page = require("webpage").create();

page.open("http://www.viget.com", function(status) {

    if (status !== 'success') {
        console.error("Failed to open", page.frameUrl);
        phantom.exit();
    }

    page.injectJs("lib/mocha.js");
    page.injectJs("lib/sinon.js");

    page.injectJs("src/reporter.js");
    page.injectJs("src/rules.js");

    page.injectJs("tests/home-test.js");

    page.evaluate(function() {

        // Undefine GA
        window._gat = undefined;
        window._gaq = [['_setAccount', 'UA-00000000-1']];

        // Run tests
        window.mocha.run();
    });

});

page.onCallback = function(data) {
    data.message && console.log(data.message);
    data.exit && phantom.exit();
};

This is a lot to take in all at once. Thinking in terms of regular browser use, here's what happens:

Line 1: Create a new tab.

Line 3: Navigate to http://www.viget.com

Lines 5-8: Could we successfully connect? If not, notify the user and exit PhantomJS

Lines 10-16: Add the JavaScript that runs our tests to the page

Lines 18-26: page.evaluate lets us run JavaScript on the current page. Here we unset the current Google Analytics setup so that we don't accidentally push our results. Once this has all been configured, we run our tests on line 26.

Lines 30-33: PhantomJS 1.6 added the ability to send messages from the client back to PhantomJS. This activity can be seen within the log function of the custom test reporter. Here we define a function to handle the event that is fired when this method is envoked.

Give it a spin

With all of this complete testing can begin. Use the following command to test the rules we've created:

phantomjs test-runner.js

You should see the following output:

Take aways from PhantomJS testing

I found that testing Google Analytics in this way is significantly faster and provides continuous confirmation that events are behaving as expected. Given the sheer volume of pages you are able to test, refactoring can be done safely. Reproducing and testing fringe issues also becomes much easier.

The code for this article was simplified for brevity, however JavaScript remains at its core. Modification should be straightforward and I encourage you to use this code in other forms of front-end testing. These methodologies provide exciting opportunities for advancing the quality of tests for JavaScript applications and demonstrate the leaps the community has made to advance JavaScript as a language in the past few years.

Viget Intern Prank: How to Improve any Website

Eli Fatsi — Mon, 01 Oct 2012 14:08 GMT

During my internship, I was given a summer checklist in the form of a so-called “Vigtories card” — the idea was to accomplish as many non-work-related activities (some quirkier than others) as possible. You can see the interns’ final Vigtories on the intern tumblr blog.

Throughout the summer, I made some decent progress (bike to work, go hiking with a fellow Viget, learn something new about Brian or Andy), but one particular task remained unchecked for the majority of the summer - “pull one harmless prank in coordination with your mentor”.

THE IDEA

While celebrating Intern Appreciation Day at a local brewpub with the Boulder developers, we decided to hijack Viget’s DNS and redirect the intern tumblr blog to something funny. Weeks went by however and as the internship neared its end, we’d yet to decide what would be something funny. With one day left however a formal plan was crafted: we would manipulate the intern blog itself.

INITIAL IMPLEMENTATION

Since the task at hand was fairly lightweight, I went with building up a simple Sinatra proxy app. The basic idea was to load the original source using Nokogiri and fiddle with all the images to our liking. We targeted author images first, but what to do with them? Mustachify.

The following snippet was the first implementation. It flipped through the HTML and replaced every image of class ‘author-photo’ with the mustachifyed version of itself, then returned the new HTML for eyes of the world to see.

get '/' do
  doc = Nokogiri::HTML(open("http://vignettesfromviget.tumblr.com/"))
  doc.css('.author-photo img').each do |image|
    image['src'] = "http://mustachify.me/?src=" + image['src']
  end

  halt 200, {'Content-Type' => 'text/html'}, doc.to_html
end

STEPPING THINGS UP

Things then got real as Pat jumped in with some MiniMagick and Digest::MD5 craftiness. This time the manipulation was targeted towards images in the bodies of every post. Since the new site was already riddled with mustaches, we decided to flip these images upside down. Using Nokogiri still to pluck all the posted images from the HTML, MiniMagick was used to make flipped copies of everything, and Digest::MD5 was used to save the new and improved copies to our own directory.

def flip_images
  document.css('.content img').each do |image_node|
    image_source = image_node['src']
    extension    = File.extname(image_source)
    output_name  = Digest::MD5.hexdigest(image_source) + extension

    image = MiniMagick::Image.open(image_source)
    image.flip
    image.write "./public/#{output_name}"

    image_node['src'] = output_name
  end
end

The last line in this method adjusts the HTML so pictures are loaded from our new directory and tada! - images were flipped! But not all of them…

ONE LAST ROADBLOCK

If you upload multiple images into one Tumblr post, an iframe is created in the post instead of a simple image. This difference was allowing these iframe photos to slip under the radar. To fix this, first we broadened the scope of our image flipping function from '.content img' to '.content img,.photoset_row img' allowing us to capture, flip, and save the iframe images with the rest. We then wrote a method to change the source of the iframe images away from the tumblr blog server and to our own directory.

def rewrite_iframe_source
  document.css('iframe.photoset').each do |iframe_node|
    iframe_node['src'] = iframe_node['src'].sub!("http://vignettesfromviget.tumblr.com", "")
  end
end

THE RESULT

A glorious manipulation of the intern tumblr blog. The last step in the prank was to hijack the Viget DNS and redirect anyone who visits vignettesfromviget.tumblr.com to the IP of the modified site. This requires some administrative access to the DNS and thus was a job for Pat. But, after all, this was a “prank in collaboration with my mentor”. This prank was purely local as we could only redirect people to the new IP address if they were on a Viget network. But, for your viewing pleasure, feel free to enjoy the fruits of our labor - before and after

BONUS - CAGEME

You’ll notice a few posts do not have a mustachifyed young adult picture as the author, but instead a mustachifyed picture of Nicolas Cage as someone else. This indicates a post by Jack: one of the other Rails interns here this last summer, and the creator of one of the greatest websites around - http://cageme.herokuapp.com

Viget.com Blogs

The Neo Fallacy: How Our Mental Model of Developers Leads to Bad Bug Reports

Simplify your build-out process using Layouts with Views

Code Examples

Download

In Conclusion

JavaScript Execution Patterns for Non-Web Apps

Everything in document.ready

Dom Based Routing

Garber-Irish Implementation

Feature-Based Execution

Feature-Based Execution + Script Loader

jQuery Meetup

What Did We Learn?

Why I Chose to Learn C

Preventing Interface-Induced Backbone.sync Firestorms

Only you can prevent forest-fires, by debouncing

Beautiful and Simple

Arduino, Redis, Dropcam + Holiday Cheer = Jinglebots!

Step 1 - Instant Communication

Step 2 - Receiving Messages/Tweets

Step 3 - The Fun Part

Step 3.5 - The Really Fun Part

Step 4 - The Web Site

Step 5 - The Jingle Report

Step 6 - The Auto Pilot

Summary

Using Time Zones with Rails

Method 1 - Quick, simple, 90% right

Method 2 - Quick, a little less simple, 100% right

Confirming

Rails Engine Testing with RSpec, Capybara, and FactoryGirl

Rescuing exceptions without notifications or: How to fail without knowing

1. Rescuing a very generic exception class

2. Making a connection to an external service

3. Mystery code

Simple App Stats with StatBoard

Refactoring Patterns: The Rails Middleware Response Handler

Server Maintenance Mode for Rails, Capistrano and Apache2

Client-Side Separation of Concerns: Are We Doing It Wrong?

The Classics: CPB

Are they the right ones?

New concerns

Layout

Unseparated:

Separated:

Page

Unseparated:

Separated:

Component

Unseparated:

Separated:

Content

Unseparated:

Separated:

Window

Unseparated:

Separated:

Tracking

Unseparated:

Separated:

Closing

Further reading:

Testing Google Analytics with PhantomJS

What is a headless browser?

SinonJS

Mocha

Digging in to the code

Building the sample rules

Creating the tests

The Custom Reporter

Building the PhantomJS test runner

Give it a spin

Take aways from PhantomJS testing

Viget Intern Prank: How to Improve any Website

THE IDEA

INITIAL IMPLEMENTATION

STEPPING THINGS UP

ONE LAST ROADBLOCK

THE RESULT