The Napkin ~ A Blog By Highgroove Studios

The Rapid Feedback Loop

Thu, 25 Feb 2010 22:25:00 EST

One of the best features of Ruby on Rails development is the “Rapid Feedback Loop” that allows developers to quickly make a change and see right away how that new feature, enhancement, (or bug fix) behaves and looks in real-time.

This Rapid Feedback Loop actually becomes even cooler when you loop in the client—the product owner requesting a new feature or enhancement. At Highgroove, we do outsourced development and feature additions for many different projects. We’re always looking for ways to enhance that feedback loop with our clients. Here are few tools for quickly enabling an even more rapid feedback loop, that we can’t live without….

Skype – Skype’s Screenshare capability is fantastic, it works anywhere, between Macs and PCs, and is so quick to setup. We use it every day to say “is this what you mean?”

Skitch – Plastiq’s Skitch is a screenshot tool that can go from screenshot to a private, shareable link you can send to anyone in about 2 seconds. It’s the bee’s knees for sending over a fix or showing off a new feature. Here’s a sample that we sent a client just recently pointing out a few quick changes:

Capistrano and the Capistrano Extensions – Capistrano enables us to setup production and staging servers. We can push changes to a staging server in one command, so everyone can actually kick the tires on a new feature before pushing out to production in another one line command, with no downtime.

The Extra Desk in our Office – one of our favorite clients reminded me that they love the fact that can come sit with us in our office. When we say “is this what you mean,” we can simply point to our screen.

Summary

The quicker we can enable our clients to see and feel changes, the quicker they can realize real value from a rapidly evolving project. We don’t run into the problems traditional software companies face where developers can go off into a room and code for days before presenting their black box back to the stakeholders. Even with all the specifications in the world, nothing beats kicking the tires and rapidly iterating on a feature until it’s perfect.

Iteration Based Contracts

Tue, 23 Feb 2010 20:42:00 EST

On the rails-business list, someone asked recently about Iteration Based Contracts.

“Iterations” in the software development world are a simple, yet powerful concept for developers and businesses. An Iteration is a pre-defined timeframe, usually 1 or 2 weeks, a pre-defined cost, and a general overview of the functionality and goals of an iteration (see also: Iterative and Incremental Development).

Some examples of Iterative development might have one goal:

Develop a simple back-end server for an iPhone application that allows user registration, signup, and stores user profile information.

Or may include a package of goals, usually building on a previous iteration:

Add a simple Software-as-a-Service payment processing system to user registration.

Most businesses looking for development really don’t care about the hourly rates when it really boils down to work (i.e., $20/hr for 5 hours of development for one developer or team == $100/hr for 1 hour of development for another developer or team). They really just want their product or service to work, and they would really like to know how much it will cost and how long it will take.

There is an old adage (backed up by some empirical evidence and real-world proof) that says:

“you can build software 1) on time, 2) on budget, and 3) on scope, but you can only pick two….”

This is not to say that no developer team could never do all three, but it’s actually a lot more manageable (and fun) for businesses and developers to fix “time” and “budget”, and alter the scope in Iteration based development.

You might be thinking that we’re still back in Square 1—i.e. aren’t those pesky developers just packaging up a set of hours and time—how will I know what will get developed in that time and for that cost? But, something very cool happens when you really embrace those constraints.

With an iteration, both parties are agreeing to a fixed amount of money and time, and the developers (and product owners) will develop as much functionality and business value within the Iteration. If development on an Iteration begins, and then the business owners suddenly have a fantastic new idea (which usually happens right after the first demo, once they have working software in their hands), they can simply ask the developers to make it happen. Similarly, if the developers think of a great way to simplify a process, leverage an existing plugin or library, or add (or remove) an additional feature, they can propose it without recourse. Both parties know how it affects the Iteration, and thus, the bottom line. There is no concept of the “change-order” and the rapid-feedback-loop created from iterating makes sure that what is being developed by the developers is exactly what the business owners want.

At Highgroove, we have a Master Services Agreement for Iteration Based Contracts, and without posting all the nitty gritty details, the meat of it revolves around the compensation and deliverables:

Compensation.  Highgroove shall be compensated for Services at the rate of: 
  Development per Iteration (Ruby/Rails/JavaScript/HTML/CSS): $xxx

Deliverables are always outlined as attachments to the agreement, and are usually story-based (see BDD).

Iteration Based Contracts require quite a few more mechanisms to ensure that developers are not just packaging up X number of hours—there has to be agreed upon functionality and time-frames surrounding those deliverables within the Iteration(s).

We’ve found that our best clients are really seeking a proven process, and a team that can execute on that process: being transparent and delivering real business value at every step. Iteration Based Contracts are a great way to structure this relationship.

Making the Case for Redis

Thu, 18 Feb 2010 14:55:00 EST

Most database-backed applications, especially Ruby applications written in Rails do fine with a SQL Database, like MySQL. Adam Wiggins of Heroku does a great job explaining how SQL Databases are an Overapplied Solution. There are definitely a few cases we’ve seen where a NoSQL solution like Redis can really shine.

At Highgroove, we’ve got several projects utilizing the Redis key-value store. Here are a few reasons you might want to look into Redis:

1) ”Data is frequently written, infrequently read” – if you are making tons of writes and MySQL can’t keep up, Redis has been clocked at 110000 SETs (the INSERT equivalent) per second!

2) ”Data can be expired” – if you have data that can be expired on a regular basis, over time, or explicitly, like stats, logs, and session data, Redis can explicitly expire whole keys or quickly trim large lists of data in no time.

3) ”Data is a collection of COUNTs, or SUMs of other data.” – if you have ever written an UPDATE statement that adds +1 to a record, you probably know that it is quite expensive, and could possibly be wrong. Redis has built in support for incrementing and decrementing values, quickly.

We love SQL Databases, so you won’t see us abandoning them any time soon, but we enjoy adding tools like Redis to our toolbelt.

More Information on Redis:

Redis Project Page – http://code.google.com/p/redis/

Redis (wikipedia entry) – http://en.wikipedia.org/wiki/Redis_)

A Collection of Redis Use Cases – http://www.paperplanes.de/2010/2/16/a_collection_of_redis_use_cases.html

Production Server Sysadmin Essentials from the Scout Team

Tue, 2 Feb 2010 08:40:00 EST

Deploying Rails applications has definitely become easier with the use of tools like Capistrano and Phusion Passenger (a.k.a. mod_rails/mod_rack), but really keeping them serviceable, maintainable, and always humming along can require a bit of work.

Andre over at Scout has written a fantastic guide—a checklist, really for putting a Rails or Sinatra application in production and keeping it up in tip-top shape.

Read the Production Server Sysadmin Essentials or, as Andre likes to call it: “Sysadmin Eye for the Dev Guy”.

Open-Source in Action

Sun, 24 Jan 2010 17:23:00 EST

A new client of ours had a big problem. The site they built was getting too many searches (a very good problem to have). The searches all used Andre’s geokit gem and the geokit-rails plugin to provide local results.

Even with the library’s multi-geocoder support (Google, with failover to Yahoo), the site was hitting the limits imposed by both services every day!

So, we quickly implemented a query caching mechanism that caches geocoding lookups that don’t change very regularly, saving the site from making all those API calls. But, the cool part is, we actually added this functionality to the open-source library itself, and the client’s application now directly benefits.

Several things will now happen, because we contributed back to the open-source library, instead of keeping this “addition” to ourself (and the client):

other developers facing this same problem can now leverage our code when they use geokit
other developers can now submit even more functionality, fixes or enhancements to our code, and even better support for problems we may have one day, meaning that we will eventually benefit from this too
our client now knows that the code they relied on us to develop now has even more developers eyeing it, making sure it works
our client (through us) is contributing to open-source, and can feel good about using open-source technologies, having fulfilled their part of the agreement they implicitly made, by leveraging the gains (and price) of implementing a solution based on open-source software

Truly leveraging open-source technology, when done right, can be a huge win for everyone: the client, vendor, and community.

Atlanta Startup Weekend Nov 13-15

Fri, 23 Oct 2009 09:57:00 EST

Alanta’s (3rd?) Startup Weekend is on Nov 13 – Nov 15 at ATDC.

We’re big fans of Startup Weekend for a couple of reasons:

the constraints—successful businesses are built on limited resources. A single weekend to come up with ideas, plan, execute, and launch a business is a daunting and fantastic lesson for anyone interested in starting a business.
the people—what happens when you get 15 crazy developers, 20 biz-dev folk, sprinkle in some opinionated graphic designers and some marketing know-it-alls? Sheer madness, of course! I don’t think you could build a better simulation for working with a diverse team towards a common goal.
the camaraderie—the business(es) coming out of Startup Weekend may not ever become the next Google (or Skribit), but the ideas and lessons learned by all are sure to make the tide rise here in Atlanta!

A very big thank you to our friends at The Kauffman Foundation, ATDC (and all the other sponsors) for their support of this Atlanta event.

Part II: We Just Undid Three Months of Dev work. Here's What We Learned.

Tue, 20 Oct 2009 19:38:00 EST

We published Part 2 of our lessons learned from undoing 3 months worth of development work on Scout, our server monitoring service.

As developers, we love technically beautiful solutions. But sometimes the best features are the ones that get the job done—and prove themselves by providing real business value.

We hope you enjoy us sharing these lessons, if so: up-vote us on Hacker News and be sure to subscribe to the Scout RSS Feed or Follow Us on Twitter.

We Just Undid Three Months of Dev work. Here's What We Learned.

Wed, 7 Oct 2009 13:22:00 EST

Scout, our server monitoring service, has grown quite a bit in 2009.

We’ve documented 2 big lessons we’ve learned on the Scout Blog.

If you find it helpful, give us a vote on Hacker News.

Teaching Ruby on Rails

Sat, 22 Aug 2009 17:27:00 EST

At last month’s Atlanta Ruby User Group (Meetp), I gave a presentation on “Teaching Ruby on Rails.”

I’ve taught Ruby on Rails for the Big Nerd Ranch for almost 4 years now, and given on-site trainings all over the world, from Wells Fargo to the New York Times, to startups and even government agencies. I’ve also done this talk before at the first Acts As Conference (Ruby on Rails local conference in Orlando, Florida), but I have refined it a bit based on more experience teaching Rails at various organizations.

Being an effective teacher, and thus, Teaching Rails (and for that matter, any real technical programming language and framework) boils down to 4 main points:

Define Your Purpose
Know Your Audience
Give Relevant Examples
Teach How to Learn

We’re hoping to get video of the presentation up soon, but if you just want access to the slides with notes, I’ve provided them in the footnotes.

Lessons from the Trenches at Acts As Conference Highgroove Article on Acts As Conference Talk
Teaching the Rails Slides and Notes from Atlanta Ruby User Group Talk

Apache Page Caching and Multiview

Tue, 4 Aug 2009 17:24:00 EST

I just wanted to post a quick note to anyone searching for problems with Rails Page Caching with Apache.

We just helped a client through a tricky issue that manifested itself through some strangely generated cached files.

The Problem

A visit to the resource:


http://example.com/articles

Would correctly generate the page:


/path/to/app/public/articles.html

Subsequent visits would load the cached page.

However, if you then tried to visit a page like:


http://example.com/articles/2

Where a route used that for pagination, or simply any action off that resource, you apache would try and request a file like:


/path/to/app/public/articles.html/2.html

This is problematic, and does not work.

If, however, you removed the cached file and instead created the directory first, like so:


/path/to/app/public/articles/

When you requested the resource:


http://example.com/articles

it would correctly generate an index file:


/path/to/app/public/articles/index.html

And then correctly generate any cached pages inside the directory like 2.html, 3.html for pagination (if setup in rails routes correctly).

The Solution

The problem was that Apache has the MultiViews option turned on.

We changed the server configuration file from this:


<Location />
Options All FollowSymLinks MultiViews -Indexes
</Location>

to this:


<Location />
Options All FollowSymLinks -Indexes
</Location>

When you remove this option from the configuration, you get the correct caching behavior, which is that requests to:


http://example.com/articles

Still generate the cached file:


/path/to/app/public/articles/index.html

But now, requests to the slash-ending url and resources off that route:


http://example.com/articles/
http://example.com/articles/2

Now correctly generate a folder with an index.html and 2.html, which caches just fine.

I hope this helps someone else out!