Engine Yard vs.

Managing Amazon EC2 instances directly: we’ve built our own server images for Amazon EC2 in the past and managed them, so we know first hand the time and effort involved with a DIY-approach. Engine Yard has a mature, sophisticated stack that runs on top of Amazon EC2. Coupled with an easy-to-use Web-based dashboard, Engine Yard Cloud makes it super easy for development-centric organizations like ours to deploy and manage client systems without having to deal with the complexity of EC2. In addition, data is persisted to Amazon EBS, so we don’t have to worry about setting up our own internal processes or writing custom scripts for handling backups and persistence.

Other hosting service providers: if you are running your own virtualized servers on behalf of clients or for your own needs, there are many things you have to deal with. Configuring the server, compiling custom components, setting up monitoring, configuring maintenance processes and tasks (e.g. log rotation), and especially security. Over time, we’ve been moving away from handling these details ourselves so that we can focus on the things that we do best and keep our operations lean. We prefer a sophisticated cloud management solution instead. On top of that, it’s refreshing to receive security advisories from Engine Yard about patches to Nginx or similar — these patches are deployed through a single-click in our Engine Yard Cloud dashboard.

Other cloud management solutions: the problem with these other options is they don’t have the resources on staff who understand Ruby-based architectures the way EY’s engineers do. Engine Yard has experienced, world-reknown Ruby engineers behind their stack like Ezra. Other companies claim they support Ruby but when complexities arise, they simply can’t deliver the level of support the way EY does. All it requires is a simple comparison of the content in the knowledge base and customer support forum managed by Engine Yard with their competitors. It’s clear that Engine Yard has Ruby developers at the top of its priority list.

Over the last few months, we’ve been migrating existing clients as well as some of our own apps to Engine Yard. We appreciate their awesome service and support. This is why we’re proud to be one of their partners.

Naturally, uploading photos in a Ruby on Rails application usually involves a robust gem/plugin such as Thoughtbot’s Paperclip. We found that it’s very easy to configure it to seamlessly upload to CloudFront.

Here’s a sample polymorphic photo model that uses Paperclip:

class Photo < ActiveRecord::Base
  has_attached_file :data,
    :styles         => { :lrg => "300x300&gt;", :thumb => "100x100>", :micro => "50x50>" },
    :storage        => :s3,
    :s3_credentials => YAML.load_file("#{RAILS_ROOT}/config/amazon_s3.yml"),
    :s3_host_alias  => "djoo9p6eu3n0g.cloudfront.net",
    :bucket         => "listing-photos",
    :url            => ":s3_alias_url",
    :path           => ":class/:id/:style/:filename"
 
  belongs_to :attachable, :polymorphic => true
end

The relevant configuration keys include :bucket, :storage, :s3_credentials, :s3_host_alias, and :url. The best thing is that your helper methods, e.g. photo.data.url(:lrg), continue to work the same way. Paperclip abstracts the file storage mechanism so that your views don’t need to have this responsibility.

Getting Started

1. Signup with CloudFront at http://aws.amazon.com/cloudfront/
2. Once you’re in there, you can use the AWS Management Console to define and deploy your CloudFront distribution bucket.

3. Setup Paperclip to use CloudFront as described above.

You’re done!

map.with_options :controller => "people" do |people|
    people.connect "/people",     :action => "index"
    people.connect "/people/:id", :action => "show"
end

But realize that with_options extends the Ruby Object class (source code) so it can be used practically in any context.

I’d like to keep my configuration file leaner when specifying gem dependencies.

config.with_options(:source => 'http://gemcutter.org') do |c|
  c.gem 'acts_as_fu'
  c.gem 'factory_girl' 
  c.gem 'mocha' 
  c.gem 'redgreen'
  c.gem 'shoulda'
  c.gem 'test_benchmark'
end
 
# not yet in gemcutter 
config.gem 'stubble'

However, the gotcha is the Rails::Configuration class is loaded before ActiveSupport. So if you’d like to use with_options in your config, be sure you load the module environment.rb before the Rails::Initializer block is hit.

require 'active_support'

The above is moot with the new gem bundling features in Rails 3, read more.

The key goal of your functional tests should be isolating what controllers are expected to do. Don’t let pseudo unit tests leak into this realm. Don’t waste your time writing controller tests in which a post containing specific attributes fails because of a validation rule on the instance. Instead focus on what the controller should do if the post resulted in an invalid object state.

Stubble makes things super easy by automatically stubbing out the key Active Record class and instance methods. This means less code in your tests.

Here’s an example that mixes Stubble and Shoulda:

Notice I did not need to set any attributes as part of the post. Again, Stubble automatically stubs the most common AR methods so the attributes would be ignored in the first place. Functionally, all I care about is what happens when a post results in a valid or invalid condition. Either I’ll be redirected (to the photos page) or back to the form itself (the :new action). I’ve completely decoupled the implementation details of my model from my controller test. So when my model changes in the near future, I most likely won’t have to update my functional tests.

Check out the Stubble GitHub project page for more examples.

After installing the Stubble gem, you’ll need to update test_helper.rb to leverage it:

I could definitely see myself re-using this layout in other places (and I can extract it into a Rails plugin if I start needing it across multiple apps). So let’s think about how to turn this into a Rails view helper (I’ll assume we’re using ERb).

What I’m aiming for is the following markup:

<span class="category">
  <p>Great Deals</p>
  <p>Breads &amp; Grains</p>
  <p>Dried Beans &amp; Legumes</p>
  <p>Dried Fruits &amp; Nuts</p>
  <p>Eggs &amp; Dairy</p>
</span>
<span class="category">
  <p>Fish &amp; Seafood</p>
  <p>Flowers</p>
  <p>Fruits</p>
  <p>Herbs &amp; Spices</p>
  <p>Juices &amp; Drinks</p>
</span>
<span class="category">
  <p>Meat &amp; Poultry</p>
  <p>Oil &amp; Vinegar</p>
  <p>Plants &amp; Seeds</p>
  <p>Specialty Items</p>
  <p>Vegetables</p>
</span>

The above would likely be generated by iterating through an instance variable representing an array of products from the database. A rigid implementation in Ruby could take this form:

<span class="category">
<% @products[0..4].each do |p| -%>
  <%= p.name %>
<% end -%> 
</span>
<span class="category">
<% @products[5..9].each do |p| -%>
  <%= p.name %>
<% end -%> 
</span>
<span class="category">
<% @products[10..14].each do |p| -%>
  <%= p.name %>
<% end -%> 
</span>

And you could ditch plain old HTML completely and adopt this form:

<% content_tag :span, :class => 'category' do -%>
  <% @products[0..4].each do |p| -%>
    <%= p.name %>
  <% end -%>  
<% end -%>
<% content_tag :span, :class => 'category' do -%>
  <% @products[5..9].each do |p| -%>
    <%= p.name %>
  <% end -%>  
<% end -%>
<% content_tag :span, :class => 'category' do -%>
  <% @products[10..14].each do |p| -%>
    <%= p.name %>
  <% end -%>  
<% end -%>

No matter what, the above is just ugly. We’ve got duplication of code and assumptions about the size of the collection that translate into the need for ongoing code maintenance.

So to consolidate the above into less code, we have to think of a clever way to wrap the markup into a helper method. Ideally, I’d love to be able to do something as simple as:

<% display_as_columns(@products, {:cols => 3, :tag => 'span', :class => 'category'}) do |v| %>
  <p><%= p.name %></p>
<% end %>

…or better yet:

<% display_as_columns(@products, :class => 'category') do |v| %>
  <p><%= p.name %></p>
<% end %>

You may be thinking, how do I handle a block (of markup) like that within a helper method? Easy - you can define a method signature as follows:

def display_as_columns(object_array, options = {}, &markup)

…where the ampersand in &markup denotes this is a block. But the beautiful thing about Ruby is that methods support an optional, implicit block as the last parameter. So unless you need to do something special with the block after it’s passed into the method, you don’t need the last bit. Instead you have:

def display_as_columns(object_array, options = {})

Our helper method implementation looks like this:

def display_as_columns(object_array, options = {})
  cols = options[:cols] || 3
  tag = options[:tag] || 'span'
  css_class = options[:class] || 'column'
 
  rows = (object_array.size.to_f/cols.to_f).ceil
  cols.times do |i|
    concat("<#{tag} class='#{css_class}'>")
      object_array[(rows*i)..(rows*i+(rows-1))].collect { |item| yield(item) }
    concat("</#{tag}>")
  end
end

The first 3 lines initialize a series of parameter defaults, if not set already. So if you want your columns to be wrapped into div’s, you’d pass in the :tag key in the options hash. Otherwise leaving it out will default to a span tag. Same thing goes for the number of columns. I arbitrarily default to 3 columns but you may want this to be an explicit parameter, unless a 3 column layout is standard across your views.

rows = (object_array.size.to_f/cols.to_f).ceil

The above line is simply calculating the number of rows but applying the Ruby “ceiling” numeric transformation to ensure nothing gets cut out in case your numbers don’t divide equally.

cols.times do |i|
  concat("<#{tag} class='#{css_class}'>")
    object_array[(rows*i)..(rows*i+(rows-1))].collect { |item| yield(item) }
  concat("</#{tag}>")
end

Finally, the last section is where the magic occurs. The concat method, part of Rails’ ActionView Helpers, adds your string to the output buffer. The collect method is part of the Ruby enumerable mixin. The interesting part is the yield call. It is passing the current ‘item’ instance to the markup we defined as part of the original block passed into the helper:

<p><%= p.name %></p>

Enjoy.

We have a client whose application was subject to false positives when emails were sent to other users with Gmail or Google Apps-hosted accounts. The client is using the Google Apps email service for both sending and receiving emails. Per spec, they have a valid SPF record that includes an “MX” definition for Google’s mail servers, authorizing Google to send emails on their behalf. However, we noticed that emails sent from the domain to other Google emails were “soft failing” — in other words, Google was failing emails originating from their own servers.

The fix is easy — be sure your SPF record has “include:_spf.google.com”; example:

v=spf1 a include:_spf.google.com ~all

Hope that helps!

Telephony applications can be developed easier than ever before with Twilio, a cloud-based API for voice services. During the first half of this year, we’ve been playing, building, and now wrapping in the form of a Ruby gem we’ve finally polished off. I’d like to take you through an example to show off their amazing service offering.

Twilio: How it Works

Twilio’s REST API enables your application to make calls, playback audio, transcribe calls, and much more. With most operations such as outgoing and incoming calls, Twilio communicates with your application by pinging a URL that you designate. Your application then needs to respond according to the Twilio Markup XML (TwiML) specification, which is comprised of Twilio “verbs” such as dial, gather, play, record, and say. These verbs instruct Twilio what to do during the call.

Example Use Case: Phone-Based Identity Verification

In two recent customer Web applications, we were required to add functionality to be able to verify the ownership of business phone numbers. The idea was simple: during the user registration process, the system phones the listed business number, user answers phone and hears a 4-digit PIN number, and then is required to enter this code in the Web form to continue with registration. With Twilio’s service and our Twilio Gem, this is ridiculously easy, especially within a Ruby on Rails application.

Inside your Rails user registration controller, you’ll need to generate the 4-digit random code and stick it inside a session object:

@random_code = rand.to_s[2..5]
session[:random_code] = @random_code

Now, set your Twilio credentials by calling the connect method. Because these get cached in the Twilio module, you don’t have to deal with connection instances.

Twilio.connect(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN)

Finally, tell your app to call Twilio’s service. We’ve wrapped all of the Twilio REST resources into Ruby classes, each with their own methods that wrap the entire interface. In this example, we’re set the “from” number to a constant representing the phone number associated with our Twilio account. However, Twilio allows you to use other registered Twilio numbers, thus this value can be dynamic. The last parameter is where it gets interesting and represents the callback URL that Twilio should contact to communicate with your application.

Twilio::Call.make(CALLER_ID, user.phone, 
  verify_phone_url(:pin => @random_code))

The above command instructs Twilio to phone the user’s number, showing your registered number as the caller id. As soon as Twilio connects to the number, it will also ping your URL.

Now we need to handle the response. In our example, we need to “say” the PIN code to the person on the phone. Before we created our gem, we would use XML Builder to render a response inside a .builder template. But the gem simplifies things greatly. Inside another controller action, just add:

random_code = params[:pin].gsub(/\d/) {|s| s + ' '}
render :xml => Twilio::Verb.say("Your pin code is #{random_code}", 
  :loop => 3, 
  :pause => true)

Remember that the callback URL we initially sent to Twilio included a ‘pin’ parameter, so Twilio sends this back to us. The reason we add spaces between the digits is that Twilio speech module will speak “1234″ as “one thousand two hundred thirty four” while “1 2 3 4″ will be spoken as “one two three four.” The loop and pause options result in a better experience for the person on the phone. Other options include the ability to set the voice to female, as well as use other languages.

The last thing remaining is to handle the user’s response. At this point, they hear the PIN code over the phone and enter them into the Web registration form. All that you need to do is confirm that their input matches the random_code session value.

Testing Strategies

With functional testing, you can resort to mock interfaces to guide you along. If you want to emulate the XML response handlers in your controllers, then take a look at the FakeWeb gem, which makes things very easy. In fact, most of Twilio gem’s tests leverage it.

But at some point, you’ll want to play around with real network connectivity. What I recommend is creating an SSH tunnel between a server you may be operating and your own development machine. This is painless if you manage with a rake task. First, create tunnel.yml and place in your Rails application’s config folder.

development:
  public_host_username: sugar
  public_host: staging.webficient.com
  public_port: 8868
  local_port: 3000

Now, save this rake task as lib/tasks/tunnel.rake:

namespace :tunnel do
 
  desc "Start SSH tunnel to remote host"
  task :start => :environment do
    SSH_TUNNEL = YAML.load_file("#{RAILS_ROOT}/config/tunnel.yml")[RAILS_ENV]
 
    public_host_username = SSH_TUNNEL['public_host_username']
    public_host          = SSH_TUNNEL['public_host']
    public_port          = SSH_TUNNEL['public_port']
    local_port           = SSH_TUNNEL['local_port']
 
    puts "Starting tunnel #{public_host}:#{public_port} to 0.0.0.0:#{local_port}"
    system "ssh -nNT -g -R *:#{public_port}:0.0.0.0:#{local_port} #{public_host_username}@#{public_host}"
  end
 
end

Then, it’s as easy as typing rake tunnel:start in your command line. If it doesn’t work, be sure your server has the following values set in the sshd_config file:

AllowTcpForwarding yes
GatewayPorts clientspecified

Once the tunnel is running, point your browser to the public URL, and you’ll be actually browsing your local machine. This setup is very handy when debugging Twilio interactions.

Endless Possibilities

Kyle, one of our lead engineers, created an app that calls you when a server is down. In another project, we implemented click-to-call functionality, in which a user can enter their phone number and the application will connect them to the business they are calling. You may have seen this in action on sites like YellowPages.com — same idea and wonderfully easy with Twilio. In the near future, I’ll share a few more technical examples and other capabilities of the Twilio gem.

FOLSOM and SAN FRANCISCO, CALIFORNIA – June 4, 2009 –Webficient, a leading web application development firm, and New Relic, the leading provider of software as a service application performance management solutions, today announced the integration of New Relic’s RPM™ Rails application performance management software with an Amazon Machine Image (AMI) created by Webficient. The AMI created by Webficient is optimized for deployment of Ruby on Rails applications running in the Amazon Elastic Computing Cloud (EC2.)

“We have been impressed with New Relic’s RPM for monitoring and troubleshooting applications for our clients. They provide the ideal tool for managing applications in the cloud,” said Phil Misiowiec, Founder of Webficient. “We concluded others using our AMI would find RPM as valuable as we do. So we included the RPM agent plugin in the AMI and enabled automatic account creation. This will make our AMI even better.”

“When we looked at AMI’s for Rails, we found that the Webficient AMI was rated 5 Stars (highest) by EC2 customers who had used it,” said Bill Lapcevic, vice president of business development at New Relic. “So we are very pleased that Webficient has included RPM in the AMI to provide instant application management for their users.”

Availability
The Webficient AMI is available immediately. AMI users who already have an RPM account can enter their license key when prompted. New customers can create a free RPM Lite account.

About Webficient
Webficient enables businesses to lower the costs of developing and maintaining their technology platform through “on demand” programming services and by utilizing agile practices and open source technologies. They build, maintain, and improve Web sites and applications for customers such as ad agencies, sole proprietors, and start-ups. They are located in Folsom, California and can be found on the web at www.webficient.com.

About New Relic
New Relic, Inc. offers RPM™, a Rails Performance Management solution that enables developers to quickly and cost effectively detect, diagnose, and fix application performance problems in real time. More than 1,800 top Rails organizations use New Relic RPM to monitor and optimize 18,000 production application instances, including customers 37signals, AboutUs.org, iOffer, Getty Images, GitHub, LightHouse, Liz Claiborne, Intuit, Starwood Hotels, Butterball Turkey, Ourstage, and Shopify; and partners Engine Yard, Pivotal Labs, Blue Box, RightScale and Sun. Funded by Benchmark Capital and Trinity Ventures, New Relic is a private company headquartered in San Francisco, California, USA. To learn more about New Relic, visit www.newrelic.com.

Press Contacts:

Phil Misiowiec
Webficient
phil@webficient.com

Stephanie Breslin for New Relic
650-248-9590
stephanie@newrelic.com

Just add this mapping in routes.rb:

map.audio '/audio/:item', :controller => nil

And now you get Rails’ automatic path helpers:

audio_path('greeting.mp3') # "/audio/greeting.mp3"
audio_url('greeting.mp3') # "http://www.example.com/audio/greeting.mp3"

You probably learned a lot of what you know about Javascript and how to write it through various examples on the blogs of your favorite Rails gurus, or maybe by copying examples from prototypejs.org.

The problem with a lot of Javascript examples and snippets on the Internet are that they work fine for “proof of concept” samples with a dozen objects, but they fall flat on their face performance-wise once the code is in a real application with hundreds or thousands of elements.  Frequently this is a result of the code never being run with a sizeable amount of data.  Also keep in mind that the code you’ll find on blogs was written in a, “look what I just did,” moment and usually does not include the, “oh crap, sorry folks!” feedback after being put in production.

The Prototype.js library is a great library that has opened a lot of doors, but it can be very easy to write slow Javascript with it.  The problem isn’t the library itself, rather its how easy it makes it to write complicated code (kind of like Rails and SQL queries!).  You may start off with an example you found that iterates over some DOM elements and passes them to a Class to get instantiated (having a “good” Object Oriented Design).  Then you later come back to the code and add a few more things inside the loop and to the class, and before you know what happened, IE users are calling support complaining about their browser freezing for 60s after a page load!

So here is a basic skeleton of something you might see:

// Create a nice self-contained class for our menu behaviors
var SlowMenu = Class.create({
  initialize: function(menu_el){
    this.menu = menu_el;
    this.menu.observe('click', function(){//click behavior});
  }
});
 
// Add the behaviors to all DOM elements
var flyouts = $$('li.flyout_menu').map(function(el){
  return new SlowMenu(el);
});

A project I worked on had a user directory with hundreds of elements on the page that the Javascript touched.  Just a simple decision such as creating a new object with attached behaviors for each element brought the browser to a crawl.  Speeding it up required leveraging Javascript’s functional nature and keeping the design going with the grain of Javascript’s out-of-the-box features.

It is pretty common to say, “Ok, I need a flyout menu… I’m going to make a class that contains the menu behaviors, then grab all the links on the page and make new objects with them.”  This results in Javascript fail.  In the example above, if you decide to show 100 menus on the page, and each flyout menu has, say, a dozen links in it that all need behaviors, you’re up to thousands of objects for simple functionality!

A seemingly better way is to have one loop that just attaches one behavior to each element.  That way you’re at least not instantiating new classes inside the loop, right?

$$('li.flyout_menu').each(function(el){
  el.observe('click', function(){//click behavior})
});

Actually, the best way is to not even mess with the elements themselves!  Instead just let any events you’re interested in bubble up the DOM and only make one top-level function that knows what to do by inspecting the target of an event.

$('menu_container').observe('click', function(event){
  if (event.element().hasClassName('flyout_menu')) {
    // Menu behaviors go here.
  }
});

This sample code only observes click events on the parent element, and inspects the incoming elements on the fly.  Instead of performing all of the necessary computation when the Javascript is first executed by the browser, this will delay any computation until the last possible moment the user needs it.  This will make page loads feel snappy for the user, and save their CPU if they’re unfortunate enough to be using IE.  Your browser is already downloading and interpreting the script so you want to avoid doing any more work while this is being done.

These design choices don’t apply to just Prototype.js.  This style is just as effective when using jQuery.  Here is how the above snippet would look using jQuery:

$('#menu_container').click(function(event){
  if ($(event.target).hasClass('flyout_menu')) {
    // Menu behaviors here
  }
});

So when you’re faced with some slow Javascript, chance are that you’ll deny it.  I responded that way the first few times this happened to me.  I use Firefox for most of my development, and the latest Firefoxes (and Safaris, and now Chrome) have really fast Javascript interpreters.  You may not even notice a delay, whereas someone running IE will experience the same code taking a second or more to run.

While developing and benchmarking Javascript, I’ve found IE to generally be 10-100x slower than Firefox running the same code.  Make sure to open up VirtualBox and feel the pain of IE every once in awhile when developing.

Ultimately, the best way to speed up your Javascript is to write less Javascript! A majority of the time you don’t need classes and inheritance in your Javascript code like you are used to with languages that have more traditional object systems.  Keep in mind first of all that events bubble up the DOM (if you’re using a library), and consider turning your code’s design inside-out.  If you’re attaching the same behavior to every element, try instead to have one instance of the behavior that can be parameterized with an element.

The other thing to remember is that modern applications dealing with thousands and millions of objects are going to be running on IE6 (or IE7 if you’re sane) so try give those users a decent enough experience.  Though if you can afford to tell them to take a hike, have a beer for me!

Our guest blogger series recognizes talented developers who’ve been around the block or two and offer practical advice on today’s Web dev challenges. They are also available for hire as part of the Webficient project team. Contact Webficient Now