ELC Code Feed

Testing invocation of external executables, %x{}, `

David Palm — Thu, 09 Jul 2009 10:17:17 +0000

If you ever found yourself trying to write tests for code that invokes an external executable, such as:

  class Beef < ActiveRecord::Base
    def diskspace(flags = 'sh')
      `du -#{flags} .`.split.first
    end
  end

...and wondered how to write a spec that ensures the du command is called with the right options, you might have tried something like:

  before do
    @beef = Beef.new
  end

  it "has diskspace with humanized multipliers" do
    Kernel.should_receive(:`).with("du -h .")
    @beef.diskspace('h')
  end

That doesn't work.

After some headscratching and calls for help on the ruby-talk mailing list, I learned how the Kernel#` method is not being called directly. The Ruby object hierarchy has Object at the top-level, and thus all your objects eventually descends from Object. As Object mixes in Kernel, all your objects have all the methods defined in that module, so it's not Kernel that receives your call to "`", but it's the current 'self'. In the example above it's the @beef instance.

Consider:

  class Sheep
    def mytick(arg)
      puts "tickety-tick: #{arg.inspect}"
    end
    alias :"`" :mytick

    def tick_it!
      `cheese`
    end
  end

  Sheep.new.tick_it!

  $ ruby sheep.rb
  tickety-tick: "cheese"

So, to test the original code, here's the right way:

  it "has diskspace with humanized multipliers" do
    @beef.should_receive(:`).with("du -h .")
    @beef.diskspace('h')
  end

All of the above also applies to the "%x[]"-syntax (just an alias for Kernel#`).

Kudos to Brian Chandler for helping out on this, and happy hacking!

Solr vs. Sphinx - fuzzy search

John Eberly — Tue, 07 Jul 2009 01:08:22 +0000

First the results....

Fuzzy search accuracy for commonly misspelled drug names using sphinx and solr with different stemmers

A little background:

We are about to add full text searching to our rails project, and we thought we would take a look a few different solutions for search. We have been using solr on an older project and were fairly happy with the results, just a little disappointed in the indexing speed. Also, solr plugins for rails seem to be a little fragmented at this point. acts_as_solr has no real home other than github where you have to play "pick a fork". There are a few other libraries for solr (eg. Sunspot) in rails but none of them seem to be mature or very active.

We decided to try sphinx with the thinking_sphinx plugin. Our first impressions seemed to be very good. Extremely fast indexing, nice support for indexing associated models, and all around very easy to work with as a developer. But....

Fuzzy Search:

We are mostly searching non-English words (eg. last names). While using sphinx for a little while, we began to "feel" the search results were not as good as they had been in solr. With search results and open source search tools, your results basically boil down to what stemmer (algorithm) the tools are using to find the relevant results.

In order to get a quantifiable "score" for search results, I decided to write a little test script that would test the different configurations against a known set of drug names and there common misspellings found here. The script indexed the properly spelled drug names and then searched using the misspelled name. If the result was found in the top 10 results it was considered a match.

Above are the results using different stemmers (sphinx calls this "morphology" options) Solr obviously had the best results.

Solr uses the Levenshtein distance by default if you use the "~" fuzzy search operator. Unfortunately, sphinx does not have the option to use the Levenshtein distance, just soundex, metaphone among others.

So for now we have decided to move back to Solr and most likely a fork of acts_as_solr. Any other options out there for rails and solr?

Ruby FreshBooks Integration

Jonathan Siegel — Mon, 29 Jun 2009 02:06:39 +0000

Having trouble integrating FreshBooks? So were we.

Some gotchas:

The README is often incorrect.
Update doesn't work. Use our gem.
Status for estimates is an integer, not string: 1 draft, 2 sent, 3 viewed, 4 replied 5 accepted, 6 invoiced
Check the result for invoice.update. If it's false, there's an error. Edit the gem to turn on debug statements around lib/freshbooks/connection.rb:call_api
To download the pdf, you must set the status to 'sent' for a draft (remember to set it back!).
Install it: gem install elc-freshbooks.rb --source http://gems.github.com

Full code to download an invoice or estimate PDF:

  def get_pdf(inv_or_est)
    is_draft = inv_or_est.status == 'draft'
    if is_draft
      inv_or_est.status="sent"
      update_success = inv_or_est.update # Fail if this is false
      raise "FreshBooks API has changed. Document could not be readied." unless update_success
    end
    logged_in_session_to_view_doc_url = inv_or_est.links.client_view
    cookiejar = Tempfile.new("cookies")
    # Note this is particularly ugly because we require session cookies + redirect handling + bad SSL chain (thanks godaddy!). Bailed on Net::HTTP approach.
    html_doc = `wget "#{logged_in_session_to_view_doc_url}"  --no-check-certificate --cookies=on --keep-session-cookies --save-cookies='#{cookiejar.path}'  -O -`
    # https://elc.freshbooks.com/getPDF.php
    if inv_or_est.class == FreshBooks::Estimate
      pd = "estimate"
    else
      pd = "invoice"
    end
    pdf_file = Tempfile.new("pdf")
    cmd = "wget 'https://#{self.freshbooks_api_host}/getPDF.php' --post-data='#{pd}id[]=#{inv_or_est.number}' --cookies=on --keep-session-cookies --load-cookies='#{cookiejar.path}'  --no-check-certificate -O '#{pdf_file.path}'"
    pdf = `#{cmd}`
    logger.debug "Get the actual PDF: #{cmd}"
    cookiejar.delete
    if is_draft
      inv_or_est.status="draft"
      update_success = inv_or_est.update;
      raise "FreshBooks API has changed. Document could not be readied." unless update_success
    end
    pdf_file
  end

See more at: http://github.com/elc/freshbooks.rb/tree/master

Auto delta parent records in ThinkingSphinx

Max, John — Fri, 26 Jun 2009 17:44:25 +0000

Auto updating sphinx delta on child record updates

Thinking Sphinx is quickly becoming the de-facto rails search plugin. In the past there have been a few other plugins that gained popularity and lost it almost as quickly by either being orphaned, outdated, etc. Thinking Sphinx has a clear path and a very involved/attentive maintainer.

The way sphinx does it's "Live Updates" is through delta indexing. This allows indices to be added without rebuilding the master index. Thinking Sphinx handles new and updated records to be added to the delta through a couple of facilities, the one I will be focusing on is the delta boolean which is added to your index definition block like so:

Class UsedVehicle < ActiveRecord::Base
  define_index do
    indexes [manufacturer, model], :as => make_and_model
    indexes :trim_level
    indexes :year
    indexes :color
    indexes :condition
    indexes :price
    
    #index relations
    indexes option_package.options, :as => :options
    indexes accessories.package, :as => :accessory_package
    
    #this is to allow for delta index updating on record modification
    set_property :delta => true
  end
  
  ...
  
end

In the example above we're indexing option_package.options and accessories.package which are ActiveRecord relations. Thinking Sphinx will update the parent UsedVehicle records in the delta when it sees them changed, through the 'delta' column on UsedVehicle, but what if you're updating the child records? Simple:

Class OptionPackage < ActiveRecord::Base
  after_update :set_parent_delta
  
  def set_parent_delta
    self.used_vehicle.update_attributes(:delta => true)
  end

  ...

end

Now you'd want to do the same to the Accessory class to get that updating also. And it's that simple.

Scraping Images from Twitpics

Alex Chee — Fri, 26 Jun 2009 00:15:09 +0000

Recently, I've been scraping Images and videos from Twitter and one site that has not been too easy to grab pics from is Twitpics. Here's a snippet of code that I've been using to grab the image from Twitpic with Hpricot:

require 'net/http'
require 'hpricot'

def rip_twitpic(url)
  begin
    code=url.match(/[\w]+$/).to_s
    unless code.blank?
      uri=URI.parse(url)
      resp=Net::HTTP.get_response(uri)
      html=Hpricot(resp.body)
      html.at("#photo-display")['src']
    end
  rescue Exception => e
    puts "Error extracting twitpic: #{e}"
    url
  end
end

Note: Thanks to Stephen Boisvert for showing my typo. That's what I get for rushing something out and drinking too many cups of coffee.

Coworking Visas: Offices Without Borders

Nicholaus — Thu, 25 Jun 2009 17:19:34 +0000

Listen up, Señor T is talking to you. If you have ever tried to spend 8 hours working from a coffee shop, then I don't have to tell you that it's not all that it's cracked up to be. Yes, its a fun dynamic working environment, where people will make you coffee and tasty snacks. However, spotty internet connections, loud music, chatty patrons and lack of power outlets get old quick. But for a cybernomad like me, plopping down a monitor on a desk just isn't an option.

Shared Workspaces

In recent years, shared community orientated workspaces have been growing in numbers. Places like Souk, which we jokingly refer to as the ELC Portland office, are great places to drop in and work. They have flexible memberships and sport all the accommodations of a regular office (except a boss). I am currently at Office Nomads in Seattle and they even have a kegarator, a 48 inch TV and people doing Yoga.

Coworking Visas

But you still don't want to be stuck in the same city in the same shared office, because a shared office is still an office. Enter the Coworking Visa, which is a network of shared workspaces like Souk and Office Nomads that allow you to float between them if you have a membership at one location. This isn't a small network either. They are in most major and some not so major US cities and 13 countries around the world, including Buenos Aires where I spent the last month. Anyways, its totally amazing and I encourage you all to pick up your computer and wander with it. Sorry the pictures are so bad, I broke my camera.

Yes that is egg-in-a-pancake-hole, I had it for breakfast.

ReCSS - AJAX friendly CSS testing

Gregor Martynus — Wed, 24 Jun 2009 23:02:35 +0000

Back in the days when the web didn't yet have a version number, David from dojotoolkit.org came up with a neat solution to reload your css without the need of reloading your page: ReCSS.

It's just a simple bookmarklet that walks through all linked stylesheets and reloads them. This omits page reloads in order to test your CSS changes, very helpful for AJAX heavy sites. Here is what the code looks like:

javascript: void(function() {
    var i, a, s;
    a = document.getElementsByTagName('link');
    for (i = 0; i < a.length; i++) {
        s = a[i];
        if (s.rel.toLowerCase().indexOf('stylesheet') >= 0 && s.href) {
            var h = s.href.replace(/(&|%5C?)forceReload=\d+/, '');
            s.href = h + (h.indexOf('?') >= 0 ? '&': '?') + 'forceReload=' + (new Date().valueOf())
        }
    }
})();

This is one of the small things that makes my life easier, and I want to make sure that it does the same for you. Drag the following link to your Bookmarks toolbar and become a CSS rockstar: ReCSS

jQuery ajax quick tip: $.ajax abort()

Cary Dunn — Mon, 22 Jun 2009 18:47:43 +0000

Dealing with ajax has become so easy and high-level that it is pretty easy to forget about the details. A nice habit to get into when dealing with an ajax-heavy frontend is to manage the event listeners and connections that a user can initiate.

Something simple to remember is that you can't rely on your requests to complete FIFO because your requests are getting routed across multiple processes/machines that may or may not be under load. Alot of users are rather click-happy (my mom still double-clicks links) which can lead to problems with content that was requested on the first click being loaded after the content from the most recent click.

In the past I have always included sequence numbers as request ID's and then have the server include the request ID it is responding to in the JSON response. This gives the event handlers some context of the response it just received in relation to the most recent action the user has performed on the page.

This works well, but sometimes you just want to throw out any response that is not in response to the most recent request. This is where the abort() method is useful.

var current_conn;

function getSome() {
  if(current_request) {current_request.abort();}
  current_request = $.get('/events',
    { team : "Ramrod" }, 
    function(resp) { alert(resp); }
  );
}

If you need to, you could use the sequence number approach and create a response queue or other more complex solution. Here are a few neat ideas as well...

http://www.protofunc.com/scripts/jquery/ajaxManager/

Simple queue idea

iPhone Tethering Without Jail-breaking Your iPhone

Brandon Trebitowski — Mon, 15 Jun 2009 15:22:16 +0000

With the 3.0 update coming out for the iPhone, people have been talking about one of the new features called tethering. Tethering is the process of using your mobile device's internet connection on your computer. This allows you to have internet on your computer anywhere there is cell service.

While phones like the Blackberry and Palm have had tethering for a few years, Apple has finally updated the new iPhone to include this feature. The problem is the US carrier for the iPhone (AT&T) will not support tethering for quite some time. And when they do decide to support it, they will charge an additional monthly fee.

Up until now, the only way to get tethering on your iPhone was to hack it. An app also came out a while ago that allowed you to do this, but it was quickly pulled from the app store.

So, the crew over at 9to5mac have discovered a way to have tethering with any iPhone running the 3.0 update. They give you step by step instructions on how to do it. All it requires is iTunes 8.2, Their tethering app , and the 3.0 iPhone update.

While this solution is only temporary I'm sure as Apple/AT&T is bound to catch on and fix the vulnerability, it is still very interesting. So be sure to check it out.

Distributed Sphinx

Dylan Stamat — Wed, 10 Jun 2009 01:41:22 +0000

While installing Sphinx for a largish application, I wasn't able to find many documented examples that touched on distributed deployments. The examples I did find were either a bit complicated when it comes to setup and execution, or, they didn't necessarily meet my needs. I "think" my needs are pretty simple:

I need indexing to be fast, searching to be fast, result sets to be up-to-date, and, this painlessly deployed across a large and dynamic array of machines.

That's reasonable, right? Well, luckily for us, Sphinx actually handles all that, and more :)

My only caveat is that the distributed deployment scenarios listed on the Sphinx site require quite a bit of setup and configuration. It's really not that much configuration, but, I want to take the path of least resistance for my particular scenario. I'm going to list my requirements (which are most likely a common set), and some the options laid out for me. If there are other solutions to this set of requirements in terms of deployment, I'd love to hear them, so comment away!

An Example Scenerio

We have about 50-60 application machines, a master database, some slaves, and a bunch of machines that play their own unique roles. There are less than 50m records (with limited fields) that need indexed, so doing full nightly indexes won't be a big deal.

Here are some of the approaches I've come across while surfing the interwave:

Sharding

The official Sphinx documentation outlines a method for horizontally partitioning out indexes across a cluster of machines. Patrick Galbraith has a great rundown of this approach as well. The searchd processes are able to connect to configured remote agents, run the search, create a unique result set, and return the results. This is a great approach, and it has been used to search through billions of records. Impressive!

Pros: Great performance, master/slave configurable for high availability.
Cons: Hefty configuration. No auto-discovery on nodes.

SCP

I've read quite a few cases of people enabling delta's across their cluster, doing a complete index on one machine, SCP'ing the index to all the machines in the their cluster, then HUP'ing all of the searchd's. Phew... that was a sentence full! This is definitely an iffy approach, but it works. You'll run into inconsistent deltas across the cluster, and some hefty configuration, but if this isn't a problem for you, go for it :)

Pros: Straightforward.
Cons: Hefty configuration. Out of synch deltas. SCP.

NFS

Your machines can have NFS mounts that point to both the primary index, as well as a single delta index. This doesn't require any special agent configuration, and your delta's will never be out of synch. This is a win, but you may have significant write contention on your delta's, and, there will be some network overhead.

Pros: Light configuration. Up-to-date delta's.
Cons: Delta contention on write heavy applications. NFS.

For what it's worth, we're going to most likely roll with the NFS approach. Our data set is minimal, the fields are not write heavy, and we don't want to worry about too much configuration (no agent configuration in sphinx.xml, just cycle up/down machines, and let NFS handle the rest).

I'm sure I'm probably missing some other snazzy setups that are out there in the wild, and I would love to add to this list. Feel free to comment about other techniques that you may have seen, heard of, or have had experience with, and I'll add it to the list above!

Upgrade GIT on OS/X

Jonathan Siegel — Mon, 08 Jun 2009 09:54:19 +0000

There's a new GIT version out with at least 10 reasons to upgrade. OS/X users of MacPorts can get the GIT upgrade with:

$ sudo port selfupdate
$ sudo port upgrade git-core # not git!

If you try this on your MacBook Air like I did be prepared for massive sluggishness for up to 2 hours.

Don't Forget

To remove that annoying message you now get with git push:

$ git push
warning: You did not specify any refspecs to push, and the current remote
warning: has not configured any push refspecs. The default action in this
warning: case is to push all matching refspecs, that is, all branches
warning: that exist both locally and remotely will be updated.  This may
warning: not necessarily be what you want to happen.
warning: 
warning: You can specify what action you want to take in this case, and
warning: avoid seeing this message again, by configuring 'push.default' to:
warning:   'nothing'  : Do not push anything
warning:   'matching' : Push all matching branches (default)
warning:   'tracking' : Push the current branch to whatever it is tracking
warning:   'current'  : Push the current branch

To keep the default setting and remove the warnings, just type:

# Sets ~/.gitconfig which applies as default across all working dirs:
$ git config --global push.default matching
$ cat ~/.gitconfig 
[push]
	default = matching

Or if you'd like to set this on a project-by-project basis, make sure you are in a GIT working directory and:

# Per project setting:
$ git config push.default matching

Your STI Broke My Sphinx

ceberz — Sat, 06 Jun 2009 04:19:55 +0000

If you're using Ultrasphinx in your rails application and you have models using ActiveRecord support for Single Table Inheritance, it might not work they way you;d like. Actually it will break.

# == Schema Information
#
# Table name: fruits
#
#  id              :integer(4)      not null, primary key
#  type           :string(255)
#  size           :integer(4)

class Apple < Fruit
  is_indexed :fields => ['size']
end

If you try to run an Ultrasphinx call with the above definition, it will search an index that was built from the entire fruits table, returning ID's that will raise an ActiveRecord::RecordNotFound exception when put through Apple.find(). To fix it, use the conditions argument to change the sql query that sphinx uses to build its index.

class Apple < Fruit
  is_indexed :fields => ['size'], :conditions => "fruits.type = 'Apple'"
end

Stubbing a method for all tests

David Palm — Thu, 04 Jun 2009 10:28:37 +0000

If you ever need to stub out a method for all tests in your test suite, for example a before filter in ApplicationController that goes takes a while to run, here's a neat trick.

Stick the following in your spec_helper:

Spec::Runner.configure do |config|
  config.before(:each, :type => :controller) do 
    controller.stub!(:blog_feed).and_return([])
  end 
end

If you need to override the above for some specs, just put a similar block on top of the spec file. (Question: how can I remove the stub completely for just a few tests?)

It might not be the most elegant solution known to man, and it sure isn't very clean and could come back and bite you one day, but if you know what you're doing... ;)

Update: Something like this might work for your unstubbing needs, but if the above is an ugly hack, this is a really ugly hack and YMMV:

class Object
  def unstub!(method_name, *args, &blk)
    self.send("proxied_by_rspec__#{method_name}", *args, &blk) if self.respond_to?("proxied_by_rspec__#{method_name}")
  end
end

If you need to poke even deeper (and you really shouldn't), this might be useful to you:

rspec_guts = my_instance_of_something.send(:__mock_proxy) # rspec works its magic on this object, which is a Spec::Mocks::Proxy
puts rspec_guts.instance_variable_get("@proxied_methods").inspect

pdf2swf on Mac OS X and ASVM mismatch

David Palm — Tue, 02 Jun 2009 19:50:18 +0000

The current version of swftools in macports is 0.8.1 and for many cases it's not good enough. In particular, for pdf2swf to generate swf files that are recognized as ASVM3 (i.e. Flash version 7+ I think, but do correct me if I'm wrong) you need swftools 0.9.0 or greater. Loading an external swf movie from within another is considerably more cumbersome if the virtual machine used in the two files versions differ.

You know you ran into the above VM mismatch error if your Flash logs are telling you something like this:

  TypeError: Error #1034: Type Coercion failed: cannot convert flash.display::AVM1Movie@1e3d9e01 to flash.display.MovieClip. 
    at com.elctech::PdfViewer/doneLoading()

The above error will show only if -- only if -- you have a debugger enabled flashplayer and logging enabled.

But as we said, the current macports version isn't recent enough, so what to do? One option is to install everything from source, which for a package such as swftools entails a lot of dependencies. Another option is to install the current macports version anyway to get all the dependencies installed and then compile just the swftools package from source (http://www.swftools.org/download.html). Not ideal perhaps, but IMHO better.

If you get complaints about missing libraries towards the end of the ./configure phase, like this:

  ***************************************************
  * The following headers/libraries are missing:  jpeglib ungif jpeglib.h gif_lib.h
  * Disabling pdf2swf tool...
  * Disabling jpeg2swf tool...
  * Disabling gif2swf tool...
  ***************************************************

... try this:

  LDFLAGS="-L/opt/local/lib" CPPFLAGS="-I/opt/local/include -I/opt/local/include/lame" ./configure

...to make sure your macports installed libs/headers are picked up.

If/when make fail with something like:

  ld: in ../libgfxswf.a, archive has no table of contents 
  collect2: ld returned 1 exit status 
  make[1]: *** [pdf2swf] Error 1 
  make: *** [all] Error 2

...run:

    ranlib lib/*.a
    make

...and the compile will continue. 'sudo make install' to finish.

I'd love to write a more thorough explanation for the ranlib magic here, but I honestly don't have a clue. It's key not to run 'make clean' or './configure' again here, just the ranlib stuff and it will pick up where it got confused. Somehow. If anyone knows of a good resource for ranlib I'd love to dig further into this.

Deep Linking Pagination with jQuery Address

Cary Dunn — Tue, 02 Jun 2009 19:17:36 +0000

With most sites today so heavily reliant on loads of asyncronous requests with javascript, json/p, flash, etc. It can be quite frustrating when trying to navigate a JS image gallery or RSS reader while also making sure never to accidently hit back/forward/refresh so that you do not lose your place in browsing. Want to send a link to the specific photo you are browsing in a JS gallery? Good luck getting your friends to click the link and then hit "Next" 32 times to see the specific photo.

Facebook does a great job at keeping all of their heavy javascript pages (such as the photo gallery) deep-linked so this is not an issue.

In the past, the most popular library for deep-linking was swfAddress and was primarily used for Flash. swfAddress now supports AJAX though the same team has also released a lightweight version for jQuery called jQuery Address.

To demostrate deep-linking with jQuery Address I put together a quick demo of deep-linked pagination of a Flickr photo gallery as an example. You should be able to use your back/forward/refresh/bookmark functions just fine.

View Demo

Source

  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
  <html xmlns="http://www.w3.org/1999/xhtml">
    <head>
      <title>jQuery Address + Flickr Pagination Demo</title>
      <meta content="text/html; charset=utf-8" http-equiv="content-type" />

      <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
      <script type="text/javascript" src="jquery.address-1.0.min.js"></script>
      <script type="text/javascript" src="jquery.flickr-1.0-min.js"></script>

      <style type="text/css">
        div#flickr_photos ul { overflow: hidden; list-style-type: none; margin: 0px; padding: 0px; }
        div#flickr_photos ul li { float: left; }
        div#flickr_pagination a, div#flickr_pagination span { margin-right: 10px; }
      </style>

      <script type="text/javascript">
        /*<![CDATA[*/
        var current_page = 1;

        function refreshPagination(ele) {
          $("#flickr_loader").remove();

          var current_page = parseInt($(ele).find("input:eq(0)").val());
          var total_pages = parseInt($(ele).find("input:eq(1)").val());

          if(current_page <= 1) {
            $("<span>").html("Previous").appendTo("#flickr_pagination");
            $("<a>").attr("href", "#").html("Next").bind("click", nextPage).appendTo("#flickr_pagination");
          } else if (current_page >= total_pages) {
            $("<a>").attr("href", "#").html("Previous").bind("click", previousPage).appendTo("#flickr_pagination");
            $("<span>").html("Next").appendTo("#flickr_pagination");
          } else {
            $("<a>").attr("href", "#").html("Previous").bind("click", previousPage).appendTo("#flickr_pagination");
            $("<a>").attr("href", "#").html("Next").bind("click", nextPage).appendTo("#flickr_pagination"); 
          }
        }

        function previousPage() {
          $.address.value("?page="+(current_page-1));
          return false;
        }

        function nextPage() {
          $.address.value("?page="+(current_page+1));
          return false;
        }

        function getPhotos(page) {
          $.address.title($.address.title().split(' | ')[0] + ' | page '+page);

          $("#flickr_photos").html($("<img/>").attr("src", "spinner.gif").attr("id", "flickr_loader"));
          $("#flickr_pagination").html("");

          $("#flickr_photos").flickr({
      			api_key: "f28804be7a09c5845676349c7e47d636",     
      			per_page: 5,
      			page: page,
      			type: "photoset",
      			photoset_id: "72157600037079977",
      			callback: refreshPagination
      		});
        }

        function addressChangeHandler(evt) {
          current_page = $.address.parameter("page");
          if(current_page) {
            current_page = parseInt(current_page);
          } else {
            current_page = 1;
          }

          getPhotos(current_page);
        }

        $(document).ready(function(){
          // Setup jQuery Address Listener
          $.address.change(addressChangeHandler);
        });
        /*]]>*/
      </script>
    </head>
    <body>
      <div id="flickr_photos"></div>
      <div id="flickr_pagination"></div>
    </body>
  </html>

The case of mysterious user interface

Rick — Mon, 01 Jun 2009 19:06:22 +0000

Until web coders and designers become one and the same, how does one deal with designing a website that is user interface heavy? Does it mean that all designers should learn to code? Or should all coders learn how to design?

I often come across site specification that came with both feature description and mock designs attached to it. In the olden days when everything is static, this was perfect. Each mock is treated as a definite state and you can easily see the flow operational flow from one mock to the next. For example, if you have a form submission page, then you have a submission result page. As a user, you are trained to think that everything works like a two-step cha-cha: submit, result, submit , result, Ba da bing, ba da boom.

This web behavior, however, is changing. As functionally heavy sties such as social network sites start to emerge, people start to demand instantaneous feedback. Out goes the dedicated landing page for submission result, and in comes the thank you popup powered by Ajax. Although the underpinning of the web infrastructure hasn't changed, the change of behavior provides a sense of orientation and expedited response that makes users feel more empowered.

Now, what does it all mean for the designers? No longer is it enough for a designer to just make things look pretty. These days things need to be both pretty and functional. Without actually coding to demonstrate the user interaction, designers will be forced to spend incredible amount of time documenting and create a mock for all possible interactions. Worse than being inefficient in using the designers' time is that coders struggle with the case mysterious user interface and make up their own interaction.

As web design is starting to lean towards the user interface design, I have no doubt that the line that separates a web graphic designer and a web coder will be blurred even more. Until then, I think the best compromise is for people to rely heavily on the communication between the designers and the coders. Some may compare the separation between coders and designers like the model and view, and argue that this separation is necessary. I certainly agree that it's necessary when it comes to software engineering. I would even agree with that on the theoretic level. But when it comes to human engineering, I would still say that talking is the best user interface there is.

RightSignature Featured on TechCrunch

Daniel LaBare — Sat, 30 May 2009 03:25:51 +0000

Check out RightSignature

Merging Hash and HashWithIndifferentAccess

Daniel LaBare — Thu, 28 May 2009 22:23:37 +0000

Ran into a gotcha the other day I thought I'd share. It came about in some code that was trying to give some default values in a hash, then merge in some params before creating an object. It looked something like this:

   Thing.new({:is_cool => true}.merge(params[:thing])

The gotcha here is that params is a HashWithIndifferentAccess, but {:is_cool => true} is not. Merging these two will actually drop the return value to just Hash. Observe.

   >> params = HashWithIndifferentAccess.new(:thing => {:is_cool => false, :name => 'Cool Thing'})
   => {"thing"=>{"name"=>"Cool Thing", "is_cool"=>false}}
   >> params.class
   => HashWithIndifferentAccess
   >> ({:is_class => true}.merge(params[:thing])).class
   => Hash

When you look at what is created when merging the two, you notice that there are actually two separate keys of the same name, one a symbol, one a string.

    >> {:is_cool => true}.merge(params[:thing])
    => {"name"=>"Cool Thing", :is_cool=>true, "is_cool"=>false}

And then when you pass this hash to your ActiveRecord model, you can see that it favors the symbol.

   >> thing = Thing.new({:is_cool => true}.merge(params[:thing]))
   => #<Thing id: nil, name: "Cool Thing", is_cool: true, created_at: nil, updated_at: nil>

The solution is to either use a string as your key:

   >> thing = Thing.new({'is_cool' => true}.merge(params[:thing]))
   => #<Thing id: nil, name: "Cool Thing", is_cool: false, created_at: nil, updated_at: nil>

Or make the initial hash a HashWithIndifferentAccess:

   >> thing = Thing.new(HashWithIndifferentAccess.new(:is_cool => true).merge(params[:thing]))
   => #<Thing id: nil, name: "Cool Thing", is_cool: false, created_at: nil, updated_at: nil>

So yea, keep an eye out for this gremlin.

Agile sighting in LAX

ceberz — Thu, 28 May 2009 18:23:32 +0000

Well...sort of an agile sighting...I mean, not really but sort of ;-)

Copy S3 assets with right_aws

Alex Chee — Thu, 28 May 2009 17:33:15 +0000

Lately, I've been using right_aws to interact with S3. One thing that I found helpful was copying assets between buckets and keeping the same permissions on them. However, it's not as simple as just copying the assets over. You need to get the Access Control Policy from the source and put it in the copied asset.

Here's a snippet of code that does the magic.

require 'rubygems' 
require 'right_aws' 

s3=RightAws::S3Interface.new(S3_KEY, S3_SECRET)
s3.copy(SOURCE_BUCKET, SOURCE_PATH, DESTINATION_BUCKET, DESTINATION_PATH, :copy, {"Cache-Control" => 'max-age=315360000', "Expires" => '315360000'})
acl_prop=s3.get_acl(SOURCE_BUCKET, SOURCE_PATH)
s3.put_acl(DESTINATION_BUCKET, DESTINATION, acl_prop[:object])

Ruby 1.9 isn't always faster: rake tab-completion revisited

David Palm — Wed, 27 May 2009 16:20:32 +0000

Rake is awesome in many ways and we're all using it for a plethora of tasks. It's easy to use, fast to code and reliable.

A while back I blogged about tab completion for rake tasks and the other day I set out to speed it up, as the project I was working on had a huge amount of tasks and retrieving the list of tasks took a loooong while.

As it turns out, speeding up the task enumeration is not a trivial task. My first thought was to ParseTree but it was horrendously slow and quite complex. Then I thought I'd use a fast language, such as OCaml or perhaps even C to scan the files and become known worldwide for my wicked coding skillz.

Didn't happen. As I'm sure you've already figured out, the rake library allows you to define raketasks in many different ways and people out there all seem to have different opinions on how to code up their tasks. Several important libraries define the rake tasks dynamically, ruling out any pattern matching approaches.

In other words I failed. But I also learned a lot and thanks to ruby-talk I also found a trick to make things spiffier: Ruby 1.9 isn't always faster, Rake::Application is a singleton and String#hash isn't what it used to be.

Read on for details.

Rake tab completion

The first step to setup rake tab completion is to tell bash to invoke your script when 'rake' is the command. Drop this in your ~/.bashrc:

  complete -C ~/bin/rake_completion.rb -o default rake

The next step is the ruby executable script that actually looks up the rake tasks. The script:

  #!/usr/bin/env ruby
 
def rake_silent_tasks
  dotcache = "#{File.join(File.expand_path('~'), ".raketabs-#{Dir.pwd.hash}")}"
  if File.exists?(dotcache)
    Marshal.load(File.read(dotcache))
  else
    require 'rubygems'
    require 'rake'
    load 'Rakefile'
    tasks = Rake.application.tasks.map(&:name)
    File.open(dotcache, 'w') { |f| f.puts Marshal.dump(tasks) }
    tasks || []
  end
end
 
exit 0 unless File.file?(File.join(Dir.pwd, 'Rakefile'))
exit 0 unless matches = ENV["COMP_LINE"].match(/^rake\s+(.*)/)
 
after_match = matches[1]
task_match = after_match.strip.empty? ? nil : after_match.strip.split.last
 
if task_match
  tasks = rake_silent_tasks.grep /^#{Regexp.escape task_match}/
 
  # handle namespaces
  if matches = task_match.match(/^([-\w:]+:)(.*)/)
    upto_last_colon = matches[1]
    after_match = matches[2]
    tasks = tasks.map { |t| (t =~ /^#{Regexp.escape upto_last_colon}([-\w:]+)$/) ? "#{$1}" : t }
  end
  puts tasks
end
 
exit 0

We need to make this executable with:

  chmod +x ~/bin/rake_completion.rb

Let's go through the code. First comes the 'rake_silent_tasks' top-level method. More on that later. Next a couple of checks to ensure we're somewhere where it makes sense to run 'rake' (i.e. a directory with a 'Rakefile' present). Then we check the $COMP_LINE environment variable. This is set by bash and we just need to match it against the string 'rake' at the beginning of the command line. Easy.

The remaining code is just your average regexp juggling to match the string you enter, e.g. 'rake db:mi[TAB]'.

What is interesting here is all in the 'rake_silent_tasks' method. In order for lookups to be fast we really want to avoid invoking rake and parse the Rakefile (and all the .task files that are commonly present).

The code first checks if we've been running rake+TAB in this directory before and if so, loads an unmarshal the tasks into a Ruby Array. Using Marshal.load/.dump is a big speedup in itself.

If we need to lookup all the rake tasks from scratch, a full parse of all task defining ruby files is necessary. Previous versions of this code used a backtick invocation of rake like so:

  `rake --silent --tasks`

That ouputs the full list of tasks and their descriptions and is quite slow.

To bypass the necessity for backticks I read through the source code for rake and asked ruby-talk to help out and sure enough, there is a way to get around the necessity to spin off a whole new ruby process, and the following returns a list of tasks as an Array:

  require 'rake'
  load 'Rakefile'
  Rake.application.tasks

Pretty straightforward, uh? Sure, after the fact it's obvious, but I spent a long while trying to instantiate a new Rake::Application object until told that rake is implemented as a singleton. This snippet will crash so badly that IRB dies and throws you back to the shell:

  require 'rake'
  a = Rake::Application.new
  a.init

(maybe it's not even a crash, just plain wrong usage but it's really hard to tell and can be quite confusing...)

Anyways, retrieving the rake tasks this way is a lot faster. How much?

  100 rake task enumerations for "/Users/david/projects/big_rails_app"
                    user     system      total        real
  backticks     0.160000   0.330000  59.680000 ( 61.323611)
  rake direct   0.590000   0.070000   0.660000 (  0.676324)

Two orders of magnitude! Yay!

Interestingly, Ruby 1.9 is quite a bit slower than good'ol 1.8:

    100 rake task enumerations for "/Users/david/projects/big_rails_app"
                      user     system      total        real
    backticks     0.230000   0.390000  59.940000 ( 61.596335)
    rake direct   0.960000   0.110000   1.070000 (  1.089892)

While I was investigating the various options to speed up tab completion I first of all grabbed for Ruby 1.9. After a while I discovered that the implementation above never hits the cache and the reason for that is a subtle change in the way the little used String#hash method works.

In 1.8 the hash for a given String is always the same, while in 1.9 the implementation changed to use MurmurHash and now the hash is identical only within the same ruby process. This change is not documented anywhere to my knowledge and is a deal breaker for ruby scripts used as command line executables (and maybe elsewhere as well).

I tried to replace the 'Dir.pwd.hash' snippet above with 'Digest::MD5.hexdigest(Dir.pwd)' so it would work under 1.9, but it's a lot slower than Sting#hash, and we're aiming for speed here, so...

All in all I'm happy with my investigation. As said, learned a lot and in the end I gained a fast-ish rake task tabber script. Many thanks go to Sebastian Hungerecker, Robert Klemme, Ryan Davis and of course to the original implementors of the rake tab completion script.

If you see any other means of speeding this up, please leave a comment! :)

Update: fixed escapes in code, as per commenters request (1 Jun 2009)

First RubyConf.China

Daniel Lv — Wed, 27 May 2009 07:01:26 +0000

The first Ruby conference in Chinae was held at May 21, 2009. I have had the honor to be the host of the event, and it turned out better than I expected! Not only was Ruby's Father, Matz, delivered an informative and educational speech, other illustrious developers such as @robbinfan, @rubbinlu, and YeZheng have also contributed their wealth of knowledge to the conference.

The conference was sponsored by JavaEye.com (the second largest tech community in China) and ShangHaionRails.org (a local ruby&rails organization located in ShangHai.) The conference room was originally booked to seat 200 people, and over 400 have attended this event. The event was a great success and you can check out more photos and videos on rubyconfchina.org and links below.

Photos

Videos

Matz -- Why Ruby?
Design Pattern from java to ruby
Deep secret in JavaEye
Ruby and Rails Pitfall
Ruby/Rails for Enterprise Development
Open Source & Scrum
Blue Ruby* A Ruby VM for the ABAP Web Application Server

Corrupt Attachments with AR_Mailer

Dylan Stamat — Fri, 22 May 2009 23:08:36 +0000

AR Mailer and Corrupt Truncated Attachments

After changing our system to send out emails via ar_mailer, the attachments we were sending out were corrupted.

The "mail" column of the ar_mailer schema is a TEXT column, but that won't work for most attachments, thus, we simply bumped it up to a LONGTEXT (we're using mysql on this project), and everything started working perfectly again.

class EmailsMailTextToLongtext <  ActiveRecord::Migration                                                                                                                                           
  def self.up                                                                                                                                                                                      
    execute "ALTER TABLE emails CHANGE mail mail LONGTEXT"                                                                                                                                         
  end                                                                                                                                                                                              
                                                                                                                                                                                                   
  def self.down                                                                                                                                                                                    
    execute "ALTER TABLE emails CHANGE mail mail TEXT"                                                                                                                                             
  end                                                                                                                                                                                              
end

Multi-part emails sent from a cron job delivered as text/plain only

Daniel LaBare — Thu, 21 May 2009 23:20:46 +0000

Had an interesting problem today. We use multipart emails in our client project snapizzi so users can get pretty little emails if their client supports html. However, we found that any email sent from a cron task was being sent with a mime-type of text/plain, but rendering only the html text. Fugly.

Turns out this is an active ticket in rails which can be found here, with a patch to fix it that can be found here.

We use Rails as a git submodule, so rather than applying the patch and freezing, I just added an actionmailer_fix.rb to my initializers. At least until this patch makes it into Rails.

Mozilla Labs Launches JetPack!

Brandon Trebitowski — Thu, 21 May 2009 17:53:16 +0000

With Mozilla Firefox having such a large following, it is obvious why one would want to extend the browser. Currently, it requires some intimate knowlege of their API and there is quite a bit of a learning curve when it comes to developing browser plugins (useful ones anyway)

Well, the Mozilla Labs team aims to solve this by creating an API that can be understood by anyone who has ever created a webpage. According to their website:

In short, Jetpack is an API for allowing you to write Firefox add-ons using the web technologies you already know.

The easy to use API is based on Javascript and HTML. It also comes complete with the JQuery library to do fancy AJAX and Animation effects.

Here is some sample code that you might expect to write to extend the browser.

jetpack.statusBar.append({ html: "Boom<i>!</i>" });

Sure, this code just puts the word Boom! in your status bar. But this is the most basic example. One thing I want to point out here is the fact that it accepts HTML markup. I was able to put the italics tags on an element in the status bar!! You could do all sorts of things, even put an iFrame there (gross).

While JetPack is still in pre-release, it still packs a punch. Be sure to check it out! https://jetpack.mozillalabs.com/