Locomotivation - The code blog from Squeejee

Global Settings With Mongo and Rails in 15 Lines of Code

Wed, 01 Jul 2009 01:52:00 -0400

There is a nice ActiveRecord Rails plugin called rails-settings that does the following:

Settings is a plugin that makes managing a table of global key, value pairs easy. Think of it like a global Hash stored in you database, that uses simple ActiveRecord like methods for manipulation. Keep track of any global setting that you dont want to hard code into your rails app. You can store any kind of object. Strings, numbers, arrays, or any object.

We have used this plugin in the past for global key / value pairs and it works great. However, for our Floxee app, we are using MongoDB / MongoRecord so this plugin would not work.

Lucky for us, MongoDB is already a document store so I was able to recreate similar code in about 15 lines of code inside of a “method_missing” method.

With this, we are now able to do things like:

>> AppSetting.jim = "is cool"
=> "is cool"

>> AppSetting.arr = ["jim", "was", "here"]
=> ["jim", "was", "here"]

>> AppSetting.arr.first
=> "jim"

>> AppSetting.hsh = {"hash_1"=>1, "hash_2"=>"demo"}
=> {"hash_1"=>1, "hash_2"=>"demo"}

>> AppSetting.hsh["hash_2"]
=> "demo"

MongoDB - Ruby friendly document storage that doesn't rhyme with ouch

Tue, 02 Jun 2009 22:30:27 -0400

I had the opportunity to share MongoDB with the Dallas.rb group this evening. Thanks @mully and @jnunemaker for contributing slides!

MongoDB - Ruby document store that doesn’t rhyme with ouch

View more PDF documents from Wynn Netherland.

Same train, new station

Mon, 01 Jun 2009 10:18:22 -0400

At RailsConf recently we met a lot of folks who gave us kudos on this blog. Many were surprised to learn that Locomotivation is brought to you by the Squeejee team.

So, we’ve decided to integrate our code blog a bit more with our web site. Sorry if the move caused your RSS reader to panic. Hopefully, once the dust settles we’ll have more regular content from our crazy ride on Rails.

Installing CouchDB From Source On Mac OS X

Thu, 12 Mar 2009 00:00:00 -0400

Wynn and I started dabbling with CouchDB this week. As with any technology, step one when dabbling is installing the software.

For Mac OS X Leopard Intel users, there is a very nice CouchDB packaged zip file that can be downloaded from the Couch Projects Google Code site. The huge positive of going this route is that it is drop dead simple to install. Simply download the .zip file, extract the code, and double-click the “CouchDBX” file. You then simply click the “Play” icon and you are off and running with CouchDB.

The negative going this route is that, as the time of this blog post, this package runs CouchDB version 0.8.1 which is about 7 months old.

In order to run the new 0.9 version of CouchDB, you will need to pull the latest source code from the SVN trunk, install some prereqs, and compile from source. Here are the exact steps I took to get up and running.

NOTE: For details on each of these steps, you can look at the README file that is downloaded with the couchdb source code.

From the command line:

Browse to the following URL in a browser to view your CouchDB Futon admin tool:

http://127.0.0.1:5984/_utils/

Ruby on Rails Integration Testing with Integrity

Wed, 25 Feb 2009 00:00:00 -0500

I did some research on what was new in the Continuous Integration testing field in regards to Ruby this week. We have used CruiseControl.rb in the past which worked fine, but I remember the setup being a bigger pain than I thought it had to be. I was hoping for something simpler.

With this in mind, I decided to give Integrity a shot. Their title of “The Easy and Fun Automated Continuous Integration Server” was exactly what I was looking for. After installing, configuring, and playing around with it for the past day or so, I have not been disappointed.

With that being said, I did have to jump around to a few different sources to get everything running correctly on a server with Phusion Passenger, so I thought I would document the steps I had to take here.

Installation:

This will create an Integrity application directory with a couple files on your server, mainly config.yml, config.ru, and a “public” directory we will point our Apache virtual host to.

Edit config.yml to suit your needs. This file is very will commented and your edits should be self-explanatory. I kept the default Sqlite database configuration to keep things simple. However, due to this, I had to install Sqlite on our Ubuntu server via:

Setup a DNS entry to point to this server and create an Apache virtual host as you would any other Rails or Sinatra web application:

Once this is all complete, simply restart apache to access your new Integrity instance.

Project Creation:

Create a Test Database:

(You can skip this test if you can use Sqlite databases for testing. Our code inevitably has MySql specific SQL syntax in it which requires us to use MySql for our tests.)

Create a MySql user that will be used for Integrity tests:

Note that “IDENTIFIED BY PASSWORD” is optional. If it is not included, your user can log in without a password.

Project Creation:

Assuming the above was successful, you should now be able to browse to the Integrity URL you created. From here, click the “Add new project” link.

Name: Unique name for the project you will be tracking
Git Repo: .git URL of the project you will be tracking. For example, for an internal project on GitHub, the url would be something like: git@github.com:acme/my_cool_application.git
Branch: The git branch Integrity should be tracking
Build Script: The rake command to use for testing after the code is pulled into your CI (Continuous Integration) server.
- rake spec (for rspec)
- rake test (for test unit)
- rake features (for Cucumber)
- etc

Customizing Your Build Script

The build script can be a tad bit tricky. It is our practice not to keep database.yml in Git for password security purposes. Because of this, Integrity will not know the test database configuration settings needed to run the tests. To get around this issue, we created a database.yml.test file with the db credentials Integrity would need to run a test similar to this:

Using MySql:

Using Sqlite:

In my Rails project, I then created the below custom Rake task in “lib/tasks/my_cool_application.rake”:

Once this is done and the files are added/committed/pushed to git, go back to your edit project page in Integrity and change your “Build Script” to be “rake test:move_test_db_config features” for Cucumber tests, for example.

Manual Build

After all of the above is complete, you should be able to click the “manual build” link for your new project to pull your code and run your tests!

Automated Tests Via Git Service Hooks

Integrity tests are automated with Git Service Hooks. Adding a service hook is drop-dead simple with GitHub. To do so:

Browse to your project in GitHub
Click on the “Admin” tab
Click on “Service Hooks” in the header
In the “Post-Receive URLs” box, add a URL similar to this:
- http://username:password@integrity.example.com/my_cool_application/push
- where “username” and “password” are the HTTP authentication credentials setup in the integrity/config.yml file

Once this is setup, your CI Integrity test will be run everytime you code is pushed to the appropriate branch in GitHub.

Notifiers

Integrity also provides a notification system to inform you when tests are run.

Current notifiers include email, Jabber, Campfire, IRC, Twitter, and Basecamp. These are installed independently of Integrity via Gems. Once a gem is installed on your CI server, just log back into your project within your Integrity web site and setup your notifier credentials. You are also free to write your own custom notifiers for applications that are not supported yet.

Resources:

Thanks to the following sites for assisting me with my original setup and this blog post:

Workstreaming with Present.ly

Sat, 20 Dec 2008 00:00:00 -0500

Having fled the cubicle jungle of Corporate America, we have a strong distaste for status reports. That’s not to say we don’t believe in letting others on the team know what we’re doing. We just prefer to workstream our activities and let our work announce itself to the rest of the team.

Present.ly - our workstreaming tool

I hate to put Present.ly in a box by calling it “Twitter for Business” (because it is so much more than that), but that’s the easiest way to describe it.

Familiar Twitter feel

If you’re comfortable using Twitter, you’ll feel at home with Present.ly because it supports @replies, direct messages, and hashtags.

Questions and urgent messages

Present.ly adds some smarts to the display of messages or “tweets.” Messages that end in a question mark are displayed as questions to the group:

Need to mark a message as urgent? Simply add three exclamation marks:

Private and secure

Unlike Twitter, only your users see your messages.

Groups!

I really hope Twitter builds functionality for groups one day and when they do, I hope they rip off Present.ly’s implementation. Admins control who belongs to what groups. Perhaps the coolest feature of Present.ly groups is the broadcast. Broadcasts are like direct messages to a group except they are displayed as if sent from the group itself, which gets its own avatar. The following broadcast message

b system Elvis has left the building

would display as if it came from the System itself:

How we use it

Much of our Present.ly use is still manual. We tweet each other questions and comments just like we do on Twitter. In fact, Present.ly let’s you cross-post to twitter by using the t! command:

t! This message would go to Twitter, too!

But, increasingly, we’ve automated much of our workstreaming by sending broadcast messages at certain points in our development cycle.

Source control checkins

One of the most important things we do as a development team is add code to source control. We use “GitHub”:http://github.com for our code repositories. GitHub has a nifty feature called Service Hooks, which allows you to send a message to a number of third party services for code commits to your repo. Setting up Present.ly really easy:

Once set up, GitHub will send a new broadcast message from the group you specify with each new code push. The message includes the commit message and a link to the commit back on GitHub:

Deploy notices

Before Present.ly we had been sending our deployment notices to Campfire using Tinder. While this works well, we really preferred to have our workstream in one spot and leav eour Campfires for discussion and chat. Since Present.ly implements the Twitter API, wiring up the deploy notices from Capistrano was a piece of cake.

Now everybody knows when the build is done:

How are you using it?

We’d love to hear how you’re using Present.ly in your workflow? Any cool uses we’re missing?

People, sites, and software I am thankful for this year

Thu, 27 Nov 2008 00:00:00 -0500

I’ve been building web applications for a dozen years now and I’m convinced there has never been a better time to be a web developer! I can’t imagine doing my job if it weren’t for the impact of some great people, sites, and software. This post isn’t a Ten Biggest style article. I won’t be thanking Tim Berners-Lee, although we all should.

Rather, this list is simply the extraordinary people and things that have made my job a joy over the past year.

People I am thankful for

John Resig

Creator of jQuery, JavaScript ninja, John casts a large shadow on the web development landscape. Looking at John’s code has taught me more about JavaScript than every book I’ve ever bought on the subject.

Chris Wanstrath aka defunkt

Half of the err the blog duo, a third of the GitHub posse, Chris is the most energetic Rubyist I’ve seen. Just watching the amount of software his company puts out will either inspire you or send you into a shame spiral.

The Less Everything guys, Steve and Allan

Makers of cool, free, and downright funny software, these guys write more less code than anybody around. I’m thankful that guys like this are around to keep us from taking ourselves too seriously.

Jean-Philippe Lang

I’ve never seen anyone single-handedly will a project into existence like JP. While there is no doubt that Redmine has a large and growing community behind it, I’m sure everyone would agree that Redmine exists because of the sheer will of Jean-Phillipe.

Sites I am thankful for

GitHub

No other technology has impacted me more this year than Git, and I don’t think it would have happened without GitHub. We first signed up for GitHub because we wanted a turnkey central repository for our projects. But GitHub is so much more. GitHub brings social networking to source control. This means you can see what other folks are working on and discover new projects and tools all the time. It also means it’s never been as easy to contribute to open source. Want to improve an open source project? With GitHub, you just fork it and send a pull request. If the original owner goes off the grid, no worries, you’ve got your own repo and keep on chugging.

Twitter

I’ll have to admit at first I didn’t get Twitter. I mean, take a look at the public timeline. Who cares when random folk are bathing? I’ve since learned that there is no better tool for finding those other folks that are just as weird as you. You know, those long lost members of your tribe(s). Folks who are interested in those things you bug everyone else about.

Presently

Soon after I climbed on the Twitter train, I longed for something as easy as Twitter to workstream all those source control checkin notices, build statuses, and “what I am doing” updates. The folks at Intridea have hit a homerun with Present.ly. The groups and hashtag functionality is awesome. Try it for your team!

Software I am thankful for

Redmine

We’ve used a lot of issue trackers. Some great ones. Some workhorses. But we’ve never used anything as extensible as Redmine. Plugins, themes, and custom fields, oh my!

Webby

I want to thank Tim Pease for building such a cool little tool. I’m using Webby to write this post!

Blueprint CSS

Blueprint has been a part of my standard toolbox since it came out. Blueprint brings sanity to my layout and typography while make it so easy developers can do CSS!

This year I’ll be looking to Compass to semantify my Blueprint.

jQuery

I wasn’t in the market for a new JavaScript framework when I discovered jQuery, but I quickly fell in love with this one. jQuery lets me do more powerful things with less code than any other framework I’ve used. With comprehensive CSS3 (and bonus) selectors, chaining, and plugin architecture it’s a joy to use.

How about you?

What are the people, sites, and projects you’re thankful for this year?

Javascript and Ruby Frameworks at Lone Star Ruby Conf 2008

Sun, 07 Sep 2008 00:00:00 -0400

I had the opportunity to give a talk at Lone Star Ruby Conf 2008. There are a lot of hardcore Rubyists here writing great web applications in Rails, Merb, and Waves. It was fun to knock around ideas with so many folks about how Javascript fits into the mix.

View SlideShare presentation or Upload your own. (tags: ruby javascript)

Until the slides are posted on the official LSRC site you can grab them from SlideShare.

Simplify ActiveRecord Aggregates and Other Goodies via named_scope

Mon, 25 Aug 2008 00:00:00 -0400

We’ve been huge fans of named_scope since its introduction in Rails 2.1. If you are new to the named_scope concept, please read Ryan Daigle’s intro to named_scope and watch Ryan Bates’ named_scope RailsCast. Ryan Daigle also had a good follow up article explaining his utility scope gem.

With a recent project, we really started utilizing extensions with named_scope which is very powerful and cleaned up our code considerably.

For example, let’s say you have an issue tracking system that has an Entry model that belongs_to both a User model and a Project model. Using named_scope, we can easily access Entries for a date range filtered to both the Project and User level using named_scope lambdas:

To find the Entries for the last week for the first user, we would simply do:

u = User.first
u.entries.by_date(1.week.ago, Time.now)

To find the Entries for the last week for the first project, we would simply do:

p = Project.first
p.entries.by_date(1.week.ago, Time.now)

What happens if we want to see the sum of hours worked for a date range? This is where extensions come in very handy. Let’s add a “total_duration” extension to our named_scope:

Now to get the duration for the past week for the first user we simply do:

u = User.first
u.entries.by_date(1.week.ago, Time.now).total_duration

with similar logic at the project level.

To group all of the entries by project, we would do:

If you need the lambda arguments inside your named_scope extension, you will need to use “self.proxy_options”. For example, let’s say you need the from and to dates in order to calculate the average duration per weekday. That code would look something like this:

In the above example, “weekday_count” is a custom method we created to calculate the number of weekdays between 2 given dates.

The opportunities to simplify code using named_scope is great. As I was writing this post, one of our team members, Jason Derrett, created a named_scope to wrap a Xapian full text search into a Lesson model:

In this example, using Xapian to search for a keyword becomes as simple as:

Lesson.find_with_xapian "fishing"

How cool is that?!?

Are there any other cool named_scope tricks out there that any of you are using?

Simple Ruby on Rails Full Text Search Using Xapian

Wed, 23 Jul 2008 00:00:00 -0400

In my previous blog post, I wrote about the several full-text search engines that are available to us as Rails programmers and how we ultimately decided to go with Xapian.

Background

I am by no means a Xapian expert. I first heard about it less than a week ago. However, we have been very impressed with its combination of ease of installation and robust features.

Xapian is written in C++ and has been around in its current form since 2001. Its code base is derived from a previous search tool that went closed source.

Written in C++
Feature rich including relevance feedback, boolean search operators, stemming, wildcard searching, and similar results

Its user list includes del.icio.us and GMane both of which are indexing tens of millions of records, so scalability should not be an issue. Here is a quote about scalability from the site:

People often want to know how Xapian will scale. The short answer is “very well” - a previous version of the software powered BrightStation’s Webtop search engine, which offered a search over around 500 million web pages (around 1.5 terabytes of database files).

Searches took less than a second.

The largest recent installation we’re aware of is probably gmane, which currently indexes over 50 million mail messages.

On the other hand, indexing time could take a while if you have a huge set of documents. According to this article, it took about 5 hours to index the entire Wikipedia database using Xapian for offline Wikipedia access and searching.

Acts_As_Xapian is the Rails plugin for the Xapian. It was written in May 2008 by Francis Irving after he experienced some frustrations with Solr. Acts_As_Xapian has a google group at http://groups.google.com/group/acts_as_xapian.

Xapian Installation:

For Ubuntu Users:

sudo apt-get install libxapian15 libxapian-ruby1.8

For Mac / Other: (taken from the xapian.org install docs)

(version = 1.0.6 at the time of this writing)

wget http://oligarchy.co.uk/xapian/1.0.6/xapian-core-1.0.6.tar.gz
wget http://oligarchy.co.uk/xapian/1.0.6/xapian-bindings-1.0.6.tar.gz

Alternately if you haven’t installed wget on the mac:

curl -O http://oligarchy.co.uk/xapian/1.0.6/xapian-core-1.0.6.tar.gz
curl -O http://oligarchy.co.uk/xapian/1.0.6/xapian-bindings-1.0.6.tar.gz

tar zxvf xapian-core-versionnumber.tar.gz
tar zxvf xapian-bindings-versionnumber.tar.gz

cd xapian-core-versionnumber
./configure --prefix=/opt
make
sudo make install

If you don’t have root access to install Xapian, you can specify a prefix in your home directory, for example:

./configure—prefix=/home/jenny/xapian-install


cd xapian-bindings-
./configure XAPIAN_CONFIG=/opt/bin/xapian-config
make
sudo make install

Acts_As_Xapian Installation:

Install the plugin: script/plugin install git://github.com/frabcus/acts_as_xapian.git

Generate and execute the acts_as_xapian migration:

script/generate acts_as_xapian
rake db:migrate

Basic Model Code

To include fields to your index, you will need to add an “acts_as_xapian” call to the model you want to index.

class Lesson < ActiveRecord::Base
  acts_as_xapian :texts => [:name, :description]
end

In this case, only the data found in the “name” and “description” attributes of our Lesson model will be searched.

A Quick Test Of Your Index

At this point, you can quickly test that your Xapian install, plugin, and model code are all working together via the following rake commands:

To build the index:

rake xapian:rebuild_index models="Lesson" RAILS_ENV=development

(You can add more models with a space delimeter. “Lesson User Tag”) To update index:

rake xapian:update_index RAILS_ENV=development

To test index:

rake xapian:query models="Lesson" query="golf" RAILS_ENV=development

The Basic Search

Adding the this line of code @search = ActsAsXapian::Search.new([Lesson], @lesson_search_params)

will return a ActsAsXapian::Search object with the following (From the acts_as_xapian README):

description – a techy one, to check how the query has been parsed
matches_estimated – a guesstimate at the total number of hits
spelling_correction – the corrected query string if there is a correction, otherwise nil
words_to_highlight – list of words for you to highlight, perhaps with TextHelper::highlight
results – an array of hashes each containing:
- :model – your Rails model, this is what you most want!
- :weight – relevancy measure
- :percent – the weight as a %, 0 meaning the item did not match the query at all
- :collapse_count – number of results with the same prefix, if you specified collapse_by_prefix

So, if we want to return an ActiveRecord model object we could do something like this: @lessons = @search.results.collect {|r| r[:model]}

To get the recommended spelling for a misspelled word(s) in a search, we would do: @corrections = @search.spelling_correction

Similar Results

If you have a requirement to show similar or “You may also like” results based on a search result set or a single result, this can be done with a single line of code:

@similar_lessons = ActsAsXapian::Similar.new([Lesson], @lessons).results.collect {|r| r[:model]}

where @lessons in this case would be the list of lessons returned in the original search.

Updating The Index

Unfortunately, one drawback to Acts_As_Xapian is that the indexers are not updated automatically when data in an indexed model is added, modified, or deleted. When model data is changed, Xapian will put a record in the table ‘acts_as_xapian_jobs’ in your database to notify the update task to update the index. A cron job or something of the like will be needed to call the following rake task periodically to keep the index up-to-date:

rake xapian:update_index

Francis, the acts_as_xapian creator, commented on this on my original Mulling Over Our Ruby On Rails Full Text Search Options blog post.

with Xapian you can update and search simultaneously, and updates are immediate. However, only one thread can update a Xapian database at the same time. Since I wanted offline indexing anyway (as my index operation is risky, complex and slow, involving parsing Word documents, PDFs etc.), I didnâ€™t try to find a solution that causes a second thread in the web application to, say, wait for the database lock. So acts_as_xapian currently only supports offline indexing.

Conclusion

This is a very basic tutorial meant to highlight the steps needed to get up and running with simple Xapian searches quickly. There are quite a few topics that I did not touch on such as:

Sorting and grouping with the :values option in the model
Advanced searching with the :terms option in the model
Adding extended attributes from other models into a searched index. (Such as adding lesson comments to the Lesson index)
Filtering searched data (in our case, we had to filter our results so only “active” lessons were returned)
Highlighting keywords in the search results

Smells Like a Google Search - Easily Grab Inbound Search Terms in Rails Using search_sniffer

Thu, 17 Jul 2008 00:00:00 -0400

Sure you can mine your referrers in Google Analytics, but why not tailor your site to your users by peeking at what they were looking for when they found your site? The search_sniffer plugin makes it too easy to do just that.

Make your site sticky, give ‘em more of what they want

Once you get folks to your site, the goal is to keep them there. On MindBites, we wanted to provide additional suggested content based on any search terms that led them to the site. The good news is that getting these terms is as easy as parsing the HTTP_REFERER from the HTTP request. Unfortunately, the format of this url differs between search engines.

With nearly 70 percent market share let’s take look at a Google example. A search for “Ruby on Rails Houston” from the Firefox search box results in the following URL:

http://www.google.com/search?q=ruby+on+rails+houston&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a

We only care about the q parameter, “ruby on rails houston.” Since “on” is such a common word, we really only care about “ruby rails houston.” The SearchSniffer plugin includes a list of common English words that are removed from the search terms by default.

How do I get it?

Installation is easy from Github

script/plugin install git://github.com/squeejee/search_sniffer.git

How do I use it?

class ApplicationController < ActionController::Base
    before_filter :sniff_referring_search

    ...
end

The plugin populates the @referring_search object containing info that can be passed to a keyword highlighter or internal site search engine to pull related content.

@referring_search.search_terms 
  => "ruby rails houston" 
@referring_search.raw
  => "ruby on rails houston
@referring_search.engine
  => "google

The plugin supports all the major search engines including Google, Yahoo!, MSN, and more.

Let us hear from you!

If you find it useful, or have ideas for improvement, let us know!

Mulling Over Our Ruby on Rails Full Text Search Options

Wed, 16 Jul 2008 00:00:00 -0400

There are quite a few choices when it comes to adding a full text search in a Ruby on Rails application. We thought that had considered all of our options when we ultimately settled on using Sphinx / Ultrasphinx. We learned otherwise, though, after stumbling across Xapian / Acts_As_Xapian while trying to find a fix for our Sphinx implementation after a production build.

Here are the details of our thought process and how we ultimately ended up deploying with Xapian.

While looking for a full-text search engine when Mindbites was released a year ago, we were looking for something easy and quick. We ended up going with Douglas Shearer’s Acts_As_Indexed which worked out great. It was written entirely in Ruby and very easy to implement with automatic indexing. (ie No cron jobs needed to keep the index up to date.) If you have a simple site and want to implement a basic search very quickly, definitely give Acts_As_Indexed a look.

However, over the past month or so we decided it was time to find something a little more robust. We wanted the features that the full-blown full-text search engines give such as spell correction, stemming (ie “connection / connecting / connected” would all search for words containing “connect”), finding “similar” results, and the ability to work across multiple servers. We were also having a few index corruptions with Acts_As_Indexed that we were wanting to get away from with another tool.

We went through the usual suspects that are often seen in the Rails Community:

Solr / Acts_As_Solr

Very robust search server based on the Lucene Java library with a mature acts_as_solr Rails plugin. I was blown away by Erik Hatcher’s “Solr On Rails” talk at RailsConf 2007 and thought this could be a good fit. However, we decided to check out other options because, all things being equal, we would rather not deal with installing Java on our servers.

Ferret / Acts_As_Ferret

We nixed Ferret fairly quickly after hearing a few horror stories about corrupted indexes among other issues with Ferret on production servers.

Sphinx / Ultrasphinx

Sphinx appears to be the new defacto standard for full text search among Rails developers. From what we read, it is very powerful, very fast indexing, and easy to use with Ultrasphinx. This was our choice to replace Acts_As_Indexed. However, after about a week’s worth of development and a deployment to our production server, there still were a lot of mysteries to the Sphinx.

We were not thrilled with all of the config files involved. With Ultrasphinx, you will need a xxx.base file which is accessed via a rake task to generate a config file that Sphinx can use. This works, but I was hoping for something a little bit simpler.

We were also not big fans of the daemons that run in the background. In our ITG environment, the daemons decided to stop running on a couple of occasions which caused a 500 error to be thrown when searching. (We later fixed this issue by reindexing and restarting the daemon with each Capistrano deployment.) Lastly, I read this quote from the Ultrasphinx deployment notes about recommended cron jobs for Sphinx on your production server:

“The first line reindexes the delta index every 10 minutes. The second line reindexes the main index once a day at 4am. The third line will try to restart the search daemon every three minutes. If it is already running, nothing happens.”

Kicking off a job every 3 minutes just to make sure another job is running did not seem right to me.

Xapian / Acts_As_Xapian

I had never heard of Xapian before this past Thursday. I happened to stumble across this article on on Evan Weaver’s blog while I was researching a Sphinx issue with our production build. Always looking for new and better ways to do things, I took a closer look at Xapian. Within 15 minutes, I had Xapian installed on my local Ubuntu machine and was successfully searching Mindbites lessons using the acts_as_xapian plugin. I spent the rest of the weekend replacing our Sphinx code with Xapian.

Installation, configuration, and deployment all went so well over the weekend that we deployed our completely revamped search code to our Mindbites production server this past Monday with spelling corrections, stemming, and “You may also like” functionality intact.

As a side note, that same Evan Weaver blog post has a comment about a plugin called act_as_searchable using the Hyper Estraier full-text search system. I have not looked into this solution, but I would be curious to hear from other readers who have tried this.

In part 2 of this blog post, I will go into detail about our Rails Xapian implementation.

It's Double Coupon Days! Generate and Redeem Coupons With the acts_as_redeemable Rails Plugin

Thu, 10 Jul 2008 00:00:00 -0400

Plugins are an excellent way to share functionality across your Rails applications. When we needed the same coupon and invite functionality across multiple apps, we decided to create acts_as_redeemable. Hopefully you’ll find it useful, too!

What is it?

acts_as_redeemable adds redemption capability to a model for items like coupons, invitation codes, etc. Each redeemable gets a unique code upon creation that can be sent in an email or printed as a coupon code.

Installing

You can install the plugin from Github:

script/plugin install git://github.com/squeejee/acts_as_redeemable.git

or if you’re on an older version of Rails, you can download the tarball.

How do I use it?

If you don’t have a model created already, go ahead and generate one:

script/generate redeemable Coupon
rake db:migrate

Make your ActiveRecord model act as redeemable

class Coupon > ActiveRecord::Base acts_as_redeemable :valid_for => 30.days, :code_length => 8 # optional expiration, code length end

Create a new instance

Now that you’ve created and configured your model, it’s time to create your coupon.

c = Coupon.new
c.user_id = 1 # The user who created the coupon 
c.save
c.code 
# "4D9110A3" 
c.created_at  
# Fri Feb 15 14:56:37 -0600 2008
c.expires_on 
# Fri Mar 16 14:56:37 -0600 2008

Of course, you’ll have other business logic for your coupon, discount, invite or other redeemable item. We just spot you the generation, redeeming, and expiration functionality. If you’re sending coupons via email, be sure and check out Slantwise Design’s excellent acts_as_invitation plugin.

Easily Switch Between Rails Development Sites With Phusion Passenger

Tue, 01 Jul 2008 00:00:00 -0400

~~mod_rails~~ Phusion Passenger is gaining some steam as it makes deploying Rails apps easier. Passenger also makes it easier to switch between your Rails projects during development.

Multi-tasking, Rails style

If you’re like me, you often find yourself flipping between two or three different Rails projects. Each time you flip, you do the same little dance in Terminal:

wynn$ cd projects/blog 
wynn$ mate . 
wynn$ script/server

You code along happily on the first project, knock out a couple of bugs, and then it’s time to switch. Either you stop your first mongrel server and dance the terminal jig all over again, or you just cmd+T a new terminal tab and crank up project two on a different port, like :3001 (and remember to specify the new port when you crank up your browser).

Phusion Passenger, not just for your production boxes

Passenger eliminates the need to start up your own mongrel server for each of your development Rails apps. Just like your PHP days, you simply set up an Apache virtual host for each app, and Passenger does the rest.

Step 1: Set up your host names

To set up your host aliases for each domain you want to use, edit your /etc/hosts file. Here’s mine:

127.0.0.1 beta.mindbites.local
127.0.0.1 mindbites.local

Step 2: Install Phusion Passenger

Assuming you have Apache2 installed (bundled with Leopard, an apt-get install away on Ubuntu):

sudo gem install passenger 
sudo passenger-install-apache2-module

The install script will chug away for a few minutes. Once on the other side, you’re only a couple steps away from multi-tasking bliss.

Step 3: Set up your virtual host

On OSX, you’ll need to use Headdress or modify /etc/apache2/httpd.conf with your virtual hosts:

NameVirtualHost *  
 
        DocumentRoot /Users/wynn/projects/mindbites/public 
    ServerName mindbites.local 
    ServerAlias beta.mindbites.local 
    RailsEnv development 
        
        Options FollowSymLinks    
        AllowOverride None    
        Order allow,deny    
        Allow from all 
     
  

    DocumentRoot /Users/wynn/projects/blog/public 
    ServerName locomotivation.local 
    RailsEnv development 
    
        Options FollowSymLinks    
        AllowOverride None
        Order allow,deny
        Allow from all
      
  

    DocumentRoot /Users/wynn/projects/goalistics/public 
    ServerName goalistics.local 
    RailsEnv development 
    
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny 
        Allow from all

After initially setting up my virtuals, I discovered that only the first one worked. A quick look on Google pointed me to the NameVirtualHost * directive which solved the problem. Note that the DocumentRoot /Users/wynn/projects/goalistics/public directive points to the public folder within your Rails app, not the root. Second, you’ll need to include the directory overrides in the Directory block to allow Apache to serve up your static content. (I was shocked when my app was served up naked the first time).

Also note that the RailsEnv directive determines the RAILS_ENV for the application and is production by default.

Wrap-up

I’ve been really pleased with Phusion Passenger so far. I discovered one hiccup, though. Passenger does not seem to care for inline ERB code in your database.yml so I had to side step some dynamic configuration we were doing. Error pages from Phusion are much nicer on the eyes though than the standard Rails versions. Also, restarting Rails applications are as easy as bouncing Apache or:

sudo touch tmp/restart.txt

from the project root. Passenger is definitely worth a look on your development machine.

Redesign Your Site in Place Using Rails Custom Mime Types

Sun, 29 Jun 2008 00:00:00 -0400

After reading Ben Smith’s classic iPhone on Rails 2 article and implementing our own iPhone site after the jump to Rails 2, I was struck at how easy it is to create you own custom mime types to serve up different content to different devices.

When the time came to redesign the main MindBites site, I didn’t look forward to a lot of branching and merging, no matter how easy Git makes it. Then it hit me. Why not create a custom mime type and serve up a new layout and views for beta users? Let’s take a look at how we’re doing just that.

Step 1: Register your custom mime type

The first thing we need to do is tell Rails we’ve got a new format type to serve up:

# config/environment.rb

Mime::Type.register_alias "text/html", :iphone
Mime::Type.register_alias "text/html", :beta

We used beta, but you could use any value here not already in use.

Step 2: Detect a beta request

In order to let only the cool folks into our new beta site, we’ll need a big burly bouncer at the door keeping out the riff-raff:

# application.rb
before_filter :adjust_format_for_beta

...

def adjust_format_for_beta
  request.format = :beta if beta_request? 
end

def beta_request?
  return (request.subdomains.first == "beta" || params[:format] == "beta")
end

The before_filter executes before each request and if ‘beta’ is the first subdomain or passed in as a format param, Rails will set the format to :beta.

Step 3: We’re in da club, now what?

Once we’ve identified our beta users, we now need to give them the new eye candy. We do this simply by dropping in a new view named xxxxxx.beta.erb. The bulk of the changes for our redesign are in a new application layout which serves up a new basic layout and CSS.

application.html.erb → application.beta.rb

Other views (even partials) are swapped out in a similar manner:

show.html.erb → show.beta.erb

_lesson.html.erb → _lesson.beta.erb

And that’s it! You don’t have to do every view in the site because Rails will fall back to .html.erb for any views without a .beta.erb version. Most of our redesign is accomplished in CSS. Where we have massive markup changes, we simply swap out the view templates.

See update 2 below

A gotcha to watch out for

In a few cases, we had respond_to blocks in our controllers to handle differences between the iPhone and regular versions of the site. Anywhere that you have these, you’ll need to add the beta mime type as well:

respond_to do |format|
  format.html
  format.beta # <= You'll get a 406 Unprocessable entity error without this line
  format.iphone do 
    render :layout => false if request.xhr?
  end
end

If an action does not already have a respond_to block, you’re OK. No changes are required in your controller.

UPDATE

Thanks, Ryan Heneise, for reminding me about the last step:

Step 4: What is new is old again

When you’re ready to launch, you’ll want to fold all those beta views back into your project as the default .html.erb views. Simply rename them from .beta.erb to .html.erb. For us, it’s a bit easier since we don’t have to overwrite any .html.erb files. We’re still using .rhtml!

Thanks, Ryan!

UPDATE 2

Paul Canavese turns in some excellent QA work and informs me that Rails will not fall back to .html.erb. Our templates are still .rhtml. So it would appear you will have to touch each template or use .rhtml. Sorry!

jQuery Parallax Scrolling - Build Your Own 1980's Video Game!

Wed, 18 Jun 2008 00:00:00 -0400

What is old is new again. Inspired by the recent crop of css parallax techniques. I thought it would be cool to take it one step further and introduce parallax scrolling. Thus the jQuery Parallax plugin was born.

What the heck is a parallax anyway?

Paul Annett of Clearleft does a far better job of explaining the css parallax technique than I could. If you’re unfamiliar with this layer-stacking technique, jog over to Vitamin and get the scoop.

That’s pretty cool, but why do I have to resize my browser to enjoy it?

For no practical reason whatsoever, I thought it would be cool to animate the layers in a CSS parallax and give the scene a depth of field that you can’t convey with static layers. As with all my jQuery endeavors so far, I found this surprisingly easier than I thought.

Step 1: Build the layered scene

Just in case you’re reading this and were served one of the other parallax scenes on the site, here’s a look at the finished product.

For our simple western landscape, we’ll construct the scene from four layers:

poles.png Through the magic of repeat-x, a single telephone pole with power lines becomes a never-ending array of energy delivery
logo.png Adding the logo to the scene creates an even greater illusion of depth. The slight gradient also conveys depth as items further in the distance have more light
windmill.png A lone windmill is set off in the distance
mountains.png Just like our telephone pole, this mountain range goes to the end of the earth
sky.png Finally, that pretty blue sky is a tiny image only one pixel wide, repeating on the x-axis

Step 2: Add the required Javascript references

To animate our parallax we’ll use my jQuery Parallax plugin which requires jQuery 1.2.x and the jQuery Easing plugin (for acceleration and deceleration).

Step 3: Wire up the parallaxes (what a fun word to say!)

Now it’s time to attach our parallax behavior to the container and each slice in our scene.

jQuery(document).ready(function() {

    jQuery("#header").parallax("create");
    jQuery(".scene-1 .slice-1").parallaxSlice("create", {speed: 5});
    jQuery(".scene-1 .slice-2").parallaxSlice("create", {speed: 3});
    jQuery(".scene-1 .slice-3").parallaxSlice("create", {speed: 1});

});

Step 4: Sit back and enjoy the show!

Again, this little demo has very little practical value. Maybe I should do something more constructive with it like, recreate the first level of Contra

ImageMagick OS X Installer

Thu, 20 Mar 2008 00:00:00 -0400

Installing ImageMagick / RMagick on OS X does not have to be a hastle. There is a very handy install script on RubyForge that does all of the work for you.

http://rubyforge.org/projects/rmagick/

Just be sure you have the following prereqs already installed on your Mac:

XCode Tools
X11 SDK
X11.app

Also, even though the script is supposed to include RMagick Ruby library, that piece did not work for me. Luckily, a simple “sudo gem install rmagick” worked like a champ for me.

Syntax Highlighting Blog Posts

Sat, 30 Jun 2007 00:00:00 -0400

After setting up this blog on the amazing new Expression Engine 1.6, we needed a way to share code snippets on blog posts. I found a number of utilities that will preformat your code, injecting markup around keywords that you can then style to taste. However, most of these solutions were language specific and cumbersome. One of the best tools I found was Pastie. With its built-in API, it would have been a nice fit had I been running a Rails-based CMS.

Most of the tools I found were Wordpress plugins which doesn’t help out an EE fan like me. I did find a great EE plugin for PHP code, but we tend to sling much more Ruby (and Rails), SQL, HTML, Javascript, and CSS on this blog.

Perhaps we were asking too much but we simply wanted to format our code within an entry with:

No special markup required
Multi programming language support (including the ones I haven’t learned yet)
CSS based styling

The solution

I remember liking how Javascript Fu Master Dan Webb formatted code within posts on his blog and was pleased to find a link to his CodeHighlighter project on his site. CodeHighlighter is a 100% client-side solution, using Javascript to perform configurable Regular Expression matches to inject CSS classes around keywords, constants, and other language constructs. Supported languages include Ruby, HTML, CSS, and Javascript, however you can add support for your favorite language in literally a few minutes. CodeHighlighter doesn’t offer SQL highlighting out of the box, so that’s exactly what we did.

Step one: set up the scripts Simply copy the CodeHighlighter.js and other language specific scripts to your server and add the script tags to your page:

Step two: insert your code block Next, add your code block to your post:


  your code here

Note the CSS class on the code element. This determines the language formatting that CodeHighlighter should apply to your code.

Step three: set up your styles The last step to syntax highlighting bliss is to add language specific CSS rules to your stylesheet:

.javascript .comment, .ruby .comment, .sql .comment { color : #9c6; font-style: italic; }

.javascript .string, .ruby .string, .sql .string { color : #9c6; }

.javascript .keywords, .ruby .keywords, .sql .keywords, sql-keyword { color : #679ef1; }

.javascript .global { color : blue; }

.javascript .brackets, .ruby .brackets { color : red; }

.css .comment { color : gray; }

.css .properties { color : navy; }

.css .selectors { color : maroon; font-weight : bold; }

.css .units { color :red; }

.css .urls { color :green; }

…

Adding support for SQL (well actually MySql)

To support SQL, I simply cloned the Javascript rules (javascript.js) as sql.js and edited the Regular Expression rules as necessary.

CodeHighlighter.addStyle(“sql”,{ comment : { exp : /(—[^n](n|$))|(/[^]*+([^/][^]*+)/)/ }, string : { exp : /’[^’]’|”[^”]*”/ }, keywords : { exp : /b(ADD|ALL|ALTER|ANALYZE|AND|AS|ASC|ASENSITIVE|BEFORE|)b/ }, global : { exp : /b(NULL)b/ } });

All I had to do was copy this file to the server and Jim’s first MySql post just got a bit more purty.

Drawbacks

While CodeHighlighter is a super easy, unobtrusive way to provide syntax highlighting in a server agnostic way, it’s client-side nature does mean you may see a slight delay before your code becomes formatted as the page loads before the script initiates. Also, as you can see with my SQL example, the rules files may get quite large, 8k in our case. If any RegEx gurus are reading, let me know if you can match a set of words in upper, lower, and mixed case for case insensitive languages like SQL. That would cut the sql.js rules file in half!