Patrick Crosby

Write Facebook apps using Google AppEngine

Sun, 20 Apr 2008 19:33:15 GMT

I hacked up minifb in order to get it to work with Google AppEngine. The result is gminifb. It uses Google's urlfetch library instead of urllib2.

To get it to work, download gminifb and place it in your application's directory. It requires simplejson, so put that in your app's directory as well.

Here's a sample webapp.py action that uses the library:

class FacebookAction(FbHandler):
  def get(self):
    arguments = gminifb.validate(FB_SECRET_KEY, self.request)
    session_key = arguments["session_key"]
    uid = arguments["user"]

    usersInfo = gminifb.call("facebook.users.getInfo",
        FB_API_KEY, FB_SECRET_KEY, session_key=session_key,
        call_id=True, fields="name,pic_square",
        uids=uid) # uids can be comma separated list
    name = usersInfo[0]["name"]
    photo = usersInfo[0]["pic_square"]

    self.render('facebook.html', {'name': name, 'photo': photo})

Thanks to Peter Shinners for writing the original minifb library. My changes were very minimal...

Firefox search keyboard shortcut saves years of your life

Tue, 08 Apr 2008 03:51:23 GMT

Cmd-k (or ctrl-k) takes you to the search box, but opens the results in the current tab. Go to about:config and change browser.search.openintab to true and now your search results go in a new tab.

Firefox 3 + del.icio.us + firebug

Mon, 07 Apr 2008 13:36:50 GMT

Firefox 3 is now usable because there are versions of the Firebug and del.icio.us extensions that work:

Firebug: http://www.getfirebug.com/releases/firebug/1.2/

del.icio.us: http://tech.groups.yahoo.com/group/delicious-firefox-extension (Join the group, go to the Files section)

vim indent block of code

Sat, 05 Apr 2008 16:51:00 GMT

Summary: Vjjjjjj=

Enter visual line mode: V

Select a block of code: jjjjjjj (or whatever)

Indent it: =

Heaven...

rails + vim tip: :RSunittest

Sat, 05 Apr 2008 16:29:08 GMT

Using rails.vim, there are a bunch of ways to navigate around your project using :R commands: :Rview, :Rcontroller, :Rmodel, :Runittest, etc.

You can also do :RS with any of these and they will open up in a split window. So if you are in a model, you can do :RSunittest and vim will open your unit test in a split window.

Open a new Terminal window using Quicksilver

Wed, 26 Mar 2008 20:02:00 GMT

Create the following AppleScript file:

tell application "Terminal"
  do script ""
end tell

I named mine newterm.scpt and put it in ~/bin, where I put all my little shell scripts.

Once Quicksilver reindexes its catalog (you can force this), you can activate this script anywhere. I've trained Quicksilver to use Ctrl-space nt.

Automatic production.log performance reports: RAWK + logrotate

Tue, 18 Mar 2008 19:31:00 GMT

Killing two birds with one stone here...rotating your Rails production.log (in the default scenario, it just grows forever) and monitoring the performance of your application.

First, get rawk.rb. This little script analyzes your production log and produces nice reports showing which requests are taking the most time. Put it somewhere like /usr/local/bin.

Next, you want to create a logrotate config file for your app's production.log. On a debian system, they live in /etc/logrotate.d. Here's an example in a file named rails-example:

/var/www/example.com/shared/log/production.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    copytruncate
    prerotate
      /usr/local/bin/rawk.rb < /var/www/example.com/shared/log/production.log | mail -s "[example.com] rawk report" somebody@example.com
    endscript
}

This tells logrotate to rotate the production log every day, keep 14 days worth of copies, and before it rotates the log file, it runs rawk on it and mails the result to somebody.

How to make (non-stretched) square thumbnails with attachment_fu and ImageScience

Wed, 05 Mar 2008 02:55:46 GMT

By default, if you try to make a square thumbnail from a non-square image with attachment_fu, it will stretch/squash the image to make it square instead of cropping it.

ImageScience has a method called cropped_thumbnail that will make a correct square thumbnail, but attachment_fu doesn't use it.

To fix this, change the resize_image method in vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/processors/image_science_processor.rb like this:

def resize_image(img, size)
  # create a dummy temp file to write to
  # ImageScience doesn't handle all gifs properly, so it converts them to
  # pngs for thumbnails.  It has something to do with trying to save gifs
  # with a larger palette than 256 colors, which is all the gif format
  # supports.
  filename.sub! /gif$/, 'png'
  content_type.sub!(/gif$/, 'png')
  self.temp_path = write_to_temp_file(filename)
  grab_dimensions = lambda do |img|
    self.width  = img.width  if respond_to?(:width)
    self.height = img.height if respond_to?(:height)
    img.save self.temp_path
    self.size = File.size(self.temp_path)
    callback_with_args :after_resize, img
  end

  size = size.first if size.is_a?(Array) && size.length == 1
  if size.is_a?(Fixnum) || (size.is_a?(Array) && size.first.is_a?(Fixnum))
    if size.is_a?(Fixnum)
      img.thumbnail(size, &grab_dimensions)
    else
      if size[0] == size[1]
        img.cropped_thumbnail(size[0], &grab_dimensions)
      else
        img.resize(size[0], size[1], &grab_dimensions)
      end
    end
  else
    new_size = [img.width, img.height] / size.to_s
    img.resize(new_size[0], new_size[1], &grab_dimensions)
  end
end

Then in your model, if you specify a thumbnail (or a resize) with square dimensions, it will use cropped_thumbnail and create a proper square thumbnail:

class Photo < ActiveRecord::Base
  has_attachment :content_type => :image, 
                 :storage => :file_system, 
                 :max_size => 1.megabytes,
                 :resize_to => '320x200>',
                 :thumbnails => { :small => [20, 20], :thumb => [50, 50] },
                 :processor => 'ImageScience'

  validates_as_attachment
end

PostgreSQL primary key sequence out of sync

Sat, 23 Feb 2008 21:01:02 GMT

I'm not sure how this happened, but I kept getting

PGError: ERROR:  duplicate key violates unique constraint...

messages while doing a simple INSERT using ActiveRecord with a PostgreSQL database, and the key was the primary key for the table. My guess was that the primary key sequence was out of sync.

SELECT MAX(id) FROM posts;

returned 4, but

SELECT nextval('posts_id_seq');

returned 4 as well.

I found two ways to fix it. The first odd way is to run the INSERT statement manually in psql. I don't know why it works in psql and not through ActiveRecord, but I don't know why there was a problem in the first place.

The second is to update the sequence manually:

SELECT setval('posts_id_seq', (SELECT MAX(id) FROM posts)+1);

I never tried the second method as the problem hasn't returned.

FeedTools database cache

Sun, 17 Feb 2008 00:51:00 GMT

The ruby gem FeedTools has a built-in caching mechanism. Unfortunately, it doesn't quite work out of the box, and the documentation doesn't explain it very well. Despite what it says, you have to tell it to explicitly use the cache in your code:

#!/usr/bin/env ruby

require 'rubygems'
require 'feed_tools'

FeedTools.configurations[:feed_cache] = "DatabaseFeedCache"

puts "Getting feed 1st time..."
f = FeedTools::Feed.open('http://www.slashdot.org/index.rss')
puts "1. live? #{f.live?}"
puts "Getting feed 2nd time..."
f = FeedTools::Feed.open('http://www.slashdot.org/index.rss')
puts "2. live? #{f.live?}"

Assuming FeedTools can find a database.yml file and the database has the cached_feeds table described in the FeedTools documentation, the above script will cache the feed after the first time and f.live? will return false the second time you run it.

I also added an index on the href column in cached_feeds. All the lookups are done on that column and the migration that comes with FeedTools doesn't have any indices. Here's my migration for cached_feeds:

class AddFeedToolsTables < ActiveRecord::Migration
  def self.up
    create_table :cached_feeds do |t|
      t.column :href, :string
      t.column :title, :string
      t.column :link, :string
      t.column :feed_data, :text
      t.column :feed_data_type, :string
      t.column :http_headers, :text
      t.column :last_retrieved, :datetime
      t.column :time_to_live, :integer
      t.column :serialized, :text
    end
    add_index :cached_feeds, :href
  end

  def self.down
    drop_table :cached_feeds
  end
end

Apache 2 virtual host redirect

Wed, 13 Feb 2008 04:32:24 GMT

For one of my sites, I own .org, .net, and .com for the domain and I want all the requests to go to the .org domain. I switched to Apache 2 and this is the configuration I used:

<VirtualHost *:80>
  ServerName india.bigpatents.com
  ServerAlias india.bigpatents.com 

  RedirectMatch (.*) http://india.bigpatents.org$1

</VirtualHost>

Postgresql slow query log

Fri, 08 Feb 2008 23:16:40 GMT

Set the following variable in postgresql.conf to log all queries that take longer than 100 milliseconds:

log_min_duration_statement = 100

They will show up in the main postgresql.log file.

pgFouine looks like an intriguing postgresql log file analyzer, except it's written in PHP??!?

Ping google blog search using ruby

Fri, 08 Feb 2008 22:29:13 GMT

With the following class, you can ping Google's blog search whenever you update your blog:

require 'xmlrpc/client'

class BlogPing
  def self.ping(site_name, site_url, page_url, feed_url)
    begin
      server = XMLRPC::Client.new2("http://blogsearch.google.com/ping/RPC2")
      server.call2('weblogUpdates.extendedPing', site_name, site_url, page_url, feed_url)
      return true
    rescue => detail
      puts "ping failed (#{detail})"
      return false
    end
  end
end

With any luck, it just worked on this post!

Daemontools scripts for sphinx

Fri, 08 Feb 2008 04:02:48 GMT

Here are scripts to use daemontools to manage sphinx.

The run script (/service/sphinx/run):

#!/bin/sh
echo starting
exec 2>&1
exec /usr/local/bin/searchd --console --config /var/www/example.com/current/config/sphinx/production.conf

the --console flag is key: it keeps searchd running in the foreground.

The log/run script is standard (/service/sphinx/log/run):

#!/bin/sh
exec multilog t ./main

And finally a script to reindex everything:

#!/bin/sh

/usr/local/bin/svstat /service/sphinx | awk -F ')| ' '{print $4}' > /var/www/example.com/current/tmp/pids/searchd.pid
/usr/local/bin/indexer --config /var/www/example.com/current/config/sphinx/production.conf --all --rotate

The searchd.pid file is referenced in the sphinx production.conf file and is used by indexer. It would be nice to just call indexer --all and then svc -h /service/sphinx, but indexer requires the --rotate flag to reindex without taking searchd offline, so you have to do the awk junk to get the pid first.

How to fix unsupervised daemontools log scripts

Fri, 08 Feb 2008 03:30:00 GMT

I am using daemontools and was having trouble with the log files. The log script was not being supervised, nothing I did seemed to make it work. svstat /service/test/log returned unable to open supervise/ok: file does not exist.

The problem was that svscan had picked up on my service before the log directory existed. And no matter what I did (including restarting the service), it wasn't noticing the log directory. I couldn't get it to notice the filesystem change because I made real directories in /service for each service, not symlinks.

One solution would be to reboot, but rebooting a unix box is equivalent to admitting failure. I stopped my service, moved the service files to /var/local/services/test, then symlinked the service directory into /service.

gettext 1.10.0 and Rails 2.0

Thu, 07 Feb 2008 21:14:00 GMT

The ruby gem gettext 1.10.0 doesn't work with html.erb files. It's simple to fix. In the file .../gems/gettext-1.10.0/lib/gettext/parser/erb.rb change the following:

module GetText
  module ErbParser
    @config = {
      :extnames => ['.rhtml']
    }

module GetText
  module ErbParser
    @config = {
      :extnames => ['.rhtml', '.erb']
    }

With this change, the gettext system will parse all your erb files.

Rails fragment caching with multiple accounts

Wed, 30 Jan 2008 23:38:19 GMT

In Episode 90: Fragment Caching railscast, there is a great, simple explanation of how to do fragment caching.

I like using strings as the identifiers for the cached portion of a view, but if your site has accounts with users signed in and you want to cache fragments that change based on the user, you need to do something more. If the user id is exposed in the url, you can use the url to generate the cache identifier. But if the user id is hidden, then you need to do something more.

First, it should be noted that if you put slashes in the cache identifier, it will treat those as directories. So

<% cache 'some/deep/fragment' do %>
Hello
<% end %>

will put the fragment in RAILS_ROOT/tmp/cache/some/deep/fragment.cache. This is good because if you have a lot of users, you won't want all their cache files in the same directory as that would slow down the filesystem dramatically.

To be clear, if you did

<% cache 'recent_contacts' do %>
<%= render :partial => "/shared/contact", :collection => @contacts %>
<% end %>

this would not work for multiple accounts as once it is cached, every account will see the same contact list.

In my project, there are many accounts. Each account has many users, but the fragments can be cached for all the users in an account, so putting the account id in the cache identifier is enough to make them unique.

So you could do

<% cache "recent_contacts_#{@account.id}" do %>
<%= render :partial => "/shared/contact", :collection => @contacts %>
<% end %>

This is fine, but once you get a lot of accounts, you are going to have a lot of files in the cache directory, one for every account. Besides slowing down the filesystem, this could even stop working if you go above the number of allowed files in a directory.

My approach was to add a cache_dir method to the Account model that splits the accounts into many subdirectories. There are many ways to do this that balance the directories better (hash the id, then split it up into directories to randomize where it goes), but this way is simple and makes for an easy explanation:

class Account < ActiveRecord::Base

  # ...

  def cache_dir
    unless @cache_dir
      x = sprintf("%08d", self.id)
      @cache_dir = x[0..1] + "/" + x[2..3] + "/" + x[4..5] + "/" + x[6..7]
    end
    @cache_dir
  end

end

Then the cache statements in the views look like:

<% cache "#{@account.cache_dir}/contacts/recent" do %>
<%= render :partial => "/shared/contact", :collection => @contacts %>
<% end %>

So the recent contacts for account id 1 would go into tmp/cache/00/00/00/01/contacts/recent. It keeps it fairly clean and keeps all the cache files separated by account ids. Once I hit 10,000,000 accounts, I'll need to adjust this, but that's a good problem to have.

Rails 2: Custom source annotations

Wed, 30 Jan 2008 10:24:00 GMT

Rails 2.0 has the new rake notes:... tasks built in. If you annotate your code with OPTIMIZE, FIXME, or TODO comments, then rake notes will show all of them, rake notes:fixme will show just the FIXME comments.

What if you want annotations of other comments? I've been using XXX for a few years now, so I wrote the following in lib/tasks/notes.rake:

require 'source_annotation_extractor'

namespace :notes do
  desc "Enumerate all XXX annotations"
  task :xxx do
    SourceAnnotationExtractor.enumerate "XXX"
  end
end

Now I can run rake notes:xxx and see all my XXX comments in the project. They don't show up in the global rake notes task, but...

git: fatal: empty ident not allowed

Wed, 30 Jan 2008 10:01:00 GMT

I use git like I used to use svn: one central repository, clients push/pull to it. I recently migrated the repository from a crappy Virtuozzo VPS (horrible, horrible disk access times...had to wait many seconds for ls to run) to a Xen one.

Everything seemed fine until today when I did my first git push, which returned the error:

fatal: empty ident &lt;git@example.com&gt; not allowed

It took me a while to figure out as .git/config had my user information set, and I even had a ~/.gitconfig file with default user information set. The git faq has an entry for this problem, but doesn't mention server set-up.

The problem was the git user on my new server didn't have any name information in /etc/passwd. It didn't have this on the old server either, but for whatever reason it needed it on the new one (the old server is too slow to bother investigating). I changed /etc/passwd on the server, giving the git user the full name of 'Git Git' and everything worked fine. It's weird that it cares...in the log, this user never shows up. Must be something to do with merging? Anyway, it works...

How to get acts_as_audited to work with Rails 2

Tue, 11 Dec 2007 16:50:00 GMT

The audit.rb file in the great acts_as_audited plugin requires acts_as_list, which in Rails 2 is now a plugin. So install the acts_as_list plugin, then add the following to environment.rb:

config.plugins = [:acts_as_list, :all]

This will force rails to load acts_as_list before acts_as_audited and everything will work well...