smallroomsoftware.com - Everything

Getting into github. ActiveRecord#previous_changes gem

2009-02-14T23:20:00Z

Not using it as my main VCS but I'm appreciating the power of Git more and more. Subversion definitely still has it's place but Git is pretty damn compelling for any open source project and any closed source project of any real scale. And it is _so_ nice to have my old unmaintained projects on github where I can leave people to fork with no help from me.

As github makes it super-easy to distribute ruby gems I thought I would write one for one for once. previous_changes adds an instance method, previous_changes, to ActiveRecord objects to retrieve what the changes would have returned before a call to save.

Shopping carts on Rails

2008-10-10T21:51:00Z

For a while now I've been looking at options for changing the software on my Dad's online store. We've been using LiteCommerce for a few years and although it's generally worked well, (and I would still recommend it) it's been hacked on quite a bit to add features and modify the way it behaves. Fiddling with partially encrypted PHP code isn't my idea of fun and upgrading is also a real pain. Also, for a PHP application, it isn't particularly fast. It's coded in a clever object oriented way but I would say the per request overhead is quite high. I looked at some other PHP shopping carts but nothing looked much more compelling that LiteCommerce. It might be nice to have someone else worry about the whole problem but I couldn't find a good hosted solution (and what's the fun in that!) I looked at Shopify for a while and then realised it was pretty underpowered and kind of expensive.

Ideally I'd like a Rails based shopping cart and I've been following the progress of Spree for a while now. Spree seems to gaining momentum but it's still missing a lot of features that I need. Also, I suspect that a lot of effort is going to go into building user interface for store configuration. Things like, configuring of shipping, tax, payment gateways etc. I actually think this is the wrong approach for Spree. Spree is almost exclusively going to be used by Rails programmers and Rails programmers are a lot happier configuring these kind of thing in code. I mean, that's just normal practice in Rubyland. Actually, I would count being able to configure my shipping rules with a bit of Ruby code a feature! Same goes for tax calculations. So I guess I would actually like Spree not to be a finished product in itself. Well maybe a finished product when you add some basic extensions. The extension system does seem really good and I hope that they use extensions for more core components (permutations, inventory control). For anything that might not quite fit a potential user's needs (like me!), so they can swap the default out and code their own.

(Update: Sean points out below that Spree's tax and shipping calculation to plug in as Ruby code. I think it's great that Spree is taking this developer-centric approach and not aiming to be a turnkey solution. Just to be clear, I think Spree is by far the most promising shopping cart system I've seen for a programmer to pick up and adapt for their client.)

So in the end, I realised that in the long-run, building a custom cart is going to be less of a hassle and more of a benefit than migrating to some other software that has it's own quirks and distinctive way of viewing the world. Actually, it has turned out not be as much work as I had expected. As I mentioned above, lots of the complexity of reusable cart software comes from having to provide enough flexibility and configuration UI. I get to ignore all this and essentially hard-code things like shipping rules and tax calculations. So I can quickly code up a shipping exceptions for certain products. I also don't need a sand-boxed template editor - I just use normal erb templates.

So I've probably put about 6 weeks of work into this and I already have the new site running. I think that's a pretty good advertisement for how fast you can build a fairly non-trivial application with Rails. I haven't implemented all of the features I have planned but it effectively does everything for my dad's business that Litecommerce did and a bit more. Building and deploying a Rails app from scratch is certainly a smoother process than it was a couple of years ago (more on that later).

Here are some screenshots:

Testing your SSL requirements with Webrick in development mode

2008-08-18T19:28:00Z

Testing SSL pages (login pages etc) in development mode doesn't seem to be supported out of the box with Rails. Turns out that webrick includes an SSL server and it isn't too hard to set. You need this script (mentioned here). I changed the :ip setting and then saved it in my project as script/ssl_server. The trick is to run both the script/server and the script/ssl_server scripts at the same time. The problem is that you'll be using non-standard ports so the ssl_requirement plugin will fail to provide the correct redirects (it assumes ports 80 and 443). There's a ticket and patch about this but it doesn't look like it's going to be applied anytime soon. Let's just patch the code ourselves, replacing ssl_requirement's ensure_proper_protocol method.

def ensure_proper_protocol
  return true if ssl_allowed?

  if ssl_required? && !request.ssl?
    redirect_to "https://" + request.host + (RAILS_ENV == 'development' ? ':3001' : '') + request.request_uri
    return false
  elsif request.ssl? && !ssl_required?
    redirect_to "http://" + request.host + (RAILS_ENV == 'development' ? ':3000' : '') + request.request_uri
    return false
  end
end

So I ran into another problem with my SSL pages, this time in production. If you use the assets host feature, you'll find your SSL protected pages reference assets with normal unsecured http URLs. This will cause a lot of security warning dialogs for IE users so you have to deal with it. You need all referenced media to be server over SSL. Luckily, you can supply a Proc to generate asset URLs and this Proc receives the current 'request' object which we can interrogate. Turns out that this is such a common situation, the example in the rails docs gives the solution. Let's get our four hostname pipelining back too:

ActionController::Base.asset_host = Proc.new { |source, request|
  if request.ssl?
    "#{request.protocol}#{request.host_with_port}"
  else
    "#{request.protocol}asset#{(source.hash % 4)}.rentability.com"
  end
}

Non-stop most blue skies

2008-07-28T18:32:00Z

Finally, I have Most Blue Skies running reasonably well on one of our servers. Here's the "live" world view:

... with some including current "bluest" sky here.... At the moment, the server struggles to run enough sky calculations to keep the simulation up-to-date. I think it runs about 10 per second. That's on a dual 2Ghz G5 Xserve. We currently have 500,000 places in the database to test which will take about 14 hours to run through. Lets say half of those are in the dark and half or the remainder are under cloud. That's still several hours worth of calculations. So I need to either learn Fortran and improve the SMARTS software for batch processing (not that likely), buy more servers or code some kind of Most Blue Skies @ Home system where the public or sponsors can donate some processing power to the project. Maybe I can find a Fortran to Java translator. That might be a way forward...

Java Concurrency in Practise - essential reading!

2008-07-22T23:07:00Z

Just finished reading Java Concurrency in Practise and I'd highly, highly recommend it. This book quickly scares you into thread-safety seriously! It contains a lot of information relating to Java's memory model that you're unlikely to pick up from the javadoc (without a lot of effort anyway) and is written really, really nicely. Definitely increased my appreciation for Java and for how well optimised the JVMs are nowadays.

mod_rails makes life easier

2008-04-25T13:12:00Z

Every six months there seems to be a new, preferred way to deploy a Rails application. Mongrel+modproxy has worked OK for me for a while, but it's a bit fiddly to set up and quite tricky to make robust across reboots or crashes. Even normal restarts of my mongrel clusters sometimes fails. Now there is modrails which seems to provide most of the advantages of a mongrel cluster with the simplicity of an apache module that spawns rails processes on demand. This is definitely becoming my preferred deployment method. I might even move rentability to mod_rails when I get the chance as it'll simplify our server configuration a lot.

Flowy page layouts

2008-02-18T22:48:00Z

For a long time we've wanted to get better looking property adverts onto Rentability. We thought about offering several layout templates that the owner could choose from. Then we thought about totally "floating" layouts where the user creates several "blocks" of content (text or photos) that are designed to flow down the page and be happy with different page widths and font sizes. The user doesn't have precise control over the placement of blocks. Instead, they have a bunch of blocks, of different dimensions that slot together down the page (from right to left and top to bottom) filling all available space. With fixed element widths (say a third or two thirds of the page) you can get a nice magazine/newspaper style look. Of course, a simple CSS float:left won't work all that well - you get a lot of holes where elements sort of clear:right as they wrap around down the page. So I tried to create this 'flowy' layout with javascript. The javascript measures each block, figures out how it slots into the layout, then positions it absolutely.

In the editing UI I've used scriptaculous to animate the reflowing of reordered blocks. This isn't always entirely intuitive - try the demo below - but then the whole approach is a compromise of sorts.

It's not beyond the realms of possibility that a layout will flow slightly differently on two different browsers. Actually, Sim's advert for the Pigeonnier lays out differently on IE than it does on Safari. Both looks great though! It's interesting how robust this technique can be.

Unless I've shown you a preview, you're probably wondering what the hell I'm going on about. Here's a demo page that you can play with (the changes you make are only visible by you).

Fix for missing authors problem with svnsync and trac

2008-02-15T22:12:00Z

<strike> Anyone pointing trac at a subversion repository built with svnsync might run into the same problem as me - trac shows commits but they show with author set to the username passed to the svnsync command and not the real author from the source repository. My best guess is that this happens because trac detects the checkin before synsync has copied over the revision properties (that includes changing the author). So to fix it, just add a post-revprop-change hook script (to the target repository) that tells trac to resync the revision concerned:

#!/bin/sh
REV="$2"
TRAC_ENV="/path/to/trac/environment"

/usr/bin/trac-admin "$TRAC_ENV" resync "$REV"

I also add the call to the contributed trac-post-commit-hook in there too (rather than putting it in post-commit):

/usr/bin/python /path/to/trac-post-commit-hook -p "$TRAC_ENV" -r "$REV"

</strike>

Update: 19/02/08:

After receiving a lot of duplicate emails from trac I realised that the above approach isn't quite good enough. svnsync updates revision properties more than once per commit (once for each property I guess) so you end up resyncing redundantly and, in my case, calling trac-post-commit-hook several times (resulting in duplicate ticket updates). So I decided not to use the subversion hook scripts and to do what I needed to do to trac directly after running svnsync. Here's the script I run with cron:

#!/usr/bin/env ruby

# This is script is run every 5 minutes. It syncs the svn repository onto the
# trac server. It also manually tells trac to resync each revision and calls
# the trac-post-commit-hook for each revision.

SVN_USER = "rails"
FROM = "svn://www.example.org/project"
TO = "file:///path/to/repos-mirror/"
TRAC_ENV = "/path/to/trac/environment"
POST_COMMIT_HOOK_SCRIPT = "/path/to/trac-post-commit-hook"

sync_output = `svnsync sync --non-interactive --username "#{SVN_USER}" "#{TO}" "#{FROM}"`

unless sync_output.empty?
  revisions = sync_output.scan(/Committed revision (\d+)/).flatten
  puts sync_output

  for rev in revisions
    puts "Telling trac about revision #{rev}..."
    puts `/usr/bin/trac-admin "#{TRAC_ENV}" resync "#{rev}"`
    puts `/usr/bin/python "#{POST_COMMIT_HOOK_SCRIPT}" -p "#{TRAC_ENV}" -r "#{rev}"`
  end
end

Trac 0.11 installation on Centos/Fedora

2008-02-14T14:56:00Z

How come every time I install trac I do it a different way? Mostly, I'm installing on Centos or Fedora these days. The RPMs available rpmforge only go up to trac 0.10. It seems sensible to go with the 0.11 beta for a new installation - the trac admin web UI is included by default, there are fewer dependencies and the default ticket workflow has changed a bit. After some fiddling about with the manual download, realising that trac.cgi is missing (maybe it's deprecated) I realised that installation is fairly straightforward if you get setuptools (some strange python "egg" installer) and use modpython rather than cgi/fcgi (it's more efficient and easier anyway).

So here are my steps so I don't forget them:

# As root
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py

easy_install Pygments
easy_install Genshi
easy_install Trac

trac-admin /srv/trac/myproject initenv
chown -R apache:apache /srv/trac/myproject

nano /etc/httpd.d/trac.conf

# And add something like:

<Location "/trac">
  SetHandler mod_python
  PythonInterpreter main_interpreter
  PythonHandler trac.web.modpython_frontend 
  PythonOption TracEnvParentDir /srv/trac
  PythonOption TracUriRoot /trac

  # The following gives basic HTTP authentication
  AuthType Basic
  AuthName "Trac"
  AuthUserFile /srv/users.htpasswd
  Require valid-user

  # ... which should be used with SSL:
  SSLRequireSSL
</Location>

service httpd restart

Brilliant anti-(blog)spam technique

2008-01-03T17:52:00Z

This works brilliantly!

Update: Well it worked perfectly for a couple of months. Now some spambots seem to cope with it (they check for non-visible form elements I guess). I might try this rails plugin.

transport.js with prototype 1.6

2008-01-03T17:17:00Z

Recently I've been working on embedded enquiry forms that Rentability users can place on their own website to take enquiries via their Rentability account. We wanted the person enquiring to remain on the owners homepage after submitting their enquiry (rather than be redirected to our site) so an AJAX submission was called for. Of course, when you try to make an ajax request from one domain to another (even if the code that is making the request came from the target domain), you find it doesn't work - most browsers veto the request for security reasons. The workaround is to use SCRIPT tags to emulate AJAX requests. Luckily, Thierry Schellenbach has written a plugin for Prototype to make Prototype AJAX requests use this SCRIPT tag technique.

Unfortunately, there are some major limitations here. For a start, you're obviously limited to using the GET method. In our case this is quite irritating as it's an "update" action that's being called (which is normally only accessible via a POST). Also, the request is going to be made with whatever request headers the browser decides to use. IE is typically braindead in this respect and will probably cause you pain if you have both JS and HTML responses for your action (as we do).

Also, the transport.js only works with prototype 1.5 out of the box. I fiddled for a while trying to fix it for 1.6 but I didn't feel much like wasting hours getting my head around how people hack OO into javascript. That stuff just freaks me out. So I just ended up copying and pasting the AJAX.Request constructor from prototype. Here you go:

    1 Ajax.Request.prototype = Object.extend(Ajax.Request.prototype, {
    2     initialize: function(url, options) {
    3         this.options = {
    4           method:       'post',
    5           asynchronous: true,
    6           contentType:  'application/x-www-form-urlencoded',
    7           encoding:     'UTF-8',
    8           parameters:   '',
    9           evalJSON:     true,
   10           evalJS:       true
   11         };
   12         Object.extend(this.options, options || { });
   13 
   14         this.options.method = this.options.method.toLowerCase();
   15 
   16         if (Object.isString(this.options.parameters))
   17           this.options.parameters = this.options.parameters.toQueryParams();
   18         else if (Object.isHash(this.options.parameters))
   19           this.options.parameters = this.options.parameters.toObject();
   20         
   21         this.transport = (!this.options.crossSite) ? Ajax.getTransport() : new scriptTransport;
   22         this.options.asynchronous = (!this.options.crossSite) ? this.options.asynchronous : false;
   23         //turns of the timed onLoad executer
   24         this.transport.respondToReadyState = this.respondToReadyState.bind(this);
   25         this.request(url);
   26         }
   27 });

Reuse existing partials in ActionMailer HTML emails

2007-11-18T13:58:00Z

This might be useful for anyone wanting to reuse action/controller templates/partials in HTML emails sent with ActionMailer. It causes templates/partials to be resolved relative to the 'views' directory rather than relative to the relevant mailer templates directory (which is the default).

module ActionMailer
  class Base
    # We have to go root relative here because of the change below
    def render_message(method_name, body)
      render :file => "#{mailer_name}/#{method_name}", :body => body
    end

    # This allows root relative partials to be located correctly
    def initialize_template_class(assigns)
      ActionView::Base.new(template_root, assigns, self)
    end
  end
end

SuperDuper and Un-FileVaulting

2007-09-30T14:09:00Z

SuperDuper is missing a key feature - the ability to read and backup the files in the current user's home directory even if the user is using Apple's FileVault feature. It should give two options: Option one should be to mount the spare image on the backup drive and do a smart update to it. Option two should be to write directly to the backup volume and turning off FileVault on the backup volume (presuming you're using FileVault because you carry your machine about with you during the day but you don't carry your backup drive).

SuperDuper can't do either of these things but you can sort of approximate option 2 by backing up in two phases to two different volumes. First backup the entire disk minus your home directory and your sparse image (stored somewhere like /Users/.tom/) then backup your home directory to another disk/partition. If you end up booting from the backup, obviously your home directory is going to be missing (or rather, it's backup up to another volume) so you'll want to create at least one non-FileVaulted account that you can log into and recreate your home directory. Actually, you could even alter the backup system so that it sees the home directory backup as your home directory. Then you don't have to do any copying/fixing to get up and running again (at least until you want to restore your internal drive). You could get SuperDuper to run a script after backup. Something like this:

nicl -raw /Volumes/SystemBackup/var/db/netinfo/local.nidb \
    -create /users/tom home /Volumes/HomeBackup
nicl -raw /Volumes/SystemBackup/var/db/netinfo/local.nidb \
    -delete /users/tom home home_loc

I tried this and it seems to work. The only odd thing that happened was that after rebooting my "HomeBackup" volume was set to "Ignore Permissions" which I had to then switch off in the Get Info window.

Simple java properties style rails localisation with Gibberish

2007-07-23T17:23:00Z

I had a session preparing the Rentability front-end for localization the other day. Luckily I found the Gibberish rails plugin which makes localizing the static text fairly painless - you can store the translated strings in yml files, and the syntax is very simple. I made a slight patch to gibberish so it'll print the key next to each string on the live app so that the human translator can see where each key is used on the running site:

module Gibberish
  module Localize
    alias_method :old_translate, :translate
    def translate(string, key, *args)
      str = old_translate(string, key, *args)
      "#{str} <span class=\"translation-key\">#{key}</span>"
    end
  end
end

You can put that in environment.rb (maybe if RAILS__ENV == 'development'). Here's some CSS:

span.translation-key {
  font-size: 11px;
  font-weight: normal;
  color: black;
  background-color: yellow;
  font-family: geneva;
  border: 1px solid #444;
  padding: 0 1px;
  text-transform: none;
  font-style: normal;
}

Perfect High Street

2007-06-26T18:15:00Z

Here's a nice animation from the idealists (or maybe complete fantasists) at Monocle magazine. I'd be happy if Woolworths came back to Cambridge...