Rails in the Country - Home

Prangz.com

2009-03-24T02:58:00Z

Yes, another Friday night coding spree.

I was itching to write an app that used real-time, location specific data and so http://prangz.com was born.

The whole app is 43 lines of code. It’s rails 2.3 and the scraper checks the RTA website every 2 minutes for new data.

Next up is an iPhone app, using the new 3.0 push capabilities. Apply some rules to the firehose so people only get what they want in the way of location and time. “Send me incidents between home and work after lunch” would be a common use I reckon.

I’ve emailed the RTA to alert them to it, let’s see how long I last.

Tweetbeer.com - Directors Cut

2008-07-27T00:41:00Z

Opening Credits

Here we see Matta doing the dishes on the night of Friday the 25th of July, 2008. As the music is streaming he pauses. He just thought up tweetbeer.com. He goes straight into the living room to IM Lachie, and so it began.

tweetbeer.com

At around 12 midnight of last night Lachie Cox and I launched a fun little app we wrote over the previous day, tweetbeer.com. Here’s some little tidbits of info

Rails 2.1
Mysql DB
Haml for layout
97 Lines of code
Approx 6 hours of dev time
git was used as our collabaration tool, worked so well
All frontend CSS and the funky-ass logo done by Lachie
Tweet parser uses search.twitter.com called via a cron every minute and run through script/runner

So from idea to launch look about 18 hours of real time with about 6 hours of actual coding. Simple ideas are fun. If you could come up with a compelling way to monetize a simple idea like this, that’d be even cooler.

Rails Camp, the Third

2008-06-24T22:53:00Z

Here we are, a few days after the Third Rails Camp in Australia.

This was my first true rails camp, I attended a day of the first one but due to having a 2 week old baby 4 days/ 3 nights was out of the question. When the 2nd one in Melbourne was on, I was in Fiji with my family, I can’t complain too loudly.

Rails Camp was just awesome. The vibe was electric. It was great putting names to faces with the guys from the #roro IRC channel that I spend alot of time in. We asked people to blog about Rails Camp so we could get the word out, here is what I hacked on:

Twetter

Whilst shivering my arse off in bed on Friday night it hit me – how hard would it be to make tiwtter.com resolve to my laptop and watch the logs for the URLs? The answer was, not hard at all. So first thing on Saturday morning I had twitteriffic running locally. It took 50 lines of code in a rails 2.1 app. Lachie had an example of the XML that the API returned and with a little help from Max M and his internet enabled CrackBerry we were able to read the twitter API docs and get the other bits and pieces. Once Lincoln woke up I had twitter.com resolving to a shiny new VM on Bigguns (the awesome server) and twitter was alive! There was no friending, everyone got the same stream. I did implement replies but left out direct messages. I think it was a hit.

Merb Caching

I met Dan Neighman for the first time in real life. We speak alot on IRC and hang our in #geekdads. He was working on a new caching mechanism for Merb and came to seek me out as the resident Caching Guru. I was flattered. We spent a good few hours over the weekend fleshing out the solutions to problems that I currently have solved in rails in a hacky, need to use cron kind of way. The outcome was background-tasks. Dan wrote it up here. This is Uber exciting and I really want to start Merb Hacking.

Managing Expectations

I decided to give a talk on Managing Expectations. In a previous life this was what our software did but in reality we all have to do it each day. I pegged it as a discussion rather than a talk and that was exactly how it panned out. I reckon there might have been 20 guys all sitting around relating stories about their experiences and how they dealt with them. I think it’s safe to say that the conclusion was, politics aside, that being open and transparent is the safest way forward, especially if you’re a one-man-band or a small shop. Clients/Bosses are generally more willing to co-operate if you play it straight. The talk was actually interrupted by lunch but quite a large proportion actually kept talking whilst eating. I think this was a great outcome for a non-tech talk at a very geeky weekend.

So, to all you guys that read this and went, blog it up people! Get it out there. Let’s start a * Camp movement.

—Matta

Going Dark. Again

2008-05-05T21:49:00Z

UPDATE: I reckon this article has nailed it

This weekend was the Anzac weekend, a time for reflection and remembering the things our Grand Fathers and Great Grand Fathers (and possibly Mothers) did for us and our way of life. I miss you Pa.

I had the opportunity on Friday night to sit down with some code, uninterrupted. No internet, therefore no Email, IM, IRC, Twitter, RSS. It was grand.

It started me thinking, how the hell can us “knowledge” (I hate that term) workers balance being as productive as you know you can with keeping in touch? The vast majority of my friends are online most of the time, at work, at home and even in between sometimes (Hi Lachlan!). I like being in the loop, I like knowing what’s going on. Twitter has made that a process alot easier than it was 12 months ago.

So I have decided that my work day is now going to consist mostly of “off-line” time. 9 – Lunch will be offline, that is, working without distractions. No Email, IRC, IM, Twitter. At Lunch time i’ll surface again for a little while, probably an hour tops. After lunch i’ll hook back in. After the kids are in bed I may come back on, maybe not. Dunno yet.

I tend to embrace things half-heartedly, this is one thing i’m going to try and stick to. If you need me urgently, SMS or Phone is where it’s at.

The other side effect I hope this will address is my new level of bored-ness with the interwebs, seriously, I can’t find anything interesting out there atm.

Apologies for the above brain dump, upon re-reading it wasn’t very well thought out. Oh Well.

Matta

Fluid.app

2007-12-17T22:21:00Z

Have been using Fluid for the past week.

It’s an OS X (Leopard only) app that lets you create standalone cocoa based web apps, basically a wrapper for web kit.

I’ve got one for gmail and unfuddle setup. It’s nice to have them survive browser crashes normally due to unruley javascript.

UPDATE: Here’s a sexy big icon the UF guys sent me

Highly resposive Unfuddlers

2007-12-06T08:23:00Z

To follow on from my last post, I have spent the day to-ing and fro-ing with the Unfuddle.com guys and I have come up with an SVN hook for non-UF-hosted svn repos.

Here’s for code for anyone interested

Move over lighthouse, Unfuddle it is

2007-11-30T02:19:00Z

So, I blogged earlier about Lighthouse and how much I was in love it with. It has all the features I like but unfortunately for me, it missed a few critical ones my Boss wanted.

Enter Unfuddle

We’ve been using this for a few months now and it works a charm. It’s a rails app and is run buy some very passionate dudes in Hawaii.

The feature that really stands out for us is the work flow for tickets. See image below.

Boss opens it, I accept it, I resolve it, boss closes it.

Boss can reopen it if need be. So easy for me to see what tickets are left to work on in this release, also what tickets need to be verified by him before I can push the release live.

Anyone who is dealing with a large ongoing site needs a decent bug tracker, this one is up there with the best i’ve used, both OSS and commercial.

Print this out, give it to your boss

2007-11-27T01:55:00Z

I don’t actually have to, he sent me the link.

This made me a very happy programmer.

Wide vs. Deep

Going Dark

2007-10-17T01:25:00Z

Not sure if it’s just me, but lately the constant barrage of IM/IRC/Twitter/Email has been making me feel less productive than I know I can be so for the majority of the day I have been Going Dark. That is, turn comms off and the music up.

I don’t go completely off net as I need my other brain to remind me of things I constantly forget.

I miss my IRC buddies from #roro on freenode.net heaps, they are a constant source of knowledge and entertainment but they are pretty good at posting to the roro tumblr, so that keeps me in the loop mostly.

I actually jump on IM and IRC once or twice during the day, mostly at lunch, or in a lull between pushing out code.

I’d like to hear how everyone else deals with the “always on” side of things?

When Testing Caching ...

2007-10-15T06:34:00Z

Don’t forget to set:

config.action_controller.perform_caching = true

in test.rb in environments. You’ll wonder why nothing is working until you do.

-33.702635,151.099434 : Hornsby

2007-09-17T11:21:00Z

So what does a north western Sydney suburb have to do with rails? Not very much, but it has lent it’s name to a very very cool app written by Lachie Cox.

The main aim of Hornsby is to get rid of the dreaded rails fixtures. As a newcomer to Rails about a year ago, testing was a new and exciting thing for me. It took a bit of discipline but I got into it. Then I got out of it. Fixtures where p*ssing me off, badly. They were a nightmare to maintain and actually made me write worse code than normal due to the fact I didn’t want to rewrite all my fixtures.

I then started using Rspec, I love writing specs but fixtures were still making me swear at my MBP.

Hornsby makes me an extremely happy Rails programmer. Here is Lachie’s description.

Here’s my code. It’s an example of setting up a “scenario” that I used for testing my Equipment

scenario :equipment do
  @manufacturer = EquipmentManufacturer.create!(:name => "Ben Hogan")
  @category = EquipmentCategory.create!(:title =>"Fairway Woods")
  @equipment = EquipmentProduct.new(:name => "Edge CFT Ti  Hybrid",:equipment_manufacturer_id => @manufacturer.id,:equipment_category_id => @category.id)
  @equipment.save(false)
end

Here is the bit from Rspec test that loads it up.

describe EquipmentController do

  before(:each) do
    hornsby_scenario :equipment
  end
   ...
end

You’ll notice I hand in “false” to the save method, this allows us to bypass all the validations and setup just the bare minimum to pass tests. This is helpful sometimes but be aware it can lead to other weird stuff than you might need to be aware of, assuming your validations are setup for a reason. Another nice side effect of Hornsby scenarios is the fact that if your DB tables are not the rails default ones you don’t have to jump though hoops to get your Db populated.

This software has made my testing life a joy yet again, knowing the exact state of your DB when testing is precisely what was missing from fixtures.

Here’s those links again:

Lachie’s write up

Source Code

Out of Bandwidth Cache Expiry

2007-08-28T21:56:00Z

So, you’ve been merrily caching away and using the cache plugin I wrote and you’re still getting some heat about your site sometimes “hanging”. I reckon the problem might be that the person complaining is getting stuck behind a slow running process that is re-caching a page on your site. So what to do?

What I’ve done is started an extra mongrel in my pack but not included it in the standard mongrel pack, my (cut down) mongrel config looks like this:

--- 
port: "2222"
environment: production
address: 127.0.0.1
pid_file: log/mongrel.pid
servers: 7

As you can see there is 7 mongrels there. Next, setup a second cluster that contains only your 7th mongrel

123	<Proxy balancer://expirecluster> BalancerMember http://127.0.0.1:2228</Proxy>

Next, tell apache to redirect any of your magic expiration URLs to that cluster

12	RewriteCond %{QUERY_STRING} magic_url_expire_param=1$ RewriteRule ^/(.*)$ balancer://expirecluster%{REQUEST_URI} [P,QSA,L]

Restart apache and you’re done. Any cron triggered expire requests will now go through the out of bandwith mongrel and not slow down visitors to you site.

This method can actually be used to segregate any URLs off to different clusters, I have a project coming up that will have a lot of slow processes that rely on an external SOAP service running. They’ll all be farmed off to a cluster running on a totally different machine with trimmed down, DB free mongrels.

Seesaw, restarting your mongrels with zero downtime

2007-08-21T12:37:00Z

Last weekend was the first Sydney Rails Group Hax Day, held by our very own Lachie. What a day! about 10 rails geeks showed up including a few guys we had either never met or only at the previous meet-up.

Max and I started the day talking about what we’d like to see in a web-based app that managed other rails apps. In discussing this I mentioned that over that iseekgolf.com we’re running getting a serious amount of traffic and a lot of that is e-commerce related, i.e stuff that would really screw me up if the mongrel is was being executed on was killed half way though a process. This led to me suggesting that it’d be great if you could restart mongrels bit by bit.

After about 30 mins of talking it though we decided that the best way was to split the mongrel pack down the middle and bounce each half separately while the other half handled the load. This would potentially slow the site down however it shouldn’t shut the site down entirely.

What we came up with was Seesaw. This does precisely what we needed including configuring the web server on the fly.

This morning I installed it live at iseekgolf.com and after a few config tweaks it was working a treat. It brought a huge smile to my face as the logs kept rolling buy as the mongrels were restarting.

One gotcha is that by default Seesaw names your mongrel cluster “mongrel_pack”, be sure to replace all instances of your mongrel cluster name in your apache config file to this new name, especially the SSL one ;)

Hopefully some people may find a use for this plugin.

We’ll be doing some minor changes to it over the coming days, including some definable pauses in between the cluster restarts and apache redirecting traffic to them to give them time to warm up.

Max has written up an awesome blog entry, with pretty pictures and all!

Lighthouse App, Svn and you

2007-08-12T22:56:00Z

Last week we decided that our ad-hoc issue tracking system wasn’t good enough for the serious development we are doing now. We tried Trac, Basecamp and iGTD with email submission. None of them really impressed us for various reasons. Trac was the best of the bunch for bug tracking but never really let us spread our wings. Basecamp is a great app for what it is meant to do but not the best for bug tracking. I setup iGTD to accept tasks from the boss via email, the problem then was closing the loop, when I was done it was up to me to let the boss know.

Enter Lighthouse

What a brilliant piece of software, it’s written by the Active Reload guys who also wrote Mephisto the blogging software this blog is written in. The brilliance of this software comes of it’s simplicity. It does the job I want it to do almost perfectly.

It doesn’t make any assumptions about things like Bug Priority and such, however it does have a standard tagging system and you can save searches into “bins” that are available on the bug searching screen, we have settled on, low,normal,high,highest and URGENT. See below.

I also implemented the SVN integration, the FAQ can be found here Now all I do is put a proper comment in the SVN commit that explains the changes and has the lighthouse syntax in it, an example might be “Changes to log form [#15 state:resolved]”. A commit with that comment in it closes the ticket as well, brilliant.

So the aim of this post was really just to send props to Rick and the team at Active Reload, it’s great to see an app by the geeks, for the geeks.

The Caching Gap(tm) ... mixin' it with Cron

2007-08-10T08:28:00Z

So the plugin i mentioned in my last post is alive and kicking, it has totally solved the problem we were having.

A new problem arose today, now that the pages are begin re-cached as part of the sweeping process, it was taking ages. Ages being up to 60 seconds to re-cache them all, also ages with the boss sitting there waiting for thing to happen. Not so good from a Usability perspective.

Again, I was throwing the idea of using DRB around on IRC and Lachie Cox suggested good old cron. We are already using cron to expire the front page on the hour to keep the info fresh so I thought i’d expand it a little.

I created a new model called CacheQueue that had two columns, cache_type and cache_data. The two types of cache I hose now are the regular esxpressions that get handed into an expire_fragment method, or the urls that get the special param handed in and get re-cached up.

The sweeper now looks like this

## The  Sweeper
class TournamentSweeper < ActionController::Caching::Sweeper
  observe ...models...

  def after_save(record)
      ...  
      CacheQueue.create({:cache_type => "url", :cache_data => '/?special_param_to_hose_cache=1'})
      CacheQueue.create({:cache_type => "regex", :cache_data => "tournaments/#{tournament.id}-*"})
      ...
  end

So we’re creating lots of entires in the cache_queues table, all having an expired = 0 column by default.

The next step was to write a method that could be called from a rake task that expires the caches, here it is

## The  CacheQueue Class
class CacheQueue < ActiveRecord::Base
  def self.expire
    begin
      require 'action_controller/integration'
      # wite a disc lock
      lockfile = RAILS_ROOT + "/log/cache.expire"
      if File.exists? lockfile
        $stderr.puts "Locked since " + File.open(lockfile,"r").atime.to_s
        return
      end
      FileUtils.touch lockfile
      caches_to_be_hosed = CacheQueue.find(:all,:conditions => {:expired => false}, :order => "cache_type DESC, id ASC")
      hosed_url_caches = []
      hosed_regex_caches = []
      sess = ActionController::Integration::Session.new
      sess.host! LIVE_HOST
      c = ActionController::Base.new 
      skipped = 0
      caches_to_be_hosed.each do |cache|
        # update the db
        cache.expired = true
        cache.save!
        case cache.cache_type
          when "url"
            if hosed_url_caches.include? cache.cache_data
              skipped += 1
              next 
            end
            sess.get cache.cache_data
            hosed_url_caches << cache.cache_data
          when "regex"
            if hosed_regex_caches.include? cache.cache_data
              skipped += 1
              next
            end
            c.expire_fragment(Regexp.new(cache.cache_data))
            hosed_regex_caches << cache.cache_data
        end
      end
      FileUtils.rm lockfile
    rescue
      FileUtils.rm lockfile
    end
    $stderr.puts ((hosed_regex_caches.size + hosed_url_caches.size).to_s + " caches removed") if (hosed_regex_caches.size + hosed_url_caches.size > 0)
    $stderr.puts skipped.to_s + " caches skipped" if skipped > 0
  end
end

## The  Rake task
namespace :cache do
  task :expire =>  :environment do
    CacheQueue.expire
  end
end

This write out to $stderr, when it actually does something it send me an email. It also implements locking so it doesn’t run over itself.

One of the cool parts of this is that it’ll actually skip over a cache if it already hosed it, so if in the space of the 4 minutes when it was run last the boss had edited the same article twice, which would schedule the re-caching of the front page and other busy pages, this method would only do each one once, your machine will thank you.