Freelancing Gods

Funding Thinking Sphinx

2009-10-28T00:01:01+00:00

Update: I’ve now hit my target. If you want to donate more, I won’t turn you away, but perhaps you should send those funds to other worthy open source projects, or a local charity. A massive thank you to all who have pitched in to the pledgie, your generosity and support is amazing.

Over the past two years, Thinking Sphinx has grown massively – in lines of code, in the numbers of users, in complexity, in time required to support it. I’m regularly amazed and touched by the recommendations I see on Twitter, and the feedback I get in conversations. The fact that there’s been almost one hundred contributors is staggering.

It’s not all fun and games, though… there’s still plenty of features that can be added, and bugs to be fixed, and documentation to write. So, what I’d really like to do is spend November working close to full-time on just Thinking Sphinx. I have a long task list. All I need is a bit of financial help to cover living expenses.

I have an existing pledgie tied to the GitHub project, currently sitting on $600. If I can get another $2000, then I won’t have to worry at all about how I’m going to pay bills or rent for November. Even $1400 will make it viable for me, albeit maybe with some help from my savings.

If you or your workplace can make a donation, that would be very much appreciated. I’m happy to provide weekly updates on where things are at if people request it – but of course, watching the GitHub projects for Thinking Sphinx itself and the documentation site is the most reliable way to keep an eye on my progress.

I’m hoping to get Thinking Sphinx to a point where the documentation is by far the best place for support, and it’s only the really tricky problems (and bug reports) that end up in my inbox.

I want it to be a model Ruby library that doesn’t get in your way, is as fast as possible, and plays nicely with other libraries.

I want the testing suite to be rock-solid. I’ve been much better at writing tests first over the last six months, and using Cucumber has made the test suite so much more reliable, but there’s still some way to go.

This is not a rewrite – it’s polishing.

I’ve been toying with this idea for a while, and it’s time to have a stab at it. Hopefully you can provide some assistance to do this.

Better Gem Publishing with Gemcutter

2009-10-05T15:30:32+00:00

If you’re working with Ruby and have been paying attention to Twitter or RSS feeds, then you’ve probably heard of Gemcutter. If not, it’s the latest flavour for publishing gems, and I’m finding the simplicity of it a delight.

Its appearance is doubly useful, as since GitHub has moved to Rackspace, automated gem building from projects has been disabled, perhaps never to return.

Getting Started

If you’ve not clicked the link to Gemcutter yet, let’s run down how easy it is to get it set up on your machine.

sudo gem install gemcutter
gem tumble

That’s it. Any future gem installs will look at Gemcutter’s growing library.

This doesn’t replace RubyForge or GitHub in your sources list, but it does set Gemcutter as the top priority – which is fine, as it has almost all of RubyForge’s gems ready for you anyway.

Publishing

Firstly, get yourself an account, click that confirmation email link, then hunt down a gem you want to publish, and run the following command:

gem push my-awesome-gem-0.0.1.gem

If it’s your first time, you’ll be asked for your login details, and then the gem is online and ready for anyone to download it. No waiting, no forms, no pain.

When you’ve got a new version, just run that same command again, pointing to the new gem file:

gem push my-awesome-gem-0.1.0.gem

One command. No authentication prompts. Available for everyone straight away. Awesome.

Migrating

If you’ve already got gems on RubyForge that you’d like to take ownership of on Gemcutter, it’s another one-step process:

gem migrate my-legacy-gem

You’ll be prompted for your RubyForge account name and password, and then Gemcutter does the rest.

Pretty easy, hey?

My Gems

Over this past weekend, I made Gemcutter the definitive source for all of my gems:

Incoming Confusion

There’s been some discussion about whether Gemcutter should replace the gem hosting facilities provided by Rubyforge. This may or may not happen, but it is confirmed that Gemcutter will be moving to rubygems.org soon.

Everything will still work fine via the gemcutter.org address, though, so don’t let that hold you back from diving in head first.

Hat-tip

The talented Nick Quaranto has been working hard on this for a while, and it’s great to see the Ruby community embrace Gemcutter so quickly. Here’s hoping it becomes the defacto gem source for all Ruby projects.

script/nginx

2009-09-27T12:00:05+00:00

This morning I decided to get Nginx and Passenger set up in my local dev environment. I needed an easier way to test of Thinking Sphinx in such environments, but also, I find Nginx configuration syntax so much easier than Apache.

And of course, if I’ve got these components there, it would be great to use them to serve my development versions of rails applications, much like script/server. So I’ve got a script/nginx file that manages that as well. Sit tight, and let’s run through how to make this happen on your machine.

Be Prepared to Think

Firstly, a couple of notes on my development machine – I’m running Snow Leopard, and I compile libraries by source. No MacPorts, no custom versions of Ruby (yet). So, you may need to tweak the instructions to fit your own setup.

Installing Passenger

Before we get to Nginx, you’ll want the Passenger gem installed first.

sudo gem install passenger

You’ll also need to compile Passenger’s nginx module (keep an eye on the file path below – yours may be different):

cd /Library/Ruby/Gems/1.8/gems/passenger-2.2.5/ext/nginx
sudo rake nginx

Installing Nginx

Nginx requires the PCRE library, so that adds an extra step, but it’s nothing too complex. Jump into Terminal or your shell application of choice, create a directory to hold all the source files, and step through the following commands (initially sourced from instructions by Wincent Colaiuta):

curl -O \
  ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-7.9.tar.bz2
tar xjvf pcre-7.9.tar.bz2
cd pcre-7.9
./configure
make
make check
sudo make install

That should be PCRE taken care of – I didn’t have any issues on my machine, hopefully it’s the same for you. Next up: Nginx itself. Grab the source:

curl -O \
  http://sysoev.ru/nginx/nginx-0.7.62.tar.gz
tar zxvf nginx-0.7.62.tar.gz
cd nginx-0.7.62

Let’s pause for a second before we configure things.

Even though the focus is having Nginx working in a local user setting, not system-wide, I wanted the default file locations to be something approaching Unix/OS X standards, so I’ve gone a bit crazy with configuration flags. You may want to alter them to your own personal tastes:

./configure \
  --prefix=/usr/local/nginx \
  --add-module=/Library/Ruby/Gems/1.8/gems/passenger-2.2.5/ext/nginx \
  --with-http_ssl_module \
  --with-pcre \
  --sbin-path=/usr/sbin/nginx \
  --conf-path=/etc/nginx/nginx.conf \
  --pid-path=/var/nginx/nginx.pid \
  --lock-path=/var/nginx/nginx.lock \
  --error-log-path=/var/nginx/error.log \
  --http-log-path=/var/nginx/access.log

And with that slightly painful step out of the way, let’s compile and install:

make
sudo make install

And just to test that Nginx is happy, run the following command:

nginx -v

Do you see the version details? Great! (If you don’t, then review the last couple of steps – did anything go wrong? Do you have the passenger module path correct?)

Configuring for a Rails App

The penultimate section – let’s create a simple configuration file for Rails applications, which can be used by our script/nginx file. I store mine at /etc/nginx/rails.conf, but you can put yours wherever you like.

daemon off;

events {
  worker_connections  1024;
}

http {
  include /etc/nginx/mime.types;
  
  # Assuming path has been set to a Rails application
  access_log            log/nginx.access.log;
  
  client_body_temp_path tmp/nginx.client_body_temp;
  fastcgi_temp_path     tmp/nginx.client_body_temp;
  proxy_temp_path       tmp/nginx.proxy_temp;
  
  passenger_root /Library/Ruby/Gems/1.8/gems/passenger-2.2.5;
  passenger_ruby /usr/bin/ruby;
  
  server {
    listen      3000;
    server_name localhost;
    
    root              public;
    passenger_enabled on;
    rails_env         development;
  }
}

script/nginx

The final piece of the puzzle – the script/nginx file, for the Rails app of your choice:

#!/usr/bin/env bash
nginx -p `pwd`/ -c /etc/nginx/rails.conf \
  -g "error_log `pwd`/log/nginx.error.log; pid `pwd`/log/nginx.pid;";

Don’t forget to make it executable:

chmod +x script/nginx

If you run the script right now, you’ll see a warning that Nginx can’t write to the global error log, but that’s okay. Even with that message, it uses a local error log. I’ve granted full access to the global log just to avoid the message, but if you know a Better Way, I’d love to hear it.

sudo chmod 666 /var/nginx/error.log

Head on over to localhost:3000 – and, after Passenger’s warmed up, your Rails app should load. Success!

Known Limitations

The environment is hard-coded to development. If this is annoying, the easiest way around it is to create multiple versions of rails.conf, one per environment, and then use the appropriate one in your script/nginx file.
You can’t specify a custom port either. Patches welcome.
You won’t see the log output. Either tail log/development.log when necessary, or suggest a patch for script/nginx. I’d prefer the latter.

Beyond that, it should work smoothly. If I’m wrong, that’s what the comments form is for.

Also, you can find all of my config files, as well as other details of how I’ve set up my machine since installing Snow Leopard, on gist.github.com.

Rails Camps - Coming to a Country Near You

2009-07-14T01:00:51+00:00

This weekend, there’s going to be a Rails Camp. In October, there’s going to be a Rails Camp. Then in November, there’s going to be a Rails Camp. That in itself is pretty freaking cool. What’s even cooler is that they’re in Maine, England and Australia respectively.

Definition

If you’re not quite sure what Rails Camps are – they’re unconference style events, held away from cities, generally without internet, on a weekend from Friday to Monday. The venues are usually scout halls or similar, so the name is slightly inaccurate – most people don’t bring tents, but sleep in dorm rooms instead.

Also, they are events for Rubyists of all level of experience – and not just focused on Rails either. Anything related to Ruby and development in general is a welcome topic for discussion.

The weekends are made up of plenty of hacking, socialising, talks, and partying. Alcohol and guitar hero usually feature. A ton of fun ensues.

Rails Camp New England

A quick rundown in chronological order: first up, from the 17th to 20th of July, is Rails Camp New England. This will (as far as I know) be the first Rails Camp in North America. We’ll be up in the middle of Maine, at the MountainView House (a bit different from most Rails Camp venues) in Bryant Pond.

Unfortunately, if you want to come to this camp, we’re all sold out. Let me know anyway, just in case someone drops out (although it is late notice).

Rails Camp UK 2

Building on the success of last year’s first UK Rails Camp, a second one has been put together by Tom Crinson out in Margate, Kent.

If you’re anywhere in the UK, or even Europe, you really should be keeping the weekend of the 16th to 19th of October free. In fact, go book your spot right now.

Rails Camp Australia 6

Last on this list is the original Rails Camp, that started back in June 2007, run by the inimitable Ben Askins. We’re returning to Melbourne (the host of the second camp, in November 2007), but this time we’re down by the beach in Somers.

November 20th to 23rd are the dates for this, and going by the names of confirmed attendees, alongside what looks to be an fantastic venue, it’s going to rock just as much as the last five (and quite possibly even more). Feel like booking your place?

For all of these events, you should beg, borrow or steal to get your hands on a ticket. The energy, intelligence and passion of past camps has been amazing (which is why I do my best to spread the word), and they are a breath of fresh air compared to the staid and structured setup of RailsConf and most other technical conferences.

Thanks to John Barton, Max Muermann, and Jason Crane for the photos above.

FutureRuby and Californian Conflict

2009-07-12T23:02:26+00:00

Update: Jesse’s talk is now online should you want to watch it.

Before this all fades from my brain in the post-conference haze, I just wanted to post a few thoughts on the final session of the amazing FutureRuby conference in Toronto. Jesse Hirsh delivered an impassioned argument against what he labels the Imperial Californian Idealogy.

As someone who is most definitely left-of-centre, the bulk of this talk appealed to me. A call to action, highlights of the flaws of the capitalism, railing against the environmental destruction caused by the pursuit of wealth – it ticked the boxes.

There were a lot of references to the prominent place in history that San Francisco holds when it comes to mining, wars and weaponry, corporations and politics. I won’t go into those, because I’ll probably get it wrong. I have no bones to pick with that part of the talk, though.

It wasn’t immediately clear to me that he was attacking (some of) the ideals put forward by Stewart Brand, Kevin Kelly and Chris Anderson – at first, I thought they were the alternative movement to the old San Francisco elite that was initially described. All three names hold some credibility for me, so that was an interesting twist.

That said – and if we take the points on Anderson’s Free as accurate – then I’m happy to buy into at least some of the criticism, particularly around the push for acceptance of waste. I agree completely that the wasteful nature of people has got us into the current ecological mess. It promotes a very narrow, selfish view, instead of a more holistic approach. Indeed, holistic solutions was the end point Hirsh was driving at.

My main issue? The revolutionary, us-vs-them vibe. It felt implied that we are the elite, the creators, the visionaries. The ones who know best, the ideal internet citizens. There’s enough division in the world as it is. Not that I think Jesse is advocating such an approach, but that’s how the message came across to me. Granted, the Snowcrash references were lost on me, so that didn’t help.

Besides, revolutions are (more often than not) ineffective routes to change. Evolution is the road I much prefer to walk down.

Anyway, I’m off to the afterparty to discuss all this further with other attendees. If you weren’t there, then this probably doesn’t make much sense to you – watch the video, see if that helps.

Getting around in Cambodia

2009-07-11T22:46:29+00:00

This is the third of my guides to Cambodia.

The focus for this post is how you can get yourself from one side of the country to the other.

Between Cities

Unless you have your own private tourist guide, you really only have two options to get between cities – buses and taxis.

By Bus

Buses are the cheap-but-slow option – you could be looking at 6 or more hours between Battambang and Phnom Penh, and a ticket is going to set you back maybe $7 USD. Phnom Penh to Siem Reap is a slightly longer journey, and Siem Reap to Battambang is shorter (around 4 or 5 hours).

Sometimes buses break down, and sometimes they’ll make a few more toilet break stops than really necessary. They are easy though – just buy a ticket, and hop on the right bus. Don’t expect air conditioning or a quiet trip – it’s likely you’ll have to put up with loud Khmer pop for the entire trip.

By Taxi

Taxis, on the other hand, can require a bit more patience to sort out. It’s not a formalised system like in developed countries – if someone with a car wants to be a taxi driver, then they just hang around the vague taxi point in a city, and try to lure passengers into their car. There’s plenty of competitiveness between drivers, so you could try and bargain for a seat. Don’t expect all drivers to speak English though.

Taxi drivers will try to fill up the car with as many passengers as possible. I’ve been in cars with four Khmer adults in the back seat of the Camry (almost all taxis are Camrys), two in each of the front seats (yes, the driver shares his seat), and in one case, one girl in the boot (kept open, thankfully).

If you’d like to have a little privacy, you pay more. For Battambang to Phnom Penh, $20 is a common rate for a Westerner to have the passenger seat to themselves. $10 will probably get you a seat in the back with two or three others. Westerners are generally given more space, because they have the money to pay for it.

You can also hire out a whole car if you’ve got plenty of cash or a group of people – expect to pay around $50. For the other major routes, prices can be modified depending on the distance (I’d guess around $60-70 for Phnom Penh to Siem Reap).

While finding drivers is pretty easy (your hotel or hostel may even help you source a driver), there is a good chance you could be waiting around for the rest of the taxi to fill up. Khmers are early risers, so as the day wears on, there’s less potential travellers. In some cases, I’ve waited over an hour before leaving. Of course, if you offer more money, then the driver will be happier to make the journey with less passengers.

So, for all this hard work, what are the advantages? Air-conditioning (indeed, you might actually be too cold) is the big one. The journey is much faster, too (I once did the Battambang to Phnom Penh trip in under four hours, but four to five hours is the usual length of time). You should still expect to be blasted with Khmer music (although some drivers will graciously accommodate and turn the radio down or off).

Either way, the roads aren’t fancy – and sometimes not even sealed – so don’t hope for a super-smooth ride.

The Boat

The one other major transport option is only for travelling between Battambang and Siem Reap, and even then, only during the wet season: the boat. There’s two different boats (I’m not sure if they’re run by separate companies), and one is apparently comfortable.

I say apparently, because I went on the other boat, which is a glorified gondola, holding about 20 people sitting on benches facing each other, with their backs to the water. The sides of this boat are low enough that you will get water flowing over the edge every now and then – nothing dangerous, but your bags may get a little wet.

Given my description thus far, it sounds like a horrible trip – and while it wasn’t super comfortable, and did take seven hours, you do get some great views of the Cambodian countryside. The boat skirts the western edge of the massive Tonlé Sap lake, and also motors through some floating villages. It’s certainly a unique experience, and I’m glad I’ve done it once.

In Cities

Unlike most cities, taxis (in their car form) aren’t used much at all in Cambodia for travelling within a city (although there is some normal taxis in Phnom Penh). What is used is a bit more old-school – tuk-tuks and motos. Motos are scaled down motorbikes – not quite as powerful, and not quite as large. Very similar to motorized scooters, if a little bit more motorbike-like. Tuktuks are motos with a little carriage on the back, which can fit four Westerners reasonably comfortably.

Motos

If you need to get somewhere by yourself, I’d recommend jumping on the back of a moto – there’s usually plenty of moto-dops (moto-taxis) around tourist areas and major intersections. It can be a little scary at first, but the drivers know what they’re doing, and it’s much faster than a tuk-tuk. The more you travel this way, the more comfortable it gets.

Prices vary between cities – I could generally get from one end of Battambang to the other for fifty cents (two thousand riel). For the same distance, a dollar is probably the least you’ll get away with in Phnom Penh (and it’s also a much larger city). Sometimes the driver will request a price, sometimes you just offer what you think it is worth – and bargaining is fine, although you may not get far due to the language barrier.

Much like the ride itself, estimating what to pay for a trip becomes comfortable over time.

Tuk-tuks

If there’s a group of you – and particularly if you have luggage – then a tuk-tuk is the better option. You’ve got a bit more space, and it’s definitely safer than a moto – but the caveat is that it’s more expensive, and it’s slower. I didn’t travel by tuk-tuk too much, so I’m not the best source for prices, but I’d expect to pay a dollar or two to get across Battambang, maybe a little more if there was a group of us.

Both moto and tuk-tuk drivers are often happy to wait for you for the return journey – if that’s what you want, don’t feel obliged to pay until you’re finished with their services. You should pay them a bit extra for waiting, though.

A simple example: If I was visiting the Russian Markets in Phnom Penh, and then heading back to my hotel, I’d probably pay three or four dollars all up for a moto-dop. For a similar visit to the markets in Battambang, two dollars should suffice. Of course, prices may have gone up since my last visit, and I can speak a little Khmer, so I’m more comfortable with bargaining. Don’t be surprised if you’re asked for double those amounts, or more.

Directions

When you’re stuck with the language barrier, it can be a bit hard to describe where you want to go (particularly in a city you’ve never been to before). Regular moto-dops and tuk-tuk drivers will know all the common tourist places, so generally you’re going to be fine.

If the driver doesn’t know where to go – and sometimes they pretend they do know – they’ll usually ask other Khmer for help. However, often the drivers hanging around tourist areas are experienced enough (particularly in Phnom Penh and Siem Reap) that you won’t have problems.

Touts

You definitely won’t struggle to find moto-dops or tuk-tuks – there’s always a few outside each hotel, and they’re quick to offer their services (read: hassle you incessantly). Sometimes, that’s fine, because you do need transport. Other times, it’s easier to just say ‘no’ (A’tay) or ‘I don’t need’ (A’trega).

You’ll also find some drivers ask if you want to visit tourist places, and offer to take you there (and if you’re busy, give you their number or ask when you’re free the following day). This can be handy – it’s nice to have a familiar driver taking you around – but if you don’t want to commit to any plans, make that clear, don’t just say yes to avoid confrontation.

Hiring Bikes

If you’d like to get around without a driver, it’s not hard to find places that hire out motos and bicycles (and perhaps dirt bikes for the out-of-town trips). Cambodia is quite flat for the most part, so it’s not hard to peddle your way through cities, but Phnom Penh’s traffic (much more so than Battambang or Siem Reap) heightens the challenge. You’ll also need to worry about locking the bikes up, so it may not be worth the hassle.

Border Crossing

Finally, if you’re planning on crossing the border overland from Bangkok, via Aranyaprathet (on the Thai side of the border) and Poipet (the Cambodian equivalent), it’s a bit of an adventure. I’ve done it a couple of times – the easiest way is by taking a bus that goes all the way to Siem Reap or Battambang (well, it’s actually two buses, since they don’t cross the border, but guides will look after you somewhat).

An alternative is to take a non-tourist-focused bus from Bangkok to the border, and then find a taxi once you cross over to Cambodia. This is more challenging – there’s some scam buses and taxis that charge a lot more in Poipet, so I’d recommend the bus.

If you’re going in the other direction though, taking a taxi to Poipet is easy enough, and then you can get a tuk-tuk to the bus stop in Aranyaprathet. No matter which direction you’re going in, Tales of Asia’s guide is essential reading.

That’s it for my generic Cambodian posts – the last two of this series will focus on dining and activities in the town of Battambang (as that’s where I spent the vast majority of my time).

Thanks to James Healy for the moto photos above.

PublicSphere: Open Government

2009-07-07T04:54:51+00:00

I’m a bit late to blogging about this (and I realise there’s plenty of other blog posts I should have written over the last few months), but a couple of weeks ago, the ACT Senator Kate Lundy and her advisor Pia Waugh ran their second Public Sphere event. I didn’t really hear of the first one – focused on the national broadband network – until after it happened. The second’s topic was open government, a topic I’m passionate about, and so I marked it in my calendar.

Of course, the event took place in Canberra, and I’m currently experimenting with a location-independant life at the moment, and was in Boston, Massachusetts at the time. However, there was a live video feed for the talks, and the twitter hashtag #publicsphere was massively popular. I was able to listen in for a good part of the day, and throw around some comments with others paying attention.

Reflection

There was a lot of talking. I probably caught about half the talks, and was feeling pretty braindead when I called it a night. I imagine those there would have found it a little tough, particularly with the addition of corridor discussions.

It was great to see not only Senator Lundy speaking, but Lindsay Tanner and Joe Ludwig as well – and generally saying the right things. By all appearances, they seem to understand the need for openness and transparency.

It will be interesting to see how all this talk translates into action though.

The Taskforce

Not that there isn’t action happening, mind you – a good portion of the talks were about what people were already doing. The big announcement from the event was the creation of a Government 2.0 Taskforce, which seems to be made up of some smart people. Mind you, their banner competition seems to be a small token towards the having an open and collaborative dialog.

I’m also hoping they can sidestep the bureacracy that so often ties governments down, and get things done fast and effectively. Given they’ve only got six months to make things happen, there’s a decent chance.

Disconnect

The event gave me some hope where little has been in the past. However, this drive towards open data and transparency doesn’t gel too nicely with the approach of Senator Conroy, who is the Minister for Communication. Why wasn’t he present?

That said, the proposed filter legislation sticks out like a sore thumb, so it’s fair to say he wouldn’t have received a particularly warm welcome. Something needs to change, though – personally, I’d love to see Kate Lundy take over his portfolio, but I’m not holding my breath.

Also, Lindsay Tanner really seemed to have his head around the open government space, and it sounded like that’s been the case for a while – so why are we only seeing actions like this now?

From a Distance

Finally, a few notes on how I found interacting with the event from afar.

Having a video feed is fantastic – far better than purely relying on live-blogging or Twitter
The twitter stream is great at providing a picture for others’ takes on what is being said.
Unsurprisingly, the timezone made it a challenge – I missed out on most of the afternoon sessions (around 2AM local time).
I also missed out on the informal discussions, in the breaks and corridors, which is where I feel a lot of the value usually is in conferences and unconferences.

I’m definitely looking forward to seeing both recommendations and actions from the Taskforce, as well as future Public Sphere events.