This all changed for a project that has been taking up all of my spare time lately, Geomium, a location-based communication tool. MySQL’s geospacial support is almost completely non-existant. PostgreSQL, on the other had, when combined with the PostGIS extension, has fantastic support. So PostgreSQL was what the project needed. I thought it’d be fairly easy to transition from MySQL to PostgresSQL but the command line clients of these two RDBMSes are very different. Here are some of the things I’ve learnt.

A few other things to note: PostgreSQL only supports single quotes in queries not double quotes, so

SELECT * FROM table WHERE column = "value"

would need to be rewritten as

SELECT * FROM table WHERE column = 'value'

PostgreSQL also defaults to being case-sensitive, whereas MySQL defaults to case-insensitive, so you might want to modify your queries to something like

SELECT * FROM table WHERE lower(column) = 'value'

although this can prevent an index on “column” from being used.

At first I thought PostgreSQL didn’t support some of the handy MySQL date functions (such as NOW() – INTERVAL 1 DAYS), but it turns out that it does support them, and more. It’s just fussier about the format. For details see the datetime documentation.

The Art of Computer Programming
TAOCP is mentioned throughout Coders at Work. Some of the developers had read it all the way through, and really got stuck into the maths. Others, such as Doug Crockford had only used the book as a reference. Almost everybody agreed that this book should be on your shelf though.

Programming Pearls
A book about good programming practice with examples in C, this classic has stood the test of time.

Beautiful Code
The tagline for this book is “Leading Programmers Explain How They Think”. The book contains lots of example code, and descriptions and commentary from the code’s author. While this book is only recommended once in Coders at Work, almost all of the interviewees stress the importance of reading code, and this book can help with that.

Structure and Interpretation of Computer Programs
SICP is recommended several times in the book. It’s supposed to be a very in depth book, it looks at programming from a functional perspective, using Scheme. Zawinski says despite being “Lispy”, it provides a great introduction to programming without teaching a language.

Purely Functional Data Structures
Describes data structures from a functional programming point of view, rather than the more common imperative perspective (such as Java and C++ data structure books).

Higher Order Perl
Brad Fitzpatrick recommends this book, which is about functional programming techniques in Perl. The reviews on Amazon seem to suggest this this is an excellent programming book, and should even be interesting for non-Perl developers.

Code Complete
Jamie Zawinski describes this as “the book you wish your idiot coworker had read”. It goes through the whole development lifecyle, including design, desbugging, testing and refactoring.

The Practice of Programming
Co-authored by one of the creators of the C programming language, this book aims to teach you good programming practices to help you write faster and more maintainable code.

The list above is just a sample of some of the books recommended in Coders at Work. The Joshua Block interview also mentions lots of interesting Computer Science papers.

The interviews are really great. Most aren’t particularly technical, but it is interesting to read about how these well-known programmers work, and to get some insight into some of the projects they’ve worked on. Coders at Work is well worth a read.

Clojure

Clojure is a Lisp dialect for the JVM. I’ve done some reading on Clojure and had a play with the REPL, so this year I’m finally going to get round to writing some code with it. From what I’ve seen so far it looks quite powerful, and the integration with the Java libraries seems to work nicely.

Factor

Factor is a concatenative programming language, meaning it uses a stack instead of named variables to pass data around. Seems like a strange concept to me. This will definitely be an interesting one!

Go

Google announed their Go language in late 2009, describing it as a cross between C++ and Python. It was co-created by by Ken Thompson, who also co-created Unix and worked on the C programming language.

Haskell

Haskell is a purely functional strongly typed language. It also uses lazy evaluation. It’s got a bit of a reputation for being difficult, but I’m hoping I’ll at least be able to pick up some of the main concepts and put a few small programs together.

Erlang

Erlang is a concurrent programming language designed for fault tolerant real time applications. I’m looking forward to learning how Erlang deals with concurrency.

Scheme

I did some Scheme, a Lisp dialect, while at university. I haven’t really touched it since though, so I’m planning to revisit it and maybe dig a little deeper. I’m thinking about trying The Little Schemer book, or perhaps the online fixnum days tutorial.

Fantom

Previously called Fan, Fantom claims to be portable across the JVM, .NET CLR, and JavaScript in the browser! It’s on object oriented language with C like syntax, so there shouldn’t be a huge learning curve here. It seems to have some very interesting features though, so I’m looking forward to trying it out.

Scala

Another JVM language, Scala is a strongly typed language that “smoothly integrates features of object-oriented and functional languages”.

OCaml

Objective Caml is a statically typed, object-oriented and functional. I’m sure I’ll pick up some new techniques and ideas from this language.

Ruby

Ruby is a scripting language similar in many ways to Python. I’ve never looked at it in any detail before because I always thought it was too similar to Python. I’d like to learn it mainly to give the Rails web framework a try and to see how it compares to Django.

Lua

A small and portable scripting language, Lua is very popular in the game programming world where it is often used for the scripting of complex 3D engines.

Prolog

I did a little bit of Prolog at University and I remember being blown away by it. It’s a declarative logic programming language, often used in the field of AI.

It’s going to be a tough challenge, learning 12 new languages in 12 months. But hopefully this time next year I’ll have picked up some interesting techniques, and have a few extra tools in my programmer’s toolbox. I’ll be blogging about my progress throughout the year, posting my thoughts on these languages along with any code that I come up with. If you’re interested in the updates then you can subscribe to get them automatically. If you’ve got any comments about any of the languages or about the challenge in general then I’d love to hear them!

I have been using Python for quite a few years, but mostly for writing one off sysadmin scripts, command line utilities, and of course PyRadio. Most of my web development work has been done with PHP. The language gets a lot of bad press, some deserved and some not so much. I’ve had my own gripes, but all-in-all I’ve been fairly happy with PHP.

Several months ago, though, I thought I’d give Django a try, a python based web framework. I was completely blown away!  Compared to the PHP frameworks I’d worked with, such as Cake, it was just so much more of a pleasure to work with. So here are 8 reasons why you should give Django a try yourself if you haven’t already. You won’t be disappointed!

1. Great Documentation

The Django documentation is well written, extremely comprehensive, and up to date. The official documentation contains details API references, loads of relevant examples, and tutorials for those getting started. If that isn’t enough there’s also a whole book that’s available for free online.

2. It’s Python

The fact that it’s Python is a huge plus point for me. It’s a great language that doesn’t suffer from many of the well documented inconsistencies that PHP does, and includes some nice features such as decorators and first-class functions. Going back to PHP you soon start to miss the little things, such as the ability to assign multiple values at once, and the simplicity of slicing lists.

3. The ORM

Django’s object relational mapper completely abstracts away the database, meaning you don’t need to worry about your database schema or constructing SQL queries. If you’re using to writing SQL queries then the QuerySet API takes a little getting used to, but it’s really worth the effort. Projects like South make the ORM even more powerful, allowing you to make schema changes and data migrations automatically.

4. Built in Development Server

Where PHP really shines is on its ease of deployment. Setting up a local development server can be a bit of a pain though, especially if you’re working on several different sites. Django comes with a built in development server though, so you can be up and running within minutes! From your project’s root directory you just do

./manage.py runserver

and access your django website from http://localhost:8000 – awesome!

5. The Admin Interface

Django’s built-in admin interface is practically a full blown CMS, allowing you to add, delete or update your data. It’s pretty much all automatic, but it’s also fairly configuration. See the documentation to see what it can do!

6. Reusable Applications

Django projects are broken up into “applications”, and there are lots of existing reusable applications that you can use for your own projects, such as those for user registration, facebook integration, blogging, and many many more.

Existing applications are great, but the whole project/application distinction also forces you to think about your own project structure and therefore more likely to make reusable components that you can use in more of your own projects, or even share for others to use.

7. Templates

I’ve always been a bit dubious about the merits of PHP template engines such as Smarty. The Django template layer is great though. The inheritance model works well, and the restrictive language really forces you to have a very clean separation of presentation and logic.

8. Forms

I usually find dealing with user input one of the most boring parts of web development. It takes time to get it right, and its repetitive. The Django Form API really simplifies things. You can define your form class, include and validation rules, and simply add a few lines to your template and few lines to your view and you’re done!

So those are my 8 reasons why you should give django a go. If you’re already a django user let me know if you have any points to add!

You can view the page source to see what’s going on behind the scenes, but for full details check out the December edition of JSMag. Give it a try for yourself, generate your own tree!

One feature I miss all the time is late static binding, or LSB. The lack of LSB means that you can’t tell tell which class in your class hierarchy was invoked when calling a static method. Here’s a simple example:

class class0 {
  public static function getName() {
      echo __CLASS__ . "\n";
  }
}

class class1 extends class0 {}

class class2 extends class1 {}

class0::getName(); // -> "class0"
class1::getName(); // -> "class0"
class2::getName(); // -> "class0"

Notice that all three method calls output “class0″, the name of the base class. There is no way to tell which class the static method was called on, and in some situations that is information we need to know.

So what can we do? One obvious solution is to re-implement the method in every subclass. This doesn’t pose much of a problem for a method as simple as the one in our example, but in reality our method is likely to be much more complex, and implementing it in every subclass will lead to lots of duplication. If we ever need to make changes to the method then changes will need to be made to every subclass too. Not fun!

And alternative solution is to have every subclass invoke the parent method, passing in the class name (or whatever variable we’re interested in). It requires only a couple of extra lines per subclass, rather than repeating the whole method. Any changes now only need to be made in one place. Here’s the code:

class class0 {
  public static function getName($class = __CLASS__) {
      echo "$class\n";
  }
}

class class1 extends class0 {
  public static function getName($class = __CLASS__) {
      return parent::getName($class);
  }
}

class class2 extends class1 {
  public static function getName($class = __CLASS__) {
      return parent::getName($class);
  }
}

class0::getName(); // -> "class0"
class1::getName(); // -> "class1"
class2::getName(); // -> "class2"

In PHP5.3 we can use the new get_called_class(), so the code becomes much cleaner:

class class0 {
  public static function getName() {
      echo get_called_class() . "\n";
  }
}

class class1 extends class0 {}

class class2 extends class1 {}

class0::getName(); // -> "class0"
class1::getName(); // -> "class1"
class2::getName(); // -> "class2"

After writing about Munin last week and mentioning some of its uses outside of system performance tracking I decided to write a collection of plugins for tracking website popularity. The full code is available on GitHub.

The plugins include:

• Alexa traffic rank
• Number of pages in the Google index
• Technorati authority
• Feedburner RSS subscribers
• Twitter followers

Munin will automatically handle the generation of daily, weekly, monthly and yearly graphs. All of the plugins support graphing of multiple sites/accounts, so you can show several sites on the same graph, as is the case with the Google index graph above, or you can decide to show a single site/account per graph. Some example graphs are shown below:

On a Debian-based system installing Munin is as simple as running the following command, and then going to http://your-server/munin/ in your browser:

sudo aptitude install munin

Munin comes with lots of monitoring plugins by default, including those for MySQL, PostgresSQL, Apache, Tomcat, Squid, and for things such a CPU and memory usage, load average, network traffic, and many more. You can also find lots of user submitted plugins on sites like Munin Exchange.

Munin doesn’t have to be used solely for monitoring server performance though. Being so easy to extend Munin is also a great tool for tracking non-server performance related trends over time. In just a few lines of code you could write plugins to track the following stats about your website:

• Number of User signups
• Google PageRank
• Pages in Google’s index
• Number of backlinks
• Number of twitter mentions
• Alexa traffic rank

The number of pages in Google’s index is actually a plugin I’ve written. Simple put the following code in your /etc/munin/plugins directory to see it in action:

#!/bin/sh
# Munin Plugin to display the number of pages in the
# google index for all of the given websites
# Ben Dowling - www.coderholic.com

# Change this to whatever sites you're interested in
websites="www.yahoo.com www.google.com www.twitter.com"

if [ "$1" = "autoconf" ]; then
        echo yes
        exit 0
fi

if [ "$1" = "config" ]; then

        echo 'graph_title Number of Pages in Google Index'
        echo 'graph_args --base 1000 -l 0 '
        echo 'graph_vlabel number of pages'
        echo 'graph_category google'
        echo 'graph_info This graph shows the number of pages in the Google index for a given website.'

        i=0
        for site in $websites
        do
                name="site_${i}"
                echo "${name}.label ${site}"
                echo "${name}.draw LINE2"
                echo "${name}.info The number of pages in the google index."
                i=$((i+1))
        done
        exit 0
fi

i=0
for site in $websites
do
        name="site_${i}"
		value=$(wget -q --user-agent=Firefox -O - "http://www.google.com/search?q=site:${site}" | grep -E "of about [0-9,]+" -o | grep -E "[0-9,]+" -o | sed "s/,//g")
        echo "${name}.value ${value}"

        i=$((i+1))
done

For more details about Munin see their homepage, which also includes detailed documentation on writing your own plugins.

Let me know if you can think of any more Munin plugins that could be interesting, or if you’ve used any yourself!

Lots of comments were left pointing out some sites which weren’t included in my list, so I’m following up here with a list of 10 more top programming puzzle websites:

1. Code Chef

Code Chef has lots of practice puzzles, and monthly competitions with cash prizes. The site officially supports over 35 programming languages!

2. SPOJ

The Sphere Online Judge contains 1871 different programming problems. More points are awarded for better performing solutions, which can be submitted in a range of languages.

3. Code Golf

The aim with code golf is to submit a solution using the fewest characters possible.Solutions can be submitted in Perl, Python PHP or Ruby.

4. Uva Online Judge

Over 2600 great programming puzzles, and also regular contests. Submissions in C, C++, Java or Pascal are automatically checked for you.

5. Timus Online Judge

An online competition site that automatically checks your submissions. Supports Java, C#, Pascal, C and C++.

6. Google Code Jam

The code jam is a programming contest from Google. The top 25 contestants get to travel to Google’s HQ in California. Entries are accepting in any programming language.

7. USA Computing Olympiad

Programming puzzles designed to provide “pre-college students with opportunities to sharpen their computer programming skills”. The puzzles are still interesting and fun even if you’ve got a CS degree!

8. Informatics Olympiad

A British version of the computing olympiad. Again aimed at school and college students, but fun and interesting for everyone.

9. Programming Challenges in C, C++ and C#

About.com’s C/C++/C# section regularly posts interesting programming puzzles. Successful solutions get acknowledged on the site once the deadline has passed.

10. Java Bat

A site dedicated to practical Java programming problems. You can type your code directly into the website, and it’ll tell you if you’ve solve the problem correctly or not.

bmd /~: ./dbSize.sh coderholic root ********
wp_comments Data: 6.60MB Indexes: .15MB Total: 6.75MB
wp_links Data: 0MB Indexes: 0MB Total: 0MB
wp_options Data: 1.57MB Indexes: .01MB Total: 1.58MB
wp_postmeta Data: .01MB Indexes: .01MB Total: .02MB
wp_posts Data: .57MB Indexes: .02MB Total: .60MB
wp_term_relationships Data: 0MB Indexes: .01MB Total: .02MB
wp_term_taxonomy Data: 0MB Indexes: 0MB Total: 0MB
wp_terms Data: 0MB Indexes: 0MB Total: 0MB
wp_tla_data Data: 0MB Indexes: 0MB Total: 0MB
wp_tla_rss_map Data: 0MB Indexes: 0MB Total: 0MB
wp_usermeta Data: 0MB Indexes: 0MB Total: 0MB
wp_users Data: 0MB Indexes: 0MB Total: 0MB
*** 12 Tables | Data: 8.78MB Indexes: .25MB Total: 9.03MB ***

With a few small modifications I’m sure it’d be possible to get the script working PostgreSQL or other RDBMSes. The full code is below:

#!/bin/bash
# Calculate the storage space used up by all tables in a given MySQL database
# Ben Dowling - www.coderholic.com
database=$1
username=$2
password=$3

if [ ${#database} -eq 0 ]
then
	echo "Usage: $0 <database> [username [password]]"
	exit
fi

if [ "$password" ]
then
   password="-p$password"
fi

mysql="mysql -u $username $password $database"

$mysql -se "USE $database";

tables=$($mysql -se "SHOW TABLES")

totalData=0
totalIndex=0
totalTables=0

for table in $tables
do
   output=$($mysql -se "SHOW TABLE STATUS LIKE \"$table\"\G")
   data=$(echo "$output" | grep Data_length | awk -F': ' '{print $2}')
   dataMegs=$(echo "scale=2;$data/1048576" | bc)
   index=$(echo "$output" | grep Index_length | awk -F': ' '{print $2}')
   indexMegs=$(echo "scale=2;$index/1048576" | bc)
   total=$(($index+$data))
   totalMegs=$(echo "scale=2;$total/1048576" | bc)

   echo "$table Data: ${dataMegs}MB Indexes: ${indexMegs}MB Total: ${totalMegs}MB"

   totalData=$(($totalData+$data))
   totalIndex=$(($totalIndex+$index))
   totalTables=$(($totalTables+1))
done

dataMegs=$(echo "scale=2;$totalData/1048576" | bc)
indexMegs=$(echo "scale=2;$totalIndex/1048576" | bc)
total=$(($totalIndex+$totalData))
totalMegs=$(echo "scale=2;$total/1048576" | bc)

echo "*** $totalTables Tables | Data: ${dataMegs}MB Indexes: ${indexMegs}MB Total: ${totalMegs}MB ***"