Jamie Matthews - Blog

Another new start for j4mie.org

After running on Wordpress on shared hosting for many years, I've finally moved this site onto my Linode VPS, and yes, I've written my own blog engine in Django.

A lot of the old posts were irrelevant and uninteresting, and didn't get any traffic anyway, so I've removed those. The more popular stuff has been migrated over, and I'm using Django's redirects app to make sure I don't lose too much Google juice. I've imported all the comments into Disqus.

This site is also using lots of shiny HTML5 and CSS3 hotness, and is therefore entirely incompatible with Internet Explorer. Oh well.

Do let me know what you think, or if you find any problems.

Idiorm - a simple ORM for PHP

Just a quick update on a little project I've been working on over the past few months. Idiorm is “a lightweight nearly-zero-configuration object-relational mapper and fluent query builder for PHP5,” and is released under a BSD license.

Idiorm on GitHub.

The starting point for Idiorm was the topic of my talk at BarCampBrighton last year: “Minimalism in Web Development”, the idea that for many purposes, simpler solutions are often preferable to complex ones.

Idiorm lets you start using an ORM in your application with almost no effort. No XML files, model classes, database introspection or code generation is required: You just need to tell the ORM how to connect to your database, and away you go.

The fluent interface allows you to build most basic queries without writing any SQL, and gets out of the way when you need to do something more complex (both raw WHERE clauses and complete raw queries are supported). Once you have one or more instances of the ORM class (representing the rows in your database tables) you can access the data in them directly as properties of the objects (using PHP's magic __get and __set methods internally), modify and save them, as well as delete them. For full documentation and lots of code samples, see the README file.

Obviously, such a simple project (the whole ORM class is only about 600 lines of code) would probably never be used in a huge, complex PHP application. The main drawback to using a generic ORM class instead of domain-specific model classes is that you can't attach your own behaviour to the models. So the (extremely sensible) idea of placing most of your business logic in your models (Skinny Controller, Fat Model) isn't possible with Idiorm as-is.

I have three answers to this problem. The first is that many applications simply don't need any business logic more complex than basic CRUD operations, for which Idiorm is perfect. Secondly, one clear use case for Idiorm is to clean up legacy applications which are littered with raw SQL statements, usually passed directly into some custom-written “database abstraction” class or set of functions. Here, the application is already a mess, and in the absence of a complete rewrite, integrating Idiorm can only make things better. Finally, there's no reason why you couldn't build a full model layer as a lightweight wrapper on top of Idiorm. This would probably take the form of a factory object which returns instances of the correct model class, using Idiorm to populate them with data. I have a rough-and-ready implementation of this idea on my laptop which I may release at some point, but it's easy enough to build yourself. If you're using PHP 5.3 (and so have access to late static binding), you could even code up a pretty nice implementation of the active record pattern around Idiorm.

Update: I have now released Paris, which is an Active Record implementation built on top of Idiorm.

So to summarise, Idiorm isn't really intended to replace something like Doctrine or Propel (or ActiveRecord, or Django's ORM, or SQLAlchemy, etc). It's designed to be a quick and easy solution to database abstraction in small-to-medium sized applications. And let's face it - most of us are building small-to-medium sized applications. We're building bicycles, not space shuttles. If Doctrine is a rocket engine, Idiorm is a bottom bracket. But if all you want to do is get to the corner shop, a bicycle is a much quicker way to travel.

Finally, if you think you'd find Idiorm useful, please let me know. Fork the project, make suggestions using GitHub's Issues system or just leave a comment on this post.

Brian Eno and Generative Art

Tonight, I'm heading to Brighton Dome to see a performance of Brian Eno's seminal album Apollo as part of Brighton Festival, which Eno is curating this year.

Two years ago, I wrote an essay about Eno (and the wider theme of generative art) as a piece of coursework for a course called “Generative Creativity” (part of the EASy MSc at the University of Sussex). I thought today might be a good opportunity to put it online for posterity. Below is Part One of the essay, which is essentially a short biography of Eno, concentrating on his contribution to generative art and music.

Part One: Brian Eno

Generative approaches are gaining popularity in the creative arts, and their application to music is one of the most promising and active. The term “generative music” was coined by Brian Eno, and his contributions to the field have been increasingly ambitious since his early experiments in the 1970s.

In the mid-sixties, Eno became interested in the school of musical minimalism, including composers such as Terry Riley, Philip Glass and Steve Reich. In particular, Reich's 1965 piece It's Gonna Rain fascinated Eno. It is based on a tape recording of an American preacher saying the eponymous phrase. The recording was looped and played on two identical tape machines simultaneously. Due to slight inconstancies in the playback speed of the machines, the two loops gradually drifted out of sync. After some time, the listener becomes entirely habituated to the speech itself, and the perception of the artefacts created by the slowly shifting phase of the recordings becomes the main focus of the work. This notion of a very simple process being used to create a constantly changing and unpredictable musical texture became a foundation of Eno's career.

His work in the early 1970s was the beginning of his foray into “ambient music” (another phrase he coined). This genre is usually described in terms of the output itself and its appreciation as a musical “background feature” of the environment [1]. However, Eno saw the underlying process, the “idea that it's possible to think of a system or a set of rules which once set in motion will create music for you,” as the guiding principle of his work. [2]

The 1975 track Discreet Music used a very similar approach to Reich's imprecise tape machines. Eno used a long-delay tape echo, fed with two simple melodic lines of different lengths from a synthesiser. The timbre of the lines was occasionally modified using a graphic equaliser, but the main interest of the piece is the interaction of the two lines with themselves and each other as they are shifted in time by the echo unit.

Similar processes underlie many of Eno's ambient albums, with varying contributions from live performers, and more “standard” studio manipulation and production techniques. On a piece from Ambient 1: Music for Airports he used six simple building blocks, composed of sung notes recorded on tape loops of differing lengths. These elements do not change throughout the piece, but their specific configurations and clusterings are highly unlikely to repeat themselves. On the record, the track is eight minutes long, but this is merely an excerpt from a potentially infinite piece of music.

In the following decades, Eno began to notice parallels between his analog, studio-based techniques and certain concepts from science and computing. Cybernetics, the study of feedback in complex systems such as living organisms, provided a way to study how stable patterns could emerge without external control. Eno saw connections to self-regulating stability in group improvisation, where musicians may have only loose guidance in what to play but environmental factors such as the resonant frequency of the room may influence the outcome of the performance. He wrote several essays on the subject, including one called “Self-Regulation And Autopoiesis In Contemporary Music.” [3]

He became fascinated with John Conway's Game of Life, in which simple rules applied to a grid of cells produce patterns which persist through time and space. He wanted the dynamics of the simulation to “become intuitive” to him. “I wanted to be able to understand this message that I'd found in the Steve Reich piece [...], in my own work, and now in this. Very, very simple rules, clustering together, can produce very complex and actually rather beautiful results.” [2]

At the same time, his own work was becoming more sophisticated. He began to move from directly interacting with patterns of sounds, to manipulating rules about how sounds were made. Computers presented an obvious tool with which to explore this idea. He became aware of a piece of software called Koan Pro, which could be used to control a hardware synthesiser on a computer's sound card. Koan follows a set of rules which specify the parameters for around 150 elements of each composition, such as the instruments used, the timbre and attack of each note, and the relative proportion of each interval in the scale used to build the music. These rules are followed probabilistically, and form what Eno calls a musical “seed,” a framework within which the machine improvises to produce unpredictable and infinite pieces of music. [4] In 1996 he released Generative Music 1, an “album” containing twelve of these seeds on a floppy disk. Listeners ran a version of the Koan software on their own computer, which generated the music based on Eno's rules.

Eno has also worked in the visual arts, often combining projector-based installations with his generative music systems to produce hybrid pieces. However, his recent work conflicts in some ways with his guiding principle of generating the complete piece from simple foundations. Rather than using the computer as a “way of growing little seeds” [2], his 2006 work 77 Million Paintings By Brian Eno comprises a corpus of more than 300 of his own hand-drawn images, and combines and overlays them to produce a continually changing visual experience.

Brian Eno sees generative music as a third alterative to recorded music and live music. “Like live music, it is always different. Like recorded music, it is free of time-and-place limitations.” [5] His contributions to the field have been varied and creative, and his future work (including a generative soundtrack to the forthcoming video game Spore) promises to continue his exploration of this new medium.

References

[1] Brian Eno. Music For Airports, Liner notes, September 1978.

[2] Brian Eno. Generative Music. Transcript of talk from Imagination Conference, San Francisco, June 1996. Available from: http://www.inmotionmagazine.com/eno1.html [cited 16th February 2008].

[3] Brian Eno. Keyboard Wizards, Winter 1985. Available from: http://music.hyperreal.org/artists/brian_eno/interviews/keyb81.html [cited 16th February 2008].

[4] Andy Oldsfield. Brian Eno's Generation Game [online]. July 1996. Available from: http://music.hyperreal.org/artists/brian_eno/interviews/ind96d.html [cited 16th February 2008].

[5] Richard Williams. The Brain of Brian. The Guardian, May 1996. Available from: http://music.hyperreal.org/artists/brian_eno/interviews/guard96a.html [cited 16th February 2008].

Maps and Parking Tickets

Over the last few months, I've been kicking around a couple of little spare-time projects, and I've finally got round to writing a blog post about them. Both are fairly simple, but each represents a stepping stone in my efforts to bootstrap my brain into understanding the architecture and design of web applications.

First, http://hereit.is, a service for creating short urls for geographical locations. The idea is that you use the map on the homepage to find a location, and then create a short url to represent that location (such as http://hereit.is/brighton). You can then share this url by email, on the phone, on Twitter, etc. When the recipient(s) of the link visits the url, they are shown a simple landing page containing the map, with a link to Google Maps from where they can plan routes, see satellite imagery, and so on.

Despite its simplicity, hereit.is is built using modern web application principles. It has an MVC architecture and its design is quite closely based on that of Django. I decided to implement it in PHP (which I spend all day using at work), partly to prove to myself that it is at least possible to construct well-architected applications with this badly-assembled mess of a language. I now know that if I had used Django itself, the whole project would probably have taken me about half an hour - but the project itself wasn't really the point. The point was to learn how frameworks are put together, and I have definitely succeeded at that.

I also wanted to get my head around REST, so hereit.is has simple API which you can use to create locations, and retrieve GeoJSON and KML representations of places. See the API documentation for more information.

For the front-end design, I took advantage nifty CSS3 tricks, such as border-radius and box-shadow. The main page of the site uses no images at all in its design, the idea of which appeals to the minimalist in me (of course, the map itself uses images). Even the logo is a Unicode character, inserted into the heading text with the :before pseudo-element.

I was invited to give a talk about hereit.is at £5 App, the brilliant monthly-ish meeting organised by Ian Ozsvald and John Montgomery. I gave a fairly high-level overview of how hereit.is works, and the talk should be fairly accessible (I hope) to the less technical types. You can see the talk on Vimeo here: http://vimeo.com/5664789

£5 App #18 - Jamie on HereIt.Is from Ian Ozsvald on Vimeo.

The second project is I Will Pay Your Parking Ticket. This is the brainchild of my friend Seb Lee-Delisle, who explains the idea much better himself.

He asked me to help out with the server-side implementation, which we decided to do with Django. It's fairly straightforward for the most part, with one or two interesting details, such as the method we use to select which tweets to show on the homepage (simple randomness turned out to be quite inadequate), and the Twitter search API integration. Fortunately, you can see how it all works for yourself, because we decided to make the whole project open source. You can follow or fork it at Github.

This experience has completely sold me on the benefits of Django. At one stage, we more-or-less completely tore the application apart and rebuilt it from scratch in a few hours. There's simply no way that would be possible with almost anything else. I've loved Python for years, so the ability to work on Web projects with it is fantastic.

Because of Seb's rock-star status in the Flash world, and his thousand-plus followers on Twitter, I Will Pay Your Parking Ticket has been pretty popular. We already have nearly six thousand votes in the database. Deciding whether or not random strangers deserve your sympathy is a strangely addictive experience, and a good way to waste your lunch hour.

Again, Seb gave a talk about the idea at £5 App, which you can see below:

£5 App #19 Seb and Jamie with I Will Pay Your Parking Ticket from IanProCastsCoUk on Vimeo.

So, that's a quick update on what I've been doing over the past few weeks. I'm trying to keep myself busy, with regular attendance at FlashBrighton and other events, as well as getting involved with the brand-new Brighton Hackerspace, BuildBrighton. I also managed to grab a ticket for BarCampBrighton 4 at the beginning of September. Hmm, I should probably decide what I'm going to talk about...

Project 365

I'm not generally one for New Year's Resolutions, but this year I decided to have a go at Project 365. The idea, in short, is to take one photo per day for a year. The claimed benefits of this vary, but boil down to improving one's photographic technique, creating a “photo diary” for yourself, and documenting your year for the purpose of sharing with others (usually on Flickr).

I was hoping for a mixture of these, but most importantly I intended to keep the creative juices flowing through the year by enforcing a tiny bit of creativity each day. I'm always looking for ways to train my brain to look at the world differently, and spending a bit of time each day framing life through a camera lens seems to be a good way to go about this. I actually bent the rules a little, and let a bit of digital art and even one short video sneak in amongst the photos. After all, the creativity is the point, not the medium.

Anyway, in case you hadn't guessed by now, I've decided to officially abandon the project. I started off well, posting a photo a day until well into April, with a maximum of two or three days backlog. Things started slipping a little in May, and then I caught up again, but I quickly got so far behind that there didn't seem to be much point carrying on. Today is the 177th day of the year, and my most recently uploaded photo was number 130.

There's little point dissecting all the reasons, but suffice it to say that it didn't have the intended effect. Once I started to run out of immediate things to take photos of, and the few generative art ideas I had bubbling away in my brain were turned into pixels and committed to the interwebs, the thought of having to do something creative every day got less and less attractive, and began to feel more like a chore. I decided to have a break to see if that would help, but it actually seemed to make things worse. My photos from my holiday in Cornwall in May let me catch up a bit, but I was still quite behind, and I've had no real motivation to take any photos since.

Interestingly, I don't regret the experience at all. I enjoyed the creative bits while they lasted, and once I began to get fed up of the project, it made me want to get on with other non-visual projects (more on this soon). And I was actually pretty happy with quite a few of the photos that I took.

You can have a look at the 130 photos in the final set, or see below for some of my favourites (click to embiggen).

BarCampLondon6

I spent the weekend at BarCampLondon6 at the offices of the Guardian near King's Cross Station. I think my brain is too full of interesting things I've picked up over the last couple of days (not to mention too deprived of sleep) to actually process any of it into a coherent blog post, but you'll get an idea of my thoughts by looking at my posts on Twitter tagged #barcamplondon6, and my Flickr set from the event. Thanks to the organisers for their hard work, and to all the people who took part.

My talk was about Processing. The photos below are by Rain Rabbit.

Me:

The audience (yes, I probably should have picked a bigger room):

You can view the slides on Slideshare:

Why should you care about Processing?

View more presentations from Jamie Matthews.

Finally, here's one of me playing with Auduino an Arduino-based synthesiser created by Peter Knight from tinker.it:

JavaScript Syntax Checking from gedit

JSLint is a tool for finding problems in JavaScript programs. Unfortunately, to use it, you need to paste your source code into a textarea on a website - less than ideal. This post will explain how to check your JavaScript directly from gedit.

It is based on these instructions for vim, combined with this method to allow piping text into JSLint from the standard input.

1. Install SpiderMonkey See the build instructions page for installing on many different architectures, or these instructions for quick installations on Ubuntu and CentOS 5.

2. Download JSLint SpiderMonkey Hack Edition Get this version of JSLint which has been modified to allow piping code from the standard input. Save it somewhere in your home directory, say /home/username/bin/js/jslinthack.js. You'll also need to make it executable by typing chmod +x /home/username/bin/js/jslinthack.js

3. Enable the External Tools plugin in gedit Open gedit and click the Edit menu, then choose Preferences. Click the Plugins tab and tick to enable the Extenal Tools plugin.

4. Set up the JSLint External Tool In gedit, open the Tools menu and click External Tools. Click the New button and give your new tool the name JSLint. Give it a descriptive name like "Check JavaScript source code". Give it an Accelerator if you wish (mine is Shift+Ctrl+j). Set the Command to the location at which you saved the hacked JSLint file, eg. /home/username/bin/js/jslinthack.js. Set Input to Current Document and set Output to Insert in Output Panel (you'll need to have the Bottom Panel enabled in gedit's View menu).

5. Profit! Open up a JavaScript source file, and either press your Accelarator keyboard shortcut, or go to Tools > JSLint. The results of the syntax check will be output in the bottom panel.

Cahors

A few months ago, I spent some time writing a MIDI sequencer for the Arduino. I finished the sequencer, wrote a couple of songs with it, and then got sucked into MSc thesis mode and had to abandon it.

Well, I've finally got round to publishing it as an open source project on Google Code, so hopefully others will be able to have some fun with it. I've also written some documentation which will hopefully make it a little easier to get started creating music with it. I've named it ‘Cahors,’ which is a word I found in The Deeper Meaning of Liff. See the FAQ page on the Google Code side for a full explanation.

http://code.google.com/p/cahors/

You can hear the first full piece I wrote with Cahors on last.fm here, or you can download the MP3 from the Google Code project page. I've not had a chance to go back and write and more music for it recently, but hopefully I'll get back into that soon.

How to create realistic smoke effects with astronomical photos

I was lucky enough to attend the Flash On The Beach conference in Brighton this week, and one of the highlights was Robert Hodgin of Flight404.com talking about his stunning work, including his famous Solar and Weird Fishes pieces. His Magnetosphere engine is now the default music visualiser in iTunes 8.

In passing, he mentioned that his smoke effects were created using photographs of nebulae. He didn't give any details about exactly how this works, but I thought it was such a good idea that I decided to have a go for myself.

The concept I came up with is simple: use a particle system to generate smoke-like behaviour, but instead of drawing each particle as a simple "blob," draw an image of a nebula. The older the particle, the larger and more transparent the image. Fairly simple, but it creates remarkably realistic effects.

The particle system I used is based on the Traer physics library (in fact, I used some of their sample code to create the smoke behaviour) but you could use any system you like.

The images I used are show below. I took pictures of nebulae I found via Google and cropped out the most interesting parts, then applied a circular gradient mask in Photoshop and saved the resulting image as a transparent PNG. The original PNG images are included in the ZIP file which is linked at the bottom of this post.

Here are a couple of videos showing the results from this process:

UPDATE: The videos have been moved to Vimeo, please see: Smoke 1, Smoke 2, Smoke 3.

The Processing source code for my simple example (along with the images used to create it) can be found in this ZIP file: smoke.zip. If you run this code, you'll first see the particles drawn as red dots. If you press the 'i' key, the images will be drawn. Pressing 'p' hides the red dots. Finally, the 'm' key toggles mouse following mode. Don't forget, you'll need the Traer physics library for this to work. I used the OpenGL drawing mode because it considerably speeds up image drawing, but it will work using the default Processing renderer.

This is just a basic sketch of how the idea works, and as I said, I have no idea if this is exactly how Robert Hodgin does it. Either way, I think it looks pretty good.

House of Cards

UPDATE: Videos have been moved to Vimeo.

I just noticed that I got linked to from Create Digital Motion, a blog about digital video production and computer animation. Cheers guys! However, the post says that there is "more at [my] personal site" and.. well, there isn't. Or wasn't, until now.

The post is about Radiohead's "House of Cards" video which has been getting a lot of coverage on t'interwebs this week. I'll spare the technical details as you're almost certainly familiar with them by now (if not, have a look at this post). My two attempts (so far) are below, along with a little info about how they were made. You can view both the videos in higher quality by clicking through to YouTube Vimeo.

House of Cards, Experiment One from Jamie M on Vimeo.

The first set of experiments is based on a really simple particle system. The x, y and z co-ordinates of each point in the data cloud are turned into a particle, which is then "bounced" off a hard surface using very basic physics. The position of each particle of each frame in the video is calculated the correct number of times to produce the animation effect. By bouncing the particles at different angles relative to the "camera," appearance of the bouncing particle cloud can be changed, creating some unusual visuals. The next section of the video is created by duplicating each point three times, to create three versions of Thom's head. Finally, every particle is given a random velocity, creating an "explosion" effect which again is modelled as a large number of bouncing physical objects.

House of Cards, Experiment Two from Jamie M on Vimeo.

The second video is simpler: no physical simulation, just a set of lines. Every point is connected to another point with a line. The number of points slowly increases over the video, so Thom's face slowly appears from randomness.

I've put some higher-quality stills from the videos on my Flickr account.