GhostStory

During all the research I did for my CSS testing talk, I couldn't help but spot another gap where a testing tool could be useful.

Cucumber

Cucumber is a technology used widely in automated testing setups, mostly for acceptance testing - ensuring that the thing everybody agreed on at the beginning was the thing delivered at the end.

This is accomplished by having a set of plain text files containing descriptions of different scenarios or aspects of the application, usually with a description of the actions performed by an imaginary user. You describe the situation (known as a 'Given' step), describe the user's action ('When') and describe the expected outcome ('Then').

The language (properly known as gherkin) used in these files is deliberately simple and jargon-free so that all the key stakeholders in the project - designers, developers, product owners - can understand but the files are also written in a predictable and structured style so that they can, behind the scenes, be turned into testable code.

What occurred to me when looking into this area was that there wasn't an agreed terminology for specifying the layout/colour/look and feel of a project in plain text. Surely this would be the perfect place to drop in some cucumber salad.

What we've got now is a project based on SpookyJS - a way of controlling CasperJS (and, therefore PhantomJS) from NodeJS - which contains the GhostStory testing steps and their corresponding 'behind the scenes' test code. There are only two steps at the moment but they are the most fundamental which can be used to build up future steps.

By the way, the name 'GhostStory' only just won out over the original 'CucumberNodeSpookyCasperPhantomJS'. It kinda makes sense because it's built using a lot of ghost-named technologies (Spooky, Casper, Phantom) and cucumber test files are sometimes called "Story" files because they describe a user story.

Implemented Steps

Here, "Element descriptor" is a non-dev-readable description of the element you want to test - "Main title", "Left-hand navigation", "Hero area call-to-action". In the project, you keep a mapping file, selectors.json, which translates between these descriptions and the CSS selector used to identify the element in tests.

Then the "Element descriptor" should have "property" of "value"

This is using the computed styles on an element and checking to see if they are what you expect them to be. I talked about something similar to this before in an earlier post. This is related to the 'Frozen DOM' approach that my first attempt at a CSS testing tool, cssert, uses but this way does not actually involve a DOM snapshot.

Then the "Element descriptor" should look the same as before

This uses the 'Image Diff' approach. You specify an element and render the browser output of that element to an image. The next time you run the test, you do the same and check to see if the two images differ. As mentioned many times before, this technique is 'content-fragile' but can be useful for a specific subset of tests or when you have mocked content. It can also be particularly useful if you have a 'living styleguide' as described by Nico Hagenburger. I've got some ideas about CSS testing on living styleguides that I'll need to write up in a later post.

Future Steps

Off the top of my head, there are a couple of other generic steps that I think would be useful in this project.

Then the "Element descriptor" should have a "property" of "value1", "value2", ..., or "valueN"

This variation on the computed style measurement allows an arbitrary-length list of values. As long as the element being tested matches at least one of the rules, the step counts as a pass. This could be used to ensure that all text on a site is one of a certain number of font-sizes or that all links are from the predefined colour palette.

Then the "Element descriptor" should look the same across different browsers.

This would build on the existing image diff step but include multiple browser runners. Just now, the image diffs are performed using PhantomCSS which is built on top of PhantomJS which is Webkit-based. This would ideally integrate a Gecko renderer or a Trident renderer process so that the images generated from one could be checked against another. I still feel that image diff testing is extremely fragile and doesn't cover the majority of what CSS testing needs to do but it can be a useful additional check.

The aim

I'm hoping this can sit alongside the other testing tools gathering on csste.st where it can help people get a head-start on their CSS testing practices. What I'm particularly keen on with the GhostStory project is that it can pull in other tools and abstract them into testing steps. That way, we can take advantage of the best tools out there and stuff it into easily digested Cucumber sandwiches.

Try it

The GhostStory project is, naturally, available on GitHub. More usefully, however, I've been working on a fork of SpookyJS that integrates GhostStory into an immediately usable tool.

Please check out this project and let me know what you think. I might rename it to distinguish it from the original SpookyJS if I can figure out exactly how to do that and maintain upstream relationships on GitHub.

Things I learnt at CSS Dev Conf

Jank

This is the word Paul Irish (@paul_irish) used to describe that jittering, stuttering effect that happens when you aren't syncing your animations with the monitor using requestAnimationFrame. Find out more about Jank at jankfree.com or requestAnimationFrame at CreativeJS.

SuperToonists

Rachel Nabors (@CrowChick) and Kyle Weems (@cssquirrel) are just as cool in real life as they are on the Twitter. Probably moreso, in fact, as there's more than 140 characters of them at a time.

Scoped Revelation

Tab Atkins (@tabatkins) explained over the buffet table the reason why the spec went for

<style scoped></style> 

rather than

<scopedstyle></scopedstyle>

It had been bugging me that using the former meant that this wasn't backwards compatible but it turns out I was totally wrong on this one. By using the attribute rather than a new tag, the styles were backwards compatible. They would still be interpreted by older browsers (although they would still spread out to affect the whole page). If they had been included in a new tag, older browsers wouldn't have read them. The affect of the styles on the page can be avoided by introducing an ID inside the block to be scoped (and all styles within the scoped block). That way, older browsers can continue to work fine while newer browsers that interpret the attribute correctly will benefit from the improved performance of having styles only affecting a small chunk of the page.

This is why I don't write specs.

Choose your elements wisely

The examples for improving selector performance for GitHub's diff pages presented in Jon Rohan's talk made me wonder "Do I really need all these divs?". Read more yourself. Also, I love the word 'Design-gineer'.

A Porter?

Pam Selle (@pamasaur) couldn't believe I had never heard of a porter beer. Is this actually a thing? Wikipedia seems to think so but then, Wikipedia can be convinced that Julianne Moore is Dudley Moore's daughter so I'll need more proof than that. Ale, stout, bitter, lager, all words I'm familiar with but 'porter'? Nope.

More than you'd think

I thought I had a fairly comprehensive list of all the CSS testing tools out there. Really. I've spent over a year researching and testing as many as I could get my hands on. There are even more. Phil Walton (@philwalton) made some excellent points around mocking content for tests that I'm going to have to follow up on.

Skyscraper

Christopher Schmitt (@teleject) is tall. Really tall. No, taller than that.

CSS Dev Conf

Last week I was at CSS Dev Conf in Hawaii - the first of an annual series of conferences. Not only was I there, I actually gave a talk!

I gave an overview of the tools and techniques available for automated CSS testing. I didn't mention it much here or on twitter because, well, I was kinda nervous and didn't want to jinx it. In my daily job, I organise Tech Talks and give pointers to people on how to make their presentations exciting and full of awsm but I tend to avoid actually standing up at the front of the room myself. Beyond introducing the speakers, that is. This is probably why I used to help organise gigs more than play them.

I won't repeat the content of the talk here. For that, you should start out by checking out the slides. The main point I finished on, however, is that the web dev community has fantastic tools for testing every aspect of web development except CSS and that it appears that every developer interested in the subject has started from scratch and built a tool that does what they need it to do. I'm hoping that I can turn http://csste.st/ into a community-driven site where people can share best-practices, techniques, tools and terminology around CSS testing. That way, we can create some sturdy processes around testing our CSS and not reinvent the wheel every few months.

The conference

Firstly, the trip to Hawaii: it was fantastic. I spent the weekend before the conference staying in a backpackers hostel on Waikiki beach. If you ever need somewhere to stay for a fraction of the price of a hotel on Waikiki, I totally recommend the 'Waikiki Backpackers Hostel'. Backpackers' hostels are the same everywhere - there are the people who hang around reception that may or may not work there, you can't quite tell; there's the Australian guy who comes down drunk for breakfast, smokes half-a-dozen cigarettes quickly then goes back to bed until the afternoon; there's the guy in his mid 70s who looks like a cross between everybody's favourite grandad and a stereotypical acid-casualty.

I swam with turtles in Hanauma Bay, climbed up Diamonhead Ridge, took a bus to the far side of downtown then spent 4 hours walking back via Iolani Palace. Like I say, it was a fantastic trip, even for someone who doesn't like the sun that much.

On the Tuesday, I moved into the Outrigger Reef on the Beach hotel for the conference. Ooh, posh. I could practically step out of my room into the elevator then step out of the elevator onto the beach. I then got to spend my remaining time in Hawaii hanging out with a huge bunch of cool, clever people (Rachel Nabors put together a Twitter list of the speakers), all of whom know an awful lot about web development.

With any multi-track conference there are always going to be a few sessions you have to miss but with CSS Dev Conf being four-track and having such a high level of quality, I kinda worry that I missed out on about three-quarters of what I wanted to see. Fortunately, everybody's been sharing their slides so I can at least get the gist of what they were talking about.

The journey

All in all, I'd totally recommend keeping an eye open for the announcement of next year's CSS Dev Conf. It might not be held in Honolulu again but, from my point of view, that's a good thing. It really is a fantastic place to spend time (seriously: turtles!) but getting there from Berlin is not the easiest thing. In fact, if you use the 'Earth Sandwich' lab on here.com, you can see how close Hawaii is to being the Exact Opposite Side of the World from Berlin. On the way out there, it didn't seem too bad as I left on Friday morning (Berlin time) and landed on Friday evening (Hawaii time) but on the way back, I left on Thursday and landed on Saturday. That was quite a journey.

jHERE playground

Every couple of months, we have a Research Week. It's kind of like the well-known Google 20% time but instead of doing it one day a week, we gather up our time and do a week of maps-based hacking. It's totally cross-discipline so we usually gather a couple of developers, a couple of UX and visual designers, a QA and build something cool. In past Research Weeks, I've built or helped build the Alien Easter Egg, Maps Labs and CoverMap.Me. I also built a maps-based JSBin fork called JotApp.

When my partner-in-crime Max started working on the latest version of his jQuery plugin (formerly called jOvi, now called jHERE), I wanted to build a new playground where people could play and build their own maps mashups and hacks really easily. My first thought was to rework the fork of JSBin again and maybe add in a slightly nicer default theme. I wanted users to be able to save to Gists so, seeing as there is already an SQLite and a MySQL adapter, I wrote a Gist adapter which appeared to the application as if it were a database but actually saved to anonymous Gists. The problem was that it was a bit too… heavy.

Don't get me wrong, JSBin is a fantastic project. It just does a lot more than I needed. I didn't need the MySQL adapter, the alternate themes, the localstorage or the user registration. Also, it's a bit too weighty for my phone. When someone tweets a link to a JSBin or a JSFiddle, I usually check it out on my phone and it's not the best experience. Seeing as HERE maps work on mobile, I wanted my playground to work, too. Rather than spend a couple of hours cutting out all the bits I didn't want from JSBin, I decided to spend a couple of hours building my own version from scratch. So, this past Sunday afternoon, that's exactly what I did:

jHERE Playground

It's written in NodeJS on top of express and works nicely on desktop, iPad and mobile.

The project is open-sourced on GitHub (naturally) and can be modified to be a general JS-playground for anything. If you fancy a simple, self-hosted JS hackspace, just change the default HTML, CSS and JS and it's ready to go.

Open Source Snacks

In my opinion, seed snacks are pretty much the perfect web dev snack: they're tasty, they're low-fat, they're vegan-friendly, they're gluten-free. I've always got a tub of some next to my monitor so, when I'm chewing over a tricky layout, I can grab a handful and chew them, too.

This repository collects some of my favourite seed recipes and I'm hoping that other web devs can clone the project, roast their own, suggest improvements and submit a pull request. If you have any other easy to make snacks (such as nut snack recipes), feel free to submit them, too.

Eating tips

From observation, seed eaters tend to fall into one of these categories:

Pour-and-snarf

Pour a few into your hand, tip them into your mouth in a oner. Good when you're in a hurry. Can lead to stray seeds falling into the keyboard.

Considerate Ant-eater

Pour a few into your hand, stick your tongue into the pile of seeds, retract.

Solo Ant-eater

Stick your tongue directly into the tub of seeds. Clean, efficient, not good for sharing.

Ice-cream scoop

Use a spoon. Good for sharing and minimises mess. Prevents multi-tasking. Feels kind of like using your mouse to go Edit > Copy when ctrl-C is right there.

Rousing call-to-arms

The stereotypical image of the geek - bottle of cola on one side, jumbo bag of chips on the other, little desire to do anything beyond the computer - has never really been true for the majority. We're all kinds of different people - mountain climbers, cyclists, needlepoint workers, knitters. The people that played on the football team and the ones who didn't get picked. We deserve more than just nachos.

Also nachos.

Clone on GitHub

Sunflower Seed designed by Dmitry Sholkov from The Noun Project

Some App.net recipes

This is a collection of code snippets for various common tasks you might need to accomplish with the App.net API. Most of these are focused on creating or reading geo-tagged posts. They require a developer account on app.net and at least one of an App ID, App Code, App Access Token or User Access Token. The calls here are implemented using jQuery but that's just to make it easier to copy-paste into the console to test them out (so long as you fill in the blanks).

An important thing to bear in mind is the possibility for confusion between a 'stream' and 'streams'. By default, a 'stream' is a discrete chunk of the 20 latest posts served at a number of endpoints. This is the open, public, global stream:

https://alpha-api.app.net/stream/0/posts/stream/global

On the other hand, 'streams' are long-poll connections that serve up any matching posts as soon as they are created. The connection stays open while there is something there to receive the response. Streams are available under:

https://alpha-api.app.net/stream/0/streams

Totally not confusing. Not at all.

Creating a user access token

Required for any user-specific data retrieval. The only tricky thing you'll need to think about here is the scope you require.

scope=stream email write_post follow messages export

should cover most requirements.

Requires

• client_id

Visit this URL:

https://alpha.app.net/oauth/authenticate
    ?client_id=[your client ID]
    &response_type=token
    &redirect_uri=http://localhost/
    &scope=stream email write_post follow messages export

Using a user access token to create a post (with annotations)

Requires

• User Access Token
• text to post

The text is essential if you don't mark a post as 'machine_only'. The annotations here are optional. Annotations don't appear in the global stream unless the requesting client asks for them.

$.ajax({
  contentType: 'application/json',
  data: JSON.stringify({
    "annotations": [{
      "type": "net.app.core.geolocation",
      "value": {
        "latitude": 52.5,
        "longitude": 13.3,
        "altitude": 0,
        "horizontal_accuracy": 100,
        "vertical_accuracy": 100
      }
    }],
    "text": "Don't mind me, just checking something out."
  }),
  dataType: 'json',
  success: function(data) {
    console.log("Text+annotation message posted");
  },
  error: function() {
    console.log("Text+annotation message failed");
  },
  processData: false,
  type: 'POST',
  url: 'https://alpha-api.app.net/stream/0/posts?access_token={USER_ACCESS_TOKEN}'
});

Using a user access token to post a machine_only post (with annotations)

Requires

• User Access Token

In this example, we're creating a post that won't show up in user's timelines and adding the 'well-known annotation' for geolocation.

$.ajax({
  contentType: 'application/json',
  data: JSON.stringify({
    "annotations": [{
      "type": "net.app.core.geolocation",
      "value": {
        "latitude": 52.5,
        "longitude": 13.3,
        "altitude": 0,
        "horizontal_accuracy": 100,
        "vertical_accuracy": 100
      }
    }],
    machine_only: true
  }),
  dataType: 'json',
  success: function(data) {
    console.log("Non-text message posted");
  },
  error: function() {
    console.log("Non-text message failed");
  },
  processData: false,
  type: 'POST',
  url: 'https://alpha-api.app.net/stream/0/posts?access_token={USER_ACCESS_TOKEN}'
});

Retrieve the global stream, including geo-annotated posts if there are any

Requires

• User Access Token

This is a very basic call to retrieve the global stream but it also instructs the endpoint to return us all annotations and include machine-only posts.

var data = {
  "include_machine": 1,
  "include_annotations": 1,
  "access_token": "{USER_ACCESS_TOKEN}"
};

$.ajax({
    contentType: 'application/json',
    dataType: 'json',
    success: function(data) {
      console.log(data);
    },
    error: function(error, data) {
      console.log(error, data);
    },
    type: 'GET',
    url: 'https://alpha-api.app.net/stream/0/posts/stream/global',
    data: data
});

Creating an App Access Token

This is necessary for many of the streams operations. It is not used for individual user actions, only for application-wide actions.

• App.net API wiki on App Access Tokens

Requires

• client_id
• client_secret

client_credentials is one of the four types of grant_type specified in the OAuth 2.0 specification. I had difficulty getting this to work when using a data object:

var data = {
    "client_id": "{CLIENT_ID}",
    "client_secret":"{CLIENT_SECRET}",
    "grant_type": "client_credentials"
};

The client_credentials kept throwing an error. Instead, sending this as a string worked fine:

$.ajax({
  contentType: 'application/json',
  data: 'client_id={CLIENT_ID}&client_secret={CLIENT_SECRET}&grant_type=client_credentials',
  dataType: 'json',
  success: function(data) {
    console.log(data);
  },
  error: function(error, data) {
    console.log(error, data);
  },
  processData: false,
  type: 'POST',
  url: 'https://alpha.app.net/oauth/access_token'
});

One other thing to note is that this bit should be done server-side. This will throw a bunch of "…not allowed by Access-Control-Allow-Origin…" errors if you do it via jQuery.

Returns

{
    "access_token": "{APP_ACCESS_TOKEN}"
    }

Creating a streams format

Now you have your app access token, you can use it to tell the service what kind of data you want back. The streams offered in the API have two quite powerful aspects. Firstly, filters allow you to run many kinds of queries on the data before it is streamed to you so you don't need to recieve and process it all. Secondly, the decoupling of filters from streams means you can specify the data structure and requirements you want once then just access that custom endpoint to get the data you want back any time.

Requires

• App access token

This first example just creates an unfiltered stream endpoint

$.ajax({
  contentType: 'application/json',
  data: JSON.stringify({"object_types": ["post"], "type": "long_poll", "id": "1"}),
  dataType: 'json',
  success: function(data) {
    console.log(data);
  },
  error: function(error, responseText, response) {
    console.log(error, responseText, response);
  },
  processData: false,
  type: 'POST',
  url: 'https://alpha-api.app.net/stream/0/streams?access_token={APP_ACCESS_TOKEN}'
});

Returns

{
    "data": {
        "endpoint": "https://stream-channel.app.net/channel/1/{LONG_RANDOM_ENDPOINT_URL}",
        "id": "77",
        "object_types": [
            "post"
        ],
        "type": "long_poll"
    },
    "meta": {
        "code": 200
    }
}

Using Filters to create a stream of geotagged posts

We'll specify some requirements for our filter now so that it only returns back a subset of posts. The rules we're specfying here are:

At least one item in the "/data/annotations/*/type" field
must "match"
the value "net.app.core.geolocation"

Requires

• User access token

The field is specified in 'JSON Pointer' format. Within the response, there is a 'data' object and a 'meta' object. The data contains an 'annotations' object which contains an array of annotations, each of which has a type. This is represented as /data/annotations/*/type.

$.ajax({
  contentType: 'application/json',
  data: JSON.stringify({"match_policy": "include_any", "clauses": [{"object_type": "post", "operator": "matches", "value": "net.app.core.geolocation", "field": "/data/annotations/*/type"}], "name": "Geotagged posts"}),
  dataType: 'json',
  success: function(data) {
    console.log(data);
  },
  error: function(error, responseText, response) {
    console.log(error, responseText, response);
  },
  processData: false,
  type: 'POST',
  url: 'https://alpha-api.app.net/stream/0/filters?access_token={USER_ACCESS_TOKEN}'
});

Returns

The filter rules you just specified, the id of the filter (remember that for later) and the details of the application used to make the request.

{
"clauses": [
    {
        "field": "/data/annotations/*/type",
        "object_type": "post",
        "operator": "matches",
        "value": "net.app.core.geolocation"
    }
],
"id": "527",
"match_policy": "include_any",
"name": "Geotagged posts",
"owner": {
    "avatar_image": {
        "height": 200,
        "url": "https://d2rfichhc2fb9n.cloudfront.net/image/4/Pr63PjEwJ1fr5Q4KeL3392BMgSnIAYlHxv8OkWwzx75V8quNfpaFp4VPpKnDRxdXtYYPtIutrDVdU9NbJn7hKApQL84T5sfB1D9bWTgtizMWInignv0WyPPfM2DpqSThQgvkB68vbPzjZ8VeKM02M2GySZ4",
        "width": 200
    },
    "canonical_url": "https://alpha.app.net/thingsinjars",
    "counts": {
        "followers": 30,
        "following": 65,
        "posts": 96,
        "stars": 0
    },
    "cover_image": {
        "height": 230,
        "url": "https://d2rfichhc2fb9n.cloudfront.net/image/4/UWZ6k9xD8_8LzEVUi_Uz6C-Vn-I8uPGEBtKb9jSVoFNijTwyEm1mJYpWq6JvnA6Jd4gzW76vFnbSWvM3jadhc1QxUl9qS4NTKiv3gJmr1zY_UpFWvX3qhOIyKrBPZckf2MrinqWay3H0h9rfqY0Gp9-liEg",
        "width": 960
    },
    "created_at": "2012-08-12T17:23:44Z",
    "description": {
        "entities": {
            "hashtags": [],
            "links": [],
            "mentions": []
        },
        "html": "<span itemscope="https://app.net/schemas/Post">Nokia Maps Technologies Evangelist; CreativeJS team member; the tech side of museum140; builder of The Elementals; misuser of semi-colons;rn</span>",
        "text": "Nokia Maps Technologies Evangelist; CreativeJS team member; the tech side of museum140; builder of The Elementals; misuser of semi-colons;rn"
    },
    "id": "3191",
    "locale": "en_GB",
    "name": "Simon Madine",
    "timezone": "Europe/Berlin",
    "type": "human",
    "username": "thingsinjars"
}
}

Listening to the geotagged post stream

This wil return a link to a long-lasting connection to the app.net stream that will only return posts with the geolocation annotation.

Requires

• filter_id from the previous call

Note: the filter_id was returned as id in the previous response.

$.ajax({
  contentType: 'application/json',
  data: JSON.stringify({"object_types": ["post"], "type": "long_poll", "filter_id": "527"}),
  dataType: 'json',
  success: function(data) {
    console.log(data);
  },
  error: function(error, responseText, response) {
    console.log(error, responseText, response);
  },
  processData: false,
  type: 'POST',
  url: 'https://alpha-api.app.net/stream/0/streams?access_token={APP_ACCESS_TOKEN}'
});

Returns

The same kind of response as the 'Creating a streams format' example except the data coming down on the stream is filtered.

https://stream-channel.app.net/channel/1/{LONG_RANDOM_ENDPOINT_URL}

Open that URL up in your browser (seeing as we're testing) and, in a different tab, create a geo-tagged machine-only post (see above). Your post will appear almost instantly after you've submitted it.

Location-based time

Inspired by the simplicity of implementing a proximity search using MongoDB, I found myself keen to try out another technology.

It just so happened that I was presented with a fun little problem the other day. Given a latitude and longitude, how do I quickly determine what the time is? Continuing the recent trend, I wanted to solve this problem with Node.JS.

Unsurprisingly, there's a lot of information out there about timezones. Whenever I've worked with timezones in the past, I've always gotten a little bit lost so this time, I decided to actually read a bit and find out what was supposed to happen. In essence, if you're doing this sort of task. you do not want to have to figure out the actual time yourself. Nope. It's quite similar to one of my top web dev rules:

Never host your own video.

(Really, never deal with video yourself. Pay someone else to host it, transcode it and serve it up. It'll will always work out cheaper.)

What you want to do when working with timezones is tie into someone else's database. There are just too many rules around international boundaries, summer time, leap years, leap seconds, countries that have jumped over the international date line (more than once!), islands whose timezone is 30 minutes off the adjacent ones...

To solve this problem, it needs to be split into two: the first part is to determine which timezone the coordinate is in, the second is the harder problem of figuring out what time it is in that timezone. Fortunately, there are other people who are already doing this. Buried near the back of the bottom drawer in very operating system is some version of the tz database. You can spend hours reading up about it, its controversies and history on Wikipedia if you like. More relevantly, however, is what it can do for us in this case. Given an IANA timezone name – "America/New_York", "Asia/Tokyo" – you can retrieve the current time from the system's tz database. I don't know how it works. I don't need to know. It works.

Node

Even better for reaching a solution to this problem, there's a node module that will abstract the problem of loading and querying the database. If you use the zoneinfo module, you can create a new timezone-aware Date object, pass the timezone name to it and it will do the hard work. awsm. The module wasn't perfect, however. It loaded the system database synchronously using fs.readFileSync which is I/O blocking and therefore a Bad Thing. Boo.

10 minutes later and Max had wrangled it into using the asynchronous, non-blocking fs.ReadFile. Hooray!

Now all I needed to do was figure out how to do the first half of the problem: map a coordinate to a timezone name.

Nearest-Neighbour vs Point-in-Polygon

There are probably more ways to solve this problem but these were the two routes that jumped to mind. The tricky thing is that the latitude and longitude provided could be arbitrarily accurate. A simple lookup table just wouldn't work. Of course, the core of the problem was that we needed to figure out the answer fast.

Nearest Neighbour

1. Create a data file containing a spread of points across the globe, determine (using any slow solution) the timezone at that point.
2. Load the data into an easily searchable in-memory data-structure (such as a k-d tree)
3. Given a coordinate, find the nearest existing data point and return its value.

Point in Polygon

1. Create a data file specifying the geometry of all timezones.
2. Given a coordinate, loop over each polygon and determine whether this coordinate is positioned inside or outside the polygon.
3. Return the first containing polygon

This second algorithm could be improved by using a coarse binary search to quickly reduce the number of possible polygons that contain this point before step 2.

Despite some kind of qualification in mathematic-y computer-y stuff, algorithm analysis isn't my strong point. To be fair, I spent the first three years of my degree trying to get a record deal and the fourth trying to be a stand-up comedian so we may have covered complexity analysis at some point and I just didn't notice. What I do know, however, is that k-d trees are fast for searching. Super fast. They can be a bit slower to create initially but the point to bear in mind is that you only load it once while you search for data lots. On the other hand, while it's a quick task to load the geometry of a small number of polygons into memory, determining which polygon a given point is in can be slow, particularly if the polygons are complex.

Given this vague intuition, I settled on the first option.

If I wanted to create a spread of coordinates and their known timezones from scratch, it might have been an annoyingly slow process but, the Internet being what it is, someone already did the hard work. This gist contains the latitude and longitude for every city in the world and what IANA timezone it is in. Score! A quick regex later and it looks like this:

module.exports = [
  {"latitude": 42.50729, "longitude": 1.53414, "timezone": "Europe/Andorra"},
  {"latitude": 42.50779, "longitude": 1.52109, "timezone": "Europe/Andorra"},
  {"latitude": 25.56473, "longitude": 55.55517, "timezone": "Asia/Dubai"},
  {"latitude": 25.78953, "longitude": 55.9432, "timezone": "Asia/Dubai"},
  {"latitude": 25.33132, "longitude": 56.34199, "timezone": "Asia/Dubai"},
  etc…

(See the original on GitHub)

All that's left is to load that into a k-d tree and we've got a fully-searchable, fast nearest neighbour lookup.

Source

The source for this node module is, of course, available on GitHub and the module itself is available for install via npm using:

npm install coordinate-tz

When combined with the zoneinfo module (or, even better, this async fork of the module), you can get a fast, accurate current time lookup for any latitude and longitude.

Not a bad little challenge for a Monday evening.

Explanating Experiment Follow-up

A year ago, I started a little experiment (and not one of my usual web experiments).

I decided to give away my book, Explanating, for free. The website has links to download the book without charge and also links through to Amazon and Lulu. I asked people to download the free one then, if they enjoyed it, they could come back and buy it.

Twelve months later, I now have the answer.

• Free: 315
• Amazon: 2
• Lulu: 0

Erm. Yeah. Not quite as successful as I would have liked. Still, the point was to measure the 'conversion rate'. Admittedly, it's a small sample size but it would appear to be about 0.6%. At this rate, I only need another 97,465,886 people to download the free one and I'll have made £1,000,000! Sweet!

Building a Proximity Search

This is the detailed post to go with yesterday's quick discussion about proximity search. All the code is available on GitHub.

This assumes a bit of NodeJS knowledge, a working copy of homebrew or something similar.

Install

• MongoDB - brew install mongodb
• NodeJS
• NPM (included in NodeJS installer these days)

These are included in the package.json but it can't hurt to mention them here:

• npm install twitter (node twitter streaming API library)
• npm install mongodb (native mongodb driver for node)
• npm install express (for convenience with API later)

Start mongod in the background. We don't quite need it yet but it needs done at some point, may as well do it now.

Create a Twitter App

Fill out the form Then press the button to get the single-user access token and key. I love that Twitter does this now, rather than having to create a full authentication flow for single-user applications.

ingest.js

(open the ingest.js file and read along with this bit)

Using the basic native MongoDB driver, everything must be done in the database.open callback. This might lead to a bit of Nested Callback Fury but if it bothers you or becomes a bit too furious for your particular implementation, there are a couple of alternative Node-MongoDB modules that abstract this out a bit.

// Open the proximity database
db.open(function() {
    // Open the post collection
    db.collection('posts', function(err, collection) {
        // Start listening to the global stream
        twit.stream('statuses/sample', function(stream) {
            // For each post
            stream.on('data', function(data) {
                if ( !! data.geo) {
                    collection.insert(data);
                }
            });
        });
    });
});

Index the data

The hard work has all been done for us: Geospatial Indexing in MongoDB. That's a good thing.

Ensure the system has a Geospatial index on the tweets.

db.posts.ensureIndex({"geo.coordinates" : "2d"})

Standard Geospatial search query:

db.posts.find({"geo.coordinates": {$near: [50, 13]}}).pretty()
(find the closest points to (50,13) and return them sorted by distance)

By this point, we've got a database full of geo-searchable posts and a way to do a proximity search on them. To be fair, it's more down to mongodb than anything we've done.

Next, we extend the search on those posts to allow filtering by query

db.posts.find({"geo.coordinates": {$near: [50, 13]}, text: /.*searchterm.*/}).pretty()

API

Super simple API, we only have two main query types:

• /proximity?latitude=55&longitude=13
• /proximity?latitude=55&longitude=13&q=searchterm

Each of these can take an optional 'callback' parameter to enable jsonp. We're using express so the callback parameter and content type for returning JSON are both handled automatically.

api.js

(open the api.js file and read along with this bit)

This next chunk of code contains everything so don't panic.

db.open(function() {
  db.collection('posts', function(err, collection) {
    app.get('/proximity', function(req, res) {
      var latitude, longitude, q;
      latitude = parseFloat(req.query["latitude"]);
      longitude = parseFloat(req.query["longitude"]);
      q = req.query["q"];

      if (/^(-?d+(.d+)?)$/.test(latitude) && /^(-?d+(.d+)?)$/.test(longitude)) {
        if (typeof q === 'undefined') {
          collection.find({
            "geo.coordinates": {
              $near: [latitude, longitude]
            }
          }, function(err, cursor) {
            cursor.toArray(function(err, items) {
              writeResponse(items, res);
            });
          });
        } else {
          var regexQuery = new RegExp(".*" + q + ".*");
          collection.find({
            "geo.coordinates": {
              $near: [latitude, longitude]
            },
            'text': regexQuery
          }, function(err, cursor) {
            cursor.toArray(function(err, items) {
              writeResponse(items, res);
            });
          });
        }
      } else {
        res.send('malformed lat/lng');
      }

    });
  });
});

If you've already implemented the ingest.js bit, the majority of this api.js will be fairly obvious. The biggest change is that instead of loading the data stream then acting upon each individual post that comes in, we're acting on URL requests.

app.get('/proximity', function(req, res) {

For every request on this path, we try and parse the query string to pull out a latitude, longitude and optional query parameter.

if (/^(-?d+(.d+)?)$/.test(latitude) && /^(-?d+(.d+)?)$/.test(longitude)) {

If we do have valid coordinates, pass through to Mongo to do that actual search:

collection.find({
  "geo.coordinates": {
    $near: [latitude, longitude]
  }
}, function(err, cursor) {
  cursor.toArray(function(err, items) {
    writeResponse(items, res);
  });
});

To add a text search into this, we just need to add one more parameter to the collection.find call:

var regexQuery = new RegExp(".*" + q + ".*");
collection.find({
  "geo.coordinates": {
    $near: [latitude, longitude]
  },
  'text': regexQuery
}

This makes it so simple, making it it kind of feels like cheating. Somebody else did all the hard work first.

App.net Proximity

This works quite well on the App.net Global Timeline but it'll really become useful once the streaming API is switched on.

Of course, the code is all there. If you want to have a go yourself, feel free.

Proximity Search

Now that geolocated posts are beginning to show up around app.net, I found myself wondering about proximity search. Twitter provides one themselves for geotagged tweets. What a proximity search does, essentially, is provide results from a data set ordered by increasing distance from a given location. This can be further enhanced by combining it with a text search either before or after the distance sorting. This would give you a way to search for a certain query within a certain area.

Here, for example, is a proximity search centred on Berlin including any tweet in 1000 miles containing the word 'toast'.

When I first started thinking about the tech required for a proximity search, I remembered Lukas Nowacki back in our old Whitespace days implementing the Haversine formula in MySQL (Alexander Rubin has a good overview of how to do this). As much as I love my trigonometry and logarithms, I must admit, I was looking around for a simpler solution. Actually, I was looking around for a copy-paste solution, to be honest. I may even have spent some time going down that route if Max hadn't pointed me in the direction of MongoDB.

I'd been putting off digging into NoSQL databases for a while because, well, I had no real reason to. Recently, I've either been focused on front-end dev or hacking away at Java and never really had any good reason to investigate any of these new-fangled technologies get off my lawn you kids.

MongoDB

After 10 minutes of messing around with Mongo, I pretty much just found myself saying “No... way. There's no way that's actually working” I'm sure those of you experience with document-oriented databases are rolling your eyes right now but for those few of us left with an entirely relational concept of databases, let me just explain it like this: you know those things you want to do with a database that are just a hassle of multiple joins and confusing references? Document databases do some of those things really really well.

The biggest selling point for me, however was the native geospatial indexing. That pretty much made the majority of my proximity search complete. All I needed to do was wrap it in a nice interface and call it a day...

I'll follow up tomorrow with a more detailed 'How-to' guide.

How App.net became useful

After Twitter started announcing changes to its API usage and people started to get worried about the future of the developer eco-system, App.net appeared. It provides a streaming data-platform in much the same way Amazon provides a web hosting platform. The reason for the Twitter comparison is that one of the things you can make with it is a Twitter-like public short message system. This was, in fact, the first thing the App.net developers made to showcase the platform: Alpha although that example seems to have convinced many people that the whole point was to build a Twitter-clone instead of a service whose purpose is to stream small, discrete blocks of meta-tagged data. The community-developed API spec is an important aspect of App.net as well although feature discussions can devolve into musings on Computer Science theory a bit too easily.

For the first couple of weeks, it was fun to just hack around with the APIs, post a bit, build a little test app (disabled now that more App.net clients have push notifications). It all became much more interesting, however, when two new features were added to the platform – Machine-only posts and Well-known Annotations.

Machine-only posts

Pretty much exactly what they sound like. These are posts that are not intended to be viewed in any human-read conversation stream. They still get pushed around the network exactly the same as all other posts but to see them, you must explicitly say 'include machine-only posts' in your API calls. For developers who have been building silly toys on Twitter for a couple of years, this is fantastic. You don't need to create a separate account purely for data transfer. This can be part of your main account's data stream. I have quite a few Twitter accounts that I just use for outputs from various applications. I have one, in fact, that does nothing but list the tracks I listen to that I created for this side-project.

By classifying these as 'machine-only', they can stay associated with the user and subject to whatever privacy controls they have set in general. This makes it far easier for the user to keep control of their data and easier for the service to track accurate usage. For devs, it also means you can hack away at stuff, even if it is to be public eventually, without worrying too much about polluting the global stream with nonsense.

Well-known Annotations

Annotations were part of the spec from the beginning but only implemented at the end of August. Annotation is just another word for per-post meta-data – each post has a small object attached which provides extra information. That's it.

The annotations can be up to 8192 bytes of valid JSON which is quite a lot of meta-data. Even with this, however, the usefulness is limited to per application use-cases until some standards start to appear. There's nothing to stop a popular application being the one to set the standard but for the more general cases, it is most fitting to continue with the community-led development model. This is where Well-known Annotations come in.

Well-known annotations are those attributes which are defined within the API spec. This means that the community can define a standard for 'geolocation', for example, and everyone who wants to use geolocation can use the standard to make their application's posts compatible with everybody else's.

Obviously, I'm quite into my geolocated data. I love a bit of map-based visualisation, I do. Here's a sample of jQuery that will create a post with a standard geolocation annotation:

$.ajax({
  contentType: 'application/json',
  data: JSON.stringify({
    "annotations": [{
      "type": "net.app.core.geolocation",
      "value": {
        "latitude": 52.5,
        "longitude": 13.3,
        "altitude": 0,
        "horizontal_accuracy": 100,
        "vertical_accuracy": 100
      }
    }],
    machine_only: true
  }),
  dataType: 'json',
  success: function(data) {
    console.log("Non-text message posted");
  },
  error: function() {
    console.log("Non-text message failed");
  },
  processData: false,
  type: 'POST',
  url: 'https://alpha-api.app.net/stream/0/posts?access_token=USERS_OAUTH_TOKEN'
});

In the same way as the machine-only posts, these annotations aren't provided on a default API request, you have to specifically ask for them to be included in the returned data. This is to make sure that the majority of use cases (public streaming, human-readable conversation) don't have to download up to 8KB of unnecessary data.

Retrieving both

This is an API call to retrieve posts marked machine-only and with annotations

Potential use cases

You might have noticed, the API call above had the machine-only attribute as well as the well-known geo annotation. If I wanted to create an app that would run on my phone and track my routes throughout the day, all I would need to do would be to run that $.ajax call periodically with my current geolocation. The data would be saved, distributed, streamed and could be rendered onto a map or into a KML at the end of the day. I could record hiking trails or museum tours, or share my location with someone I'm supposed to be meeting so they can find out where I'm coming from and why I'm late. That's just a couple of the single-user cases. Having a shared standard means that the potential for geo-tagged posts opens up to at least equal that of Twitter's. Heatmap-density diagrams showing areas of the most activity; global and local trends; location-based gaming. Add a 'news' annotation to geotagged posts and suddenly you've got a real-time local-news feed. Add 'traffic' and you've got community-created traffic reports.

There are so many clever things you can do with location-tagged data. I hope others are just as enthused about the possibilities as I am.

CoverMap - Nokia Maps on Facebook

I'm almost managing to keep to my intended schedule of one map-based web experiment per week. Unfortunately, I've mostly been working on internal Nokia Maps projects over the weekends recently so I've not had much to post here.

I can share my latest toy, though: CoverMap.me

Using just the public APIs over a couple of hours last Sunday afternoon, I made this to allow you to set a Nokia Map as your Facebook Timeline cover. The whole process is really straightforward so I thought I'd go over the main parts.

The exact aim of CoverMap.me is to allow the user to position a map exactly, choose from any of the available map styles and set the image as their cover image.

Make a Facebook App

Go to developers.facebook.com/apps/ and click 'Create New App', fill in the basic details – name of the app, URL it will be hosted on, etc – and you're done.

Facebook login

I've used the Facebook JS SDK extensively over the summer for various projects but I wanted to try out the PHP one for this. Super, super simple. Really. Include the library (available here), set your appId and secret and request the $login_url.

$facebook->getLoginUrl(array('redirect_uri' => "http://example.com/index.php"));

That will give you a link which will take care of logging the user in and giving you basic access permissions and details about them.

Nokia Maps JS API

When I'm hacking together something quick and simple with the Nokia Maps API, I usually use the properly awsm jQuery plugin jOVI written by the equally awsm Max. This makes 90% of the things you would want to do with a map extremely easy and if you're doing stuff advanced enough to want the extra 10%, you're probably not the type who'd want to use a jQuery plugin, anyway. Either way, you need to register on the Nokia developer site to get your Nokia app_id and secret.

To create a map using jOVI, simply include the plugin in your page then run .jOVI on the object you want to contain the map along with starting parameters:

$(window).on('load', function() {
  $('#mapContainer').jOVI({
    center: [38.895111, -77.036667], // Washington D.C.
    zoom: 12,           // zoom level
    behavior: true,     // map interaction
    zoomBar: true,      // zoom bar
    scaleBar: false,    // scale bar at the bottom
    overview: false,    // minimap (bottom-right)
    typeSelector: false,// normal, satellite, terrain
    positioning: true   // geolocation
  }, "APP_ID", "APP_SECRET");
});

This gives us a complete embedded map.

As I mentioned above, part of the idea for CoverMap.me was to allow the to choose from any of the available map styles. This was an interesting oddity because the public JS API gives you the choice of 'Normal', 'Satellite', 'Satellite Plain' (a.k.a. no text), 'Smart' (a.k.a. grey), 'Smart Public Transport', 'Terrain' and 'Traffic' while the RESTful Maps API (the API that provides static, non-interactive map images) supports all of these plus options to choose each of them with big or small text plus a 'Night Time' mode. Because of this, I decided to go for a two-step approach where users were shown the JS-powered map to let them choose their location then they went through to the RESTful static map to allow them to choose from the larger array of static tiles.

RESTful Maps

The RESTful Maps API is relatively new but does provide a nice, quick map solution when you don't need any interactivity. Just set an img src with the query parameters you need and get back an image.

(this should be all on one line)
http://m.nok.it/
    ?app_id=APP_ID
    &token=APP_TOKEN
    &nord       // Don't redirect to maps.nokia.com
    &w=640      // Width
    &h=480      // Height
    &nodot      // Don't put a green dot in the centre
    &c=38.895111, -77.036667 // Where to centre
    &z=12       // Zoom level
    &t=0        // Tile Style

That URL produces this image:

Upload to Facebook

Given the above, we've now got an image showing a map positioned exactly where the user wants it in the tile style the user likes. We just need to make the Facebook API call to set it as Timeline Cover Image and we're done.

You'd think.

Facebook doesn't provide an API endpoint to update a user's profile image or timeline cover. It's probably a privacy thing or a security thing or something. Either way, it doesn't exist. Never fear! There's a solution!

With the default permissions given by a Facebook login/OAUTH token exchange/etc... (that thing we did earlier), we are allowed to upload a photo to an album.

The easiest way to do this is to download the map tile using cURL then repost it to Facebook. The clever way to do it would be to pipe the incoming input stream directly back out to Facebook without writing to the local file system but it would be slightly more hassle to set that up and wouldn't really make much of a difference to how it works.

// Download from RESTful Maps
$tileUrl = "http://m.nok.it/?app_id=APP_ID&token=APP_TOKEN&nord&w=640&h=480&nodot&c=38.895111,%20-77.036667&z=12&t=0";
$ch = curl_init( $tileUrl );
$fp = fopen( $filename, 'wb' );
curl_setopt( $ch, CURLOPT_FILE, $fp );
curl_setopt( $ch, CURLOPT_HEADER, 0 );
curl_exec( $ch ); 
curl_close( $ch );
fclose( $fp );

//Upload to Facebook
$full_image_path = realpath($filename);
$args = array('message' => 'Uploaded by CoverMap.me');
$args['image'] = '@' . $full_image_path;
$data = $facebook->api("/{$album_id}/photos", 'post', $args);

The closest thing we can do then is to construct a Facebook link which suggests the user should set the uploaded image as their Timeline Cover:

// $data['id'] is the image's Facebook ID 
$fb_image_link = "http://www.facebook.com/" . $username . "?preview_cover=" . $data['id'];

Done

There we go. Minimal development required to create a web app with very little demand on the user that gives them a Nokia Map on their Facebook profile. Not too bad for a Sunday afternoon.

Go try it out and let me know what you think.

The code is now available on GitHub

Using CSS :target to control state

There can be a fair amount of discussion provoked by the phrase "CSS only" or "Built entirely in HTML and CSS" when it comes to tech demos. The discussion generally revolves around the fact that what the phrase actually means 99% of the time is "Using CSS for the graphical bits but tracking the mouse using JS" or "Using CSS transforms for most of it but using JS to calculate the values" or any number of variations on that theme.

Now, I'm not saying that's a bad thing at all, I quite happily called my interpretation of Super Mario Kart CSS-only even though without JS, Yoshi wouldn't make it to the first bend. What these demos are doing, essentially, is using CSS as the rendering layer where previously, JS would have been used. By doing so, you even get some nice hardware acceleration thrown in. Not too bad.

Why fall back?

The reason we fall back to JS for a lot of things is because CSS is a declarative language (see my post on CSS testing to learn more). However you say things should be is how they are. In CSS, you can't set a variable to one value, do something with it then set it to a different value and do something else. For a start, you don't have variables. Even if you did, the style sheet is read top-to-bottom before anything is done with it. The variable would always have the final value assigned to it, even the first time you used it. I'm simplifying a bit but not much. If you want to do anything clever, you generally need to resort to JS, an interpreted language.

Really CSS only

If you want to make something properly interactive but entirely CSS, you have to find a way to modify the state of the DOM so that different sets of styles can be applied to the same sets of elements. We actually do this all the time when we change the colour of a link on :hover. The user interacts with the page, the state of the page is changed slightly, a different style applies. There's a lot you can do now that you have a way to modify the page based on user interaction. Hooray for pseudo-classes!

An extremely important addition to the set of pseudo-classes available in CSS3 is the :target class. This becomes set on an element when the element is the target of a clicked link. Think of in-page jump links. When you click on one and the URL changes to 'blah.html#the-thing-you-clicked', the element with id="the-thing-you-clicked" gets the :target pseudo-class. Now you can affect it and its children with new styles. Now it becomes interesting, you can click on something on one bit of the page and it cause something to happen on another bit of the page.

Multiple-nested states

By nesting several elements around the thing you're intending to modify, you can now create a set of states entirely controlled by CSS. For example, with this HTML:

<div id="red">
  <div id="blue">
    <div id="green">
      <div id="yellow">
          <p>Hello.</p>
      </div>
    </div>
  </div>
</div>
<ul>
  <li><a href="#red">Red</a></li>
  <li><a href="#blue">Blue</a></li>
  <li><a href="#green">Green</a></li>
  <li><a href="#yellow">Yellow</a></li>
  <li><a href="#default">Default</a></li>
</ul>

p {
    color:black;
}
#red:target p {
    color:red;
}
#blue:target p {
    color:blue;
}
#green:target p {
    color:green;
}
#yellow:target p {
    color:yellow;
}

You can change the colour of the text by clicking on the link. Without any JS.

See this on JSFiddle.

Shiny demos

This still isn't perfect. There are still going to be many things that JS is best for, calculations being one, keyboard input being another. To try and find the best way to show this off, I tried to update CSS Mario Kart to be entirely CSS and I almost got there but wasn't 100% successful.

• CSS-only Mario Kart

I then started to build a zoetrope in CSS, found someone else had already done it and decided to modify it.

• CSS-only Zoetrope

Maybe soon, the phrase "CSS only" will really mean "CSS only. And a bit of HTML."

Web Audio API Part 1

CSSSquircle

• CSSSquircle

Sounds like a Hackathon

A couple of weeks ago, I started digging into the Web Audio API. Initially, I was just looking to find out what it was and see if any ideas for toys presented themselves but, kinda predictably, I ended up getting elbow-deep in a bucketful of audio routing graphs, gain nodes and impulse responses.

I'll write up a more detailed post about the API shortly but Max and I used it quite heavily in the 5apps hackathon we attended last week and I wanted to share the outcome of our hacking.

SoundScape

“A Nokia Map mashed together with a bunch of APIs and positional audio to create an immersive map experience.”

For a better explanation of how SoundScape works, check out Max's slides:

• http://mmarcon.github.com/slides/soundscape/

In essence, we use a Nokia Map as the base, add on a Flying Hamster, make a call to Freesound.org for geotagged audio samples, male a call to LastFM for upcoming concerts, make a call to Deezer for the most popular track by the artist playing the event reported by LastFM and put them all together in the browser using 3D positional audio. Basically.

The source is available on GitHub

The Actual Demo

SoundScape (Web Audio API-capable browser required)

Web Audio

I'm getting a bit obsessed with the Web Audio API just now. That's not a bad thing, of course, I just wish the browsers would all catch up.

Ages ago, I mentioned the SoundScape maps mashup Max and I made at the 5apps hackathon. Finally, here's the video of the two of us presenting it at a Nokia Berlin Tech Talk.

• Web Audio intro and SoundScape

Creative tutorials

This goes along with the series of Web Audio API introductory tutorials I'm writing over at CreativeJS.

Web Page Screensavers

I don't find myself using screensavers that much these days. Sure, they made sense when you needed to avoid burning a ghost of the windows start bar into your CRT monitor but with TFTs, LEDs, projectors, OLEDs and whatever else, it's rare you'll find hardware that actually needs protecting like that any more. On top of that, in my case, I'm either at my desk coding or at the coffee machine refilling. There aren't that many opportunities for me to appreciate a warp-speed starfield or some infinite pipes.

What I'm saying is: I miss screensavers.

Since I started writing for CreativeJS, I've seen a lot more examples of clever, cool, pretty and downright creative demos and toys written in JS than I ever had before. You can probably figure out where I'm heading with this: these would make cool screensavers.

A quick bit of googling later and I found a couple of applications that let you set a web-page fullscreen as your screensaver. Of course, you can't just set any old demo as your screensaver, many of them rely on user interaction which kinda defeats the purpose.

Downloads

OS X

WebSaver

Unfortunately, this uses plain-old standard WebKit so no WebGL demos. Maybe someone can fork Chromium to make it do this.

Windows

web-page-screensaver

This one seems to be based on IE so it probably won't work with the canvas-based demos below. If you can point me to a WebKit-based one, I'll include that instead.

Old-school screensavers

Flying Windows

http://dl.dropbox.com/u/16145652/flying/flying.html

Starfield by Chiptune

http://www.chiptune.com/starfield/starfield.html

Non-canvas

Insta-Art by me

http://thelab.thingsinjars.com/insta-art.html

Newsola by Nick Nicholaou

http://www.newsola.com/#/uk

Canvas

Falling blocks by Lionel Tardy

http://experiments.lionel.me/blocs

MMOSteroids by Seb Lee-Delisle

http://seb.ly/demos/MMOsteroids.html

Origami by Hakim El Hattab

http://hakim.se/experiments/html5/origami/

The Single Lane Superhighway by Aaron Koblin and Mr.doob

http://www.thesinglelanesuperhighway.com/

Ablaze by Patrick Gunderson

http://theorigin.net/ablazejs/

Visual Random by Dimitrii Pokrovskii

http://voxelrain.appspot.com/

Circle Worm by Ilari

http://style.purkki.ilric.org/projects/js_wave/

Boids by Jason Sundram

http://viz.runningwithdata.com/boids/

3D Globe by Peter Nederlof

http://peterned.home.xs4all.nl/demooo/

WebGL Needed

Just in case someone in the comments finds a WebGL-capable screensaver, here are the demos I liked that require WebGL.

Clouds by Mr.doob

http://mrdoob.com/lab/javascript/webgl/clouds/

WaterStretch by Edouard Coulon

http://waterstretch.ecoulon.com/

Further Development

The ideal screensaver would allow you to enter several different URLs to allow you to easily save them. There should also be a checkbox to mark demos as 'works offline'. That way, when the screensaver starts, it checks for connectivity displays a screensaver that doesn't require a connection.

Add your suggestions below.

Nokia Web Dev Blog

Exciting news! The Nokia Web Dev Blog is finally live!

You might remember it was one of my 2012 ToDos and I can now move the To-Do to Done.

So far, I've put two posts up, one on using background-size: cover and one on quick prototyping using python SimpleHTTPServer.

I'm not convinced a URL with '.../Blogs/blog/...' is necessarily the most memorable so maybe the next task would be to get something catchier. Any suggestions?

Superhero Libraries

Writing, writing and writing

In case anybody's wondering where I've been for the last few weeks (and I know you all have), I've recently joined the CreativeJS team writing about the coolest, shiniest things on the world-wide interwebs. My first post went up last week.

I'm also about to launch the Nokia Web Dev blog (link to come soon) where I get to write tutorials about the coolest, shiniest things we do in Nokia Maps

Don't worry, there won't be fewer articles going up here than before, I just had to spend a couple of weeks figuring out which articles were best suited to where. To make up for a quiet few weeks, here's a picture of some Kohlrabi, a really tasty vegetable barely known outside Germany.

JS Minification is seriously addictive.

Having gotten hooked on it during the first js1k, written about extreme JS minification and submitted a bunch of stuff to 140bytes, I think it's fairly safe to say I'm an addict. Because of that, there was really no chance that I could let this year's js1k go by without entering.

The theme this year is ‘Love’. Seeing as I never actually submitted my maze1k demo, I decided I'd spruce it up, restyle a bit and submit it covered top to bottom in hearts.

There's not a lot I can say about minification techniques that hasn't already been covered either on this site, on Marijn Haverbeke's site, on Ben Alman's or on the 140bytes wiki page on byte-saving techniques. The only things I will add are a couple of techniques which are new to me. I have thoroughly commented the code below, however. If you want to play along, have a look.

Left-first Evaluation

It's a feature of most programming languages that when you have a logical operator such as && or ||, the value of the left side of the operator determines whether the right side will be evaluated at all. If the left side of an AND is false, we're going to get a false for the whole thing. It doesn't matter what the right side is so we don't even need to look at it. Similarly, if the left side of an OR is true, the eventual return value will be true so we don't need to look at the right. For example, here are two statements:

coffee&&morning
wine||work

In the first example (&& – AND), we will only check morning if coffee is true. We first have a look at coffee and if it is false, we don't care whether morning is true or not, we just skip it (and, presumably, go back to bed). If coffee is true, we'll then have a look to see if morning is true. It doesn't matter if morning is a function or a variable assignment or whatever, it will only happen if coffee is true.

In the second example (|| – OR), we will only evaluate work if wine is false. We start by looking at and evaluating wine. If that is true, the JS interpreter saves us from even considering work and skips it. The right-side of the operator, work, is only considered if wine is false.

You can probably see how, in a few simple situations, this can help avoid an if(), thereby saving at least one byte. Usually.

‘A’ font

If you want to set the font-size on a canvas 2D context, you have to use the font property. Makes sense, right? Unfortunately for obsessive minifiers, you can't just set the fontSize, you also have to set the fontFamily at the same time:

canvas.getContext('2d').font="20px Arial"

Failure to set the font family means that the whole value is invalid and the font size isn't applied.

My thought process: “But 'Arial'? The word's so… big (5 bytes). There must be some way to make this smaller. If only there were a font called ‘A’ (1 byte)…”

Well, it turns out, if you set a font that doesn't exist on the user's system, the canvas will fall back on the system sans-serif by default. On windows that is... Arial. On OS X, it's Helvetica. I'm glad about that because otherwise, Helvetica wouldn't get a look-in, being 9 whole bytes.

There is always the chance that someone will have a font called ‘A’ but I'm willing to take that risk. This possibility could be avoided by using an unlikely character like ♥ for the font name.

The code

These are hosted on GitHub rather than in my usual code blocks to make it easier for you to fork them, pull them apart and get elbow-deep in them.

The almost-impossible-to-read minified submission:

The full, unminified code with thorough commenting:

The submission

My entry is up on the js1k site now. It's already getting quite busy up there after only a few days and I have to say I'm impressed by the fantastic job @kuvos has done in building it up over the last couple of years and providing me with fuel for my addiction.

ImpressJS Tools

While putting together last week's promo video for Museum140 (a vote'd be awsm, btw), I knocked up a few tools to make my life easier. As I always say, if you find something you like, make a plugin for it. Seriously, I always say that. That might even be how I proposed to my wife, I'll have to check.

Anyway.

Play

This is a simple timing helper. It just provides a little array you can push slide durations into and at the end, you call 'play'. I can't see many uses for this other than in creating videos.

ImpressJS.play(3000);  //Set the first slide duration for 3 seconds
ImpressJS.play(1000);  //Set the second slide duration for 1 second
ImpressJS.play(); //Play from the start

Edit

This is much more useful.

If this script is included in the page (after the impress.js script), you can drag your slides around, rotate and scale them into exactly the position you want. Once you're happy, you can get the current HTML output onto the console for pasting back into your original slideshow file. I could have snuck in a drag-n-drop file handler but that would make it Chrome only.

Disclaimer

These tools rely on ImpressJS having a public API which it currently doesn't have. It's obviously high on the author's priority list (given the amount of discussion it's raised) but, as too many pull requests spoil the broth, I've made a little fork of my own, added the public functions the tools require and I'll update them once the main repository's settled down a bit.

Download

These are available in the tools folder of this fork of impress.js. Each one contains an example. Hopefully, these will be updated to use the public API as soon as it's available.

Museum140 Shorty

As regular readers are probably aware, one of my side projects is helping out Museum140 with tech and design support (building websites, designing stickers, etc). Jenni's the main driving force behind some of the coolest “Museums and Social Media” projects of the past year.

The Museum140 project is in the running for a Shorty Award so I thought I'd help out by making a promo video. Of course, it's always best to play to your strengths so I built the video as a web page…

ImpressJS

HTML5 slideshows are all pretty cool (I even used one myself a few months back) but most of them deliberately emulate traditional slide presentations. When I saw ImpressJS for the first time last week, I was astounded. Its style is based on prezi.com but built using CSS3 rather than Flash. As well as being an inventive way of giving presentations, it also gave me an idea.

A couple of hours coding later and we've got a simple but stylish video with every word and phrase positioned perfectly. I wrote a little helper function to assist in creating a consistent timeline and recorded it in Screenflow. After that, I spent 10 minutes with the other kind of keyboard and came out with a nice little piece of background music, too.

The Video

There you go, ImpressJS is not only good for slideshow but also promo videos. Not bad.

Vote?

It would also be remiss of me if I didn't ask those of you with a twitter account to at least consider voting.

Umpteen ways to test for a function

After wrestling with another 140bytes challenge, I found myself wondering how many different ways you can test an object in JS to see if it’s a function. I wrote out a few of them then threw it out to my colleagues who came up with a few more. I’d love to hear from anyone who can suggest more to add to the list. Ideally, you want to find a test that will return true for all functions and only for functions. It’d be great if it’s a universal test that can be slightly modified to test for other types but that’s not essential.

Bear in mind, most of these shouldn’t be used in the real world, this is just for fun.

There are a couple of main categories and several variations within.

Duck Typing

When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck.

In other words, if an object has the methods of a function and the properties of a function, it’s probably a duck. No, that doesn’t sound right.

This simply detects whether the object we’re testing contains the methods or properties we’d normally associate with a function. The most common test is checking whether a.call exists. In most cases, this will only be defined on a function and it will be defined on all functions. It is, therefore, a good test to use.

The downside is that it can be fooled by setting a property of call on the object to be truthy. This will pass the test but still not be a function. Also, if the object is null or undefined, this will throw a TypeError (as pointed out by subzey on git).

a.call                   // Hey, this has a call property. Quack?
a.call && a.call.call    // The call also has a call so is probably also a function. Quack.
a.apply                  // As before but with apply this time
a.apply && a.apply.apply // ditto and ditto

String comparisons on typeof

This area of inspection is probably the richest for coming up with silly ideas to test. The typeof operator simply returns a string containing the type of the object. That’s it. Anything you can think of to compare a string against another string, be it RegEx, charAt, equals or threequals (===) can be manipulated to become a check for type.

typeof a>'f'&&typeof a<'g'   // Long-winded and quite silly. Performs a numerical comparison on the strings
(typeof a).charAt(0)=='f'    // Sturdy but not very useful.
typeof a==typeof eval        // May as well use eval for something, it’s a global function
typeof a==typeof dir         // Shorter but might not exist everywhere
typeof a=='function'         // The usual way to test. Everything above here is purely academic
/^f/.test(typeof a)          // Matching the string against a unique RegEx. See the table below
typeof a==typeof b           // Requires access to another variable which is a known function
(typeof a)[0]=='f'           // Small and robust but doesn’t work in IE6 or 7

Table of RegEx Patterns to match object types:

As little aside here, we’ve got a table of simple RegEx tests that do the same as the one mentioned above. They return true if the type is what you expect, false for all other types. They work by assuming things like ‘object’ being the only type to contain a ‘j’ or ‘boolean’ being the only one with an ‘a’.

Type         RegEx                 Note
boolean      /a/.test(typeof a)    // Shorter than typeof a==‘boolean’
function     /^f/.test(typeof a)   // Shorter than typeof a==‘function’
undefined    /d/.test(typeof a)    // Shorter than typeof a==‘undefined’
number       /m/.test(typeof a)    // Same length as typeof a==‘number’
object       /j/.test(typeof a)    // Same length as typeof a==‘object’
string       /s/.test(typeof a)    // Same length as typeof a==‘string’
null         /ll/.test(typeof a)   // Longer than typeof a==‘null’

Pick & Mix

This not only makes the assumption that an object is probably a function if it contains a ‘call’ but also that if that call has the same type as the object, they’re both probably functions.

typeof a==typeof a.call      // A mixture of typeof string comparison and duck typing

instanceof

In some circumstances, instanceof is going to be better than typeof as it compares types rather than strings.

a instanceof Function      // This will throw a ReferenceError if a is undefined.

The [[Class]] of the object

This comes from the JavaScript Garden where you’ll find they have a strong opinion on typeof and instanceof. This uses call to execute the toString method on the prototype of the basic Object constructor. Phew. At that point, you'd have a string ‘[Object Function]’. You can then chop off the beginning and the end using slice (treating the string as if it were an array) to get just the type. All together, it looks like this:

Object.prototype.toString.call(a).slice(8, -1);

Testing the string representation of the object

This is fairly nasty but still quite effective. Convert the object itself to a string (not its type but the actual object) and see if that begins with ‘function’. This is nowhere nearly as robust as some of the other tests as this will also pass true for any strings that begin "function..." but it’s quite cunning. Credit goes to Max for this one.

/^function/.test(a.toString())   //Test if the output of .toString() begins with ‘function’
/^function/.test(a+"")           //As above but using a shortcut to coerce the function to a string.

Running it

This isn’t so much checking whether it looks and sounds like a duck, this is more like serving it à l’orange and seeing if it tastes like a duck. The idea here is to actually try and execute it. If it throws an error, it’s not executable, if it doesn’t, it is. Or something like that. Here, we’re testing that the error is an instanceof TypeError as an undefined object would also end up in the catch.

The obvious downfall to this technique is that you don’t necessarily want the function executed when you’re testing it. In fact, you almost never want to do that. I might go as far as to say you never want that.

try {
  a();
  true;
} catch (e) {
  !(e instanceof TypeError);
}

The other big weakness in the above technique is that, even if the object is a function, the call itself might throw a TypeError. In Chrome, there's a bit more transparency as the error thrown has a type property. In that case you want to check that the type is 'called_non_callable' but that might still be a consequence of the function. In Safari, there's a similar property on the error (e.message) but the error object itself is no longer a TypeError, it is just an Error.

More…

I’m certain there are more. Many, many more. There are also several dozen that are trivial variations on those mentioned above – you could do the same string comparison tests on the [[Class]] property, for instance – but I missed these out. There’s probably a huge section missed out here (I'd forgotten instanceof existed until after the first draft of this post, for instance). If you can think of any more, let me know here or on The Twitter.

I'll also reiterate my point from earlier: most of these are deliberately useless or inefficient. The point here isn't to find better ways to do things, it's to practice doing what you do every day. The more you play while being a web developer, the less you need to work.

Dun-dun-Duuuuun

Or, to put it another way: Done.

After chatting with my co-conspirator in Museum140, I was finally convinced to do a list of stuff I started and finished in 2011. I'm usually a bit reluctant to write these kinds of things down because it borders on trumpet-blowing but at least this way, I'll have something to prompt me when I start going senile and remaking old ideas.

Personal

Before getting to the lists, I have to mention that this time last year I was living in Edinburgh, working at National Museums Scotland and being sleep-deprived by my newborn son. This year, I'm living in Berlin, working at Nokia Maps and being sleep-deprived by my teething one-year-old son.

My job at Nokia is seriously kick-ass. Aside from spending most days figuring out how to do cool stuff in clever ways, I've been getting actively involved in organising our weekly Tech Talks.

Websites

These are sites I built or helped build with Jenni or with the rest of my awesome team at Nokia.

• ceca.icom.museum
• shelvi.st (blog post)
• museum140.com
• Nokia Maps City Pages

Tools

Things I built to make my life easier which I hope others might find useful.

• Scoped CSS polyfill (blog post)
• TweetArchiver
• 8-bit Alpha (blog post)

Book

Although I first published the book last year, this year, I did try out the ‘Read now, Pay Later’ experiment this year. I'll let you know how that's going later.

• Explanating

Video

• HTML 5 for Large Public Bodies

Digital Toys

These are the most fun bits. The silly, experimental games, gadgets and fun ways to waste time.

• Insta-Art
• Knot (blog post)
• Torch (blog post)
• Sine (blog post)
• 3D CSS Mario Kart (blog post)

Still to do...

Get the Nokia Web Dev blog off the ground. Don't currently have the slightest idea how to go about it but we've got some world-class webdevs here and we should share some of those smarts.

Write articles for other people. I write a lot and, often, my only editor is myself. I have no idea if any of this is any good to anyone. The best way to find out is to try writing for another editor some time.

Present a lot more. As someone who, at one point, used to make his living standing in front of a theatre full of people being funny at them, I kinda miss that in my day-to-day.

Not move country. Seems like a simple plan but I've failed at it 3 out of the last 5 years.

Latest 140byt.es offerings

Long-term readers (a.k.a. my Mum) might remember my JS1K efforts last year and the subsequent discussion of Extreme JS Minification. The same kind of mindset that inspired qfox to create that competition also inspired Jed to create 140 Bytes. As with JS1K, the clue is in the name, you have the length of a tweet – 140 characters – to create something cool or clever in JS.

I won't go into any more detail here, if you're interested, you've either already heard about it or would do better clicking around the 140bytes site or reading the comments on the master gist.

My entries

Following last week's draggable file script, I started messing about with making it small enough to qualify and, predictably, got hooked. Here are four I made last weekend.

The scripts below are directly embedded from GitHub and are the minified, highly compressed versions. I recommend clicking through to the original gist for each of them where you'l find a file called annotated.js which talks through the code.

Create a draggable file on-the-fly

The code from last week shrunk down.

Detect Doctype

There's no simple way to access the current document's doctype as a string. There is already a document.doctype object so you might expect (or, at least I'd expect) you could do a .toString() or .asDoctypeString or something like that. Nope. Nothing. You have to manually recreate it as a concatenation of properties. A quick discussion with kirbysayshi and mmarcon came up with a few alternative methods (Max's is quite devious, actually) before eventually culminating in this.

Chainify and Propertize

This began as a very minimal implementation of Lea Verou's Chainvas. The idea is to enhance various constructors (Element, Node, etc.) so that each method returns the original object. This means that you can then chain (jQuery-style) any built-in function. Each constructor is also enhanced with a .prop function which allows property setting in a similarly chainable manner. For a better description, read through the Chainvas site.

Homemade Lanyard

I'm currently looking for new earphones as my Klipsch ones finally gave out on me last week. Until I get some (Christmas is coming, after all), I decided to rummage around in my 'retired headphones' box (everyone's got one, no?) and found this:

Back when I was using my second generation iPod nano, I needed something that combined the usefulness of the lanyard attachment but with good quality headphones. As I couldn't find anything, I got a pair of Sennheiser CX300s, an old USB connector cable and a metre of red ribbon and made my own. Once the USB end was cut off, I looped it round and glued it ends together then wrapped the join in plumbers' PTFE tape to seal it. The headphones were wired round, fastened with cable ties and then sewn together with the ribbon to make the neck bit comfortable. The iPod connector has two little clips in it which was plenty to hold the lightweight iPod Nano in place under my t-shirt and on that model, the headphone jack was on the bottom so the whole thing clipped together nicely.

The sound quality was as good as you'd expect from Sennheisers and I actually surprised myself with the build quality when I had to, yesterday, dismantle them after almost 4 years. Unfortunately, the iPhone 4 is a little bit too chunky to wear under a t-shirt. I got some odd looks when I tried.

Create a draggable file on-the-fly with JS

Here's a useful little code snippet if you're building a web application. It's a simple way of making the boundary between web-browser and platform a bit smaller. It allows you to create a file (text, html, whatever) in in your page which the user can drag onto their desktop (if their browser supports the dragstart event and dataTransfer methods).

document.getElementById('downloadElement').addEventListener("dragstart", function (e) {
  e.dataTransfer.setData("DownloadURL", "text/html:filename.html:data:image/png;base64," + btoa(fileContent));
});

A description of the code:

• attach an event listener to the draggable element you specify (downloadElement)
• when you start to drag it (dragstart),
• it creates a dataTransfer object (with the type DownloadURL)
• and sets the content of that to be whatever you pass it (fileContent)
• It uses btoa() to encode the string data as a base64-encoded string.
• When you combine this with the MIME-type (text/html),
• you can create a file with the specified name (filename.html) when the user releases the drag in their local file system.
• The fake MIME-type (image/png) is there as part of the object data to convince the browser this is a binary file (which it is, even though it's not an image).

Note: The element this is attached to (downloadElement above) should either be draggable by default (image, link or text selection) or have the draggable attribute set otherwise this'll do nothing.

Credit goes to Paul Kinlan for using this in Appmator which is where I first saw this done this way. He actually uses it alongside some extremely clever JS zip-file creation stuff, too, it's definitely worth digging through the source there.

You can find out more about the drag events on the MDN.

Simple and clever beats clever

When messing about with another little game idea, I found myself retyping (for the umpteenth time) the same little bit of value bounding code I use a lot:

var x = Math.max(0, Math.min(y, 1))

"Set x to be the value of y as long as it's between 0 and 1. 
If not, set it to 
  0 if it's smaller or 
  1 if it's larger."

Of course, 0 and 1 don't need to be the boundaries, I'm just using them for convenience.

Instead of continuing with the game, I decided to take a little tangent and see if there was any way I could rewrite this so that the syntax was a bit more obvious. I'd like to be able to use syntax like:

x = [0 < y < 1]

to mean the same. Written exactly as above, JS will try and evaluate left-to-right, changing the comparisons to booleans. The statement would become

y = 0.5

x = [0 < y < 1]
x = [0 < 0.5 < 1]
x = [true < 1]
x = [false]

Similarly:

y = -0.5

x = [0 < y < 1]
x = [0 < -0.5 < 1]
x = [false < 1]
x = [true]

My first thought was to be clever about it, I wanted to try and figure out how to partially evaluate the expression and take the different outcomes to figure out the logic required. If '0 < y' was false, then y is less than zero therefore outside our bounds, the return value should then be 0. If the first part is true and the second is false then we know the value is higher than our bounds....etc and so on.

This proved to be a logical dead-end as there was no good way to partially evaluate the statements. Not without preparsing the JS, anyway. Which leads me onto the second attempt...

Preparsing the JS

The next attack on the problem was the idea of reading the JS as a string, shuffling it around quickly and silently (not like a ninja, more like a speedy librarian in slippers) and put it back where it was.

So I began to look at ways to recreate that. I remembered from many, many years ago (two, actually) Alex Sexton creating the genius abomination that is goto.js and how that used some kind of preparsing. A quick skim through the code later and I ended up on James Padolsey's site looking at parseScripts.js.

In the end, all I needed to do was include parseScripts (which is a generalised form of the code I ended up using for the whitehat SEO question from last month) and provide a new parser function and script type.

parseScripts(/:bound/, function(unparsed) {
 return unparsed.replace(/\[(\d)\s*<\s*(\w)\s*<\s*(\d)\]/g, "Math.max($1, Math.min($2, $3))");
})

I'm not saying parseScript isn't clever because it most definitely is but I am saying it's simple. There's not always a need to branch off into deep technical investigations of partial evaluation when a simple search-and-replace does the job better and faster.

For someone always going on about bringing simplicity and pragmatism into development, you'd think I'd have gotten there faster...

Bounded Range

The final bounded range syntax code is available here (hint: view the console). It's silly but it was a fun half-hour.

Improvements?

Do you know of any better way to do this? Is there a clever parsing trick we can use instead? Is there, indeed, any language which has this syntax?

cssert – Like ‘assert’ but with CSS at the front

This continues on from my post about CSS verification testing

cssert – pronounced however you feel like it – is my attempt at bringing some kind of style verification into an automated build process. If you've read the previous article, you'll know that this falls into the second group of CSS test frameworks, style measurement and comparison. The system works exactly as I described above – your test files have a basic HTML structure created by traversing the DOM from the element being tested upwards. You could also include your entire HTML in the test file if you liked, it would just be unnecessary in most cases.

I've created a couple of (for me, at least) helpful utilities which allow these test cases to be generated from a page via a bookmarklet and then run in the browser or on the command-line. Running the tests in the browser is useful for quick human verification while the command-line interface can be integrated into an automated build system if you like that kind of thing. The test file structure is quite simple (all samples here taken from the Twitter Bootstrap project:

First, we have the test file opening structure:

<!doctype html><html><head><title>cssert test page</title><link rel="stylesheet" href="../lib/cssert.css"></head><body><h1>cssert Test cases</h1><p>click to expand test</p><script type="text/html">/*==

Then we have the name of the test:

Intro Paragraph

Then we have the skeleton DOM:

<!doctype html><html><head><meta charset="utf-8"><base href="http://twitter.github.com/bootstrap/examples/hero.html"><link href="../1.3.0/bootstrap.css" rel="stylesheet"><style type="text/css">
      body {
        padding-top: 60px;
      }
    </style><style>#cssert-style-modal {display:none;position: fixed;top: 10%;left: 50%;margin-left: -350px;width: 700px;background: #39c;color: white;padding: 10px;color: #fff;text-shadow: 0 1px 0 rgba(0,0,0,.3);background-image: -webkit-linear-gradient(-45deg, rgba(255,255,255,0), rgba(255,255,255,.1) 60%, rgba(255,255,255,0) 60%);border-radius: 5px;border: 1px solid #17a;box-shadow: inset 0 0 0 1px rgba(255,255,255,.3);}#cssert-style-modal ul,#cssert-style-modal li {margin:0;padding:0;font-size:11px;list-style:none;}#cssert-style-modal>ul>li {float:left;width:140px;font-size:13px;}#cssert-style-modal ul {margin-bottom:10px;}#cssert-pre {position:fixed;top:10px;right:10px;background:white;border:1px solid gray;width:200px;height:200px;overflow:auto;}#cssert-drag {position:fixed;top:210px;right:10px;background:white;border:1px solid gray;width:200px;height:20px;overflow:auto;}</style></head><body><div class="container"><div class="hero-unit"><p>Vestibulum id ligula porta felis euismod semper. Integer posuere erat a ante venenatis dapibus posuere velit aliquet. Duis mollis, est non commodo luctus, nisi erat porttitor ligula, eget lacinia odio sem nec elit.</p></div></div></body></html>

The CSS selector identifying the element to verify:

html body div div p

And the styles we wish to verify:

{"font-family":"'Helvetica Neue', Helvetica, Arial, sans-serif","font-weight":"200","font-style":"normal","color":"rgb(64, 64, 64)","text-transform":"none","text-decoration":"none","letter-spacing":"normal","word-spacing":"0px","line-height":"27px","text-align":"-webkit-auto","vertical-align":"baseline","direction":"ltr"}

A test file can contain as many test units as you like. At the very end is the close of the test file structure

*/</script><script src="../lib/cssert.js"></script></body></html>

You'll probably notice the crucial bit in the test structure is the base element. The CSS available from the location specified here is the thing we are actually testing. In typical test lingo, the structure we have in our test file is the mock and our tests work by asserting the values ‘output’ from applying the CSS to this structure are as expected.

Running the tests

Running the tests in-browser

Open the test page in a browser. That's it. If it's green and says 'passed', the test passed. If it's red and says 'failed', the test failed. You can see the output by clicking on the title of the test.

This works by loading the test file, creating an iframe and injecting the test case into the iframe as source. It then looks into the iframe and measures the styles. If they match those specified in the test file, it passes, otherwise, it fails. Clicking on the test title simply removes the position:absolute which is hiding the iframe.

Running the tests on command-line

The exact same test page can also be used with the command-line interface. cssert uses PhantomJS to run the tests in a headless webkit instance. You'll need to install PhantomJS into your path after downloading it. Place your test case in the tests folder and run:

$ ./cssert testcase.html

To run all tests in the tests folder at once, simply run with no arguments:

$ ./cssert

This works by, again, loading the HTML from the test files. In this case, the structure is injected into a new headless browser window. The styles are measured and the output is redirected to stdout. Each browser window test is also rendered as a PNG so you can see what failed if any did.

Limitations

I'm not saying this is the ultimate solution to CSS testing. Declarative languages don't sit well with testing. This is as close as I can get for the moment. I'm also not going to be able to head off or counter all possible complaints or criticisms but I will cover a couple.

Firstly, most of the limitations you'll run into are caused by using the automatically generated tests. They're good for creating a starting point but at the moment, they need to be tweaked for many cases.

Sibling selectors

Because the test generates the DOM via following parents up the structure, sibling elements are ignored. These styles are still testable, though. Simply add the sibling element into your HTML test block.

Styles modified by JS

The styles are measured on the element as it is when the case is generated. The test compares this against the styles provided by the CSS. If the element contains JS-only styles not added by CSS, they will not be correctly matched. Modify your test case to allow for this.

Why not use Selenium?

This, essentially does the same as Selenium would do if you took the time to set up your test cases. This makes it much easier to set up test cases, though.

@font-face!

If your @font-face declaration contains a suggested 'local' source (as recommended in Paul Irish's bulletproof syntax), a bug in QTWebkit will prevent the test case from running correctly.

Installation

Just clone the git project from git@github.com:thingsinjars/cssert.git and you're good to go.

The tests directory comes with some sample tests generated using Twitter's Bootstrap project. Put your tests in that same place.

CSS Verification Testing

It's difficult to test CSS. In Nokia Maps, we use lots of different kinds of automated tests to ensure the build is doing what it's supposed to do – Jasmine for JS unit tests, jBehave for Java acceptance tests, a whole load of integration tests and systems to make sure all along the process that we know as soon a something goes wrong.

CSS has typically been left out of this process – not just in our team but generally throughout the industry – because it's a difficult thing to test. CSS is a declarative language meaning that the CSS of a page doesn't describe how the page is to accomplish the task of turning text red, for example, it simply says 'the text should be red' and the implementation details are left up to the browser. This doesn't fit well into typical testing strategies as there is no sensible way to pass information in to check the outputs are as expected. With a declarative language, there is very little logic around which you can wrap a test. In essence, the bit you would normally test is the bit handled by the browser. About the closest CSS gets to having continuous-deployment style tests is to run your files against the in-house style guide using CSSLint.

That's not to say people haven't tried testing CSS. There are lots of opinions on how or why it should be done but with no concrete answers. There has also been a lot of thinking and quite a lot of work done in the past to try and solve this requirement. This article from 2008 and this article from 2006 both propose a potential route to investigate for testing strategies.

In summary, the two main approaches used are:

• Generate images from the rendered HTML and compare differences (c.f. css-test by Gareth Rushgrove)
• Specify expected values and compare actual values (cssUnit, CSSUnit)

There must be something about the desire to test CSS and the name Simon as both Simon Perdrisat and Simon Kenyon Shepard created (separate) unit-testing frameworks called 'CSSUnit'. And there's me, obviously.

As a related aside, there may also be something to do with working at Nokia and wanting to test CSS as Simon Shepard is the Tech Lead on Nokia Maps although I stumbled across his project completely randomly outside work. Weird.

Another important related note: there's no point in developing a CSS testing framework for Test-Driven Development. Again, this is an aspect of being a declarative language but, by the time you've written your test, you've written your code. There's no Red-Green pattern here. It either does what you intended it to or it doesn't.

In essence, the only way to test CSS is by verification testing – the kind of thing you do before and after refactoring your styles. This, essentially, involves working within your normal process until the job is done then creating 'snapshots' of the DOM and capturing the current cascaded styles. You can then refactor, rework and reorganise your CSS as much as you like and, as long as the combination of snapshot DOM plus CSS produces the same results as before, you can be confident that your entire system still appears the same way.

Get to the point...

Why the long ramble about CSS testing strategies? Well, unsurprisingly, I've had a go at it myself. My approach falls into the second category mentioned above – measure styles once finished and create secure test-beds of DOM structure which can have CSS applied to them. The test system is currently being run through its paces in Nokia Maps City Pages (my bit of maps) and, if it passes muster, I'll put it out into the big, wide world.

Whiteboard Laptop Lid

I use whiteboards a lot. Whether I'm coding, explaining some concept or other, sharing the results of a meeting, wherever. If there's a whiteboard nearby, I'm quite likely to jump up and start drawing big diagrams with lots of arrows and wavy lines. When there's not a whiteboard, I still jump up but I tend to lean more towards big handy gestures drawing diagrams in the air (I recently watched a video of myself presenting with the sound turned down and I looked like an overenthusiastic mime artist dancing to 'Vogue').

To make sure I always have a whiteboard to hand, I roped in Jenni to help with a little home craft-making.

D.I.Y. Laptop-lid Whiteboard

Blue Peter style list of things:

You'll need scissors and/or a craft-knife, double-sided sticky tape, a measuring tape, a bit of sturdy white cardboard or a thing sheet of opaque PVC and some sticky-backed clear plastic. You'll also need a laptop and an grown-up to help you with the cutting.

First, measure the top of your laptop and figure out how big your whiteboard can be and draw that on your cardboard or plastic (from now on referred to as 'white board').

Next, cut your cardboard or plastic to the right size. Remember to measure twice and cut three times or something like that. I can't remember exactly but there some ratio of measuring to cutting. Do that.

If you're using a piece of shiny PVC or something like that, you can miss this next bit. If you're using cardboard or something else, you'll need to cover it with the transparent sticky-backed plastic.

The final preparation stage is to put the double-sided sticky tape on the back and position it on your laptop lid.

Try not to take any photos one-handed while doing this step or this may happen:

There you go. A portable, take-anywhere whiteboard in 10 minutes. Apart from needing a pen, of course, and a cloth to wipe it so you don't end up with smudges all over your laptop bag, we're done.

I can now take my whiteboard with me everywhere I go for a meeting. Long may the diagramming continue.

Super Mario Kart CSS

Silly CSS Fun for a Sunday Afternoon

Yesterday, I decided to mess around with 3D CSS transforms. I've used them here and there for various things (the flip animations in Shelvi.st, for example) but nothing silly.

My mind wandered back to an early HTML5 canvas demo I saw ages ago where Jacob Seidelin had written Super Mario Kart in JS and I wondered if it would be possible to do the pixel-pushing part of that demo in CSS.

An hour later and we have this:

Super Mario Kart CSS

Yes, it's silly. Yes, it's nothing like playing Mario Kart and, no, there isn't any acceleration. That wasn't the point, however. View the source to see the description of the rotations and transforms.

Poor man's touch interface

Here's a little code snippet for today.

While I was making the 3D CSS Mario Kart, I needed a simple, drop-in bit of code to handle touch interfaces. I looked through some old experiments and found a rough version of the code below. It was based on this KeyboardController by Stack Overflow user Bob Ince.

It doesn't do anything clever. All it does is provide a simple way to attach functionality to touch events in different areas of the screen – top-left, top-center, top-right, middle-left, middle-center, middle-right, bottom-left, bottom-center, bottom-right. My apologies to any Brits for the spelling of centre as 'center'. It's the Internet, we have to.

How to use

Include this code:

function TouchController(areas, repeat) {
  var touchtimer;
  document.onmousedown = document.ontouchstart = document.ontouchmove = function(e) {
    var position;
    e.preventDefault();
    e.touches = [{'clientX':e.pageX,'clientY':e.pageY}];
    switch(true) {
    case (e.touches[0].clientY<window.innerHeight/3) : 
        position = 'top';
        break;
    case (e.touches[0].clientY>(2*window.innerHeight)/3) : 
        position = 'bottom';
        break;
    default : 
        position = 'middle';
        break;
    }
    position+='-';
    switch(true) {
    case (e.touches[0].clientX<window.innerWidth/3) : 
        position += 'left';
        break;
    case (e.touches[0].clientX>(2*window.innerWidth)/3) : 
        position += 'right';
        break;
    default : 
        position += 'center';
        break;  
    }

    if (!(position in areas)) {
      return true;
    }

    areas[position]();
    if (repeat!==0) {
      clearInterval(touchtimer);
      touchtimer= setInterval(areas[position], repeat);
    }
    return false;
  };
  // Cancel timeout
  document.onmouseup = document.ontouchend= function(e) {
  clearInterval(touchtimer);
  };
};

Now, all you need to do to attach a function to a touch event in the top-left area of the screen is:

TouchController({
  'top-left': function() { topLeftFunction();}
}, 20);

I use this for direction control in the Mario Kart experiment which maps exactly onto the cursor keys used for the normal control.

TouchController({
  'top-left': function() {  // UP + LEFT
				drawMap.move({y: 2}); 
				drawMap.move({z: drawMap.z + 2}); 
				drawMap.sprite(-1) 
			  },
  'top-center': function() {  // UP
				drawMap.move({y: 2}); 
			  },
  'top-right': function() {  // UP + RIGHT
				drawMap.move({y: 2}); 
				drawMap.move({z: drawMap.z - 2}); 
				drawMap.sprite(1) 
			  },

  'middle-left': function() {  // LEFT
				drawMap.move({z: drawMap.z + 2}); 
				drawMap.sprite(-1) 
			  },
  'middle-right': function() {  // RIGHT
				drawMap.move({z: drawMap.z - 2});
				drawMap.sprite(1) 
			  },

  'bottom-left': function() {  // DOWN + LEFT
				drawMap.move({y:  - 2}); 
				drawMap.move({z: drawMap.z + 2}); 
				drawMap.sprite(-1)  
			  },
  'bottom-center': function() { 
				drawMap.move({y:  - 2}); 
			  },
  'bottom-right': function() {  // DOWN + RIGHT
				drawMap.move({y:  - 2}); 
				drawMap.move({z: drawMap.z - 2}); 
				drawMap.sprite(1)  
			  },
}, 20);

If you need anything clever or you need two or more touches, you should use something else. This is just a simple drop-in for when you're copy-paste coding and want to include touch-screen support.

Silly CSS Gradient

Okay, here's a bit of silliness. I did a little presentation yesterday with some slides which I built using Hakim El Hattab's 3D CSS Slideshow. I decided to have some fun with the last slide.

The Looney Tunes “That's all Folks” end title using radial gradients.

I was doing my presentation using Chrome so I'm allowing myself some leeway that it's not fully cross-browser.

That Version-y Control-y Stuff

If you’re reading this site, you’re probably not a beginner. The following article is probably beneath you. Go on, read it anyway.

Just in case you don’t know, a Version Control System (VCS) is pretty much the most important thing ever in the history of anything ever. Just think about it conceptually: the ability to stand back and look at everything you’ve ever done, choose the good bits, drop the bad bits, mop up the mistakes… it gives you the freedom to “Play with it ‘til it breaks” with the safety net that you can always go back and fix it again.

You Get To Play With Time Itself!

There are many, many debates about the different methodologies you can use with version control and which technologies are best suited to which methodologies and which interfaces are best suited to which technologies and so on ad nauseam. The main concepts of working with any VCS are essentially the same, however.

1. It must be almost invisible.
2. It must be completely reliable
3. You mustn’t rely on it

It must be almost invisible

If you have never used a VCS before, it must fit perfectly into your workflow otherwise you won’t do it. It isn’t until you’ve been saved from potential disaster by a system that you will truly appreciate it and see the value in working it into your process. If you have or are currently using a VCS, think about when and how you use it. Does it require launching a new application or does it run in the background? Do you have to manually check things in or is it seamless versioning? The more effort required to mark a version, the less often you’ll do it as it breaks the flow of development

On the other hand, if the process is completely invisible, you might forget it’s there. It’s exactly the same as assuming your changes are going live on the server – the moment you assume it happens, it doesn’t. You still need some level of manual checking.

It must be completely reliable

This is fairly obvious, hopefully. You need the security that your version system is working otherwise you might again be tempted to miss it out – why take the extra step required to version a file if it’s not going to be there when I need it?

If you’re hosting your own repositories internally to your organisation, don’t put it on a server older than you are. The safest route to go down is contracted external hosting. That way, it’s someone’s job to make sure your datums are secure.

You mustn’t rely on it

Always have a backup. Always have a copy. Always have an escape route. Or something. You should have picked up the theme by now. Version Control Systems are great but as soon as it is the only thing standing between you and a complete failure to deliver a project, it will fail. Or not. It’s funny like that.

Copy-paste coding

This weekend, I had a spare couple of hours on Saturday night (as often happens when you have a kid) so I decided to pass the time with a bit of copy-paste coding.

I grabbed a bit of example code for a top-down racing game built using Box2d and GameJs. I then grabbed a Google Maps demo and a Nokia Maps demo and smooshed them together. I've seen a few racing games before so I know there's nothing innovative here, it's just a bit of silliness.

The results

• Driving on Nokia Maps
• Driving on Google Maps

Copy-Paste: Not a bad thing

There are many developers who will tell you that blindly copy-pasting is a terrible way to code. They make the valid points that you don't understand something if you don't read it properly and that you'll end up stuck down an algorithmic cul-de-sac with no way to three-point-turn your way out (although they may phrase it differently). These are valid points but...

If I'd sat down on Saturday night and thought "I want to build something in Box2D" then gone to the site, downloaded the library, read the docs and loaded the examples, by the time I had understood what I was doing, Sunday morning would have come round and I'd still have GameJS to read up on. There's absolutely no harm in blindly catapulting yourself a long way down the development track then looking round to see where you are. Sure, you'll end up somewhere unfamiliar and you may end up with absolutely no clue what the code around you does but at least you have something to look at while you figure it out. At least a few times, you'll find something almost works and a quick search of the docs later, you've figured it out and know what it's doing.

Basically, copy-pasting together a simple little demo and making it work is a lot more interesting than taking baby-steps through example code and beginner tutorials. Don't be too careful about trying to build something complicated.

An Idea

If you're in the mood for copy-paste coding, try taking the two demos above (Google and Nokia), grab the node.js server from the multi-user page experiment and figure out how to have multiple racers on the same map (by transmitting angle and geo-coords).

Sine

Another game concept prototype – Sine

Honestly, I have no idea what I was going for with this one. It started off last weekend with a vague idea about matching patterns of numbers and old-school graphics and I don't know what and ended up with this.

The idea is to make the bottom row match the top row, basically. There are several front-ends to this game so you can choose the style of play you prefer - numbers and letters, waves, colours or a generated sound wave (if you have a shiny new-fangled browser). It uses the nifty little Riffwave library to generate PCM data and push it into an audio element.

Further development

If I were to develop this further, I'd try and build it in a modular fashion so that front-ends could be generated really easily and open it to other people to see how many different ways this game could be played. It'd be an interesting social experiment to be able to play what is fundamentally the same game in a dozen different ways. You could find out if visual thinkers processed the information faster than numerical or audio-focused people. Leaderboards could allow different styles of player to compete on the same playing field but with a different ball (hmm, weak sports analogy). The rhythms of the game lend themselves well to generated drum tracks so there's probably something in that area for exploring as well.

At the moment, the code is written as the programming equivalent of stream-of-consciousness – global variables everywhere, some camel-case, some underscored, vague comments sprinkled around. There's some commented-out code for the wave mode that moves the waves closer together so that there's a time-limit set but I felt it didn't suit the game.

Warning

The audio mode is very annoying. Seriously. If I were still a full-time game designer, I would not release this due to public health concerns. As I'm now an enthusiastic amateur, here you go :D

Explanating

I never really marketed it much but I wrote a book called ‘Explanating’ a couple of years ago. I even decided to self-publish1 and organised ISBNs and everything.

1My decision to self-publish may or may not have been related to my inability to find an actual publisher, it's impossible to tell.

The book is an "illustrated collection of completely plausible but entirely untrue explanations of everyday phenomena". Basically, it's lots of the kinds of things you might make up to explain something to a kid if you really have no idea. Hence the name ‘Explanating’ – it's kinda like explaining but not quite right. It also has a rather nice cheesecake recipe in the appendix. I put it on Lulu and Amazon and didn't really do anything else with it. I did try to get it in the iBookstore but that seems to be a horribly complicated process if you aren't based in the US.

Now available for the low, low price of...

Rather than have the book sit around for another few years not doing anything useful, I've decided to try something new. You can now download the book for free from explanating.com in your ebook format of choice (PDF, ePub, Mobi). You don't have to pay anything.

Unless you want to.

If you read it and decide you like it, you can come back to the site any time and buy it from Lulu or Amazon for £1.71 (or the equivalent in USD or EUR or wherever you happen to be). Or not, you could like it and keep it and not pay anything. It's entirely up to you. And your conscience :D.

Download, read, buy

I have no idea if anyone will actually take me up on this offer but I hope some people do, at least, enjoy the book. If nothing else, make the cheesecake, it's delicious.

And that URL once more

explanating.com

HighlightBlock.vim

tl;dr: Here's a Vim plugin to highlight style and script blocks in HTML

Okay, this'll be my last vim post for a while. I just couldn't leave it alone, though.

My Cobalt theme was good but it wasn't quite enough. You'll see from the screenshots that the entire HTML page had the same background colour in Vim while the TextMate version changed the background colour inside <style> and <script> blocks. I was surprised how much that bugged me so I figured there must be a way to highlight an entire line. It turns out this isn't a trivial thing to do. Syntax matches will only match to the final character in the line ($), not the width of the screen. No amount of tweaking a colorscheme would allow highlighting all the way across.

After a lot of digging around, I found out about signs. This is a built-in feature which allows you to add a marker to a line for whatever purpose you want. It can point out debugging information or provide visual checkpoints to mark out things in your document. It's probably very handy but as I've only just started using vim, I don't really know what it's best for. However, as a side-effect, it can also apply a style to the entire screen-width of a line.

Some googling, hacking and probable violation of Vim plugin best-practice, I knocked together this plugin:

HighlightBlocks.vim

When this is installed, it will highlight any embedded CSS or JS blocks in a HTML, PHP, Velocity templates or Ruby files. Well, it will apply the syntax class 'HighlightedBlock' to the line. If your theme has a style for that, it will highlight it. Incidentally, I updated the Cobalt port to include that style.

It runs on initial load then refreshes every time you exit insert mode.

I might update it later to highlight PHP blocks in HTML or some other things like that but for my current purposes, it's finished.

Warning

• It pays no attention to existing signs. If you use them, you probably shouldn't use this. If you know of a simple way to group signs together to stop me messing with them, let me know.
• When signs are added to a file, an extra two-character column appears to the left of the line numbers. This plugin shrinks the line-numbers column by two characters if signs exist and increases it again when they are removed. This stops everything from jumping around but if you're working on a 10,000 line file, you might see some confusion down the bottom.

Installation

As with the Cobalt theme, if you're using Janus, add this to ~/.janus.rake, I still have no idea if this works. It might. :

vim_plugin_task "HighlightBlock.vim", "git://github.com/thingsinjars/HighlightBlock.vim.git"

Text Editors

Many years ago during the text-editor holy wars, I sided neither with the Vi Ninjas nor the Emacs Samurai but instead went a third way – Pico (short for Pine Composer). It was the text editing part of the e-mail program I used (Pine). For many years, this served me well. Even today, Pico's successor - Nano – is installed pretty much everywhere. It isn't , however, quite powerful enough for fast-paced web development. Serious, full-time development needs shortcuts and macros, syntax highlighting and snippets. When you spend 10 or more hours every day pressing buttons to change flashing lights, you need to optimise the way the lights flash.

After Pico, I found Crimson Editor which served me well for almost 10 years. I eventually started working on a Mac and became a TextMate user for most of the last 5 years.

In my new job, I find myself jumping from computer to computer to desktop to server quite a lot. The only constant editor available is Vi. Or Vim (Vi Improved). I've been trying to pick it up as a way to ensure I can always jump into a friendly text editor no matter where I am. Besides, these days Vim is the old-new-cool thing that all the cool-old-kids use, particularly MacVim so I thought it was worth giving it a go to see what the fuss was about.

Incidentally, the first thing that always jumped into my head whenever anybody mentioned Vi was the Vim Golf challenge two of my lecturers played while I was at Uni. It's a well-known and surprisingly popular game where two Seventh-dan Vi proficients try to accomplish an edit on a file using the fewest keystrokes. Each morning, one lecturer would lay down a challenge and the other had 24 hours to come up with a solution before presenting it and the next challenge the following day. The (possibly apocryphal) conclusion to the long-running game came when it was discovered that the lecturer who won most often was actually recompiling the network install of Vim every night to include his own custom keyboard shortcuts.

One of the biggest deciding factors in trying it out was actually fellow Edinburgh Web Dev/International Traveller Drew Neil (@nelstrom), creator and voice of the vimcasts.org series of screencasts who is actually writing a book on Vim this very second as I write this. Most people are evangelical about their choice of text editor to the point of rabid fundamentalist, frothing-at-the-mouth, intoning-keyboard-shortcuts craziness (hence my allusions to text-editor holy wars). When I mentioned to Drew that I used Pico instead, his response was long the lines of "Fair enough". This lack of confrontation actually inspired me to try it out. Well played, sir.

In an unrelated aside, I once did an illustration for the 'Collective Nouns Illustrated' project Drew ran.

Anyway, I'll give it a go and see what happens. If you're interested, I recommend reading Yehuda Katz' post 'Everyone Who Tried to Convince Me to use Vim was Wrong'. Don't worry, I'm definitely not going to try and convince anyone to use one code editor over another. You should probably stop using FrontPage, though.

Cobalt.vim

What?!

Another port of the TextMate theme 'Cobalt' to Vim?

But there are already so many!

Yes. Yes there are. However, the other ones I found were all missing something. Some had the right colours but in the wrong places, some had most of the right things highlighted but in slightly wrong colours. None of them had coloured the line-numbers. I think this is the most complete port. The initial work was done automatically using Coloration and then manually tweaked and added to. I know I should probably have just picked one of the existing GitHub projects, cloned it and pushed to it but I feel it would be a tad presumptuous of me to just turn up in someone's repository one day saying "yeah, you did alright but mine's better". Besides, I've never tried making a theme for Vim before. I've probably done something wrong (see below). If it continues to work for a while without causing any major issues, I might look at pushing it to another repo.

This was done on top of a vanilla install of the excellent Janus configuration of MacVim so whatever plugins are installed by default may have an affect on this.

There are a few limitations in the syntax files enabled by default so this includes a couple of matches, regions and things that really shouldn't be in a colorscheme file but I've included them because I felt like it and it was the only way to really match some of the highlighting TextMate allows (this is the thing I was referring to above).

I haven't really touched the NerdTree colouring much as it's probably impossible to have different background colours in different panes. Can't guarantee that, though.

Installation

There are probably some conventions to do with how to organise a GitHub project so that it can be automatically installed but I've just gone with sticking it in a directory called 'colors' so it can be pulled in from the root of your ~/.vim directory.

If you're using Janus, add this to ~/.janus.rake, I think it'll work:

vim_plugin_task "Cobalt.vim", "git://github.com/thingsinjars/Cobalt.vim.git"

If you're not using Janus, you probably know what you're doing anyway.

The GitHub project is here:

• github.com/thingsinjars/Cobalt.vim

Screenshots

The first of each pair is Vim, the second is the TextMate original.

Editing CSS

Editing HTML

Whitehat Syndication

When I refer to Google or Googlebot here, I could really be talking about any web search or web crawler. When I refer to Yahoo, I actually mean Yahoo. Don't get confused.

I recently ran into an interesting SEO problem on a project which has led to a question I just don't know the answer to:

How do you display syndicated content without triggering Google's duplicate content flag?

Hmm... intriguing.

Background

To explain the problem more fully (without giving out any project specifics), imagine you have a website. You probably already do so that shouldn't be hard. Now, imagine you fill that website full of original content. Again, that shouldn't be hard. For the sake of example, let's assume you run a news blog where you comment on the important stories of the day.

Next, you figure that your readers also want to read important related facts about the news story. Associated Press (AP) syndicates its content and through the API, you can pull in independently-checked related facts about whatever your original content deals with. So far, so good.

Unfortunately, a thousand other news blogs also pull in the same AP content alongside their original (and some not-so-original) content. Now, when the Googlebot crawls your site, it finds the same content there as it does in a thousand other places. Suddenly, you're marked with a 'duplicate content' black flag and all the lovely google juice you got from writing original articles has been taken away. Boo.

Your first thought might be to reach for the rel="canonical" attribute but that really only applies to entire pages. We need something that only affects a portion of the page.

Solution

What you need to do is find a way to include the content in your page when a visitor views it (providing extra value for readers) but prevent Google from reading it (hurting your search ranking). Fortunately, there are some methods for doing this. One involves having the content in an external JS file which is listed in your robots.txt to prevent Google from reading it. Another similar method involves having the content in an external HTML and including it as an iframe, again, preventing crawling via robots.txt. When the reader visits the page, the content is there, when Google visits, it isn't.

The Problem with the Solution

Both of the techniques mentioned here involve an extra HTTP request. You are including an external file so the visitor's browser has to go to your server, grab the file and include it. This isn't a huge problem for most sites but when you're dealing with high-traffic, highly-optimised websites, every file transferred counts. You go to all the trouble of turning all background images into sprites, why waste extra unnecessary connections on content?

There is an extra problem with the JS solution in that the content isn't available to visitors that don't have JS enabled but as I've mentioned before, that's only a problem if any of your visitors actually have JS disabled. Only you know that for sure.

Yahoo's Solution

Yahoo have a nice solution to this problem. If you include the attribute class="robots-nocontent" on any element, the Yahoo spider (called 'slurp') will ignore the content. Perfect. This does, however, only work for Yahoo. Not perfect.

My solution

My attempt at solving this problem which is a combination of SEO and high front-end performance was inspired by the technique GMail uses to deliver JS to mobile devices. In their article, Google delivers JS that they don't want run immediately in the initial payload. They figure that the cost of serving a single slightly larger HTTP request is less than the delay in retrieving data on demand.

I use HTML embedded in a JS comment in the original page which is then processed on DOMReady to strip out the comments and inject it into wherever it is supposed to be (identified by the data-destination attribute). I'm doing this on a page which already loads jQuery so this can all be accomplished with a simple bit of code.

<script type="text/html" class="norobot" data-destination=".content-destination">
 /*!
  <p>This content is hidden on load but displayed afterwards using javascript.</p>
 */
</script>
<div class="content-destination"></div>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>
<script>
 $('.norobot').each(function() {
  _this = $(this);
  $(_this.data('destination')).html(_this.html().replace('/*!','').replace('*/',''));
 });
</script>

Notes on the code

You may have noticed the type="text/html" attribute on the script block. That's to get around the fact that jQuery parses and executes any script blocks it finds when moving elements around (in an appendTo() or an html(), for example. Adding this attribute tells jQuery to process this as template code.

Also, the opening JS comment here begins /*!. The exclamation mark in this is a directive to any minifiers you might use on the code to tell them not to remove this comment block.

This is also available in a standalone page.

This is all a very long setup for my initial question. Does Google read this and, if so, does this affect duplicate content rankings?

Plus and Minus

• Minus: The duplicate content is definitely in the page.
• Plus: It's hidden in JavaScript
• Minus: we're using JavaScript to serve different content to users and to google.
• Plus: we're showing less to google than users. Spam techniques show more to increase keyword matches.
• Plus: faster response due to a single http request (Google likes fast pages)

Obviously, we could add an extra step of obfuscating the 'hidden' content by reversing it or encoding it. This would definitely hide it from google and it would be trivial to undo the process before showing it to the user but is this step necessary? Part of my reasoning for concluding that Google ignores JS comments is that thousands of sites include the same standard licences bundled with their JS library of choice and don't get penalised. This may, of course, be specific to licences, though.

I can find no definitive answer anywhere on this subject. If you have any good references, please let me know. Alternatively, if you happen to know Matt Cutts, ask him for me. If I get any conclusive answer, I'll update here.

jQuery Detach Scrollbars Plugin

tl;dr: I made a plugin: jQueryDetach Scrollbars Plugin download [Zip – 1.4KB]

A design I recently worked on called for a scrollbar which only took up half the height of the area it was scrolling. Now, I'm not going to get into a discussion about scroll behaviour, accepted standards or user expectations - fascinating though that would be - instead, Here's a jquery plugin.

The existing jQuery plugins I found that dealt with scrollbars created their own custom, themable bars. jScrollpane, for instance, gives you complete control over pretty much every aspect of the look and feel of the scrollbar. I figured this was too much for my needs, all I wanted was to be able to anchor the scrollbar to a given corner and change its height. The resulting plugin does just that. It even handles OS X Lion's disappearing scrollbars natively, too.

Usage

Note: This currently only works on vertical scrolling and doesn't have a good jQuery setup and teardown. If I needed it more than once on a page, I'd have written it a bit more robustly so it could be used more than once. Maybe I'll leave that task as an exercise for the enthusiastic reader.

If you call the plugin on an element with a scrollbar (i.e. the element has height and contains content that overflows), it will duplicate the scrollbar using the standard browser scrollbar and get rid of the original one. In fact, in its default configuration, there's pretty much no difference between using the plugin and not using it.

$(element).detachScrollbar();

It becomes more useful, however, when you pass in a couple of options. This example will move the scrollbar to the left of the content area:

$(element).detachScrollbar({anchor : 'topleft'});

You could leave the scrollbar on the right but shrink it downwards:

$(element).detachScrollbar({anchor : 'bottomright', height : '50%'});

The only behaviour of a standard scrollable area that isn't replicated by default is being able to use the mouse wheel to scroll while over the content area. If you want this behaviour (and you probably do), all you need to do is include the jQuery Mousewheel plugin in your page. This script will recognise if it's available and enable the functionality.

How it works

The script creates a container div around the original scrollable content and sets that to overflow:hidden while setting the original area to height:auto. This means that the original scrollbar disappears. It then creates another invisible div the same height as the original content and wraps that in a div with overflow-y:scroll, this creates a scrollbar that looks exactly like the original one. Add in some clever event trickery to tie the scroll position of one to the scroll position of the other and we're done. We've replicated the original functionality but can now control the styles applied to the scrollbar completely separately from the original content. This means we can position it to the left or above, maybe add a 90° CSS transform and have it look like a horizontal scrollbar, anything you like.

The plugin also incorporates “Cowboy” Ben Alman's scrollbar width plugin to make sure we're matching dimensions on whatever platform this is used on.

The options that can be passed in are:

internal: true / false (default: true)
autoposition: true / false (default: true)
anchor: 'topleft' / 'topright' / 'bottomleft' / 'bottomright' (default: 'topright')
height: Any CSS length (default '100%')

Advanced usage

The autoposition option allows you to decide whether to let the plugin handle the layout (which you probably do for most cases) or whether you want to specify everything yourself using the provided classes as styling hooks.

The other option, internal, determines the DOM structure. Specifically, it says whether everything is contained within the one element or whether the scrollbar is separate. Specifying internal: false would allow you to put the scrollbar anywhere on your page. You could have all scrollbars set as position: fixed along the top of the page if you wanted. Not sure why you would but you could.

Example and Download

jQueryDetach Scrollbars Plugin

jquery.detach-scrollbars.js

You must be able to read before you get a library card

I like JavaScript. JS. ECMAScript. Ol' Jay Scrizzle as nobody calls it.

I also like jQuery. jQ. jQuizzie. jamiroQuery. Whatever.

Ignoring the stoopid nicknames I just made up, did you notice how I referred to JavaScript and jQuery separately? I didn't say "I like JavaScript, there are some really great lightbox plugins for it" just the same as I didn't say "I wish there was a way to do indexOf in jQuery".

I'm regularly amazed at how many new (and some not-so-new) web developers either think they know JavaScript because they know jQuery or wish there was a way to do something in jQuery that they read about in an article about JavaScript. jQuery is a library written to make coding in JavaScript easier. It's made in JavaScript so you can say "jQuery is JavaScript" but only in the same way that "Simon is male". To confuse jQuery as all of JavaScript is the same as saying "Simon is all men" (don't worry, there's still only one of me).

For most web site or web app development, I do recommend using a library. Personally, I've used jQuery and Prototype extensively and decided I prefer jQuery. Libraries are designed to make coding faster and more intuitive and they can be a great productivity aid. You can get a lot more done quicker. There is a downside, however.

Downside

If you're doing what the library was intended to help with, great. Slide this panel up, pop open a modal window, scroll to the bottom of the page and highlight that header. Brilliant. The difficulties come when you're either trying to do something the library wasn't intended to do or something nobody's thought of before or you're just plain doing something wrong. If you are fluent in your library of choice but don't know the JavaScript underpinnings, your usual debugging tools can only help you so far. There will come a point where there's an impenetrable black-box where data goes in and something unexpected comes out. Okay, it's probably still data but it's unexpected data.

Don't let this point in the process be the discouragement. This is where the fun bit is.

Learning to read

Library authors are very clever groups of people. Often large groups. Reading through the unminified source of a library can be an awesomely educational experience as it's usually the culmination of many years best practice. If you want a nice introduction to some of the cool things in jQuery, for instance, check out these videos from Paul Irish:

• http://paulirish.com/2010/10-things-i-learned-from-the-jquery-source/
• http://paulirish.com/2011/11-more-things-i-learned-from-the-jquery-source/

I've dug around in jQuery many, many times to try and figure out why something does or doesn't do what it should or shouldn't. The most detailed investigation was probably Investigating IE's innerHTML during which nothing was solved but I found out some cool stuff.

Learning to write

The best way to get your head around libraries is to write your own. Yes, there are literally millions of them (not literally) out there already but you don't need to aim for world dominance, that's not the point of writing your own. Start simply, map the dollar symbol to document.getElementById. Done. You've written a tiny library.

function $(id){ 
    return document.getElementById(id);
}

Now you can add some more stuff. Maybe you could check to see if the thing passed to the $ is already an element or if it's a string. That way, you could be a bit more carefree about how you pass things around.

function $(id){ 
  if(id.nodeType) {
    return id;
  } else {
    return document.getElementById(id);
  }
}

Add in a couple of AJAX methods, some array manipulation and before you know it, you've got a full-blown web development toolkit.

If you're wanting a boilerplate to start your library off, I recommend Adam Sontag's Boilerplate Boilerplate.

Here's your Library Card

By now, you've rooted around in the jQuery undergrowth, dug through some of Moo's AJAX and pulled apart Prototype's string manipulation. You've written your own mini library, gotten a bit frustrated and wished you had a community of several thousand contributors to make it more robust. Now you're ready to start getting irked every time someone on Forrst asks if there's a jQuery plugin for charAt. Enjoy.

This article is modified from a chapter in a book Andrew and I were writing a couple of years ago about web development practical advice. Seeing as we both got too busy to finish it, I'm publishing bits here and there. If you'd like to see these in book form, let me know.

A collection of bookmarklets

Bookmarklets are the handiest little things. In case you don't know (which I'm sure you do), they're small chunks of JS that you store in your browser's Bookmarks or Favourites section which you can launch while looking at any web page. I write bookmarklets for all kinds of different tasks – navigating quickly around the build monitor at work, filling in tricky forms that my browser autocomplete doesn't handle, etc.

Here are a few of bookmarklets I find extremely useful for web development. To add them to your browser, simply drag the link on the title into your bookmark bar, hold shift down and drag or right-click and 'Add to Favourites', depending on what browser you're in.

Add JS to the page by URL

This will allow you to add any script you like to the current page. This can be particularly useful if you want to add a certain library or plugin to a page to investigate it further.

javascript:(function(){document.body.appendChild(document.createElement('script')).src=prompt('Script to add');})();

Add the latest CDN jQuery to the page

A variation on the above bookmarklet, this simply adds the latest version of jQuery. If you want to be able to play with a page and are familiar with jQuery, this will ensure that it is loaded and attached.

javascript:(function(){document.body.appendChild(document.createElement('script')).src='http://code.jquery.com/jquery-latest.min.js'})();

Add CSS to the page by URL

Add any stylesheet to the current page with a particular URL. This is handy if you want to demo variations to clients, I find. particularly if you predefine the CSS URL.

javascript:(function(d){if(d.createStyleSheet)%20{d.createStyleSheet(%20u%20);}%20else%20{var%20css=d.createElement(%27style%27),u=prompt(%27CSS%20to%20add%27);css.setAttribute(%22type%22,%22text/css%22);css.appendChild(document.createTextNode(%22@import%20url(%22+u+%22)%22));d.getElementsByTagName(%22head%22)[0].appendChild(css);}}(document))

Submit this page to the webdev subreddit

This isn't so much a web dev helper, more a general helper. I use this (or a variation thereof) to submit posts to specific reddits with fields prefilled. This bookmarklet defaults to the webdev subreddit.

javascript:window.location=%22http://www.reddit.com/r/webdev/submit?url=%22+encodeURIComponent(window.location)+'&title='+encodeURIComponent(document.title);void(0);

Add a CSS rule to the page

I can't remember whether I wrote this one or if I found it somewhere. The 'x<style>' is something from the middle of jQuery, though. Anyhow.

This allows you to add a style block directly into the page. This is useful for small CSS injections where the browser's web inspector is unavailable or incapable. Even though it's a single prompt that pops up, there's no reason why you can't past an entire stylesheet in there. I sometimes find myself doing that when I'm previewing designs on a production server.

javascript:(function(){var div = document.createElement('div'); div.innerHTML ='x<style>' +prompt('Style to add')+ '</style>';document.body.appendChild(div.lastChild);})();

Vendor-prefix questions

There's something that's always niggled me about vendor-specific prefixes on CSS.

Vendor prefix background

Just in case you don't know about vendor prefixes, there's a good summary on A List Apart.

Best practice dictates that you should always include non-prefixed properties last. This is so that when the property does become standard and the browser implements the non vendor-prefix version, it will use the standard rule as it comes later in the stylesheet than the prefixed one. The thing that has been bugging me is the assumption that the agreed standard version produces the same or better results than the prefixed one.

A convulted and completely made-up example

Imagine you have a paragraph:

<p>Made-up text.</p>

And you want it to have a white border with rounded corners. I'm fully aware you now don't need the vendor prefixes here but bear with me.

p {
  border:1px solid white;
  -webkit-border-radius:5px;
  -moz-border-radius:5px;
  -ms-border-radius:5px;
  -o-border-radius:5px;
  border-radius:5px;
}

In this scenario, the non-prefixed border-radius hasn't been implemented by any browser but you're following best practice so you include the values matching the current specification. Okay, now imagine that for Webkit's implementation of -webkit-border-radius, they decided that the radius value was actually to be divided by two. No problem, you can include the required value for Webkit. Again, not a real example but stick with me.

p {
  border:1px solid white;
  -webkit-border-radius:10px;
  -moz-border-radius:5px;
  -ms-border-radius:5px;
  -o-border-radius:5px;
  border-radius:5px;
}

You launch the site and send it out into the world.

Six months later, the standard is set, it turns out developers agree on Webkit's implementation. It becomes standard to double your radius value. A month after that, browsers start recognising the non-prefix version of the rule and rendering with the new standard. At this point, webkit, moz, ms and o are all rendering wrong because they are ignoring their vendor-specific implementation and using the non-prefixed standard. Even though webkit currently has the right value, it's being overwritten. If the rules had been included the other way round, they'd still be rendering the same as they were.

p {
  border:1px solid white;
  border-radius:5px;
  -webkit-border-radius:10px;
  -moz-border-radius:5px;
  -ms-border-radius:5px;
  -o-border-radius:5px;
}

Eventually,support for the prefixed version will be dropped and the browsers will only use the standard but that will be a little way down the line and gives you more time to correct your code or your client's code or wherever the code happens to be. Basically, prefix-last order buys the developer more time to correct any potential issues. I know that by using the vendor-prefixed rules, the developer is implicitly accepting the fact that they are using experimental properties which could change but in this scenario, it is not the prefixed properties that change but the non-prefixed one.

The reason I chose border-radius here is just because it was easier that typing up an example that might actually happen in real-life involving gradients or masks.

Open to suggestions

I'm open to any explanation as to why this scenario might never happen or comments on how I've misunderstood something.

Feed the RSS

Multi-user pages

Whenever I'm away from home, my usual stream of random ideas tends to become more focused on projects involving sharing. Usually something about creating a connection between people where the interaction point is the Internet. This is what first inspired the now defunct MonkeyTV project back in 2007 or noodler.net in 2008 – both created while I was living in Tokyo as ways to connect to people back in Edinburgh.

Until the beginning of August, I'm in Berlin finding somewhere to live while Jenni and Oskar are in Edinburgh packing up our flat (although, I'm not entirely sure Oskar is doing much more than drooling over packing boxes). The result of this is that I started to wonder about how to best show Jenni some of the flats I'm looking at remotely. What I figured I wanted was a way for us both to be looking at the same web page and for each of us to be able to point out things to the other. I tidied up my idea and posted it to the Made By Ideas site hoping that someone else would run off and make it so I could focus on apartment hunting.

The inevitable happened, I couldn't let it lie:

Multi-user page (installable bookmarklet).

If you install that bookmarklet by draggin’ it to your bookmarks bar then launch it anywhere, your cursor position and how far you've scrolled the page will be sent to anyone else viewing the same page who has also launched it. If you launch it on this page just by clicking it just now, you'll see cursors from other people reading this post who've also clicked it.

Technical

This is built in node.js with socket.io.

It heavily reuses Jeff Kreeftmeijer's multiple user cursor experiment but updated to use socket.io v0.7. I also used the socket.io 'rooms' feature to contain clients to a given window.location.href so that it could be launched on any page and interactions would only appear to users on that same page. I also removed the 'speak' feature to simplify things. I'm planning on talking via Skype when I'm using it. In theory, mouse events other than cursor coordinates and scroll could be shared – keyboard input, clicks, selects.

The day after I built this, Christian Heilmann pointed out on twitter a different solution to the same problem. Browser Mirror uses the same technology (node + websockets) but instead of passing cursor positions, it passes the entire DOM of the page from the instigator's computer to their node relay and then out to any invited viewers. This approach gets round a lot of the problems and is probably a more robust solution, all in all. They also have an integrated chat feature.

Warning

The server side is running on a borrowed VPS. It's not even been daemonised using Forever so it might fall over and not come back up. Don't rely on this for anything, just think of it as a point of interest.

The Code

I'm not really going to do any further development with it but for interest, here's the node.js server, almost entirely Jeff Kreeftmeijer's work but updated for the latest socket.io

var sys = require('sys'),
    http = require('http'),
    io = require('socket.io').listen(8000),
    log = sys.puts;

io.sockets.on('connection', function (socket) {
	socket.on('set location', function (name) {
	    socket.set('location', name, function () { socket.emit('ready'); });
		socket.join(name);
	});
	socket.on('message', function (request) {
		socket.get('location', function (err, name) {
			if(request.action != 'close' && request.action != 'move' && request.action != 'scroll') {
				log('Invalid request:' + "\n" + request);
				return false;
			}
			request.id = socket.id;
			socket.broadcast.to(name).json.send(request);
		});
	});
	socket.on('disconnect', function () {
		socket.broadcast.json.send({'id': socket.id, 'action': 'close'});
	});
});

And here's a bit of the client-side JS that I modified to connect via socket.io v0.7 (again, modified from Jeff Kreeftmeijer's):

var socket = io.connect('http://yourserver.com', {port: 8000}),
socket.on('connect', function() {
	socket.emit('set location', window.location.href);
	socket.on('message', function(data){
    if(data['action'] == 'close'){
      $('#mouse_'+data['id']).remove();
    } else if(data['action'] == 'move'){
      move(data);
    } else if(data['action'] == 'scroll'){
			clearTimeout(noscrolltimeout);
			disabled = true;
      scroll(data);
			noscrolltimeout = setTimeout('disabled = false;',2000);
    };
  });
});

If you'd like to snoop around the code more, it's all available on the lab: Multi-user pages

Props to Andrew for letting me use his server, by the way. It was enough hassle having to learn node for this little experiment, he saved me from having to build my own.

Workflow. The flow. Of work.

How to actually work

Don’t tell anyone (particularly not your clients and especially not my clients) but making websites is really very easy. Don't make it harder by deliberately hindering yourself. You should always try to travel the shortest distance between “amends needs done” and “amend is done”. Again, I hear a “Pah!” in the distance, “Well, that’s obvious.”, but is it? Are you sure you’re ‘Being all you can be’?

The shortest distance

Okay, picture the scene: you've run out of socks. You need to put on a load of washing. The washing is in a pile in the corner of the room. You hunch over, pick up all the socks, pants, t-shirts and walk, waddle and totter to the washing machine. As you walk, you drop a sock and bend down to pick it up, dropping another. You recover that one and another falls. Eventually, you make it to the washing machine after a few minutes, put everything in, throw in some powder and set it going. As you head back to your bedroom, you spot three more socks that you missed. Darn.

Okay, picture the scene: you've run out of socks. You need to put on a load of washing. Fortunately, every time you took your socks off, you put them straight in the washing machine. You wander to the kitchen, put in the powder and switch it on. Done.

Any time you try to do anything, the more steps involved between intention and completion, the more likely it is to go wrong. Whether the intention is 'wear clean socks' or 'update this website', if you can do it in a handful of steps, you'll make fewer mistakes than if you have to do it in a hunched-over armful.

Workflow. The flow. Of Work.

The next time you're making an amend somewhere, watch yourself. Watch which applications you jump between. Don't just make your changes, pay attention to how you do them. Are you jumping between text editor and FTP client? Text editor and web browser? FTP and Subversion? Just take a few minutes to think about which steps you might be able to miss out. For instance, if you use Textmate and Transmit, you can set up DockSend so that pressing ctrl-shift-f then 2 will upload the file you currently have open to the same place on the server. You can now change the editor↔FTP↔browser cycle to editor↔browser.

However...

Seamless does not imply flawless. Don't be tempted to simplify to the stage where you don't need any interaction between making a change and the change being live. If you rely on the fact that your changes 'just happen', you might be tempted not to check. That's the point at which they don't 'just happen'.

This article is modified from a chapter in a book Andrew and I were writing a couple of years ago about web development practical advice. Seeing as we both got too busy to finish it, I'm publishing bits here and there. If you'd like to see these in book form, let me know.

Testing

The web is a visual medium. Well, mostly.

There's no better way to test a visual medium than by looking at it. Look at your site in as many browsers as you can. If you've already got as many browsers installed on your development computer as you can fit, get another computer and install some more. Either that or run a Virtual Machine.

Definition: Virtual Machine (VM)

Applications like VirtualBox, VMWare or Parallels allow you to run an entire computer within your computer. It is a self-contained system that doesn't interact with your own machine meaning you can have IE6 installed on one VM, IE7 on another and IE8 on a third. All running in a window on your iMac. Shiny.

If you can't do that easily, you could use one of the growing number of browser testing services. These are server rooms packed with computers running Virtual Machines and automated systems to which you supply a URL, wait a few moments and get shown an image (or several hundred images) showing your URL in different browsers on different platforms. Some of the more sophisticated services allow you to scroll down a long page or specify different actions, text entry or mouse events you want to see triggered. These services can be exceptionally useful when it comes to developing HTML e-mails as there are some rare and esoteric e-mail clients out in the wild. Litmus does an excellent job at virtualised testing for HTML e-mails. On that note, the Campaign Monitor library of free HTML e-mail templates is a great place to start, learn and possibly finish when working on an HTML e-mail.

There is also a place for automated testing for some things. Recently, there has been a bit of a movement away from validating code as the purpose of web development is not to make it 'check a box' on a merely technical level, it is to get the message across via the design however possible. However, validation is still the best and easiest way to check your syntax. Syntax errors are still the main cause for mistakes appearing in your web sites and are the easiest thing to fix. Don't assume IE is wrong. Again, if you're keen on HTML e-mails, here's a great post on the Litmus blog.

This article is modified from a chapter in a book Andrew and I were writing a couple of years ago about web development practical advice. Seeing as we both got too busy to finish it, I'm publishing bits here and there. If you'd like to see these in book form, let me know.

Don’t be seduced by the drag-and-drop side

You don’t have to be a survivor of the vi and Emacs holy wars to appreciate the beauty of fully hand-crafted code. There was a bit of attention a couple of weeks ago on the soon-to-be-launched Adobe Muse which lets you “Design and publish HTML websites without writing code”. If you want to be a kick-ass developer, you must realise that tools like this aren't designed for you. They're designed for people who want to do what you can do but either don't have the time or the inclination to learn how. Although drag 'n' drop application do lower the barrier to entry for creating a website, there is still a need for web developers to know exactly what's going on in their pages.

I'm not saying that there will never be a time when visual design can be automatically translated into as good a product as a hand-crafted site, I'm just saying it’s not yet.

In much the same way as with JavaScript (See “You must be able to read before you get a library card”), building your HTML using that extra level of abstraction might work for almost every situation but will eventually leave you stuck somewhere you don’t want to be. By all means, pimp up your text editor with all manner of handy tools, shortcuts and code snippets but make sure you still know exactly what each bit of mark up means and does. If you structure your code well (more on that in a later post), your natural feel for the code will be as good a validator as anything automated (by which I mean prone to errors and worth double-checking).

Learn the keyboard shortcuts. If you learn nothing else from this, learn the importance of keyboard shortcuts. You might start off thinking you'll never need a key combination to simply swap two characters around but one day, you'll find yourself in the middle of a functino reaching for ctrl-T.

Also, there is no easy way to tell if a text editor is fit for you until you have tried it, looking at screenshots won’t work. You don't need to build an entire project to figure out whether or not you're going to get on with a new text editor, just put together a couple of web pages, maybe write a jQuery plugin. Do the key combinations stick in your head or are you constantly looking up the same ones again and again? Do you have to contort your hand into an unnatural and uncomfortable claw in order to duplicate a line?

The final thing to cover about text editors is that it's okay to pay for them. Sure, we all love free software. “Free as in pears and free as in peaches” (or whatever that motto is) but there are times when a good, well-written piece of software will cost money. And that's okay. You're a web developer. You are going to be spending the vast proportion of your day using this piece of software. If the people that made it would like you to spend $20 on it, don't instantly balk at the idea. Think back to the idea of web developers as artisan craftsmen. You're going to be using this chisel every day to carve out patterns in stone. Every now and then, you might need to buy your own chisel.

This article is modified from a chapter in a book Andrew and I were writing a couple of years ago about web development practical advice. Seeing as we both got too busy to finish it, I'm publishing bits here and there. If you'd like to see these in book form, let me know.

Go on, google me.

A little while back when I was still job-hunting, I saw an interesting position come up with FI in Stockholm. They're the agency behind things like Google's 20 things I learned about browsers and the web. They were looking for a Web Producer to be clever and creative and innovative and insightful and all the usual things Web Producers are. I decided to apply but with such a high-profile agency, I figured my application had better be something appropriately clever and creative to get noticed.

I spent a couple of weeks tweaking key-words in my site to manipulate search results for the phrase "Awesomest Web Producer". I then used Moo to print up a postcard featuring a google search box with that phrase entered and the mouse cursor hovering over the "I'm feeling lucky" button (this trick doesn't work quite so well now that Google have launched Google Instant).

I posted the card off to them with the job reference number on the back and the words "My application" handwritten. No name, no signature, no contact details. If the scheme worked, they'd find me, proving the value of taking me on. That was my thinking, anyway. To be honest, once I'd sent off the postcard, I figured that was the end of it. It was a cocky, big-headed move and probably wouldn't work.

It did work. Kinda. It worked perfectly on a technical level, at least: you could enter the phrase, hit the button and land on my CV site. If you didn't hit "I'm feeling lucky", you were still presented with a page of search results, each about me, my career history and some of my projects. It also worked to get FI to contact me. I went through a few stages of the recruitment process before we parted ways. I never did find out exactly why but I wasn't quite 'the right fit'.

Of course, if I'd gone there, I wouldn't be starting a kick-ass job with Nokia next week.

Torch

Concept

Another proof-of-concept game design prototype. This is kind of a puzzle game. Ish. It's mostly a simple maze game but one in which you can't see the walls. You can see the goal all the time but you are limited to only being able to see the immediate area and any items lit up by your torch. You control by touching or clicking near the torch. The character will walk towards as long as you hold down. You can move around and it'll follow.

Torch v0.1

The first 10 levels are very easy and take practically no time at all. After that, they get progressively harder for a while before reaching a limit (somewhere around level 50, I think). The levels are procedurally generated from a pre-determined seed so it would be possible to share high scores on progression or time taken without having to code hundreds of individual levels.

Items that could be found around the maze (but aren't included in this prototype) include:

• Spare torches which can be dropped in an area and left to cast a light
• Entrance/exit swap so that you can retrace your steps to complete the level
• Lightning which displays the entire maze for a few seconds (but removes any dropped torches)
• Maze flips which flip the maze horizontally or vertically to disorient you.

I worked on this for a few months (off and on) and found it to be particularly entertaining with background sound effects of dripping water, shuffling feet with every step, distant thunder rumbling. It can be very atmospheric and archaeological at times.

Slightly technical bit

The game loop and draw controls are lifted almost line-for-line from this Opera Dev article on building a simple ray-casting engine. I discarded the main focus of the article - building a 3D first-person view - and used the top-down mini map bit on its own. The rays of light emanating from the torch are actually the rays cast by the engine to determine visible surfaces. It's the same trick as used in Wolfenstein 3D but with fewer uniforms. It's all rendered on a canvas using basic drawing functionality.

The audio is an interesting, complicated and confusing thing. If I were starting again, I'd look at integrating an existing sound manager. In fact, I'd probably use an entire game engine (something like impact.js, most likely) but it was handy to remember how I used to do stuff back in the days when I actually made games for a living. Most of all, I'd recommend not looking too closely at the code and instead focusing on the concept.

Go, make.

As with Knot from my previous blog post, I'm not going to do anything with this concept as I'm about to start a big, new job in a big, new country and I wanted to empty out my ‘To do’ folder. The code's so scrappy, I'm not going to put it up on GitHub along with my other stuff, I seriously do recommend taking the concept on its own. If you are really keen on seeing the original code, it's all unminified on the site so you can just grab it from there.

The usual rules apply, if you make something from it that makes you a huge heap of neverending cash, a credit would be nice and, if you felt generous, you could always buy me a larger TV.

Torch v0.1

Knots

Game design

I had an idea around Christmas for a new game design. it was basically to be a single or multi-player game where you would tie knots in a piece of ‘string’. The single-player game would focus on discovering new patterns while the multi-player one would be a time-based challenge where each would tie a knot, pass it over to their opponent and the first to untie it would win.

I built a basic prototype and had some fun inventing a few interesting knots (my favourite being ‘Yoda’) but didn't get round to doing anything with it. As I'm tidying up all my projects before immigrating to Germany, I figured I should just go ahead and put this out into the world.

Knot v0.1

Slightly technical bit

The system is built using an invisible 3x3 grid and 4 quadratic curves. The curves' start, control and end points are the centre of a tile. When you move a tile, it basically swaps position with the tile nearest where you drop it. This can also be done with multiple tiles simultaneously if you're on a multi-touch device. You can see the tiles if you enable debug mode. You can also switch between the two colour schemes it has at the moment.

The only addition I've made to it since Christmas was to add on a system to allow players to save knots back into the system. I've only built in 22 patterns so if you make a nice, interesting or funny one, give it a recognisable name (nothing too rude, please) and save it, it will then be available to everybody who plays with it. You can also set whether the pattern is to count as ‘matched’ when rotated 90°, 180°, flipped vertically or flipped horizontally. Calculating the isomorphisms (when two knots look the same to the viewer but aren't necessarily the same to the computer) was probably the trickiest bit of the system.

Go, make.

If you're interested in taking the idea further, grab the code from GitHub and make something from it. The usual rules apply, if you make something from it that makes you a huge tonne of neverending cash, a credit would be nice and, if you felt generous, you could always buy me a larger TV.

Knot v0.1

Forrst Podcast

Design Archive

Along with (yet another) redesign, I decided to collect all my old blog designs together for posterity. The thing that surprised me is my own progression in design terms. I would still find it difficult to call myself a designer in any sense but I will admit that I'm a bit better now than I was back when I started this site.

Note: this is just a gallery of the previous designs for the blog incarnation of this site. The site itself existed for quite a while before that when it hosted my two webcomics “Things in Jars” and “Scene & Herd”.

These are all simply themes that were applied to the base CMS ‘Dooze’.

Previous blog themes

• Dooze (2007-2009)
• Underwater Slate (2009-2010)
• Felt (2010-2011)
• Desk (2011-?)

It's also a bit odd that I made this while working at a design agency and this while working for the Government.

HTML5 for Large Public Bodies

Your country needs you...

...to stand at the cutting edge of technology.

Sounds awfully impressive, don't you think?

There are quite a few regulations to bear in mind and comply with when developing a website for a Government organisation or any large public body. This has lead to a lot of sites being developed in a very defensive manner, ensuring safe compliance at the expense of a great and consistent user experience.

This video features a presentation about how large publicly-liable institutions should and can embrace the latest in web technologies without sacrificing standards. By embracing them, in fact.

The content of this was developed while planning and building the National Museums Scotland website which launched last November. The messages presented are applicable to museums, galleries, swimming pools, councils, anywhere, really.

If you're a techie person in the cultural or government sector, you might find this useful in convincing others to take advantage of the latest cool (and useful) technologies.

Video

HTML5 for Large Public Bodies from Simon Madine on Vimeo.

Links from the presentation

• PolyFills
• .NET magazine article
• (Almost) complete list of polyfills
• Dive into HTML5
• HTML5 Boilerplate
• Smashing Magazine
• HTML5 Doctor
• AlphaGov

Slideshow

The source for the slides is available online although it's mostly webkit-friendly. I realise the irony of a presentation about cross-platform HTML5 not being a great example itself of that but it does degrade adequately. If I get the time in the future, I'll tidy it up. An actual good (and relevant) example of cross-platform web technologies is the National Museums Scotland website itself which performs fantastically across all manner of platforms.

Instant art

Psychopathic megalomaniac vs Obsequious sycophant?

- or -

Tabs vs Spaces?

Which are you? Is there any middle ground?

In any modern, code-friendly text editor, you can set tab size. This means that one tab can appear to be as wide as a single space character or - more commonly - 2 or 4 spaces. It's even possible in some to set any arbitrary size so there's nothing stopping you setting tabs to be 30 spaces and

format() {
                              your code() {
                                                            like;
                              }
                              this;
}

However you do it, the point is that using tabs allows the reader personal preference.

Indenting your code using spaces, however, is completely different. A space is always a space. There's actually a simple conversion chart:

• 1 = One
• 2 = Two
• 3 = Three
• and so on.

The fundamental difference here is that the person reading the code has no choice about how it's laid out. Tabs means you get to read the code how you want to, spaces means you read the code how I want you to. It's a subtle difference but an important one.

Space indenting is therefore perfectly suited to psychopathic megalomaniacs who insist their way is right while tab indenting is for obsequious sycophants who are putting the needs of others above themselves. Sure, there may be lots of grades in-between but why let moderation get in the way of a barely coherent point?

Curiously, neither of these extremes feels a great need to comment their code. One sees no need as their code is for themselves and they already understand it, the other assumes that if they can understand it, its purpose is so obvious as to be trivial. Both are wrong here.

There's unfortunately no real middle ground here. Teams must agree amongst themselves what side of the fence they fall down on. It is entirely possible to set many text editors to automatically convert tabs to spaces or spaces to tabs on file save but if you're doing that, you'd better hope your favourite diff algorithm ignores white space otherwise version control goes out the window.

This article is modified from a chapter in a book Andrew and I were writing a couple of years ago about web development practical advice. Seeing as we both got too busy to finish it, I'm publishing bits here and there. If you'd like to see these in book form, let me know.

Up-sticks

So, after almost 2 years at National Museums Scotland, I'm moving on. And not just me, Jenni's coming too (and Oskar, of course).

After delivering a kick-ass, ground-breaking website and implementing some cool shiny stuff, I feel confident I can move on having made a bit of a difference. I initially took the job for two main reasons:

1. To bring advanced web-awesome to the cultural sector
2. To prove my technical ability to myself at an international level

I'm happy I've done that. The National Museums Scotland site looks a lot nicer above and below the surface than it did when I and the rest of the web team arrived in 2009. Hopefully the systems and techniques I've put in place will ensure it stays at the front edge of the culture sector in terms of well-considered use of technology. The rest of the web team are still there, of course, and will continue to make cool stuff. I'll be satisfying my urge to build cool cultural stuff by providing the tech behind the various Museum 140 projects.

You can read more about the first Museum 140 project ‘#MusMem’ if you're interested.

The second reason is just as important. Working on a large national organisation website made me triple-check everything I wrote but my desire to build ‘Cool Stuff’ meant that I couldn't play it too safe. Favourable reviews from .NET magazine and ReadWriteWeb suggest that we got the balance right.

Personal projects

In the last 12 months, I've also had a fair amount of success with my various personal tech projects – 8bitalpha, harmonious, The Elementals, Whithr, Shelvi.st as well as being a featured case study on phonegap.com. All of which have served to remind me I like the challenge of doing something beyond what I can already do.

Where now, then?

Now is the right time to step up from the top half of the First Division to somewhere in the middle of the Premier League (yeah, that's right, sport analogies). I'm going to be starting in July as Senior CSS Developer at Nokia in Berlin. I'll be working on the desktop interface for the services that used to be ovi.com (until a couple of weeks ago) building interface frameworks and UI components.

Nokia have been going through a bunch of changes recently so I'm excited to be joining them now when there's a great opportunity to make a significant difference on a big scale.

I've only been to Berlin twice (an upcoming blog post will give more details about that) and, despite being German, Jenni's never been so it'll be an interesting move. Berlin does, however, have over 170 museums so if there's anywhere Jenni can perform her particular brand of museum-wizardry, it'll be there. We've asked Oskar's opinion but he's not saying much. He's mostly drooling.

Über-curricular activities

I'm also hoping to give more conference talks and presentations (like yesterday's) as well as write more educational articles for this blog and others. I've got a few written that I've not had the chance to present anywhere so I might put them here at some point.

If you have any questions, throw a tweet in my direction.

Archiving Tweets

Archiving Tweets - Technical

Twitter has an amazing amount of data and there's no end to the number of ideas a coffee-fuelled mind can come up with to take advantage of that data. One of the greatest advantages of Twitter – its immediacy, its currency, its of-the-moment-ness – is, however, also one of the drawbacks. It is very easy to find out what people are thinking about now, yesterday or a few days ago but trying to find something from more than a week ago is surprisingly tricky. The standard, unauthenticated search only returns the last 7 days of results, beyond that, there's nothing. You can get around this by building custom applications which authenticate themselves correctly with Twitter and provide a level of traceability but that's quite tricky.

The easiest way to archive search results for your own personal use is actually surprisingly simple. Every search results page includes an RSS feed link1. This will be updated everytime there are new results. Simply subscribe to this feed with a feed reader (Google Reader is simple and free) and your results will be archived permanently. This is great if you're searching for personal interest stuff but doesn't work so well if you want to share the results with the public.

1 Note: The feed link only appears for searches performed on search.twitter.com, not for searches performed via the standard web interface at https://twitter.com/#!/search/

This was the problem I was presented with when I was asked to build a tweet-archiving mechanism for Museum140's Museum Memories (#MusMem) project. Jenni wanted some kind of system that would grab search results, keep them permanently and display them nicely. I didn't want to create a fully OAuth-enabled, authenticated search system simply because that seemed like more work than should really be necessary for such a simple job. Instead, I went down the RSS route, grabbing the results every 10 minutes and storing them in a database using RSSIngest by Daniel Iversen. The system stores the unique ID of each Tweet along with the time it was tweeted and the search term used to find it. The first time a tweet is displayed, a request is made to the Twitter interface to ask for all the details, not only of the tweet, but also of the user who tweeted it. These are then stored in the database as well. This way, we don't make too many calls to Twitter and we don't get blocked.

If you want your own Tweet Archive, I've put the code on GitHub for anyone to use. It requires PHP, MySQL and, ideally, a techie-type to set it up.

Archiving Tweets - Non-technical

With the technical side out of the way, we're left with the human issues to deal with. If you're automatically saving all results with a particular phrase, all a malicious person needs to do is include that phrase in a spam-filled tweet or one with inappropriate language and suddenly, it's on your site, too. If you aren't going to individually approve each tweet before it is saved, you must keep a vigilant eye on it.

The other thing which has turned out to be a problem is the Signal-to-Noise ratio. When we initially decided on the hashtag #MusMem, nobody else was using it. To use Social Media parlance, there was no hashtag collision. The idea was to encourage people to use it when they wanted their memories stored in the MemoryBank. Unfortunately, it is now being used by anyone tweeting anything related to Museums and Memories. This is particularly troublesome at the moment as this month is International Museum month, one of the themes of which is ‘Memory’ (which is why we built the MemoryBank in the first place). This means that the permanent memories we want stored (the Signal) are getting lost in the morass of generic Museum Memories (the Noise). There is no way to solve this problem algorithmically. If we are to thin it down, we actually need to manually edit the several thousand entries stored.

If anyone can think of a solution to this issue, please let everybody know – the world needs you.

Mini Zombie Pipe

Here are some mockups I did for a game design last year.

The general premise is a 2d physics game with multiple single-screen levels. When the level starts, you have a few seconds to rearrange whatever items/furniture/planks of wood you find lying around and place your hero somewhere on the screen. After the initial timer has run out, the pipe opens and hundreds or thousands of tiny zombies or robots (or robot zombies) pour into the level. You have to avoid being touched by them for 30 seconds, after which the level is cleared and you're on to the next level. The difficulty is in being able to quickly rearrange the furniture and choose your safe spot to prolong the time before they get you. As they are able to eat through most things, they would eventually get through.

There is the possibility for tweaks to the difficulty setting by changing the amount of available furniture, whether the protagonist can move mid-level, how many zombies there are, how small they are and so on.

At the time, I was too busy building The SMWS Spirit Cellars and when i had time again, I started building The Elementals so I never came back to it. Looks like it could be a fun game, though.

Travelling tales

When I was a young game designer wannabe living in St Andrews, I interviewed with a company in Manchester. For years, this was my biggest travel time:interview time ratio in that I got up shortly before 6, took a taxi, train and different train to get there by 1, had a 15 minute interview and then took two trains and a bus to get home by 11pm 1.

1Incidentally, the reason the interview only took 15 minutes was because the hiring manager delegated the responsibility of interviewing to the person he'd given the job to the week before. It was pretty much a case of “If you find out he's better than you, we'll give him the job instead...” Needless to say, I didn't get the job.

Now, the ratio is still unbeaten (31:1) but I have now definitely overtaken the basic numbers. I've recently travelled to Berlin twice to interview with Nokia's Ovi offices. I have since come to the conclusion that there is some mysterious force at work who really doesn't like me travelling through Schiphol. Note: I don't mean an omniscient being, I mean some actual arch enemy.

Trip one

Leaving on a Tuesday afternoon, I jumped on a plane at Edinburgh Airport to Amsterdam Schiphol then changed onto a plane to Berlin Tegel. I eventually arrived at my hotel in Schiphol around 11pm. After checking in, hanging up my interview shirt and scrubbing my face to get rid of the sheen of fellow air passengers, it was after midnight. Not the best prep for a full day of interviews, to be honest. The next morning, I got up, tried to partake of a German breakfast of rye bread and wurst, settled for a croissant and coffee and headed out into the -10°C weather. I won't go into the details of the interview process but by the end of the day, I'd been interviewed by 8 different people, drunk two pots of coffee myself and been told that I don't look particularly German. Not being German, I thought this statement was accurate.

I left the office, caught a taxi back to the airport and relaxed safe in the knowledge that I'd be back in Edinburgh in a few hours. An incorrect assumption, it would seem.

I arrived in plenty time to double-check the flights were all okay. There was even a flight to Schiphol before mine that some passengers were offered the chance to catch. I was relaxed, it was fine, I wasn't in a hurry, I'll catch the next one. Several hours later, I regretted that decision. The second flight was held in Schiphol before coming to Tegel due to broken air conditioning. By the time it had landed, turned round and let us board, I should have already been half way to Edinburgh and had missed the last connection. Boo.

One over night in a hotel later, I finally made it back to Edinburgh at 8am. Approximately 40 hours after I left. About 5.7:1.

Trip two

When it came to my second round of face-to-face interviews, I wasn't lucky enough to get an overnight stay, unfortunately. I had to get to Berlin and back in a day. The 2.45am start would have been bad enough under any circumstances but when you have a teething 5-month old and have generally been running on empty for the best part of 2 months, it very nearly killed me. Still...

Edinburgh to Schiphol – fine; Schiphol to Tegel – also fine. I arrived at the office only 3 minutes late for my 12 o’clock interview which isn't too bad after travelling about 800miles. Three hours of geek talk later, I'm back in the taxi on the way to Tegel, snapping photos out the taxi windows just to prove I was there. This time there's no delay at Tegel and I land in Amsterdam with about 2 hours to make it to my gate. I'm not exactly a relaxed traveller so I'm not the kind who can go via the airport pub, have a sit-down meal and casually meander to the gate in time to board. I'm more inclined to high-speed sprints and panicked departure board-scanning just to make sure I get to the gate several hours early so I can sit and do nothing. That's exactly what I did. I got to the gate in plenty of time to see the ‘Flight delayed’ ticker come up. Plenty of time to watch the ‘Estimated Departure’ go from 21:00 to 22:00 to 23:00 to 00:00. It was when it flashed ‘00:30’ it finally decided to stop.

I made it back to Edinburgh around 1am and got back to my house around 2.30am. Just in time to pick up Oskar as another night of painful teething screams started off, in fact. 7:1, this time.

I made this

Reading over recent posts, you may have gotten the impression that I'm a bit of a geek. I occasionally need to remind myself that I wasn't always so this weekend, I took a few hours and drew a picture of Oskar (the youngling).

Here are a few pictures I took of the drawing as it came along. Apologies for the quality not being great but my drawing studio isn't the best setup for photography.

Investigating IE's innerHTML

Be warned, this is a long and quite techy post that rambles a bit. It's about JS, jQuery, IE and XML namespaces. Be careful.

I'm currently working on a tool which uses JS to parse an XML document and output it as JSON. Straightforward enough, you'd think. The issue I'm fighting against crops up when the XML tags have an arbitrary namespace. True enough, some of the files I'll be processing have this namespace defined at the top of the document but the tool has to be robust enough to cope when they don't.

To cut a long story short, IE6, IE7 and IE8 have an interesting attitude to innerHTML when the tags you are trying to insert have a namespace. IE9 seems to do the job as you'd expect. I've created some jsFiddle pages which try and identify the issues. They both use QUnit and the test code is included below.

createElement nodeName vs get(0).nodeName

View this on jsFiddle.net

I started off using jQuery to help identify this as the tool uses jQuery for the XML heavy-lifting. The two tests in this demo create elements in two different ways. First, we create the element using document.createElement and grab the nodeName then we use jQuery's constructor and use get(0) to grab the bare DOM element's nodeName. Also, in this first set of tests, we're creating non-standard elements.

test("Compare elements without namespace", function() {
 var element1, element2;

 element1 = document.createElement('spud').nodeName;
 element2 = $('<spud/>').get(0).nodeName;

 equals(element1, element2, "We expect these to match");
});

The code above runs fine everywhere – IE, FireFox, Opera, Chrome, etc. etc. Good.

test("Compare elements with namespace", function() {
 var element1, element2;

 element1 = document.createElement('a:spud').nodeName;
 element2 = $('<a:spud/>').get(0).nodeName;

 equals(element1, element2, "We expect these to match");
});

This runs fine in non-IE browsers, they all report the nodeName as 'a:spud'. IE now reports the nodeName as 'spud'. Ah. I dug through the jQuery source, tracking down the bare roots of the constructor and eventually figured out that just looking at the element itself isn't going to provide any clues. The bit that does the actual string-to-elements work (somewhere around line 5619 in jQuery 1.5.2) creates a container div then injects the (slightly modified) code as innerHTML. The issue must be in IE's interpretation of innerHTML, I thought to myself. And then to you by writing it here.

.innerHTML aside

or ‘jQuery is clever’

Before we continue with this long and, ultimately, unnecessary investigation into namespaces, I have to take a small diversion to cover some smart stuff jQuery does. One thing in particular, in fact. Around that line I mentioned earlier (5619-ish), an extra bit of text is inserted into the innerHTML to cope with IE's oddity. If you are trying to create a non-standard element using innerHTML, IE will not complain but also just do pretty much nothing at all:

        var div = document.createElement('div');
        div.innerHTML = '<spud></spud>';
        alert(div.innerHTML);

The above code will alert '<spud></spud>' in most browsers but '' in IE. What jQuery does is firstly wrap your element in an extra <div></div> (producing '<DIV></DIV>') then prepends the word 'div' to that. The innerHTML reported by IE is now 'div<DIV><SPUD></SPUD></DIV>'! There it is! Next, the extra gubbins is removed by calling .lastChild and you're left with innerHTML = '<SPUD></SPUD>'. That's pretty darned clever.

.innerHTML vs document.appendChild

View this on jsFiddle.net

Back on track. Armed with this little trick, we can reliably test innerHTML in IE using non-standard elements.

module("Known elements (span)");
 test("Compare elements without namespace", function() {
  var div1, div2;

  div1 = document.createElement('div');
  div1.innerHTML = '<span></span>';

  div2 = document.createElement('div');
  div2.appendChild(document.createElement('span'));

  equals(div1.innerHTML.toLowerCase(), div2.innerHTML.toLowerCase(),
     "We expect these to match");
 });

 test("Compare elements with namespace", function() {
  var div1, div2;

  div1 = document.createElement('div');
  div1.innerHTML = '<u:span></u:span>';

  div2 = document.createElement('div');
  div2.appendChild(document.createElement('u:span'));

  equals(div1.innerHTML.toLowerCase(), div2.innerHTML.toLowerCase(),
     "We expect these to match");
 });

The first test in this pair runs fine everywhere exactly as we'd hope and expect. The second fails miserably in IE. Let us quickly run the same test with unknown elements just to make sure we're identifying the right problem:

module("Unknown elements (spud)");
 test("Compare elements without namespace", function() {
  var div1, div2;

  div1 = document.createElement('div');
  div1.innerHTML = 'div<div>' + '<spud></spud>' + '</div>';
  div1 = div1.lastChild;

  div2 = document.createElement('div');
  div2.appendChild(document.createElement('spud'));

  equals(div1.innerHTML.toLowerCase(), div2.innerHTML.toLowerCase(),
     "We expect these to match");
 });
 test("Compare elements with namespace", function() {
  var div1, div2;

  div1 = document.createElement('div');
  div1.innerHTML = 'div<div>' + '<u:spud></u:spud>' + '</div>';
  div1 = div1.lastChild;

  div2 = document.createElement('div');
  div2.appendChild(document.createElement('u:spud'));

  equals(div1.innerHTML.toLowerCase(), div2.innerHTML.toLowerCase(),
     "We expect these to match");
 });

As before, the first test in this pair works fine, the second fails. Cool. Or not. Either way, you can now see that it doesn't really matter whether the elements are standard or custom and that little diversion we took earlier really was unnecessary. Still, you know more now about some of the cleverness in jQuery than you did before.

It turns out the reason IE reports the nodeNames as the non-namespaced ones is because it has been busy behind the scenes and added an extra XML namespace prefix into our current context. The innerHTML of the div we filled up using innerHTML has been modified to:

  <?xml:namespace prefix = u />
  <u:span></u:span>

Where'd that namespace declaration come from?! Goshdarnit, IE. From its point of view, within that little context, u:span is equivalent to span

The most stripped-down example

View this on jsFiddle

Seriously, it does not get more fundamental than this.

element = document.createElement('div');
testHTML = '<div></div>';
element.innerHTML = testHTML;
element.innerHTML.toLowerCase() == testHTML.toLowerCase() 

The last line there is true for all browsers.

element = document.createElement('div');
testHTML = '<a:div></a:div>';
element.innerHTML = testHTML;
element.innerHTML.toLowerCase() == testHTML.toLowerCase() 

The last line there is true for all browsers except IE 6, 7 and 8!

In conclusion?

Ultimately, there are no winners here. Identifying the problem is quite different from fixing it. I've added a note to the relevant jQuery bug in the tracker but it's not so much a bug in jQuery as a humorous IE quirk. There's some talk of refactoring the .find() method to handle more complicated tagnames so this might get picked up then. The fix will probably be something along the lines of checking the outcome of the innerHTML doesn't have an unexpected namespace declaration when the selector has a colon in it:

div.replace( /]*>/g, '' )

I'd submit the patch myself but I'm having difficulty getting unmodified jQuery to build on any of my machines without failing most of the QUnit tests. I've probably typed something wrong.

Autogenerated Everything

After seeing this collection of the 892 different ways you can partition a 3 x 4 grid1, I was struck by a thought. If these were generated as HTML templates, they could be combined with a couple of other useful websites and become a nice, API-driven site builder2.

The process

• On the site-building webpage, you'd enter a few keywords describing the site you want and drag a slider along between 1 and 12 to specify how many content areas you want. The value from the slider would be used to pick a template randomly from the number available for that combination of panels.
• This template would be dropped into the middle of an HTML 5 boilerplate (possibly generated via Initializr)
• The keywords would be passed to ColorAPI to generate an asthetically pleasing colour-scheme
• The keywords would then be passed to FlickHoldr along with the dimensions of some of the areas from the template to get relevant imagery
• Grab some lorem ipsum of the right length from LoremIpscream to fill out the content areas of the site
• Done. Your asthetically pleasing, nicely designed site is ready to download within a few seconds.

Once this service has been created, I'm fairly sure me and the rest of the industry will be out of a job.

1 Technically, there are 3,164 ways you can partition the grid but most of them are vertical or horizontal mirrors of the 892 mentioned.

2 There are a few references to 3x4grid.com which is apparently going to be an API-accessible collection of these layouts but there doesn't seem to be anything there yet.

8 bit alpha

8-bit alpha

Shelvist

I've been a bit quiet for the last couple of weeks. Now I can reveal why!

All my spare time has been taken up building Shelvist (well, that and looking after a 4-month old). It's a way of keeping track of which books you have read, which ones you are currently reading and which ones you want to read. That's it.

Just over a year ago, I started listening to a lot of audiobooks and, 12 months after I subscribed to Audible, I decided I wanted to see just how many books I'd read*. All the existing book tracking sites (e.g. Shelfari, Goodreads) focus on recommending, rating, reviewing which just seemed like hard work.

*Listening to an audio book counts as reading. It just sounds weird if you tell people you listened to a book.

Building this was also a chance to learn some new tech. I've been wanting to play with CakePHP for a while so this seemed like the best opportunity. It's been a while since I used any kind of framework and I've never used a PHP MVC framework (although I did build one last year as an intellectual exercise).

I got over 90% of the site built in spare time over about 5 days and most of that was spent just reading the CakePHP docs. The reason for the lengthy delay between initial build and launch is down to that last 10%. As often happens with frameworks, nearly everything you'd ever need to do is bundled into some nice, easy to access functionality. That is, after all, kinda the point of a framework. It's the stuff that makes your product or website unique that proves trickier. I won't go into any details just now although I might come back to it in a later post.

More later, first of all, go shelve some books: Shelvist.

Scoped Style

1. Have a look at a demo
2. Check out the source on GitHub
3. Download it and start scoping some styles.

CSS Scoped

Update: Bam! Plugin'd

One of the things in HTML5 that caught my eye a while back was the new 'scoped' attribute for style tags. The idea of it is that you can include a style element mid-document, mark it as scoped and its declarations will only apply to the style elements parent element and its child elements. The styles won't affect anything outside this. The biggest problem with bringing in this attribute is that it's not backwards compatible. If you include a style block mid-page, its declarations will be applied to ever element whose selector matches, inside or outside scope. It is anti-progressively enhancable. This means that designers and developers can't start using it until there's enough support. What we need is another of those JS tricks to make it usable.

Possibly the idea of having a style block dropped into the middle of a semantic document strikes you as a distasteful but thinking practically, this could make CMS workflows a bit easier and keep global stylesheets clean of lots of one-off specific styles. For example, this post describing striped CSS backgrounds could have benefited from having the demo directly within the body of the post but, without going into the backend to add an extra id, this could have messed up the layouts for all other posts.

My first attempt at solving this problem with JS involved generating a unique class, adding it to the parent element and then parsing the style blocks using JSCSSP so that I could rewrite them with the new class to add specificity. This approach only worked for the most basic declarations, unfortunately. The parser worked perfectly but there's a lot of detail in CSS specificity that mean this would be a much larger problem than I thought.

My second attempt involved:

1. Allowing the style blocks to affect everything on the page (at which point, the elements in-scope look right, those out-of-scope look wrong)
2. Using JS to read the current computed styles of each element in-scope and copy them to a temporary array
3. Emptying out the scoped style element (in-scope look wrong, out-of-scope looks right)
4. Copying the styles back from the temporary array onto each element

The first couple of versions only work in webkit, the latest (below) works in mozilla, too.

This worked great unless you had more than one scoped block – step 1 allowed scoped styles to affect each other.

The current attempt involves temporarily emptying all other scoped styles before taking the computed styles from a block. I'm now just thinking that this method might not work if you have multiple scoped blocks within the same context. Oh well, there's something to fix in the future.

This is where I'm at just now.

Yes, it's a mess, yes the JS is scrappy and yes, it doesn't currently work in IE but I'll get round to that next. It took long enough to get it working in Firefox as there's no simple way to convert a ComputedCSSStyleDeclaration to a string in Mozilla unlike Webkit's implementation of cssText or IE's currentStyle. I might even make it into one of those new-fangled jQuery plugins everyone's using these days.

jQTouch Calendar Extension

Uncooked Composition 5

Uncooked Composition 4

Uncooked Composition 3

Toolkit

I wish...

iMagritte

Context-free content and content-agnostic design

Last year I wrote about the trend towards separation between the design creators and the content creators and I think it’s time to revisit that topic. The idea was that, although the tools of the future would mean there would be less need for individual content to be designed and developed by hand, there would be an increasing number of opportunities for content to be viewed within the context the consumer wants to view it rather than the way the designer wanted it viewed.

Okay, that sounds a lot more complicated than it is. Here’s a scenario:

Part One

1. I write interesting articles
2. I post them on my MySpace-inspired, geocities refugee, blink-tag-heavy page
3. You refuse to read them because the design literally makes your eyeballs itchy.

Part Two

1. Some other guy writes reeeeeeeally dull articles
2. He posts them on his beautifully-designed, subtly gradiented, drop-shadowed, Helvetica Neue Light site
3. You refuse to read them because the content is so dull your heart-rate slows almost to a stop and you forget to breathe.

The type of content/design separation I’m talking about here would mean you take my wonderfully written articles, his exquisitely-designed layout and put the two together. Technically, you might be able to create a brain-numbingly-dull but eye-ball-searing monster out of the left-over parts but you shouldn’t. At the moment, we can’t just take any arbitrary design and apply it to any arbitrary content but it won’t be that long. As I mentioned last year, the separation of design and content is not a bad thing for designers, in fact, it’s an opportunity to create a better content consumption experience than the next guy.

The last year has provided a great number of examples of this trend and it seems it can only continue. Some of the more high-profile ones are:

FlipBoard

The hype around this was so immense that everybody absolutely had to have it on launch day. It was so popular, in fact, that it was about a week before anyone could actually use it, the load on their servers bringing them grinding to a halt. It was kinda justified, though. Having all of your own tailored content pulled from Twitter, FaceBook and any number of recommended channels (groups of RSS feeds and websites) then presented to you as a personalised magazine is quite enjoyable. The creators got into a bit of trouble for screen-scraping (what they refer to as ‘parsing’) rather than using the content the way the creators intended but that seemed to die down once the content creators realised they were still getting the ad revenue.

Reeder

This really applies to any RSS reader but Reeder by Silvio Rizzi is a particularly nice and well-known example. RSS is really the best example of context-free content where, for many years now, the audience has had the ability to take in the content of articles in their reader of choice. Initially, this was for convenience – why access content in 20 different places when you could access one – but now that the RSS-consuming public are au fait with that, it has moved on to more aesthetic concerns. Reeder presents articles in off-black text on an off-white background in Helvetica. It’s possibly an overused visual cliché but it’s a nice one and it works.

Bookmarklets, UserScript, UserStyles and Extensions

There are a number of bookmarklets whose specific purpose is to apply a predefined design to any arbitrary content. Many of them use Helvetica. Although Bookmarklets have been around for a long time, their popularity never really took off with the mainstream web audience. It’s probably something to do with the conflicting messages tech-types give to non-tech-types: “Never install programs from the web, they’re almost certainly viruses” and “A Bookmarklet is a little program that runs in your browser...”. UserScripts keep pushing the boundaries for what is possible via JS but require a level of technical ability beyond the vast majority. UserStyles are a more passive way of doing many of the same things as UserScripts and Bookmarklets although whereas they run JS and actively modify the page, UserStyles take the markup they’re given and change it.

At the moment, it seems the most user-understandable way to enhance the browser experience is with browser extensions, even if they are simply used to run the same JS you’d have in a bookmarklet or a userscript.

Helvetireader

We’re getting quite meta here. If you’re using Google Reader for consuming your RSS, you’ve already brought the content into a new context. Helvetireader from Jon Hicks takes that content and overlays another design. Which uses Helvetica.

Readability

I mentioned this briefly last year. This is probably the highest-profile bookmarklet in this field. In the words of the creators, arc90, it ‘...makes reading on the Web more enjoyable by removing the clutter...’ actually, that’s a far better way of describing what I’m talking about than my whole first three paragraphs. This allows a few variations in designs giving the user a bit more control over how they want things presented.

NotForest

This is superficially similar to readability but whereas with readability, you customise the design before you install the bookmarklet, this allows you to choose your design each time it is launched.

More than words...

When I’m referring to content, I don’t just mean text. Any media – audio, video, photo – can be separated from its original context and presented in a more accessible way. QuietTube does this for YouTube videos, presenting them on a white page with no recommended videos, no channel subscription buttons and, most importantly, no comments[1]. Flickriver presents the photos from a Flickr user’s account on a black background in an endless scroll so that you can just immerse yourself in the photos without having to look for the ‘next’ button every time. Huffduffer picks up any audio files you want in webpages and gives them to you as an RSS feed that can be subscribed to in iTunes. You’re taking the audio out of its original context and consuming it how you choose.

[1]YouTube comments are a great way to remind yourself that IQ 100 is an average. Half of the population are below that.↑

Yet another wood texture

Cash machine frustrations

Cash machines, ATMs, AutoTellers, Cashpoints... whatever you call them in your bit of the world, have been around since the 1960s and yet there is still no user-interface standardisation. There’s a wide difference in user experiences, designs, hardware, layout. A certain amount can be attributed to the kind of variation you’d expect – font, colour, terminology – but after almost 50 years, there’s still a gulf between a good experience – the kind where you barely notice the machine: you have a task, you do it, you’re done – and a bad one – the kind that would prompt you to write a post about it, for example.

It may seem like a trivial thing to most people but I find I’ve actually started avoiding the machine near my house just because it is the worst offender I have ever come across in terms of consistency, design and sense. This interface was definitely planned by programmers and not designers.

Roughly, the process goes like this:

Screen one

Insert your card

Inserts card

Okay so far.

Screen two

Please type your PIN then press Enter

beep, beep, beep, beep[1], look for the Enter button...

Screen three

Please wait, processing...

Wait, I haven’t pressed enter yet! Apparently I didn’t need to press enter, did I? You were lying when you said ’press Enter’. You’re already making me dislike you.

Screen four

Would you like a receipt with this transaction?






                         yes >
                          no >

The gap between the question (at the top of the screen) and the options (at the bottom) is just big enough to make me want to double-check the question. This would be much better if the options were themselves descriptive:

       I want a receipt >
I do not want a receipt >

Even better if this was asked at the end of the transaction. Maybe I haven’t decided whether or not I want a receipt. I’ll have a better idea after my transaction. What if I say no now and change my mind later? Is it too late then? Will I get another chance? Oh no, I’m hyper-ventilating...

Related to this screen is my biggest cause for complaint. When there is no receipt paper left what does the machine do? Miss out this question? Replace it with a single notification screen? No, instead, we get this screen:

Screen four, take 2

Unable to print receipt, would you like to continue
with this transaction?






                         yes >
                          no >

If you’re in the habit of anticipating the receipt question, you’re prepped and ready to press ’no’ as in "No, I don’t want a receipt". However, if this screen comes up, even if you notice the question is different, nine times out of ten, muscle memory kicks in and you find yourself pressing it anyway with a voice in your head shouting ’Stooooop!’ at whatever’s controlling your finger. You press it, your card is spat out, you have to start again. This experience will ensure that the next time you’re presented with the screen, you’ll double-check the question then the options then the question and then the options. Twice. Even if you try to scan the text, your eyes will still pick out ’receipt’ in the middle and ’with this transaction’ at the end. That doesn’t help, you actually need to read and comprehend fully. And here was you thinking you only wanted to get some cash out.

Screen five

Please select transaction






                         View Balance >
                             Withdraw >

I don’t have too many complains about this screen except for the unnecessary distance between the question and the options. It is a touch irksome that they’ve now switched to action-based label text ("view") from the option-based labels ("yes" above)

Screen six

Please select amount to withdraw


        < £100                  Other >
        < £70	                   £50 >
        < £40	                   £30 >
        < £20	                   £10 >

I would like £30, please. *beep*

Screen seven

Please wait, processing...

About 30% of the time, this is the next screen:

Screen eight

Unable to dispense required amount. Please enter a
different amount.

   Suggested amount(s): £20, £40

   £[   ]

There are several annoyances with this screen:

1. why was I even asked if I wanted £30 if the machine was never going to give me £30?
2. what if I were to type in £50? Would the machine also be unable to dispense that amount? £20 and £40 are suggestions but there’s nothing to say why I can’t have £50.
3. was it so utterly impossible for the programmers to figure out whether they were displaying one suggested amount or two? There should never be any reason for dynamically generated text to resort to the bracket(s).

As an aside, this reminds me of an online mortgage calculator I used once where you’d enter an amount like ”£20,000“ and it would say ”Incorrect Value“ but not say what was wrong so you’d try ”20,000“, ”Incorrect Value“, ”20000“, ”Incorrect Value“, ”£20000“, ”Thank you“.

The operators of these machines must have a lot of usage statistics so rather than give equal billing to all possible options throughout the process, why not reorganise so that the greatest number of people have the smallest amount of work to do? If 90% of users use the machines to withdraw £20, put that on the front screen (just after the PIN). Don’t insist on brand colours when they’re annoyingly intrusive on the user experience. Heck, why not just set the whole thing in off-black Helvetica on an off-white background with that ubiquitous swiss red colour?

I’d rather something like this

than something like this

Am I over-reacting?

Yes, completely. That’s not the point. Well thought-out user interfaces are what separates us from the animals. Or something like that. There are now over 2 million ATMs in the world, so unifying them will probably never happen but that doesn’t give the designers of the next generation of machines free reign to start from scratch all over again.

Am I generalising?

Okay, maybe a bit. Maybe a lot, actually. I’ve successfully used ATMs in a number of countries in a variety of languages but then, I do ‘tech stuff’ for a living. Without getting my hands on those usage statistics, I couldn’t really say.

Anyway, aren’t we supposed to be paying for everything by thumbprint these days?

1Note, this is not my actual PIN, that would be ‘beep, beep, beep, beep’↑

Uncooked Composition 2

Uncooked Composition

Music, much like mathematics, is a young person's game. If you haven't made it by the time you're 25, your chances of making an impact on the world are significantly diminished. That's not to say it's impossible, it's just much less likely.

Basically, I'm beginning to come to the realisation I'm not going to be a rock star. I might not even make it into space. To that end, I've decided that, instead of scribbling away at writing and rewriting the same songs I've been trying to improve for the last 10 years, I'd go the other way. A few months ago, I put a dictaphone next to the piano and started recording the occasional random improvisation. Originally, the idea had been to pick the best bits and rework them for some reason or another but after listening back to them, I found there's some appeal in just hearing the raw first-take complete with do-overs and occasional accidental 'quotes' from other pieces.

Over the next little while, I'll be uploading some of them just so that I know I've done something with them other than leave them on a tape in the back of a drawer. When listening to them, bear in mind two things:

1. The piano needs retuned almost every week so some of it might be a bit rough.
2. This is, as the title says, 'Uncooked Composition'. I come in from work, take my shoes off, sit down at the piano, press record. There's no post-processing anywhere so there will be mistakes and do-overs.

Testing CSS3 stuff

You may have seen Google's 'Watch this space' advertising appearing all over the place. They have quite eye-catching diagonally striped backgrounds in various colours. A couple of days ago, I was wondering how easy it would be to recreate this in CSS without images. Unfortunately, the state of CSS 3 is such that some things work wonderfully, some just plain don't (scoped attribute, I'm looking at you). The following code relies on vendor extensions and so, unless you're willing to tend it and correct it after the spec is finalised, don't use this on a production server.

The most obvious thing to notice from the following code, though, is the competing syntax for the repeating linear gradient style. Mozilla have separated it into a distinct style (-moz-repeating-linear-gradient) while Webkit have built it as an option to their general gradient style (-webkit-gradient).

body {
	background-image: -moz-repeating-linear-gradient(
		-45deg,
		transparent,
		transparent       25%,
		rgba(0,0,0,0.15)  25%,
		rgba(0,0,0,0.15)  50%,
		transparent       50%,
		transparent       75%,
		rgba(0,0,0,0.15)  75%,
		rgba(0,0,0,0.15)
	);
	background-image: -webkit-gradient(
		linear,
		0% 0%,
		100% 100%,
		from(transparent),
		color-stop(0.25, transparent),
		color-stop(0.25, rgba(0,0,0,0.15)),
		color-stop(0.50, rgba(0,0,0,0.15)),
		color-stop(0.50, transparent),
		color-stop(0.75, transparent),
		color-stop(0.75, rgba(0,0,0,0.15)),
		to(rgba(0,0,0,0.15))
	);
}

To get a better idea of what this does, view source on this demo page. This includes a button to change the class on the body (using JS) which simply changes the background colour – the stripes are semi-transparent on top of that. Remember, due to the vendor prefixes, this only works in -moz or -webkit browsers.

It's supposed to look like this:

Fake girl protects fake fish from fake cat

I decided to make my desktop more dramatic...

Originally uploaded by thingsinjars on flickr

The Elementals

In one of my day jobs I do something involving education, large public institutions and web stuff and a while back I thought it might be an excellent idea to have a go at designing some cool educational toys. Learnin' 'n' Fun! At the same time! omg! And so forth!

The idea was to build the kind of thing you could use to squeeze knowledge into people's heads without them complaining. Y'see, it's never a good thing to trick people into learning. If your educational toy/game/experience relies too much on hiding the information behind the fun then the big reveal at the end – "Ha, I tricked you into learning something!" – will leave the player feeling cheated and not change their attitude towards learning. If, on the other hand, you try and push the ideas you want to get across at the expense of the core game mechanic, you'll end up with a bored user. My opinion is that you've got to be up front about the learning. You've got to say to the user "Look, this is learning and it's fun. No tricks here, it's exactly what it looks like". As for getting it to appeal in the first place, I find that very few things can beat extremely cute cartoons.

To that end, I present my first dabble in interactive educational whaddyamacallits: The Elementals, a fun periodic table where every element has its own unique personality.

It's available as an iPhone app initially but I'll be venturing into the Android Marketplace soon and putting it online as a web app.

Appington concept

Note: this is a concept sketch only. This doesn't actually exist. It'd be cool if it did, though.

Appington. Your applications brought to you.

Appington is, fundamentally, a single-application VNC client with a simple interface for task switching. Where most VNC applications present the user with the entire screen, Appington only shows a single window at any one time. This simplified interface makes interaction easier and saves on client application memory and reduces data transfer allowing the viewer to be more responsive. In some applications, this data transfer saving may be used to facilitate audio capture and transmission.

Applications list

This screen shows a list of all available applications grouped by first letter. In the lower-left, the user can toggle between listing all applications or only listing currently running applications. The right-hand panel shows more information about the selected application. In this example, Google Chrome is selected and running. The current memory and CPU usage are shown along with a note of how many active windows the application has. Because Chrome is currently running, the option to quit is shown. If we had selected an unlaunched application, this button would show the option to launch. In case of emergencies, there is always the option to Force Quit a running application.

Application window (portrait)

This shows a standard single application window. The button in the top left would return the user to the previous screen. From the right, the remaining buttons allow the user to capture a screen shot, maximize the application window to match the current iPad viewport (if possible), refresh the current screen (in case of render errors) and access the application's menu. In OS X, menu access would be accomplished by way of the accessibility API. At the moment, I'm not sure how it would work on other OSs.

Application window (landscape)

This shows a single window of an application with multiple windows. You'll notice the extra button at the top between Menu and Refresh. This menu will allow you to select which window you want to access between however many the application currently has open.

Other images

The partner application to this is a modified VNC server running on the host machine. It is responsible for window management, task-switching, menu parsing and audio capture (à la SoundFlower). If there is already a VNC server running, the partner app will leave the standard VNC functionality alone and act purely as a helper, providing the extra functionality but not using extra memory by duplicating functionality. This is a variation of noVNC using the python proxy to manage the socket connection allowing the client to be built in PhoneGap using HTML 5.

Like I said at the top, this hasn't been built yet. It'd be cool if someone did build it, though.

Tokyo Recommendations

A friend recently asked me for some recommendations for what she could do on a trip to Tokyo. I pretty much always recommend the same things so I thought I'd write them up with maps and streetview and the like.

Food

If you're going out for food the best place I can recommend is Shin Hi no Moto (a.k.a. Andy's Izakaya). It's run by a friendly English guy (Andy) and his family. It gets really busy later on so you're best to phone and book on +81 3 3214 8021. It's okay, you can book in English. They do amazing sashimi platters and big mugs of beer. Just make sure you don't order rice (there's no rice in an izakaya and they might scowl at you if you do). You can get there by taking the Yamanote line to Yurakucho.

• Andy's Shin Hi no Moto

For general daily eats, I'm addicted to Yoshinoya. Especially their Gyuu-don. Tasty, healthy and cheap. You can find Yoshinoya everywhere.

Walks

West Central Tokyo

If you're going to be there over a Sunday, you have to go to Harajuku. Even if you aren't there on a Sunday, the walk up Takeshita-dori is great fun. Here's a map of a little walk you can take up Takeshita-dori, round Harajuku and down to Shibuya:

• Takeshita-Harajuku-Shibuya

North-East Central Tokyo

If you fancy some culture, try this route. It takes in the Imperial Palace, Sumo museum and Edo-Toyko museum. You can finish off in Akihabara for sheer geek awesome or save that for another day.

• Imperial Palace-Edo-Tokyo Museum

Tokyo bay/Odaiba

Some people called me crazy for enjoying it but I like the walk across the rainbow bridge to Odaiba. Take the Yamanote to Tamachi and wander east-ish. You'll see the bridge once you're closer to the shore. You can take the lift up to the start of the walk and then wander out for some amazing views. It is, unfortunately, very noisy due to all the traffic but it's worth it. Head right across the bridge and follow it down, it'll probably take about an hour. Once you're on dry land again on Odaiba, you can wander around the shopping malls there (Aquacity, Seaside mall), take in the Statue of Liberty, go for a bite to eat and eventually head back. If you time it for getting dark, you can either get some amazing views of the bridge lit up at night or just enjoy the Yurikamome ride back (it's a completely automated train with no driver).

• Rainbow Bridge-Odaiba-Yurikamome

Views

For the best view across Tokyo, the Tokyo Metropolitan Government building (a.k.a. TMG or Tocho) really can't be beaten (especially as it's free to go up). It's in Shinjuku.

• Tokyo Metropolitan Government building

Bad tweet, go to your room

Disclaimer: I'm not a social media guru, consultant, expert, doctor, nurse, midwife, engineer, ninja or rockstar. Because of this, you know you can trust what I say about social media because I'm not trying to sell it to you. You can also ignore it because you're not paying me.

I don't tend to say much on the subject of Twitter. I also don't tend to say that much on Twitter itself. That said, I do spend a lot of time on the Internet so here are some things you should stop doing now. Like, right now. Call it ‘Social media bad practice’ if you will, or ‘Tips and Tricks for a Tidier Tweet’ if that's the kind of thing you're into. Whatever, stop doing these:

Autotweeting from 3rd party apps

When you use one of the many apps that track your weight, book-reading habits, location, tweetability or shoe-size, disable the 'post to Twitter?' option, please. When I see these posts, all I read is either:

“According to amIanEejit.com, I'm an Eejit. Are you? Try it now.”

or

“I need public validation. Did I do good? Did I?”

Tweeting a shortened URL which performs an action on the logged-in user's account

Of course, fault for this should also be spread equally between the tweeter, the website which allows GET operations to modify data and the user who isn't wary enough of shortened URLs to expand them first. The person who creates the shortened URL without being aware of the consequences is to blame and an eejit. If you're not sure what I'm talking about, here's an example:

1. You have an account on website X which you are logged into
2. I have an account on website X which I am logged into
3. Website X allows you to delete your account by going to www.example.com/deletemyaccount.php
4. You copy that URL and shorten it using bit.ly
5. You tweet “Hey, this is what I think of Website X: http://bit.ly/madeupthing”
6. I click the link
7. My account on Website X gets deleted
8. I stab you with pencils.

Using inappropriate hashtags to piggy-back on an unrelated discussion

Some people use #hashtags as tweet meta data providing an extra piece of context on the tweet – “Om nom nom #fridaymorningbaconroll” – while others use them to create fluid, transient chatrooms. Where in the past you'd have used IRC and created a relevant room, using Twitter and a hashtag, you can jump into a conversation and out again without even trying. If you attempt to barge your way in with irrelevant comments, advertising nonsense or general eejicy, you act no better than an out-and-out-spammer and I don't follow no spammers.

Flooding followers with real-time reporting

Use a separate account for this kind of thing. Macrumors do this whenever there's one of those big Steve Jobs parties – if you want to follow all the info, follow the @macrumorslive account. Similarly, Fridaymix do the same thing. Discussions happen with the @fridaymix account while the announcements of what is currently being played come from @fridaymixdj.

Related: Retweeting your other account.

If I wanted to follow the other account, I'd follow the other account.

Also related: Retweeting your own main account

I heard you. Don't be the guy at the party with one punchilne that you tell again and again. I already know that guy. I don't follow him on Twitter.

Posting quotes from conference presentations without context or grammar

This doubly applies if the quote sounds like a half-hearted Zen koan.

“Listen to the youth. They have younger voices.”

“Use torches to light the way. Technology is your torch.”

This triply applies if you use antimetabole

“Don't follow the herd, herd the followers.”

“Don't live beyond means, have meaning beyond living.”

Of course, the worst is probably tweeting about your own blog post in which you discuss tweeting as if it actually matters.

...and a salesman, too.

Writing a Plex Plugin Part III

This is the final part of my walkthrough of the Plex Media Server Transmission plugin.

Right.

We've done the required built-in functionality (preference management, for instance) and the bits that talk to Transmission itself. Basically, we're done. Anything else added here is just extra. That is, of course, the best reason to add stuff here. As I have previously ranted at length, there's no point doing anything if you aren't trying to do it as well as it possibly can be done. In this particular instance, that manifests itself in the ability to browse, search for and download torrents all within the Plex client interface.

EZTV

I love EZTV. It makes things easy. Previous versions of this plugin included an EZTV search but after rooting around in the source of the µTorrent plugin, I found some nifty code which turned me onto the clever XML parsing Plex can do.

Note: although some of this stuff looks clever, all the cleverness was done by the Plex dev team and the author of the µTorrent plugin. I'm a good copy-and-paster.

This function grabs the H2s from the page http://ezrss.it/shows/. If you go there, you'll see that the page lists every TV show in EZTV. The original µTorrent function listed everything but there are a lot of shows there now so it was actually taking a long time just to get that list. As they've split the page up by section, we can just grab the bits we want. This is going to be a full page in Plex (not a popup) so we're using a MediaContainer.

def TVShowListFolders(sender):
  dir = MediaContainer()

Using the built-in XML module, we can simply pass in a URL and get back an object containing the hierarchical structure of the entire page. Seriously, how simple is this? As it's HTML, add in the option isHTML=True.

  showsPage = XML.ElementFromURL(
                'http://ezrss.it/shows/', 
                isHTML=True, 
                errors='ignore'
              )

Now that we have the whole page structure, take the chunks of the page we want. All the sections we want (and one we don't) are divs with the class 'block' so use that in xpath to pull them out.

  blocks = showsPage.xpath('//div[@class="block"]')

The first block is the one we don't want (if you look at the page, it's the one that lists all the letters) so we remove it.

  blocks.pop(0)

For each of the remaining blocks, find the text in the first H2. That is the letter title of the section ('A', 'B', 'C', etc). Add that to Plex as a menu item then return the entire list.

  for block in blocks:
    letter = block.xpath("h2")[0].text
    dir.Append(
      Function(
        DirectoryItem(
          TVShowListSubfolders,
          letter,
          subtitle=None,
          summary=None,
          thumb=R(ICON),
          art=R(ART)
        ),
        letter=letter
      )
    )
  return dir

I hope I'm not the only one impressed with that (although I have a feeling I might be). Using just a couple of lines from the XML module and a sprinkle of xpath and we've got another menu, dynamically generated from a third-party website. If EZTV ever change their layout, it should be a simple matter of changing the xpath to match and we're done. Again.

We can now do the same again but this time, we only pull out a single section based on the letter passed in.

def TVShowListSubfolders(sender, letter):
  dir = MediaContainer()
  showsPage = XML.ElementFromURL(
                'http://ezrss.it/shows/',
                isHTML=True,
                errors='ignore'
              )
  blocks = showsPage.xpath(
            '//div[@class="block" and h2 = "%s"]' % letter
           )

Remembering to ignore any 'back to top' links, write out a list of the shows in this section. These will call the TVEpisodeList method next.

 for block in blocks:
  for href in block.xpath('.//a'):
   if href.text != "# Top":
    requestUrl = "http://ezrss.it" + href.get("href") + "&mode=rss"
    dir.Append(
     Function(
      DirectoryItem(
       TVEpisodeList,
       href.text,
       subtitle=None,
       summary=None,
       thumb=R(ICON),
       art=R(ART)
      ),
      name=href.text,
      url=requestUrl
     )
    )
 return dir

This lists all available torrents for the chosen show. By this point, you should be familiar with how this works. We're using the XML module to grab the page at the URL (this time it's an RSS feed so we don't need to parse it as HTML); we use XPath to iterate through the items in the feed; we generate a menu item from the data which will call a function when selected; we append that to a MediaContainer then return the whole thing to Plex. Done. The AddTorrent function was defined higher up.

def TVEpisodeList(sender, name, url):
 dir = MediaContainer()
 feed = XML.ElementFromURL(url, isHTML=False, errors='ignore').xpath("//item")
 for element in feed:
  title = element.xpath("title")[0].text
  link = element.xpath("link")[0].text
  dir.Append(
   Function(
    DirectoryItem(
     AddTorrent,
     title,
     subtitle=None,
     summary=None,
     thumb=R(ICON),
     art=R(ART)
    ),
    torrentUrl=link
   )
  )
 return dir

Adult considerations...

There is currently a section in the plugin which will allow you to search IsoHunt. This might get dropped in future versions of the plugin as results from IsoHunt are almost exclusively...ahem...adult, regardless of search terms. Sure, that might be exactly what you were looking for but if you were actually looking for Desperate Housewives, you might be surprised when your file comes down and it's actual 'desperate housewives'...

Search EZTV

The final part is a straightforward search of EZTV. The interesting thing to note is that this uses a different type of menu item. Where normally, you'd use a DirectoryItem in a Function, this uses an InputDirectoryItem in a Function. This type of menu item will pop open an on-screen keyboard before calling the target function giving you the opportunity to grab some user input.

It's appended to the menu in the usual way:

 dir.Append(
  Function(
   InputDirectoryItem(
    SearchEZTV,
    L('MenuSearchTV'),
    "Search the TV shows directory",
    summary="This will use EZTV to search.",
    thumb=R(SEARCH),
    art=R(ART)
   )
  )
 )

By the way, I think there's a minor bug in the InputDirectoryItem in that it doesn't like it when subtitle is passed as a named argument. I should probably file that as a bug with Elan.

When the user has entered their input and submitted, the named Function SearchEZTV is called with the standard argument sender and the extra argument query containing the user's input.

This function was a lot longer in the previous version of the Framework. It was so much simpler this time round.

def SearchEZTV(sender, query=None):
  dir = MediaContainer()
  url = "http://ezrss.it/search/index.php?simple&mode=rss&show_name="
  if query != None:
   url += "%s" % query
  feed = XML.ElementFromURL(
          url, 
          isHTML=False, 
          errors='ignore'
         ).xpath("//item")
  if feed == None:
    return MessageContainer("Error", "Search failed")
  if len(feed) == 0:
    return MessageContainer("Error", "No results")
  for element in feed:
    title = element.xpath("title")[0].text
    category = element.xpath("category")[0].text
    link = element.find("enclosure").get("url")
    size = prettysize(int(element.find("enclosure").get("length")))
    dir.Append(
      Function(
       DirectoryItem(
        AddTorrent,
        title,
        subtitle=None,
        summary="Category: %s\nSize: %s" % (category,size),
        thumb=R(ICON),
        art=R(ART)
       ),
      torrentUrl=link
     )
    )
  return dir

Done

That's it. The only other little thing to mention is how handy it is to use the built-in Log function. The first argument is a standard Python string, the second is 'Should this only turn up in the console when in debug mode?' to which the answer will almost always be 'True'. There is a third argument but unless you're messing with character encodings, you don't need to worry about it.

Log("Message to log is: %s %d" % (errorString, errorCode), True)

Go, make...

If you made it to the end here, you're probably either keen to start making your own Plex plugins or my wife who I am going to get to proofread this. Assuming you're the former, here are some handy links:

• Online plugin development manual

  There are plenty of bits missing but it's still the best reference available for the framework.

• Plex forums

  Particularly the Media Server Plugins forum

• Plex Plugins Lighthouse

  This is where bugs are filed, suggestions made and final plugin submission happens. It's handy for picking little tips if someone else has had the same problem as you.

If you want to use the plugin, it's available in 'Plex Online' in Plex/Nine or 'Plex App Store' in Plex/Eight. If you'd like to read through the complete source, you can download the zipped .Bundle.

Transmission Plugin for Plex Media Server v1.0 [Zip - 1.1MB]

Writing a Plex Plugin Part II

This is part two of my walkthrough of the Plex Media Server Transmission plugin.

The previous part dealt with the basic required functions and preparing the main menu. This bit goes through the torrent control and the next will cover the built-in site-scraping functionality. To be honest, I'm not sure how much of this middle section will be of use to anyone but other torrent client plugin makers. The cool stuff really happens in the next one. Think of this as the difficult second album that you have to listen to before the return-to-form third.

Listing the torrents

This is the main interface to Transmission. Using the RTC method from before, this prepares a request to send via HTTP and reacts depending on the result (or error) we get back. First of all, we say we want information ('torrent-get') and we specify what info we want (the 'fields')

def TorrentList(sender):
  error, result  = RTC("torrent-get",
    { "fields": [
      "hashString","name","status",
      "eta","errorString",
      "totalSize","leftUntilDone","sizeWhenDone",
      "peersGettingFromUs",  "peersSendingToUs",  "peersConnected",
      "rateDownload",      "rateUpload",
      "downloadedEver",    "uploadedEver"
    ] }
  )

If we get an error back, we check what it was.

  if error != None:
    if error != "Connection refused":
      return MessageContainer(
          "Transmission unavailable",
          "There was an unknown error."
      )
    else:
      return MessageContainer(
          "Transmission unavailable",
          "Please make sure Transmission is installed and running."
      )

Now we have our information, we create a MediaContainer to display it. We'll be building these entries up as if they were standard library MediaItems although the final action will not be to play them.

  elif result["torrents"] != None:
    dir = MediaContainer()

For each set of torrent information we get back, we need to prepare the info and make it pretty.

    for torrent in result["torrents"]:
      progress    = 100;
      summary = ""
 
      if torrent["errorString"]:
        summary += "Error: %s\n" % (torrent["errorString"])

If we have some time until we're done and we're not seeding, display the progress as "12.3 MB of 45.6 GB (0%)". We add this to the MediaItem's summary field. This is where we use the prettysize and prettyduration functions we imported at the top. They take a computer-friendly value (1048576 bytes) and return a human-friendly one (1MB).

  if torrent["leftUntilDone"] > 0 and
    torrent["status"] != TRANSMISSION_SEEDING:
    progress = ((torrent["sizeWhenDone"] - torrent["leftUntilDone"]) /
          (torrent["sizeWhenDone"] / 100))
 
    summary += "%s of %s (%d%%)\n" % (
        prettysize(torrent["sizeWhenDone"] - torrent["leftUntilDone"]),
        prettysize(torrent["sizeWhenDone"]), progress
      )

Similarly, if there's an estimated time until the file is finished downloading, add that to the summary as "3 days remaining"

    if torrent["eta"] > 0 and torrent["status"] != TRANSMISSION_PAUSED:
      summary += prettyduration(torrent["eta"]) + " remaining\n"
    else:
      summary += "Remaining time unknown\n"

Display download status ("Downloading from 3 of 6 peers") and download and upload rates ("Downloading at 3KB/s, Uploading at 1KB/s").

  if torrent["status"] == TRANSMISSION_DOWNLOADING:
    summary += "Downloading from %d of %d peers\n" % (
      torrent["peersSendingToUs"],
      torrent["peersConnected"])
    summary += "Downloading at %s/s\nUploading at %s/s\n" % (
      prettysize(torrent["rateDownload"]),
      prettysize(torrent["rateUpload"]))

For all other downloading statuses, we don't need extended information so we just return a human-friendly version of the status we get back (we do this via another method below).

  else:
    summary += TorrentStatus(torrent)

If we're seeding (the torrent has finished downloading and we're just uploading now), write out information about the uploading.

  else:
    if torrent["status"] == TRANSMISSION_SEEDING:
      summary += "Complete\n"
      progress=100
      if torrent["downloadedEver"] == 0:
        torrent["downloadedEver"] = 1

"45.6GB, uploaded 22.8GB (Ratio 0.50)" and some detail about the people we're uploading to.

  summary += "%s, uploaded %s (Ratio %.2f)\n" % (
    prettysize(torrent["totalSize"]),
    prettysize(torrent["uploadedEver"]),
    float(torrent["uploadedEver"]) / float(torrent["downloadedEver"]))
  if torrent["status"] == TRANSMISSION_SEEDING:
    summary += "Seeding to %d of %d peers\n" % (
      torrent["peersGettingFromUs"],
      torrent["peersConnected"])
    summary += "Uploading at %s/s\n" % (
      prettysize(torrent["rateUpload"]))

Icon generation

The next addition was a bit of a tricky point for this version of the plugin. Previous versions generated the thumbnail icon dynamically using the Python Imaging Library (PIL). It would create a progress bar showing the exact percentage and write the name of the file on the icon. In order to be able to achieve this, PIL had to be imported which generated a whole bunch of deprecation warnings. There are rumours that a future version of the plugin framework will include some functionality to generate images on-the-fly (possibly a variation of PIL itself) but until then, I decided the best way forward would be to generate the images by hand and include them in the plugin. This meant that I could either generate 101 images (0% - 100%) or display the percentage rounded off. In order to save space, I went with rounding to the nearest 10%.

  nearest = int(round(progress/10)*10)

The last thing to do in this loop (remember, we're still looping through the torrent information we received all the way back up at the top of the page) is to actually add this item. It is added as a PopupDirectoryItem so that selecting it will display a context menu of action choices specified in the TorrentInfo method below. With that, we also add the summary we've spent so long crafting, the percentage icon as the thumb and a couple of extra bits of information to help later functions know what to do.

  dir.Append(
    Function(
      PopupDirectoryItem(
        TorrentInfo,
        torrent["name"],
        summary=summary,
        thumb=R("%s.png" % nearest)
      ),
      name = torrent["name"],
      status = torrent["status"],
      hash = torrent["hashString"]
    )
  )

To finish this menu, we add the same functions that are available to individual torrents but acting on all – 'Pause all' and 'Resume all' – then return the menu to Plex for display.

  dir.Append(
    Function(
      DirectoryItem(
        PauseTorrent,
        L('MenuPauseAll'),
        subtitle=None,
        summary=None,
        thumb=R(PAUSE),
        art=R(ART)
      ),
      hash='all'
    )
  )
  dir.Append(
    Function(
      DirectoryItem(
        ResumeTorrent,
        L('MenuResumeAll'),
        subtitle=None,
        summary=None,
        thumb=R(RESUME),
        art=R(ART)
      ),
      hash='all'
    )
  )
  return dir

Here's the TorrentStatus lookup. Again, this uses the built-in localisation function 'L' to display the text and, again, I still haven't actually translated any of it so there's still only english. I must get round to that eventually.

def TorrentStatus(torrent):
  if torrent == None or torrent["status"] == None:
    return L('TorrentStatusUnknown')
  elif torrent["status"] == TRANSMISSION_WAITING:
    return L('TorrentStatusWaiting')
  elif torrent["status"] == TRANSMISSION_CHECKING:
    return L('TorrentStatusVerifying')
  elif torrent["status"] == TRANSMISSION_PAUSED:
    return L('TorrentStatusPaused')
  elif torrent["status"] == TRANSMISSION_DOWNLOADING:
    return L('TorrentStatusDownloading')
  elif torrent["status"] == TRANSMISSION_SEEDING:
    return L('TorrentStatusSeeding')
  else:
    return L('TorrentStatusUnknown')

Torrent action menu

This is the popup menu displayed when you select one of the listed torrents. The only thing to notice from these is that the option to pause is only shown for active torrents and the option to resume is only shown for paused torrents. The hash mentioned here is the id of the torrent which will be needed later.

def TorrentInfo(sender, name, status, hash):
  dir = MediaContainer()
  dir.Append(
    Function(
      DirectoryItem(
        ViewFiles,
        L('MenuViewFiles'),
        subtitle=None,
        summary=None,
        thumb=R(ICON),
        art=R(ART)
      ),
      hash=hash
    )
  )
  if status == TRANSMISSION_PAUSED:
    dir.Append(
      Function(
        DirectoryItem(
          ResumeTorrent,
          L('MenuResume'),
          subtitle=None,
          summary=None,
          thumb=R(ICON),
          art=R(ART)
        ),
        hash=hash
      )
    )
  else:
    dir.Append(
      Function(
        DirectoryItem(
          PauseTorrent,  
          L('MenuPause'),      
          subtitle=None,
          summary=None,
          thumb=R(ICON),
          art=R(ART)
        ),
        hash=hash
      )
    )
    dir.Append(
      Function(
        DirectoryItem(
          RemoveTorrent,
          L('MenuRemove'),
          subtitle=None,
          summary=None,
          thumb=R(ICON),
          art=R(ART)
        ),
        hash=hash
      )
    )
    dir.Append(
      Function(
        DirectoryItem(
          DeleteTorrent,
          L('MenuDelete'),
          subtitle=None,
          summary=None,
          thumb=R(ICON),
          art=R(ART)
        ),
        hash=hash
      )
    )
  return dir

Torrent action functions

Each of the torrent action functions called (ViewFiles, ResumeTorrent, etc.) could have been references to a more generic function with an action option passed in but I decided to keep them distinct and separate so that any extra customisation that might be done later would be easier to do rather than hacking it in. This isn't so much a problem with this plugin but if this were to be adapted for another torrent client, there might be specific hoops that needed jumped through.

I won't go through each of them in detail as they are all very similar. Instead, I'll just describe one of them – RemoveTorrent.

Each Function menu item (pretty much every menu item in this plugin) takes at least one argument: sender. This tells Plex where it's supposed to return control after it's finished here. The second argument is the id of the torrent we want to act on.

def RemoveTorrent(sender, hash):

We define the action to perform and the arguments to pass to Transmission.

  action = "torrent-remove"
  arguments = { "ids" : hash }

Call Transmission's RPC via the RTC method defined earlier catching the results and any errors returned.

  error, result = RTC(action, arguments)

If there's an error, any error, display it. Otherwise, display a success method. Both of these are displayed as a popup MessageContainer.

  if error != None:
    return MessageContainer("Error", error)
  else:
    return MessageContainer("Transmission", L('ActionTorrentRemoved'))

Okay, so the middle section of the plugin might not be that interesting. Next time I'll cover the clever built-in XML parsing bits and everything'll be cool again. I promise.

Writing a Plex Plugin Part I

In my awesome home cinema setup (about which I'll need to blog sometime), I use many pieces of inter-connected software and hardware. The hub, though, is the OS X computer in the middle running the Plex Media Server and the Transmission BitTorrent client. This post describes the plugin which ties them together.

I've just released version 1.0 of the Transmission plugin for the Plex Media Server. As with all good software projects, there were actually many releases before 1.0 but I thought this was the right time to write up a walk-through of what the code looks like and does. I didn't write the initial version of the plugin but I've maintained it since v0.7 and now pretty much rewritten everything a couple of times. I'll post my walkthrough in a few installments because it's quite long.

The code as it currently stands (v1.0) is available for download below but within a few days you should also be able to download it within 'Plex Online'

Transmission Plugin for Plex Media Server v1.0

As Michael A. pointed out in the comments, I haven't actually mentioned anywhere below what it is that the plugin does. For a quick overview, check out the Plex wiki Transmission page.

This plugin is built to be compatible with Plex Plugin Framework v1 which initially looked like quite a major change from the previous version of the framework but isn't really that different. For Plex plugins, every time an action is performed, the system generates a URL and passes it down through its own menu structure until it gets to a plugin that handles that URL. The plugin then deals with the info in the URL however it likes. In the previous versions of this plugin, the URL was parsed manually and split into strings separated by '/'

For example the URL:

/video/transmission/id/status/action/

would first cause Plex to look in its 'video' menu item and find the plugin that said it could handle 'transmission' URLs. The plugin would take the rest of the string and separate it out. First contact the Transmission application, ask for all information about the torrent called 'id', check its 'status' and then perform the 'action' if possible.

The new plugin does pretty much the same except I no longer manually parse the URL. The plugin still registers with Plex to say it can handle URLs starting '/video/transmission' but then it passes functions through instead of URL-catching menu items. If you're familiar with JavaScript, it's like passing an anonymous function to handle something instead of catching the event manually.

Anyway, here's the python code with a running commentary:

Imports

First we import the Plex Media Server framework:

from PMS import *
from PMS.Objects import *
from PMS.Shortcuts import *

These are a couple of handy functions from the very first version of the plugin which make the outputs much more readable.

from texttime  import prettyduration
from textbytes  import prettysize

This next line actually causes Plex to issue a warning. These libraries won't all be available in the next version of the framework. Instead of urllib and urllib2, developers are to use the built-in HTTP module. Unfortunately, HTTP doesn't allow access to the headers of responses and the Transmission authentication system relies on exchanging a session ID via headers.

import urllib,urllib2,base64

Declarations

Set up some constants to save typing later on

PLUGIN_PREFIX = "/video/transmission"
PLUGIN_TITLE = "Transmission"

This is the first call to the Localisation module. Plex plugins allow for complete string localisation via a JSON file (I've only included English in this version because my torrent-related German and Japanese are poor). This will look in the JSON file for the key 'Title' and return whatever value is associated with it (or 'Title' if there is none).

NAME = L('Title')

More shorthand

ART        = 'art-default.jpg'
ICON      = 'icon-default.png'
SETTINGS  = 'settings-hi.png'
PAUSE      = 'pause-hi.png'
RESUME    = 'resume-hi.png'
SEARCH    = 'search-hi.png'
TV        = 'tv-hi.png'

TRANSMISSION_WAITING      = 1
TRANSMISSION_CHECKING      = 2
TRANSMISSION_DOWNLOADING  = 4
TRANSMISSION_SEEDING      = 8
TRANSMISSION_PAUSED        = 16

Definitions

Required

Here is where we start the plugin code. This is one of the standard functions which gets called when the plugin is initialised.

def Start():

Tell Plex we can handle '/video/transmission' URLs and that our main function is called 'MainMenu'

  Plugin.AddPrefixHandler(
    PLUGIN_PREFIX, 
    MainMenu, 
    PLUGIN_TITLE, 
    ICON, 
    ART)

  MediaContainer.art = R(ART)
  MediaContainer.title1 = NAME
  DirectoryItem.thumb = R(ICON)

Another standard function, this handles the preferences. To connect to Transmission, you need the URL and port it is running on (127.0.0.1:9091 if it's on the same machine as the Plex Media Server) and the username and password if you have set them.

def CreatePrefs():
    Prefs.Add(id='hostname', type='text', default='127.0.0.1:9091', label='Hostname')
    Prefs.Add(id='username', type='text', default='', label='Username')
    Prefs.Add(id='password', type='text', default='', label='Password', option='hidden')

This is called immediately after the preferences dialog is submitted. This is the most basic checking you can do but it could include a call to Transmission to verify the info provided.

def ValidatePrefs():
    u = Prefs.Get('username')
    p = Prefs.Get('password')
    h = Prefs.Get('hostname')
    if( h ):
        return MessageContainer(
            "Success",
            "Info provided is ok"
        )
    else:
        return MessageContainer(
            "Error",
            "You need to provide url, username, and password"
        )

You'll notice the return here is a MessageContainer. That's Plex's version of an alert. It doesn't generate a new page, just pops up a little window.

Custom

That was the end of the predefined functions, the plugin proper starts here. As Transmission requires a username, password and a short-lived session ID (since Transmission v1.53) to perform actions, we define a function which will attempt to make a connection with just username & password. Transmission will then send back a 409 Conflict response to basically say "Urk, that's not quite right. If you want to talk to me, you'll need this:" and give us our session ID in a header.

def GetSession():
  h = Prefs.Get('hostname')
  u = Prefs.Get('username')
  p = Prefs.Get('password')
  url = "http://%s/transmission/rpc/" % h
  request = { "method" : "session-get" }
  headers = {}
  if( u and p and h):
    headers["Authorization"] = "Basic %s" % 
      (base64.encodestring("%s:%s" % (u, p))[:-1])
    try:
      body = urllib2.urlopen(
        urllib2.Request(
          url, 
          JSON.StringFromObject(request), 
          headers
        )
      ).read()
    except urllib2.HTTPError, e:
      if e.code == 401 or e.code == 403:
        return L('ErrorInvalidUsername'), {}
      return e.hdrs['X-Transmission-Session-Id']
    except:
      return L('ErrorNotRunning'), {}

Once the HTTP module allows access to returned headers, we will be able to use something like this to set global authorisation once and forget about it:

response = HTTP.Request(
    url, 
    { "method" : "session-get" }, 
    headers={}, 
    cacheTime=None
    )
HTTP.SetPassword(h,u,p)
HTTP.SetHeader(
  'X-Transmission-Session-Id', 
  response.headers['X-Transmission-Session-Id']
  )

Remote Transmission Calls

This uses the RPC API of Transmission to do everything we need. We pass into the function 'What we want to do' and 'Who we want it done to' basically.

def RTC(method, arguments = {}, headers = {}):
  h = Prefs.Get('hostname')
  u = Prefs.Get('username')
  p = Prefs.Get('password')
  url = "http://%s/transmission/rpc/" % h

  session_id = GetSession()

  request = {
    "method":    method,
    "arguments":  arguments
  }

Setup authentication here because, even though we've already gotten the session ID, it's useless if we don't actually use it.

  if( u and p ):
    headers["Authorization"] = "Basic %s" %
      (base64.encodestring("%s:%s" % (u, p))[:-1])

  headers["X-Transmission-Session-Id"] = session_id

Now that we've built our instruction, throw it at Transmission and see what comes back.

  try:
    body = urllib2.urlopen(
      urllib2.Request(
        url, 
        JSON.StringFromObject(request), 
        headers)
      ).read()
  except urllib2.HTTPError, e:
    if e.code == 401 or e.code == 403:
      return L('ErrorInvalidUsername'), {}
    return "Error reading response from Transmission", {}
  except urllib2.URLError, e:
    return e.reason[1], {}

  result = JSON.ObjectFromString(body)

We don't do error handling here as we want this function to be as generic as possible so we send anything we receive straight back to the calling function.

  if result["result"] == "success":
    result["result"] = None

  if result["arguments"] == None:
    result["arguments"] = {}

  return result["result"], result["arguments"]

Menus

Right, we've got our helper methods set up, we're ready to make our first menu. This is the main one we mentioned earlier.

def MainMenu():

You can set your menu screen to be laid out as “List”, “InfoList”, “MediaPreview”, “Showcase”, “CoverFlow”, “PanelStream” or “WallStream”. I'm keeping it simple here. Also, there's an extra call to GetSession just to check everything's fine and wake the app up.

    dir = MediaContainer(viewGroup="List")
    GetSession()

Pretty much all the menu items throughout the rest of this plugin are added using the same code which boils down to:

  dir.Append(
    Function(
      DirectoryItem(
        FunctionName,
        "Pretty Menu Item Name",
        subtitle="Short subtitle",
        summary="Longer menu item summary and description",
        thumb=R(ICON),
        art=R(ART)
      )
    )
  )

Starting in the middle, this reads as:

• Create a DirectoryItem.
• When this item is selected, use the function FunctionName to handle it.
• Display the text "Pretty Menu Item Name" for this item
• Display the text "Short subtitle" underneath this item (or None)
• Display the text "Longer menu item summary and description" for this item if required (or None)
• Use the resource called ICON (mentioned above) as the icon for this item
• Use the resource ART as the background
• This is a Function menu item
• Finally, Append this to the current menu

The first two main menu items are built exactly like that:

  dir.Append(
    Function(
      DirectoryItem(
        TorrentList,
        "Torrents",
        subtitle=None,
        summary="View torrent progress and control your downloads.",
        thumb=R(ICON),
        art=R(ART)
      )
    )
  )
  dir.Append(
    Function(
      DirectoryItem(
        SearchTorrents,
          "Search for a torrent",
          subtitle=None,
          summary="Browse the TV shows directory or search for files to download.",
          thumb=R(SEARCH),
          art=R(ART)
        )
      )
    )

This is a special 'Preferences' item that will call the Prefs functions defined at the top.

  dir.Append(
    PrefsItem(
      title="Preferences",
      subtitle="Set Transmission access details",
      summary="Make sure Transmission is running and 'Remote access' is enabled then enter the access details here.",
      thumb=R(SETTINGS)
    )
  )

A quick note: the function 'R' here, much like the localisation one 'L' above is a built-in helper function. It handles resources such as images. If you're developing a plugin and can't understand why your icon images are caching so much, it might be because you're going through this function.

Send the directory (or Menu) back to Plex

    return dir

The rest of the code deals with torrent control and some clever built-in site scraping functionality which I'll cover later.

Maze 1k

• Random perfect maze generation, solution and rendering in 1011 bytes

Okay, this is the last one for a while. Really.

Unlike my previous 1k JavaScript demos, I really had to struggle to get this one into the 1024 byte limit. I'd already done all the magnification and optimization techniques I knew of so I had to bring in some things which were new to me from qfox.nl and Ben Alman and a few other places. This, combined with some major code refactoring, brought it down from 1.5k to just under 1. In the process, all possible readability was lost so here's a quick run through what it does and why.

First up, a bunch of declarations.

These are the colours used to draw the maze. Note, I used the shortest possible way to write each colour (red instead of #F00, 99 instead of 100). The value stored in the maze array (mentioned later) refers not only to the type of block it is but also to the index of the colour in this array, saving some space.

u = ["#eff", "red", "#122", "rgba(99,255,99,.1)", "#ff0"],

This is used for the number of blocks wide and high the maze is, the number of pixels per block, the size of the canvas and the redraw interval. Thanks to Ben Alman for pointing out in his article how to best use a constant.

 c = 21,

Here is the reference to the canvas. Mid-minification, I did have a bunch of function shortcuts here - r=Math.random, for instance - but I ended up refactoring them out of the code.

m = document.body.children[0];

For most of the time when working on this, the maze was wider than it was high because I thought that made a more interesting maze. When it came down to it, though, it was really a matter of bytesaving to drop the distinct values for width and height. After that, we grab the canvas context so we can actually draw stuff.

m.width = m.height = c * c;
h = m.getContext("2d");

The drawing function

The generation and solution algorithm is quite nice and all but without this to draw it on the screen, it's really just a mathematical curio. This takes a colour, x and y and draws a square.

l = function (i, e, f) {
  h.fillStyle = i;
  h.fillRect(e * c, f * c, c, c)
};

Designing a perfect maze

"You've got 1 minute to design a maze it takes 2 minutes to solve."
- Cobb, Inception.

Apologies for the unnecessary Inception quote. It's really not relevant.

Algorithmically, this is a fairly standard perfect maze generator. It starts at one point and randomly picks a direction to walk in then it stops picks another random direction and repeats. If it can't move, it goes back to the last choice it made and picks a different direction, if there are no more directions, all blocks have been covered and we're done. In a perfect maze, there is a path (and only one path) between any two randomly chosen points so we can make the maze then denote the top left as the start and the bottom right as the end. This particular algorithm takes 2 steps in every direction instead of 1 so that we effectively carve out rooms and leave walls. You can take single steps but it's actually more of a hassle.

For more on how this kind of maze-generation works, check out this article on Tile-based maze generation.

Blank canvas

This is a standard loop to create a blank maze full of walls with no corridors. The 2 represents the 'wall' block type and the colour #122. The only really odd thing about this is the code f-->0 which is not to be read 'as f tends to zero' but is instead 'decrement f by 1, is it greater than zero?'

g = function () {
  v = [];  //our stack of moves taken so we can retrace our steps.
  for (i = [], e = c; e-- > 0;) {
    i[e] = [];
    for (f = c; f-- > 0;) i[e][f] = 2
  }

By this point, we have a two-dimensional JavaScript array filled with 2s

Putting things in

  f = e = 1;    // our starting point, top left.
  i[e][f] = 0; // us, the zippy yellow thing

Carving out the walls

This is our first proper abuse of the standard for-loop convention. You don't need to use the three-part structure for 'initialize variable; until variable reaches a different value; change value of variable'. It's 'do something before we start; keep going until this is false; do this after every repetition' so here we push our first move onto the stack then repeat the loop while there's still a move left on the stack.

  for (v.push([e, f]); v.length;) {

P here is the list of potential moves from our current position. For every block, we have a look to see what neighbours are available then concatenate that cardinal direction onto the strong of potential moves. This was originally done with bitwise flags (the proper way) but it ended up longer. It's also a bit of a nasty hack to set p to be 0 instead of "" but, again, it's all about the bytes.

  p = 0;

Can we walk this way?

These are all basically the same and mean 'if we aren't at the edge of the board and we're looking at a wall, we can tunnel into it.'.

 if (e < 18 && i[e + 2][f] == 2) p += "S"
 if (e >= 2 && i[e - 2][f] == 2) p += "N";
 if (f >= 2 && i[e][f - 2] == 2) p += "W";
 if (i[e][f + 2] == 2) p += "E";

 if (p) { //    If we've found at least one move
  switch (p[~~ (Math.random() * p.length)]) { // randomly pick one

If there was anything to note from that last chunk, it would be that the operator ~~ can be used to floor the current value. It will return the integer below thye current value.

Take two steps

This is a nice little hack. Because we're moving two spaces, we need to set the block we're on and the next one to be 0 (empty). This takes advantage of the right-associative unary operator 'decrement before' and the right associativity of assignment operators. It subtracts 1 from e (to place us on the square immediately above) then sets that to equal 0 then subtracts 1 from the new e (to put us on the next square up again) and sets that to equal the same as the previous operation, i.e. 0.

  case "N":
      i[--e][f] = i[--e][f] = 0;
      break;

Do the same for s, w and e

    case "S":
      i[++e][f] = i[++e][f] = 0;
      break;
    case "W":
      i[e][--f] = i[e][--f] = 0;
      break;
    case "E":
      i[e][++f] = i[e][++f] = 0
    }

Whichever move we chose, stick that onto the stack.

    v.push([e, f])

If there were no possible moves, backtrack

  } else {
    b = v.pop(); //take the top move off the stack
    e = b[0]; // move there
    f = b[1]
  }
 }

End at the end

At the very end, set the bottom right block to be the goal then return the completed maze.

  i[19][19] = 1;
  return i
};

Solver

This is the solving function. Initially, it used the same algorithm as the generation function, namely 'collect the possible moves, randomly choose one' but this took too much space. So instead it looks for spaces north, then south, then west, then east. It follows the first one it finds.

s = function () {

Set the block type of the previous block as 'visited' (rgba(99,255,99,.1) the alpha value makes the yellow fade to green).

  n[o][y] = 3;

A bit of ternary background

This next bit looks worse than it is. It's the ternary operator nested several times. The ternary operator is a shorthand way of writing:

if ( statement A is true ) {
  Do Action 1
} else {
  Do Action 2
}

In shorthand, this is written as:

Statement A ? Action 1 : Action 2;

In this, however, I've replace Action 2 with another ternary operator:

Statement A ? Action 1 : ( Statement B ? Action 2 : Action 3 );

And again, and again. Each time, it checks a direction, if it's empty, mark it as visited and push the move onto our stack.

(n[o + 1][y] < 2) ?
  (n[++o][y] = 0, v.push([o, y])) :
    (n[o - 1][y] < 2) ?
      (n[--o][y] = 0, v.push([o, y])) :
        (n[o][y - 1] < 2) ?
          (n[o][--y] = 0, v.push([o, y])) :
            (n[o][y + 1] < 2) ?
              (n[o][++y] = 0, v.push([o, y])) :

If none of the neighbours are available, backtrack

                (b = v.pop(), o = b[0], y = b[1]);

Show where we are

Finally, set our new current block to be yellow

  n[o][y] = 4;

Are we there yet?

If we are at the bottom right square, we've completed the maze

  if (o == 19 && y == 19) {
  n = g();    //Generate a new maze
  o = y = 1; //Move us back to the top right
  s()     //Solve again

If we haven't completed the maze, call the solve function again to take the next step but delay it for 21 milliseconds so that it looks pretty and doesn't zip around the maze too fast.

  } else setTimeout(s, c);

Paint it black. And green. And yellow.

This is the code to render the maze. It starts at the top and works through the whole maze array calling the painting function with each block type (a.k.a. colour) and position.

    for (d in n) for (a in n[d]) l(u[n[d][a]], a, d)
  };

Start

This is the initial call to solve the maze. The function s doesn't take any arguments but by passing these in, they get called before s and save a byte that would have been used if they had been on a line themselves.

s(n = g(), o = y = 1)

Done

This little demo isn't as visually appealing as the Art Maker 1k or as interactive as the Spinny Circles 1k but it is quite nice mathematically. There are now some astounding pieces of work in the JS1K demo competition, though. I do recommend spending a good hour playing about with them all. Don't, however, open a bunch of them in the background at the same time. Small code isn't necessarily memory-efficient and you could quite easily grind your computer to a standstill.

Art Maker 1K

Even though the rules for js1k only let me make one submission, I couldn't stop myself making another. This one is inspired by those pseudo-romantic pictures that you get all over tumblr that get reblogged endlessly (actually, it was inspired by the blog That Isn't Art by someone with the same opinion as myself).

It randomly creates a sentence, adds some softly-moving almost bokeh coloured-circles and look, I made an art! Wait for the sentence to change or click (or touch) to change it yourself.

Art Maker 1k

And of course, don't forget the original spinny-circles 1k

The quest for Extreme JavaScript Minification

As described in detail previously, I've recently taken part in the JS1K competition where you have to squeeze something cool and clever into 1024 bytes of JavaScript. The quest to condense has become quite addictive and I found myself obsessing over every byte. This is the kind of stuff that the Closure Compiler does quite well automatically but there are some cases where you just need to get in there and manually tweak.

Here are some of the tricks I've picked up in my struggle for extreme minification:

Basic improvements

Use short variable names.

This one's fairly obvious. A more useful addition to this is:

Shorten long variable names.

If you're going to be accessing an element more than once, especially if it's a built-in element like 'document', you'll save a few bytes every time you reference it if you create a shortened reference to it.

  document.body.style.overflow="hidden"
  document.body.style.background="red"
  (74 characters)

can shorten to

  d=document;b=d.body;s=b.style;
  s.overflow="hidden";
  s.background="red"
  (69 characters)

and any references to s after are going to save 19 characters every time.

Remove whitespace

This one's so obvious, I don't need to mention it.

Set Interval

The extremely handy setInterval function can take either a function or a string. If you give it an anonymous function declaration:

  setInterval(function(){x++;y--},10);

You will use up more characters than if you give it just the inside of the function as a string:

  setInterval('x++;y--',10);

But the outcome will be the same.

Little-used aspects

Not many people use JavScript's scientific notation unless they're doing scientific stuff but it can be a great byte saver. The number 100 is equivalent to 1 * 10^2 which is represented in JavaScript as 1E2. That's not a great saving for 100 but 1000 is 1E3, 10000 is 1E4. Every time you go up a factor of 10, you save 1 byte.

Fight your good style tendencies

In the war against space, you have to bite the bullet and accept that you may need to sacrifice some of your hard-earned practices. But only this once. Don't get in to the habit, okay?

No zeroes

  0.5  = .5

Yeah, it looks ugly but it works and saves a byte.

Naked clauses

  if {
    :
    :
  } else y

The y looks so naked out there. No braces to keep it warm. But if you only have one statement in your else clause, you don't need them...

No semi-final. . . final-semi. . . Semi-colon. No final colon.

You don't need a semi-colon on your last line, even if it does make it look as though you've stunted its growth.

The final few bytes

Operator precedence

You don't need brackets. Brackets are handy for you as the programmer to remember what's going on when and to reduce ambiguity but if you plan correctly, most of the time you won't need brackets to get your arithmetic to work out.

  b.getMilliseconds()/(a*250) 
      is the same as
  b.getMilliseconds()/a/250 

Shorthand notation

  l=l+1;l=l%14;
  l++;l%=14;
  l=++l%14;

The three lines above are equivalent and in order of bytes saved.

Shorthand CSS

If you need to set some CSS values in your script, remember to pick the most appropriate short form. Instead of s.background='black', use s.background='#000' but instead of s.background='#F00', use s.background='red'. In the same vein, the statements margin="0px" and margin=0 mean the same but the latter saves bytes.

Don't be generic

One final thing to mention is that these little challenges are not the norm. If you find yourself trying to squeeze code down like this you're probably working on a very specific project. Use that to your advantage and see if there are any savings to be made by discarding your usual policies on code reuse. In the JS1K challenge, we're provided with a specific HTML page and an empty script tag. One good saving made here (and mentioned in my previous post) was the way I grabbed the reference to the canvas element. The standard method is to use the id assigned to the canvas.

  d.getElementById('c')

Which is a good generic solution. No matter what else was on the page, no matter what order stuff was in, this would return the canvas. However, we have a very specific case here and the canvas is always going to be in the same place - the first child of the body element. That means we can do this instead:

  b.children[0]

This makes use of the reference we grabbed to the body earlier. If the page were rearranged, this would stop working but as it won't, we've saved 8 bytes.

In conclusion

Yes, this is all quite silly but it's also fun and tricky. Attempting these kinds of challenges keep us developers mindful of what it is we actually do and that makes it an extremely productive silly hobby.

Elementally, my dear JavaScript

The Angry Robot Zombie Factory launched its second iPhone/iPad app this week. I haven't mentioned it much yet because I spotted a minor typo in the final version after it had been approved so I submitted an update immediately. To get an early copy (like those misprinted stamps where the plane is upside down), go check out The Elementals. It's free, too. It's a simple, cartoonish periodic table.

Yesterday, the 1k JavaScript demo contest (#js1k) caught my eye. The idea is to create something cool using 1024bytes of JavaScript or less. I rootled around in the middle of The Elementals, grabbed the drawing function and 20 minutes later had made my entry.

The code I submitted is quite minified but isn't obfuscated. When it's unfolded, you can follow the flow fairly easily.

var d = document,
b = d.body,
s = b.style,
w = innerWidth,
h = innerHeight,
v = b.children[0],
p = 2 * Math.PI,
Z = 3,
x = tx = w / 2,
y = ty = h / 2;

The above is a bunch of declarations. Using things like d = document and b = d.body allows reuse later on without having to resort to the full document.body.style and saves a bunch of characters. When you've got such a small amount of space to play with, every character counts (mind you, the ZX81 only had 1k of RAM and look what you could do with that). Now that I'm looking at it, I think I could have tidied this a bit more. Darn. The sneaky bit about this code is the way we grab the reference to the canvas. The code d.getElementById('c') uses 21 characters but if we look at the provided HTML, we can use the fact that the canvas is the first child of the body element. The code b.children[0] uses 13 characters instead.

s.margin = "0px";
s.background = "black";
s.overflow = "hidden";
v.width = w;
v.height = h;
t = v.getContext("2d");

This sets the provided canvas to be the full width and height of the window then grabs the drawing context of it so we can make pretty pictures.

zi = function () {
 Z++;
 Z %= 14
};
m = function (X) {
 return (X * 200) % 255
};

Functions to be reused later. zi increases the number of spinning circles and is used by onmousedown and ontouchstart (oh yes, it works on the iPad, too). m is a mapping of the index of the circle to a colour. The 200 is arbitrary. I played about a bit until I found some colour combinations I liked.

 d.ontouchstart = function (e) {
 zi();
 tx = e.touches[0].pageX;
 ty = e.touches[0].pageY
};
d.onmousemove = function (e) {
 tx = e.clientX;
 ty = e.clientY
};
d.onmousedown = zi;

Setting the event handlers.

function r() {
 t.globalCompositeOperation = 'lighter';

I played about with the various composite operations. Lighter seemed the nicest.

 t.clearRect(0, 0, w, h);
 t.save();
 x = x + (tx - x) / 20;
 y = y + (ty - y) / 20;
 t.translate(x, y);

Originally, the circles followed the mouse pointer exactly but it lacked any life. By adding in this bit where the movement is delayed as if pulling against friction, it suddenly became a lot more fun and dynamic.

 var c = new Date();
 for (var i = 1; i       
]]>
Wed, 04 Aug 2010 00:00:00 GMT
home


http://thingsinjars.com/post/291/phonegap---the-drupal-of-app-development/
http://thingsinjars.com/post/291/phonegap---the-drupal-of-app-development/

        
PhoneGap - The Drupal of App development
I'm a fan of Drupal even though I don't use it that often. I like that I can see exactly what's going on. I can easily follow the execution from URL request to page serve.
What I usually end up doing on any Drupal project is:

	
build the majority of the site in a few hours
	
find one small piece of functionality missing that's absolutely essential
	
dig into the core to make it happen
	
find a simpler way of doing it and step out of the core a bit
	
find an even simpler way and step back a bit more
	
figure out how to do it in a single module file and put the core back the way it was.

That probably seems utterly inefficient but it has served me well since Drupal 4 and it means I've got a really good picture in my head of the internal workflow.
This is in stark comparison to other systems, particularly some .NET CMSs where a request comes in, something happens and the page is served. There are even some PHP frameworks and CMSs where everything is so abstracted, the only way you can get an accurate picture of what is happening is to already have an accurate picture of what is happening.
I've used several different ones and I keep coming back to Drupal (also, recently, Perch, but that's besides the point here).
“What on earth does this have to do with PhoneGap?” I hear you ask. Quite rightly, too.
When I was planning Harmonious, I looked at various frameworks for turning a combination of HTML, CSS & JavaScript into an app - PhoneGap, Appcelerator Titanium, Rhomobile. Rhomobile (or the Rhodes Framework) is built on Ruby so I didn't investigate too far. That's not to say it's not a good framework, I couldn't say either way. The idea behind using one of these frameworks is to save you the time of having to learn Objective-C and seeing as I've only done very minimal amounts of Ruby, I'd be replacing 'learn Objective-C' with 'learn Ruby'. That said, I've always thought Ruby developers opinion of themselves was slightly too high.
The first framework I properly spent some time with was Appcelerator. It seemed quite shiny and I liked having single interface access to compilation for iPhone and Android but I wasn't so keen on having to sign up for an account with them for no obvious reason. Some further investigation suggested that this was so you could send your project to their servers for proprietary cross-platform compilation of your desktop app. This is less useful, however, if you're developing just for iPhone and Android as for both, you need the SDK installed locally and the compilation is done on your own machine.
The main thing that I wasn't comfortable with in Appcelerator was that there seemed to be a lot happening behind the scenes. This is not necessarily a bad thing, of course, but it started that little buzz in the back of my head that I get when working on .NET. When I press 'compile', I want to know exactly what it's doing. I want to know exactly how it takes my JavaScript and embeds it, when does it include its own API files and what do I change to make it do stuff it doesn't do by default?
After that, I moved to PhoneGap (version 0.8.3) and found myself immediately falling into my Drupal workflow. The app fell into place in less than an hour (with a liberal sprinkling of jQTouch and the Glyphish icons). I then needed to take a screenshot and couldn't see an obvious way to do it but, due to the nature of PhoneGap being completely open-source, it was easy to spot where to jump into the code. I hacked in a screenshot function in another hour, spent another half hour making it better and another making it simpler. Just to complete the cycle, I have now wrapped up all my code into a plugin and removed my hacking from the core. Hmmm... that all seemed eerily familiar.
That's not to say PhoneGap is perfect. All the benefits of a completely open-source project referred to previously also come with all the drawbacks. The current version (0.9.0) is fiendishly difficult to download and get started with. It has been split into one parent project and several child projects (one per mobile platform) and it's no longer obvious what you do. It's easy enough if you're already set up but actually getting there is tricky. The most common failing of any open-source project is also true: poor documentation. There's a wiki but it's mostly out-of-date. There's a section on phonegap.com called 'docs' but they're also out-of-date. There's an API reference but it's autogenerated from comments and is also out-of-date. The only place to get accurate information is the Google group but that's not documentation, that's solutions to problems.
There have also been some claims that PhoneGap is unstable and crashes but personally, I haven't seen that. It's possible that the crashes and performance issues are the result of memory leaks in the JavaScript. Appcelerator automatically runs JSLint on your code before compilation so it will highlight any problems. If you can fit that into your standard workflow, you might be able to avoid some of the instability.

Additional comments

It seems that Disqus (the commenting system I'm using below) has some problems with Safari 5 & Chrome so this comment was sent via gist (I knew I shouldn't have stopped using Noodle).
I'll respond later. I've just got back from the Apple store and have toys to play with.


Comment from Jeff Haynie (@jhaynie)
A few comments about Appcelerator.
1. We're completely open source and you can see all our active development every single commit on github.com/appcelerator.  We have plenty of outside core open source contributors.
2. Yeah, to do what we're doing, it's complicated - much more than Phonegap - so it does mean with complexity it's hard to grok.  however, the source is all there. Also, it's full extensible through our SDKs and we this SDK as the way we build Titanium itself.
3. For Desktop, we _only_ upload to our build servers as a convenience to cross-platform packaging.  Nothing mysterious and all the scripts we run are included (and open source) so you can run them on your own. Plenty of our customers do this behind the firewall.  When you're developing locally (say on a OSX machine), it's all local during dev. Only cross-platform packaging is done as a convenience to developers.  We have to pay for this bandwidth and storage and we do it to make it easier.  And it's free.
Hope this clarifies some of the above.   Phonegap's a great project and we love the team - but I think we're trying to do different things and come at it from different approaches. In the end, this is good for developers as it gives everyone more choice based on their needs.
      
]]>
Thu, 17 Jun 2010 00:00:00 GMT
home


http://thingsinjars.com/post/290/licencesxml/
http://thingsinjars.com/post/290/licencesxml/

        
licences.xml
JavaScript libraries and CSS frameworks are very popular these days. With each library, plugin, extension and template, comes another licencing statement. For most of these licences (MIT, for instance), you must include the licence statement in order to be able to use the code. In many cases, you also have to provide the original source and your own modifications. While, for uncompiled technologies such as these, this is a trivial matter, both this requirement and that of including the licence are awkward to implement if you like to minify your code. The licence is usually kept in an uncompressed comment at the top of the library (the YUI compressor specifically takes this into account with comments marked /*!  */ ) and, although anyone can read your modifications to whatever you've used, post-minification code is much harder to follow (cf. three of my last four blog posts) and is really not 'in the spirit' of sharing your code.
I'd like to be able to bundle all the licences and sources together outside the production files. Somewhere the interested user would be able to look them up if they liked but not somewhere that would automatically be downloaded on a standard visit. To that end, I have looked around for an established standard for this and not found anything. If you know of one, please let me know. Until I do find a good standard, here's my suggestion – a simple XML file located at /licences.xml in the format outlined below. It contains details of the file the licence pertains to, the uncompressed source (optional), the title of the licence and a URL where the full licence text can be found (on opensource.org or creativecommons.org, for instance). It also includes a (probably superfluous) shortname for the licence. I might remove that bit. You can optionally include this meta in your HTML if you want an explicit link between your source and the licence file:
<meta name="licences" value="/licences.xml" />
 
I'm currently undecided as to whether to go with XML or JSON. They're fairly interchangeable (barring XML attributes) but JSON takes less space. Then again, there's not as much need to save space in this file. Anyone have any recommendations? The entire format is, of course, up for discussion. Have I forgotten anything? Have I included anything unnecessary? I'm going to start using this in my projects until someone points out some major legal problem with it, I think.
 
XML
 
 
<licences>
 <licence>
  <source>
   <url>
    /includes/script/jquery/1.4/jquery.min.js
   </url>
   <uncompressed>
    /includes/script/jquery/1.4/jquery.js
   </uncompressed>
  </source>
  <deed>
   <title>
   MIT License
   </title>
   <url>
    http://www.opensource.org/licenses/mit-license.php
   </url>
   <shortform>
   MIT
   </shortform>
  </deed>
 </licence>
 <licence>
  <source>
   <url>
    /includes/script/custom/0.1/custom.js
   </url>
   <uncompressed>
    /includes/script/custom/0.1/custom.min.js
   </uncompressed>
  </source>
  <deed>
   <title>
   Attribution Share Alike
   </title>
   <url>
    http://creativecommons.org/licenses/by-sa/3.0
   </url>
   <shortform>
   cc by-sa
   </shortform>
  </deed>
 </licence>
</licences>
 
JSON
 
 
{
 licences:{
  {
   source:{
    url:'/includes/script/jquery/1.4/jquery.min.js',
    uncompressed:'/includes/script/jquery/1.4/jquery.js'
   },
   deed:{
    title:'MIT License',
    url:'http://www.opensource.org/licenses/mit-license.php',
    shortform:'MIT'
   }
  },
  {
   source:{
    url:'/includes/script/custom/0.1/custom.min.js',
    uncompressed:'/includes/script/custom/0.1/custom.js'
   },
   deed:{
    title:'Attribution Share Alike',
    url:'http://creativecommons.org/licenses/by-sa/3.0',
    shortform:'cc by-sa'
   }
  }
 }
}
      
]]>
Fri, 20 Aug 2010 00:00:00 GMT
home