The Particle Lab

Track traffic exiting your site, without jQuery

Fri, 07 Aug 2015 00:00:00 +0100

A few years ago (well, over five years ago!) I published an article that showed you how to use Google Analytics to track traffic exiting your site with a little bit of jQuery. Now while jQuery is great, for smaller sites it’s usually overkill, especially if only included for this one use. Recently on one of these small sites I came up with a different way of doing it, without jQuery.

So how can we do this without jQuery? Well many of jQuery’s features are available in some form in most modern browsers. For example animation can be easily implemented with CSS3’s transition and/or animation features. Most importantly there is widespread support for querySelectorAll in browsers.

Did you know a large part of jQuery is a library called Sizzle which is just used for selecting elements with CSS selectors. This is exactly what querySelectorAll does.

With this knowledge let’s start this again without jQuery. Firstly, we’ll put something in the markup to indicate which links should be tracked. Last time, I used a custom jQuery filter to find all external links but by adding something to the markup we can be more explicit about what is tracked, as well as also tracking non-link elements (buttons and forms for example). I’m going to use a HTML5 custom data attribute for this.

<a href="http://google.com" data-ga-track>Let me google that for you</a>

That little data-ga-track is all we need. Now here’s the javascript, I’ll go through it all in a moment.

;(function(){
  if (!document.querySelectorAll) return;
  if (!Array.prototype.forEach) return;

  function track() {
    var href = this.getAttribute('data-ga-track') ||
               this.getAttribute('href');

    ga('send','pageview', '/out/'+href);
  };

  var trackables = document.querySelectorAll('[data-ga-track]');

  Array.prototype.forEach.call(trackables, function(el) {
    el.addEventListener('mousedown', track);
  });

})();

I’m using an immediately invoked function expression (IIFE) to keep the global scope clean and within the function, bailing quickly if the browser does not support a couple of key features. Obviously we need querySelectorAll and over 97% of browsers support this which is great, but IE8’s support is a little buggy. I’m also checking for the ES5 Array function forEach which is supported by over 95% of browsers. I’m using this later on as a simple way to iterate through the matching elements, but more importantly this is not supported by IE8 so that won’t run the rest of the code even though it does (partly) support querySelectorAll.

Next up there’s the function that actually logs to Google Analytics, which will look for either a value for the data-ga-track attribute or if there isn’t one, look for the href attribute. This value is then sent as a page view to Google Analytics prefixed with /out/. Last time we sent events to Google Analytics rather than a pageview but that’s just down to my current preference, you could easily change this to send an event instead.

I then use querySelectorAll to find all elements with the attribute data-ga-track and stash them away in a local variable (the IIFE keeps this variable local to the function and out of the global scope). Lastly we use the forEach function to iterate through each element and add an event handler for the mousedown event to call the track function defined earlier. I’m using mousedown here to ensure that the tracking happens before the browser starts navigating away.

The mousedown event also fires when a right click occurs on the element, some of these clicks could be someone trying to ‘open in a new tab’ but some won’t be. This will result in a few false positives.

And that’s it. There’s not much too it really. Do you think it can be made shorter, simpler, better? Or do you have any other neat little standalone javascript snippets? Let us know!

Building the new Neutron Creations site

Wed, 25 Feb 2015 00:00:00 +0000

In case you missed it, we launched the new Neutron Creations site last week. Here’s a little behind the scenes look at how we built some of it.

If you haven’t had a look yet, go and have a quick poke around now. It’s quite a change from our initial single page design by Tim Van Damme and the later refresh by Elliot Jay Stocks.

This latest design comes from the thoughts of James Ritchie who we’ve had the pleasure of working with on several projects over the past few years and whom we trusted to come up with something modern & clean but with the scope to show off our development abilities.

The most visual element of the design was the logo animation on the home page. This went through many concepts and iterations, usually delivered to us as video renders from Adobe After Effects, which we then prototyped to see how they would perform. Here are a few of those prototypes.

This is the first animation we considered using on the final site. The logo is built using SVG and animated in javascript using requestAnimationFrame() in an attempt to animate as smoothly as possible. The blur effect and rotation animation are implemented with an SVG filter and CSS animation respectively.

While on the final site we use fallbacks and feature detection to ensure devices get an experience suitable for them, these prototypes do not and as such will probably only work in modern browsers. Please do dig around the code and ask any questions you have.

Run now — View on JS Bin

Next we have a very different animation concept with the inner pieces of the logo locking together and the outer pieces appearing one-by-one. This time instead of using javascript, we tried a CSS animation only approach. Using some complex SASS and an unhealthy dose of trigonometry we created individually named animations for the transforms required for each of the inner segments. We used SASS loops to create the sequentially timed animations for the outer segments.

If you try viewing this prototype in Safari you’ll notice that the outer segments don’t animate smoothly due to a SVG animation bug in Safari.

Run now — View on JS Bin

Iterating on this a little we dropped the locking animation and used the same animation effect from the outer segment on the inner segments. We then dropped in (literally!) the navigation and link to the portfolio and were all pretty happy with it.

Run now — View on JS Bin

And being happy with that (albeit with a few minor tweaks) we moved on to the other, somewhat simpler, aspects of the design. But we’ll leave those for another time.

As always if you have any question do ask them below, and if you haven’t already do check out the site .

Building WordStream

Mon, 26 Jan 2015 00:00:00 +0000

Last week I spent a couple of hours playing with some new technology and built wordstream, a real-time word cloud generated from the twitter sample stream. Here’s how.

The twitter streaming APIs are a very efficient way of having the tweets you’re interested in pushed to you. For example you can use the filter endpoint to have tweets matching your filter (author, hashtag, keywords etc) but for this I was more interested in the sample endpoint which sends out about 1% of all public tweets. This endpoint does however have some limitations:

A set of credentials (app/user combination) can only have a single connection open (any further connection attempts will terminate the previous ones). So in order to use it I would either need to have each visitor authenticate with the app in order to create their own streaming connection, or build some sort of servers-side proxy.
The API response is actually quite large and when combined with the hundreds of tweets per second received results in a large amount of data being retrieved (during testing on a Friday morning I was getting a fairly consistent 2 Mbps of data from the API).

Here’s a quick example (capturing the stream for about 5 seconds resulting in 1.3 MB of data, I’ve shown just the first ~1000 lines here, a sample of the sample you could say) of the streaming API data:

{ created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954651971600,
    id_str: '559747954651971584',
    text: 'Мосгорсуд оставил под арестом до 16 апреля Александра Кольченко, фигуранта дела ...',
    source: '<a href="http://ifttt.com" rel="nofollow">IFTTT</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2687442584,
       id_str: '2687442584',
       name: 'Галина Никандровa',
       screen_name: 'Byce6A',
       location: '',
       url: null,
       description: null,
       protected: false,
       verified: false,
       followers_count: 210,
       friends_count: 121,
       listed_count: 1,
       favourites_count: 0,
       statuses_count: 73725,
       created_at: 'Mon Jul 28 12:45:30 +0000 2014',
       utc_offset: null,
       time_zone: null,
       geo_enabled: false,
       lang: 'ru',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'C0DEED',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '0084B4',
       profile_sidebar_border_color: 'C0DEED',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://abs.twimg.com/sticky/default_profile_images/default_profile_1_normal.png',
       profile_image_url_https: 'https://abs.twimg.com/sticky/default_profile_images/default_profile_1_normal.png',
       default_profile: true,
       default_profile_image: true,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'ru',
    timestamp_ms: '1422289286660' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954639384600,
    id_str: '559747954639384577',
    text: 'Beautiful life is so much better than Carry you tbh',
    source: '<a href="http://twitter.com" rel="nofollow">Twitter Web Client</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2974152997,
       id_str: '2974152997',
       name: 'Sandra Young',
       screen_name: 'edwardalazobuy1',
       location: 'West Virginia',
       url: 'http://optimizedirectory.com/',
       description: '1D / Glee / T-Swizzle / Narnia / Criminal Minds / KSS 8 / Lucky #18/ #23 / #24 / Directioner / MATTHEW GRAY GUBLER FOR DA WIN! / Louis\' pants',
       protected: false,
       verified: false,
       followers_count: 0,
       friends_count: 1,
       listed_count: 0,
       favourites_count: 0,
       statuses_count: 37,
       created_at: 'Sun Jan 11 06:10:53 +0000 2015',
       utc_offset: null,
       time_zone: null,
       geo_enabled: false,
       lang: 'en',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'C0DEED',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '0084B4',
       profile_sidebar_border_color: 'C0DEED',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/559450280236830720/fGI9TXLt_normal.png',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/559450280236830720/fGI9TXLt_normal.png',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/2974152997/1422261339',
       default_profile: true,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'en',
    timestamp_ms: '1422289286657' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954672943100,
    id_str: '559747954672943104',
    text: 'Saints win 2-0! Enppi are 0-0 so double chance looking good on this one too.',
    source: '<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2960224947,
       id_str: '2960224947',
       name: 'The Secret Tipster',
       screen_name: 'Secret_Tipster_',
       location: '',
       url: null,
       description: 'FREE betting tips and £10-£1,000 challenges! \n\n5pts - Strong tip (high stakes)\n3pts - Good tip (medium stakes)\n1pt   - Fair tip (low stakes)',
       protected: false,
       verified: false,
       followers_count: 343,
       friends_count: 1588,
       listed_count: 2,
       favourites_count: 104,
       statuses_count: 290,
       created_at: 'Sun Jan 04 14:09:31 +0000 2015',
       utc_offset: 0,
       time_zone: 'London',
       geo_enabled: false,
       lang: 'en-gb',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: '000000',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '89C9FA',
       profile_sidebar_border_color: '000000',
       profile_sidebar_fill_color: '000000',
       profile_text_color: '000000',
       profile_use_background_image: false,
       profile_image_url: 'http://pbs.twimg.com/profile_images/551742687452229634/Q2rfimMq_normal.png',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/551742687452229634/Q2rfimMq_normal.png',
       default_profile: false,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'en',
    timestamp_ms: '1422289286665' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954647793660,
    id_str: '559747954647793666',
    text: '3 hours of oral anatomy on a Monday morning... Bad idea #confusedfordays',
    source: '<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 481348971,
       id_str: '481348971',
       name: 'Sara Gallagher',
       screen_name: 'Saraaa08',
       location: '',
       url: null,
       description: 'Life is about not knowing, having to change, taking the moment and making the best of it, without knowing what\'s going to happen next.',
       protected: false,
       verified: false,
       followers_count: 86,
       friends_count: 86,
       listed_count: 0,
       favourites_count: 763,
       statuses_count: 2007,
       created_at: 'Thu Feb 02 16:48:17 +0000 2012',
       utc_offset: -21600,
       time_zone: 'Central Time (US & Canada)',
       geo_enabled: false,
       lang: 'en',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'E113F0',
       profile_background_image_url: 'http://pbs.twimg.com/profile_background_images/633347123/p8t7222tk4jxb2l437zv.jpeg',
       profile_background_image_url_https: 'https://pbs.twimg.com/profile_background_images/633347123/p8t7222tk4jxb2l437zv.jpeg',
       profile_background_tile: true,
       profile_link_color: '9F16CC',
       profile_sidebar_border_color: '65B0DA',
       profile_sidebar_fill_color: '7AC3EE',
       profile_text_color: '3D1957',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/445991802739249152/8LSUaeA8_normal.jpeg',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/445991802739249152/8LSUaeA8_normal.jpeg',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/481348971/1376020262',
       default_profile: false,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [ [Object] ],
       trends: [],
       urls: [],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'en',
    timestamp_ms: '1422289286659' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954660376600,
    id_str: '559747954660376576',
    text: 'Don\'t twerk if you don\'t have a butt. It is that simple',
    source: '<a href="http://twitter.com" rel="nofollow">Twitter Web Client</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2982012311,
       id_str: '2982012311',
       name: 'Ronald Garcia',
       screen_name: 'christo41542623',
       location: 'in the airport Agh',
       url: 'http://hardcorewebdir.info/',
       description: 'What we do here echoes in eternity.',
       protected: false,
       verified: false,
       followers_count: 0,
       friends_count: 1,
       listed_count: 0,
       favourites_count: 0,
       statuses_count: 25,
       created_at: 'Sat Jan 17 01:42:36 +0000 2015',
       utc_offset: null,
       time_zone: null,
       geo_enabled: false,
       lang: 'en',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'C0DEED',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '0084B4',
       profile_sidebar_border_color: 'C0DEED',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/559440775339966464/L9Jleb81_normal.png',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/559440775339966464/L9Jleb81_normal.png',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/2982012311/1422255160',
       default_profile: true,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'en',
    timestamp_ms: '1422289286662' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954668757000,
    id_str: '559747954668756994',
    text: 'RT @Viccent22: Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar. Dan, lagi-lagi harus …',
    source: '<a href="http://www.writelonger.com" rel="nofollow">Write Longer</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 1851111336,
       id_str: '1851111336',
       name: 'Ann!',
       screen_name: 'PipitAnnisaF',
       location: 'Purbalingga-Gombong-Bandung',
       url: null,
       description: 'Bismillahirohmanirrohim... Allahumma yassir wa laa tu\'assir... Ya Allah,permudahkanlah jangan dipersulit. {Ab Ann}',
       protected: false,
       verified: false,
       followers_count: 400,
       friends_count: 196,
       listed_count: 1,
       favourites_count: 70,
       statuses_count: 13273,
       created_at: 'Tue Sep 10 10:57:46 +0000 2013',
       utc_offset: 25200,
       time_zone: 'Bangkok',
       geo_enabled: true,
       lang: 'id',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'FCEBB6',
       profile_background_image_url: 'http://pbs.twimg.com/profile_background_images/378800000115095569/094796ad031c0de4fdb006f3de3c197a.png',
       profile_background_image_url_https: 'https://pbs.twimg.com/profile_background_images/378800000115095569/094796ad031c0de4fdb006f3de3c197a.png',
       profile_background_tile: true,
       profile_link_color: 'CE7834',
       profile_sidebar_border_color: '000000',
       profile_sidebar_fill_color: '78C0A8',
       profile_text_color: '5E412F',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/556377564667604993/uFr5rOI__normal.jpeg',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/556377564667604993/uFr5rOI__normal.jpeg',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/1851111336/1413637653',
       default_profile: false,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweeted_status:
     { created_at: 'Mon Jan 26 16:20:24 +0000 2015',
       id: 559747692579262460,
       id_str: '559747692579262465',
       text: 'Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar Sabar. Dan, lagi-lagi harus sabar :\')',
       source: '<a href="http://www.hootsuite.com" rel="nofollow">Hootsuite</a>',
       truncated: false,
       in_reply_to_status_id: null,
       in_reply_to_status_id_str: null,
       in_reply_to_user_id: null,
       in_reply_to_user_id_str: null,
       in_reply_to_screen_name: null,
       user:
        { id: 812104176,
          id_str: '812104176',
          name: 'Hantu TimeLine',
          screen_name: 'Viccent22',
          location: 'FOLLOW MY INSTAGRAM: Viccent22',
          url: 'http://ask.fm/viccenttt',
          description: 'Jangan follow, nanti naksir(•̯͡.•̯͡) | Contact Us: ☎087770123452 / contactme22222@gmail.com',
          protected: false,
          verified: false,
          followers_count: 850235,
          friends_count: 0,
          listed_count: 355,
          favourites_count: 698,
          statuses_count: 10579,
          created_at: 'Sun Sep 09 02:02:12 +0000 2012',
          utc_offset: 25200,
          time_zone: 'Jakarta',
          geo_enabled: false,
          lang: 'en',
          contributors_enabled: false,
          is_translator: false,
          profile_background_color: '080808',
          profile_background_image_url: 'http://pbs.twimg.com/profile_background_images/747931822/ebd2c3137e9dcbd9361b1b163fd21481.png',
          profile_background_image_url_https: 'https://pbs.twimg.com/profile_background_images/747931822/ebd2c3137e9dcbd9361b1b163fd21481.png',
          profile_background_tile: false,
          profile_link_color: 'F00E0E',
          profile_sidebar_border_color: '000000',
          profile_sidebar_fill_color: 'F6F6F6',
          profile_text_color: '333333',
          profile_use_background_image: true,
          profile_image_url: 'http://pbs.twimg.com/profile_images/477404179636289537/gNRRvecD_normal.jpeg',
          profile_image_url_https: 'https://pbs.twimg.com/profile_images/477404179636289537/gNRRvecD_normal.jpeg',
          profile_banner_url: 'https://pbs.twimg.com/profile_banners/812104176/1402835276',
          default_profile: false,
          default_profile_image: false,
          following: null,
          follow_request_sent: null,
          notifications: null },
       geo: null,
       coordinates: null,
       place: null,
       contributors: null,
       retweet_count: 33,
       favorite_count: 0,
       entities:
        { hashtags: [],
          trends: [],
          urls: [],
          user_mentions: [],
          symbols: [] },
       favorited: false,
       retweeted: false,
       possibly_sensitive: true,
       filter_level: 'low',
       lang: 'in' },
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [ [Object] ],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'in',
    timestamp_ms: '1422289286664' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954639372300,
    id_str: '559747954639372290',
    text: '🍀@NiallOfficial hi sunshine!🍀\nHow are u? I hope u\'re fine😉\n🌸I love u so much, can u follow me?🌸\nU\'re my everything.\nu saved me. 💕😭 x 3.384',
    source: '<a href="http://twitter.com" rel="nofollow">Twitter Web Client</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 1033575770,
       id_str: '1033575770',
       name: 'nialler ',
       screen_name: 'nixalley',
       location: '020714',
       url: null,
       description: 'if u loved me why\'d u leave me? || i\'m in love with p and n',
       protected: false,
       verified: false,
       followers_count: 3039,
       friends_count: 2385,
       listed_count: 12,
       favourites_count: 2460,
       statuses_count: 52990,
       created_at: 'Mon Dec 24 22:29:10 +0000 2012',
       utc_offset: 3600,
       time_zone: 'Warsaw',
       geo_enabled: true,
       lang: 'en-gb',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'FFFFFF',
       profile_background_image_url: 'http://pbs.twimg.com/profile_background_images/452216615522676736/vkqDYOOn.png',
       profile_background_image_url_https: 'https://pbs.twimg.com/profile_background_images/452216615522676736/vkqDYOOn.png',
       profile_background_tile: false,
       profile_link_color: '9266CC',
       profile_sidebar_border_color: 'FFFFFF',
       profile_sidebar_fill_color: 'EFEFEF',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/551491161215361025/kVwuzBr6_normal.png',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/551491161215361025/kVwuzBr6_normal.png',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/1033575770/1420316329',
       default_profile: false,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [ [Object] ],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'en',
    timestamp_ms: '1422289286657' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954651959300,
    id_str: '559747954651959296',
    text: 'أستغفر الله العظيم الذي لا إله إلا هو الحي القيوم و أتوب إليه http://t.co/UwtV9SLroN',
    source: '<a href="http://knzmuslim.com" rel="nofollow">knzmuslim كنز المسلم</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2807986238,
       id_str: '2807986238',
       name: 'Salman',
       screen_name: 'alghamdisalman1',
       location: '',
       url: null,
       description: 'كنت ولا زلت وسأظل اعشق الكيان الازرق معاك على الحلوه والمره يا زعيم',
       protected: false,
       verified: false,
       followers_count: 325,
       friends_count: 417,
       listed_count: 0,
       favourites_count: 4,
       statuses_count: 2605,
       created_at: 'Sat Sep 13 18:59:35 +0000 2014',
       utc_offset: null,
       time_zone: null,
       geo_enabled: false,
       lang: 'ar',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'C0DEED',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '0084B4',
       profile_sidebar_border_color: 'C0DEED',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/540117613884366848/hsp1dn2e_normal.jpeg',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/540117613884366848/hsp1dn2e_normal.jpeg',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/2807986238/1414873049',
       default_profile: true,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [ [Object] ],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'ar',
    timestamp_ms: '1422289286660' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954651971600,
    id_str: '559747954651971585',
    text: '(ما قدروا الله حق قدره إن الله لقوي عزيز) [الحج:74] http://t.co/muW1xfBOGo',
    source: '<a href="http://qurani.tv" rel="nofollow">تطبيق قرآني</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2190909662,
       id_str: '2190909662',
       name: 'لعله يشفع لي⛅️',
       screen_name: 'for_mona3',
       location: '@xMonaAlx ☀️',
       url: 'http://www.tvquran.com',
       description: 'اذكروه يذكركم',
       protected: false,
       verified: false,
       followers_count: 5,
       friends_count: 2,
       listed_count: 0,
       favourites_count: 1,
       statuses_count: 9791,
       created_at: 'Tue Nov 12 19:25:01 +0000 2013',
       utc_offset: null,
       time_zone: null,
       geo_enabled: false,
       lang: 'ar',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'C0DEED',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '0084B4',
       profile_sidebar_border_color: 'C0DEED',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/416745923730235392/YDAesMzC_normal.jpeg',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/416745923730235392/YDAesMzC_normal.jpeg',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/2190909662/1384714039',
       default_profile: true,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [ [Object] ],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'ar',
    timestamp_ms: '1422289286660' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954651963400,
    id_str: '559747954651963395',
    text: 'RT @ilknurBasli: 18509ihtiyacla Lideringilizce 51',
    source: '<a href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2906151585,
       id_str: '2906151585',
       name: 'Begum Ozsahin',
       screen_name: 'bbggmmssrrkknn',
       location: '',
       url: null,
       description: null,
       protected: false,
       verified: false,
       followers_count: 36,
       friends_count: 12,
       listed_count: 0,
       favourites_count: 5,
       statuses_count: 1896,
       created_at: 'Fri Dec 05 09:57:56 +0000 2014',
       utc_offset: null,
       time_zone: null,
       geo_enabled: false,
       lang: 'tr',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'C0DEED',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '0084B4',
       profile_sidebar_border_color: 'C0DEED',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/557186002578866176/dGheKKpE_normal.jpeg',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/557186002578866176/dGheKKpE_normal.jpeg',
       default_profile: true,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweeted_status:
     { created_at: 'Mon Jan 26 15:54:40 +0000 2015',
       id: 559741218863992800,
       id_str: '559741218863992833',
       text: '18509ihtiyacla Lideringilizce 51',
       source: '<a href="http://twitter.com" rel="nofollow">Twitter Web Client</a>',
       truncated: false,
       in_reply_to_status_id: null,
       in_reply_to_status_id_str: null,
       in_reply_to_user_id: null,
       in_reply_to_user_id_str: null,
       in_reply_to_screen_name: null,
       user:
        { id: 1599517687,
          id_str: '1599517687',
          name: 'ilknur BAŞLI',
          screen_name: 'ilknurBasli',
          location: '',
          url: null,
          description: null,
          protected: false,
          verified: false,
          followers_count: 778,
          friends_count: 852,
          listed_count: 0,
          favourites_count: 712,
          statuses_count: 11538,
          created_at: 'Tue Jul 16 23:24:47 +0000 2013',
          utc_offset: null,
          time_zone: null,
          geo_enabled: false,
          lang: 'tr',
          contributors_enabled: false,
          is_translator: false,
          profile_background_color: 'C0DEED',
          profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
          profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
          profile_background_tile: false,
          profile_link_color: '0084B4',
          profile_sidebar_border_color: 'C0DEED',
          profile_sidebar_fill_color: 'DDEEF6',
          profile_text_color: '333333',
          profile_use_background_image: true,
          profile_image_url: 'http://pbs.twimg.com/profile_images/511566148530618368/V4WS62lr_normal.jpeg',
          profile_image_url_https: 'https://pbs.twimg.com/profile_images/511566148530618368/V4WS62lr_normal.jpeg',
          profile_banner_url: 'https://pbs.twimg.com/profile_banners/1599517687/1418137237',
          default_profile: true,
          default_profile_image: false,
          following: null,
          follow_request_sent: null,
          notifications: null },
       geo: null,
       coordinates: null,
       place: null,
       contributors: null,
       retweet_count: 6,
       favorite_count: 1,
       entities:
        { hashtags: [],
          trends: [],
          urls: [],
          user_mentions: [],
          symbols: [] },
       favorited: false,
       retweeted: false,
       possibly_sensitive: false,
       filter_level: 'low',
       lang: 'tr' },
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [ [Object] ],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'tr',
    timestamp_ms: '1422289286660' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954660368400,
    id_str: '559747954660368385',
    text: 'You\'re not clever you fucking Greb',
    source: '<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2427830686,
       id_str: '2427830686',
       name: '⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀',
       screen_name: 'RAFAELUDX',
       location: '',
       url: 'http://www.unorthodoxrepublic.bigcartel.com',
       description: 'IG:RafaelUDX\nSC:Rafaellol',
       protected: false,
       verified: false,
       followers_count: 1124,
       friends_count: 379,
       listed_count: 1,
       favourites_count: 895,
       statuses_count: 2116,
       created_at: 'Fri Mar 21 22:52:37 +0000 2014',
       utc_offset: 3600,
       time_zone: 'Amsterdam',
       geo_enabled: false,
       lang: 'en-gb',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: '000000',
       profile_background_image_url: 'http://pbs.twimg.com/profile_background_images/468511827144364033/CKxYF1fx.jpeg',
       profile_background_image_url_https: 'https://pbs.twimg.com/profile_background_images/468511827144364033/CKxYF1fx.jpeg',
       profile_background_tile: false,
       profile_link_color: 'FFFFFF',
       profile_sidebar_border_color: '000000',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/558565834415931392/OhRnr15H_normal.jpeg',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/558565834415931392/OhRnr15H_normal.jpeg',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/2427830686/1422007563',
       default_profile: false,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [],
       symbols: [] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'en',
    timestamp_ms: '1422289286662' }
  { created_at: 'Mon Jan 26 16:21:26 +0000 2015',
    id: 559747954651959300,
    id_str: '559747954651959297',
    text: 'RT @JeuneAthena: Hommage à Zac Evans, anglais de 19 ans, tué à coup de machette pour protéger des filles.\nLe multiculturalisme tue.\n\n- http…',
    source: '<a href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>',
    truncated: false,
    in_reply_to_status_id: null,
    in_reply_to_status_id_str: null,
    in_reply_to_user_id: null,
    in_reply_to_user_id_str: null,
    in_reply_to_screen_name: null,
    user:
     { id: 2190207191,
       id_str: '2190207191',
       name: 'artigouha herail ن ',
       screen_name: 'HerailArtigouha',
       location: 'languedoc/midi-Pyrénées',
       url: null,
       description: 'après mes enfants j\'ai élevé mon vin! maintenant:3ème vie à inventer...',
       protected: false,
       verified: false,
       followers_count: 1139,
       friends_count: 1161,
       listed_count: 26,
       favourites_count: 34075,
       statuses_count: 100572,
       created_at: 'Thu Nov 21 17:52:30 +0000 2013',
       utc_offset: null,
       time_zone: null,
       geo_enabled: true,
       lang: 'fr',
       contributors_enabled: false,
       is_translator: false,
       profile_background_color: 'C0DEED',
       profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
       profile_background_tile: false,
       profile_link_color: '0084B4',
       profile_sidebar_border_color: 'C0DEED',
       profile_sidebar_fill_color: 'DDEEF6',
       profile_text_color: '333333',
       profile_use_background_image: true,
       profile_image_url: 'http://pbs.twimg.com/profile_images/547124242555609088/6SJm4nu4_normal.jpeg',
       profile_image_url_https: 'https://pbs.twimg.com/profile_images/547124242555609088/6SJm4nu4_normal.jpeg',
       profile_banner_url: 'https://pbs.twimg.com/profile_banners/2190207191/1419279829',
       default_profile: true,
       default_profile_image: false,
       following: null,
       follow_request_sent: null,
       notifications: null },
    geo: null,
    coordinates: null,
    place: null,
    contributors: null,
    retweeted_status:
     { created_at: 'Mon Jan 26 13:43:53 +0000 2015',
       id: 559708303853641700,
       id_str: '559708303853641728',
       text: 'Hommage à Zac Evans, anglais de 19 ans, tué à coup de machette pour protéger des filles.\nLe multiculturalisme tue.\n\n- http://t.co/r06ncd1meg',
       source: '<a href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>',
       truncated: false,
       in_reply_to_status_id: null,
       in_reply_to_status_id_str: null,
       in_reply_to_user_id: null,
       in_reply_to_user_id_str: null,
       in_reply_to_screen_name: null,
       user:
        { id: 576201752,
          id_str: '576201752',
          name: 'Jeune Athéna ',
          screen_name: 'JeuneAthena',
          location: 'ELSASS, Europa',
          url: null,
          description: 'Européenne de souche de 21 ans | Nationaliste identitaire | Hostile aux #GrandRemplacement, multiculturalisme et égalitarisme. PEGIDA!',
          protected: false,
          verified: false,
          followers_count: 8868,
          friends_count: 567,
          listed_count: 128,
          favourites_count: 7788,
          statuses_count: 19658,
          created_at: 'Thu May 10 12:44:30 +0000 2012',
          utc_offset: 7200,
          time_zone: 'Athens',
          geo_enabled: true,
          lang: 'en',
          contributors_enabled: false,
          is_translator: false,
          profile_background_color: 'B06121',
          profile_background_image_url: 'http://pbs.twimg.com/profile_background_images/495015736893403136/nVOACylL.jpeg',
          profile_background_image_url_https: 'https://pbs.twimg.com/profile_background_images/495015736893403136/nVOACylL.jpeg',
          profile_background_tile: false,
          profile_link_color: 'D5842B',
          profile_sidebar_border_color: '000000',
          profile_sidebar_fill_color: 'DDEEF6',
          profile_text_color: '333333',
          profile_use_background_image: false,
          profile_image_url: 'http://pbs.twimg.com/profile_images/542640579713458176/32Zao_II_normal.jpeg',
          profile_image_url_https: 'https://pbs.twimg.com/profile_images/542640579713458176/32Zao_II_normal.jpeg',
          profile_banner_url: 'https://pbs.twimg.com/profile_banners/576201752/1415484544',
          default_profile: false,
          default_profile_image: false,
          following: null,
          follow_request_sent: null,
          notifications: null },
       geo: null,
       coordinates: null,
       place: null,
       contributors: null,
       retweet_count: 70,
       favorite_count: 22,
       entities:
        { hashtags: [],
          trends: [],
          urls: [],
          user_mentions: [],
          symbols: [],
          media: [Object] },
       extended_entities: { media: [Object] },
       favorited: false,
       retweeted: false,
       possibly_sensitive: false,
       filter_level: 'low',
       lang: 'fr' },
    retweet_count: 0,
    favorite_count: 0,
    entities:
     { hashtags: [],
       trends: [],
       urls: [],
       user_mentions: [ [Object] ],
       symbols: [],
       media: [ [Object] ] },
    extended_entities: { media: [ [Object] ] },
    favorited: false,
    retweeted: false,
    possibly_sensitive: false,
    filter_level: 'low',
    lang: 'fr',
    timestamp_ms: '1422289286660' }

Here’s a few things to note:

There is a lot of metadata about tweets included which I don’t need.
There are quite a few native retweets which include the retweeted text prefixed with RT in the new tweet. Should they be excluded or should the retweet count towards the word count?
There are many different languages, in order to have something meaningful for myself (I only speak english fluently, plus a couple of other european languages poorly at best) I decided to only process english tweets.

All of this meant that it made sense to build a simple back-end service/proxy that created a single streaming connection, processed this data and fed a far more condensed amount of data out to the browser(s). I chose to build something with node.js.

First we need to get the data out of the streaming API. I found a npm module called node-tweet-stream that worked with the filter endpoint, and with a little butchery was able to hook it up to the sample API instead.

var twitter = require('./twitter-stream'), //
    stream;

stream = new twitter({
  consumer_key: 'xxx',
  consumer_secret: 'xxx',
  token: 'xxx',
  token_secret: 'xxx'
});

stream.on('tweet', function(tweet) {
  console.log(tweet);
});

stream.connect();

I often use Heroku for hosting small things like this and Heroku encourages you to store as much of the application configuration as possible in the environment rather your application code respository. To manage this in my Ruby projects I use dotenv to allow me to keep such configuration in a .env file locally (excluding this from the source control). I was very pleased to find such functionality also exists for developing in node. A quick install of the dotenv npm module and a simple require and it was working here.

Logging things out to the console is great for debugging things but no real use. To get the data out to a browser I started to build a simple express app as I’d had some experience with this before but something reminded me of web sockets and socket.io so I thought I’d try playing with them. Again, all that was required was another install/require and a couple of extra lines and now we have tweets being proxied through to the browser(s). The code was now looking like this:

var app = require('express')(),
    dotenv = require('dotenv'),
    server = require('http').Server(app),
    io = require('socket.io')(server),
    twitter = require('./twitter-stream'),
    stream;

dotenv.load();

stream = new twitter({
  consumer_key: process.env.TWITTER_CONSUMER_KEY,
  consumer_secret: process.env.TWITTER_CONSUMER_SECRET,
  token: process.env.TWITTER_TOKEN,
  token_secret: process.env.TWITTER_TOKEN_SECRET
});

server.listen(process.env.PORT || 5000);

stream.on('tweet', function(tweet) {
  io.emit('tweet', tweet);
});

stream.connect();

The main reason for proxying the data was to reduce the amount sent out to the browsers, so now was time to take those massive responses and reduce them to some word lists. Again I found a couple of great npm modules to help with this; keyword-extractor for extracting the important words (or more accurately, excluding the non-important words), and franc for determining the language of the tweet (keyword-extractor only works with english, much like my brain).

While writing this I noticed that the twitter response actually contains a lang field, negating the need to use franc. I hadn’t noticed this at the time, oh well!

Plugging these in, along with some exclusions myself (links, retweets, replies) gives us the final code that was deployed to Heroku:

var app = require('express')(),
    dotenv = require('dotenv'),
    server = require('http').Server(app),
    io = require('socket.io')(server),
    xt = require('keyword-extractor'),
    franc = require('franc'),
    twitter = require('./twitter-stream'),
    stream;

dotenv.load();

stream = new twitter({
  consumer_key: process.env.TWITTER_CONSUMER_KEY,
  consumer_secret: process.env.TWITTER_CONSUMER_SECRET,
  token: process.env.TWITTER_TOKEN,
  token_secret: process.env.TWITTER_TOKEN_SECRET
});

io.set('origins', '*:*');

server.listen(process.env.PORT || 5000);

function exceptions(word){
  if (word.match(/https?:/)) return false; // links
  if (word.match(/^@/)) return false; // replies
  if (word.match(/&|\/|"/)) return false; // random punctuation

  return true;
}


stream.on('tweet', function(tweet) {

  // ignore retwets
  if (tweet.retweeted_status || tweet.text.match(/^RT/)) return;

  // only english for now
  if (franc(tweet.text) != 'eng') return;

  // parse that tweet, extract words
  words = xt.extract(tweet.text,{
    language:"english",
    remove_digits: true,
    return_changed_case:true
  }).filter(exceptions);

  if (words.length > 0) io.emit('tweet', words);
});

stream.connect();

So with less than 50 lines of code we have live tweets being parsed for words and those word lists being sent out to the browser. Now let’s get the browser to render them.

This is going to be almost entirely javascript powered so I’m going to concentrate on that, if you’re interested in the HTML and CSS then take a look at the source and ask me any questions you might have.

Firstly we’ll use socket.io to connect to the web socket and start grabbing the words as they come in.

I’m using the underscore.js library here to get access to some simple helper functions

var socket = io.connect('wss://twitter-word-stream.herokuapp.com/');

socket.on('tweet', function (data) {
  _.each(data, function(word) {
    console.log(word);
  });
});

And there we go, the words are being spat out to the browser’s console, but of course this is of no practical use. Lets count the occurences and displaying that visually. We’ll do this by throwing the words and their counts in to an object and then displaying the most popular ones periodically.

var socket = io.connect('wss://twitter-word-stream.herokuapp.com/'),
    word_counts = {},
    text_nodes = {},
    frame = 0;

function render() {
  var max = 0,
      displayed_words = [];

  // increment frame counter
  frame++;

  _.each(word_counts, function(count) {
    if (count > max) max = count;
  });

  // filter them to just the most popular ones
  displayed_words = _.sortBy(_.keys(word_counts), function(word) {
    return max - word_counts[word];
  }).slice(0,30);

  _.each(displayed_words, function(word) {
    var size = words[word] / max,
        text, node;

    // create the text node if need be
    if (!text_nodes[word]) {
      text = document.createTextNode(word);
      node = document.createElement('span');

      // position kind of in the middle somewhere
      var top = 80*Math.random();
      var left = 70*Math.random();

      // give it a random pastelly colour
      node.setAttribute('style', "top: " + top + "%; left: " + left + '%; color: hsla('+360*Math.random()+',50%,50%,0.75)');

      node.appendChild(text);
      document.body.appendChild(node);
      text_nodes[word] = {
        updated: frame,
        node: node
      };
    } else {
      text_nodes[word].updated = frame;
    }

    // clear expired words
    _.each(text_nodes, function(obj, word) {
      if (obj.updated < frame) {
        obj.node.remove();
        delete text_nodes[word];
      }
    });

    // size it relative to it's occurence
    text_nodes[word].node.style.transform = 'scale(' + (0.2 + size*0.8) + ')';
    text_nodes[word].node.style.webkitTransform = 'scale(' + (0.2 + size*0.8) + ')';

  });

}

setInterval(render, 500);

socket.on('tweet', function (data) {
  _.each(data, function(word) {
    word_counts[word] = (word_counts[word] || 0) + 1;
  });
});

There’s a few things to explain here:

A scale transform is being used instead of font-size to change the size of the words as this results in a GPU accelerated transform, which we can then enhance with transitions with very little impact on performance.
The created DOM nodes are being cached in the text_nodes object so we don’t have to recreate them each time or try to find them.
A frame number is used to note when the elements were last updated so that it’s easy to remove any words that are no longer popular.
The colour of the words are randomised using hsla() as this only requires a single number to be generated (the hue) instead of the multiple numbers required to use rgba().

This works great, but it will count occurences since you first loaded the page, I wanted it to only consider the most recent words (lets say only the last 5 minutes) so I need to store the word lists in such a way that I can easily and quickly remove the older ones. I could have stored the time of each occurence of each word but that would get complicated. I decided instead to store the word occurences in several different objects (I called them buckets), with the one that was incremented being rotated every few seconds. The render method would then only use the buckets covering the last 5 minutes worth of occurences.

var socket = io.connect('wss://twitter-word-stream.herokuapp.com/'),

    text_nodes = {},
    frame = 0,

    current_bucket = {},
    buckets = [current_bucket],

    bucket_count = 30, // how many buckets to remember
    bucket_width = 10; // how many seconds worth of words to keep in the buckets

function render() {
  var max = 0,
      words = {},
      displayed_words = [];

  // increment frame counter
  frame++;

  // get counts of words across all buckets
  _.each(buckets, function(bucket){
    _.each(bucket, function(count, word) {
      words[word] = (words[word] || 0) + count;
      if (count > max) max = count;
    });
  });

  // filter them to just the most popular ones

  displayed_words = _.sortBy(_.keys(words), function(word) {
    return max - words[word];
  }).slice(0,30);

  _.each(displayed_words, function(word) {
    var size = words[word] / max,
        text, node;

    // create the text node if need be
    if (!text_nodes[word]) {
      text = document.createTextNode(word);
      node = document.createElement('span');

      // position kind of in the middle somewhere
      var top = 80*Math.random();
      var left = 70*Math.random();

      // give it a random pastelly colour
      node.setAttribute('style', "top: " + top + "%; left: " + left + '%; color: hsla('+360*Math.random()+',50%,50%,0.75)');

      node.appendChild(text);
      document.body.appendChild(node);
      text_nodes[word] = {
        updated: frame,
        node: node
      };
    } else {
      text_nodes[word].updated = frame;
    }

    // clear expired words
    _.each(text_nodes, function(obj, word) {
      if (obj.updated < frame) {
        obj.node.remove();
        delete text_nodes[word];
      }
    });

    // size it relative to it's occurence
    text_nodes[word].node.style.transform = 'scale(' + (0.2 + size*0.8) + ')';
    text_nodes[word].node.style.webkitTransform = 'scale(' + (0.2 + size*0.8) + ')';

  });

}

function rotate_buckets() {

  current_bucket = {};
  buckets.push(current_bucket);

  while (buckets.length >= bucket_count) buckets.shift();

}

setInterval(rotate_buckets, bucket_width*1000);
setInterval(render, 500);

socket.on('tweet', function (data) {
  _.each(data, function(word) {
    current_bucket[word] = (current_bucket[word] || 0) + 1;
  });
});

And there we have the (more or less) finished code, and here it is running:

There’s still a few things I’d like to improve when I can:

The positioning of the words is random, which often results in excessive overlapping, the translucency helps with that but it sometimes is quite bad.
It would be nice to have it be a little more customisable, maybe the source being a hashtag, a user or your timeline instead of the sample stream.

It was fun to spend a couple of hours playing around with some new things, everyone needs to be able to do that occasionally.

What new technologies are you most excited about playing with?

Safari SVG CSS animation bug

Fri, 21 Nov 2014 00:00:00 +0000

While building the new Neutron Creations site (coming very soon!) we came across a particularly frustrating bug in Safari involving animation of SVG elements using CSS3 transforms and opacity simultaneously.

Safari will not animate both attributes at the same time, instead choosing to animate the opacity with the correct timings and when that animation is complete the transform jumps to its correct position, or the translation is just ignored completely.

We’ve produced a minimal testcase below. On the left we have SVG elements and on the right an equivalent set of DOM elements. Both are animated using the same CSS rules.

If you’re using Safari you’ll see the DOM element smoothly animate its position and opacity but the SVG element will only animate its opacity.

There is an open Chromium bug filed for this but it looks like the Chromium team fixed this elsewhere as it does not affect the current Chrome/Opera releases. This fix has not been submitted back to WebKit or at least not included in Safari as it does affect the current Safari 8 as well as the previous version 7 (and probably even older versions that support CSS3 animations and inline SVG).

There is a workaround on Stack Overflow that suggests you add a fill property to your @keyframes declaration. This was not suitable for us for two reasons; 1) we are animating our logo which has specific colours (yes we could change them only very slightly but see the next point) and 2) unlike opacity and transforms, fill cannot be GPU accelerated meaning the numerous animations we are performing would all end up being processed only by the CPU, instead of mostly being recomposited by the GPU, and that would slow things down significantly!

So how did we fix it? We wrapped every SVG element we wanted to do this to in another <g> element and animated the translation on this element and the opacity on the original (now child) element. Unneccessary markup is never ideal but in this case it doesn’t bloat things too much and lets us keep the higher performance GPU rendering.

Have you had to work around any obscure/frustrating/interesting (delete as appropriate) browser bugs? Let us know.

My favourite Ruby Gems

Mon, 01 Sep 2014 00:00:00 +0100

Many tasks that would otherwise take hours or days to build can be easily implemented by simply including the relevant Ruby Gem in your project. Here’s my pick of interesting, useful and most-importantly time-saving gems:

What’s a Ruby Gem? Ruby and particularly Rails have an excellent developer community providing numerous open-source (usually) libraries called Gems. Ruby, Gem, geddit?

Firstly, this very blog would not exist without the wonderful Jekyll, a blog-aware static site generator. I also often use Sinatra for smaller sites that don’t require a large framework like Rails.

When it comes to authentication my first port of call is always Devise. This, along with it’s various extension gems, covers any sort of authentication you could want: Username/Email & Password, Oauth (Facebook, Twitter etc), and Mozilla Persona to name but a few.

FlagShihTzu allows you to store numerous boolean fields in a single numeric database field within ActiveRecord and access them all as though they were individual fields. Kaminari has replaced will_paginate as my pagination gem of choice.

cache_digests enabled a russian-doll style nested caching mechanism within view that can produce huge speed improvements. Read more about this on the signal vs noise blog post how basecamp next got to be so damn fast, under point #2.

ActiveAdmin build beuatiful admin interfaces quickly and simply, which is a great place to start when building the back-end services required for any web application. lograge reduces the Rails log output of most request down to a single easily-parsable line, great for searching and troublshooting.

I’m a big fan of Heroku so obviously I use a few gems that are particularly useful here: rails_12factor enforces a couple of aspects of the Twelve Factor App that Rails/Heroku don’t by default, font_assets ensures your font assets are served with the correct headers to enable browsers (mainly Firefox) to use them correctly, and heroku-deflater gzips just the content that would benefit from such compression without wasting CPU cycles compressing content that does not (it will also serve any existing .gz file).

Many of my favourite gems are solely for use during development. quiet_assets will keep those noisy asset related request lines out of your development log files. meta_request and its RailsPanel Chrome extension gives you access to information about each request in Chrome’s developer tools, here it is in action:

better_errors makes the built-in rails error pages about a thousand percent better by giving you a fully interactive console in your browser at the point of the error, or any parent frame. I recorded a short screencast demonstrating this last year:

These are just the tip of the iceberg, RubyGems.org has nearly 90,000 gems listed. What are your favourite gems? Let us know.

Photo: jobafunky

Basic testing for mobile devices in a desktop browser

Fri, 22 Aug 2014 00:00:00 +0100

My browser of choice for development is Google’s Chrome Canary, a very frequently updated bleeding edge build of Chrome allowing you to preview upcoming features. One feature that I’ve been very fond of since its first introduction is the device emulation, and that’s what I’m going to talk about here.

It’s been around for quite a while, here you can see it in the current stable channel of Chrome (version 36 as of publishing), and I was delighted to find a completely redesigned and improved interface land in a recent build of Canary.

All the old features are still there, including overriding of the user agent, screen size, geolocation and orientation. But the most used ones are now presented in a clean new interface which allows you easily change the viewport size without resizing the whole browser and visualises the break points from you media queries. There’s also a brand new feature to throttle the network connection to simulate slower and/or more unreliable network connections.

Here’s a quick demonstration of the new interface:

Of course this doesn’t replace true device testing, either on the actual browsers/devices themselves in a device lab or using a service like BrowserStack or Sauce Labs, but it does allow you to get a good idea if the design “works” on those devices.

What’s your device testing strategy? Sound off in the comments.

Lab Report #6

Tue, 05 Aug 2014 00:00:00 +0100

With summer well and truly here (in the northern hemisphere anyway) lets look at what’s hot on the web at the moment. By ‘the moment’ I mean the last couple of months as we’ve had somewhat of an unintended hiatus.

Jason Santa Maria’s new book is now available from A Book Apart
Fontface Ninja can help you identify and play around with web fonts
Erik Romijn writes about SSL certificates — why your certificate authority rarely matters and expensive certificates are not safer
The Grace Hopper documentary reached its crowd funding target
Facebook moved 20 billion Instagram photos, did you notice?
Find a ‘great’ new domain using the latest TLD with DUMB domains

And finally… here’s something completely different: The USA is safely looking after its horde of nuclear weapons right?

Lab Report #5

Wed, 14 May 2014 00:00:00 +0100

Here’s our next barrel of links. On a related issue, what is the correct collective noun for links?

Some snazzy off-canvas navigation animations.
Use Rails 4.1’s variants to create multi-device web apps.
iOS 7.1’s minimal-ui viewport option helps regain screen real estate on iPhones. Which could be good or bad (thanks @m0bile_mike).
We didn’t get to Handheld 2013, but the videos are available.
Barry Meade (of Fireproof, makers of The Room & The Room 2) writes on how the seeming success of free-to-play games is affecting the gaming industry.

Lab Report #4

Wed, 19 Mar 2014 00:00:00 +0000

Back once again for the Renegade Master, here are some links. Please, click on them. We do web development, not song lyrics.

We built a nifty little campaign page for #WithSyria
Grid - a simple guide to responsive design.
A strategy guide for life
Crafting extends to building rollercoasters - who knew?
A tediously accurate scale model of the solar system
An interactive guide to just how big space is from the BBC
A lovely visualisation of European air traffic

Win a ticket to "The Web Is"

Wed, 12 Mar 2014 00:00:00 +0000

Today is the 25th anniversary of the web, and what better way to celebrate than giving away a free ticket to The Web Is, a conference celebrating the web itself with an absolutely spectacular line-up.

All you have to do to have a chance of winning is to teach us about something interesting. It could be something you’ve learnt recently (or not so recently), something you’ve created or a link to an oscure but fascinating wikipedia entry.

Leave a comment below with your fascinating thing before the end of August. We’ll pick our favourites and publish them in future Lab Reports. One will be picked at random to win the ticket in September.

Good luck! (and make sure your disqus profile has a way of contacting you!)

UPDATE We’ll accept submissions via twitter too, just tweet a link and description @NeutronUK and we’ll consider it.

Lab Report #3

Thu, 06 Feb 2014 00:00:00 +0000

Well I for one am glad that January is over. It’s a shame the horrid weather here in the UK has not gone with it though. If anyone is currently visiting the UK, it’s not normally this wet I promise. Here’s what’s interested us since last time.

Facebook launches Paper, a new way to consume your facebook news feed.
… which shares it’s name with FiftyThree’s Paper
… who have been on the other end of this before with Figure 53.
Mapbox have built a custom map design for foursquare.
Just how much can you store in the various browser local storage systems, this quota research will tell you.
Code Club needs your help
In-app purchases are destroying the gaming industry.
37signals are changing their name to Basecamp.

Lab Report #2

Thu, 23 Jan 2014 00:00:00 +0000

We hope you are all having an enjoyable January. If you live in the Northern Hemisphere, please remain indoors where you will be warm and safe. While you are hunkered down, why not keep your mind occupied with some interesting reading? We’ve got you covered:

SpecKing Mark up your PSDs with labels for measurements, typography and so on, directly from Photoshop.
So the sum of all natural numbers is minus a twelfth‽ Holy Zeta Regularisation, Batman.
Vertical alignment with just 3 lines of CSS
Have your face replaced with those of Clooney, Cage and more, right in your browser.
Fred Wilson’s dystopian prediction of VC pitches in a year or two if net neutrality falls.

As always, please send us anything else you find that you think we should be looking at for the next post.

Back to Front show #20

Tue, 14 Jan 2014 00:00:00 +0000

I recently had the pleasure of standing in for Kieran to guest co-host episode 20 of the Back to Front show with Keir. Head over there to listen to me rambling vaguely coherently about back-end things.

This was my first podcast but hopefully the nerves won’t show. If you have any questions (or corrections!) for me drop a comment below.

Lab Report #1

Tue, 07 Jan 2014 00:00:00 +0000

Happy New Year everyone!

Welcome to the new look Particle Lab (now with added responsiveness) and the inaugural ‘Lab Report’ where we’ll publish in brief what we’ve been producing and consuming recently. We’ll do our best to keep posting these regularly but won’t commit to any schedule just yet.

Some of our favourite people have written up their reviews of 2013 – Bruce Lawson, Geri Coady, MailChimp, Petra Gregorová, Remy Sharp, Simon Collison, Tito.
The Unicorn Institute announced a few more details of their soon to be launched school.
Rachel Andrew has a new book ~~coming~~ out ~~soon~~.
Edward Snowden’s Alternative Christmas Message broadcast on Channel 4.
Reservoir Dogs retold in (re)tweets
The Internet Archive’s console living room provides playable in-browser emulation of old console games no longer under copyright

That’s all this time around. If you see anything you think might interest us; leave us a comment, drop us an email or tweet @NeutronUK.

Reserve your username

Thu, 10 Mar 2011 00:00:00 +0000

Several web application have launched recently with an invite system where you not only give them an email address to notify you but also reserve your username. Some even go a little further and ask you to connect with Facebook, Twitter etc. The latest one to do so is connect.me, which I signed up for when I had heard about it.

This triggered a brief discussion with a friend about this:

Don't know what connect.me really is, but I'm reserving my username now! http://cxt.me/sKQC8g #connectme
— Marc Roberts (@marcroberts) March 10, 2011

@marcroberts Why?
— Justin Stach (@juter) March 10, 2011

@juter cos I'm a web app whore, and there are other 'marcroberts' out there. So on the off chance it takes off I want that to be me
— Marc Roberts (@marcroberts) March 10, 2011

@marcroberts And you're happy to give them access to your Facebook/Twitter account based on that? Seems a little one-sided to me.
— Justin Stach (@juter) March 10, 2011

@juter it can have access to twitter for now, if it does anything nefarious its gone though. Probably need to explain more via other means
— Marc Roberts (@marcroberts) March 10, 2011

This discussion couldn't really continue on Twitter so has spawned this post.

I am lucky in that my full name is not that common, however I have not been able to get my preferred username of 'marcroberts' on both Facebook and Flickr. This has lead me to try to register with new web application early in order to secure my username, and why I described myself as a "web app whore" (I've also been an active member of Web Apps Stack Exchange site from conception, through its private beta to now so please don't interpret 'whore' as a bad thing!).

But it's not just about reserving a username, here at Neutron Creations we know the pains of launching a web app and so I like to encourage others. If this means signing up my email address, connecting with twitter or facebook then I'm ok with that assuming they don't do anything unwanted. Now obviously I can't know in advance whether they will, and I guess that's a risk I'm willing to take to help and support fellow web application developers.

As long as you haven't given your Twitter/Facebook password to them (and you shouldn't as OAuth should be used for both) you can remove that app from your Facebook and Twitter accounts easily and prevent any further access to your account. Of course this won't remove any data the app has retained, the app itself should provide a way to remove this data or delete/remove your account. There are also term of use with the Twitter/Facebook APIs which specify data retention rules that the app should honour.

There is one golden rule that all web apps that have connections to Facebook/Twitter should never EVER break, they should never post from your account without your explicit permission. Now my first tweet in the conversation may seem like an 'auto-tweet', and in fact if you search twitter for #connectme it seems like connect.me could be breaking this rule, however this was a purely optional step in the signup process (although heavily encouraged) and the text was editable (I did alter my tweet from the standard text).

What are you thoughts on connect.me and this trend of reserving usernames, or encouraging connections with social networks? Drop a comment below or give me a mention on twitter.

More about connect.me

Good API Documentation

Mon, 07 Feb 2011 00:00:00 +0000

So you're a developer (you're not a developer? then maybe this post isn't for you, but feel free to read on), and as with most developers you spend a lot of your time looking up things on Google, Stack Overflow or various API documentation (API docs).

The quicker we can find the information we need in these API docs, the quicker we can build the next game-changing web app. So what makes good API docs? Read on and I'll tell you.

We've all been there; you're "in the zone", just written loads of totally awesome code but can't remember the parameters for a function you need to use. No problem, you bring up Google, enter the name of the function (and maybe the language you're using or something else to get a better result, you know the drill) and BAM the top result sounds perfect. However it's not, the information is out of date and of course you don't find this out until you actual test your code some time later and it takes you hours to track it down to this one bad source of information.

So let's start at the beginning.

Accurate & Up to date

This is the most important aspect of any API docs and thankfully most producers also realise this and do their best to ensure theirs stay up to date. The best of these are any that are built automatically from the API source; javadoc and rdoc, for Java and Ruby respectively, are great examples.

What about maintaining API docs for multiple versions? Well, either have the complete set of docs for each version available separately like the Java API docs (1.4.2 / 1.5.0 ) or show clearly which version(s) a particular page/function/method applies to, check out the documentation for the jQuery .live() function.

Clear, concise and well structured

Having accurate and up to date documentation is next to useless if developers can't find anything. Unless the entire API docs take up barely more than one screen (exactly what 'one screen' or 'the fold' is can be saved for another time) there has to be a good excuse to not have a search function. Search has its place but some people prefer to navigate to what they're looking for so the API docs need to be very well organised, with a clear hierarchical system if necessary. Both of these need to be consistent, if something can be found via the search system it really should be able to be navigated to also, and vice versa.

There are some great examples of this all around the web; the latest jQuery API site is excellent, as is APIdock (which gets bonus points for also having archives of every past version) and the vzaar API docs.

Examples

Being well structured is all well and good if you know what you're looking for but for first time use the best place to start is with a tutorial or examples. Many times I have found that just looking over an example can be far more useful than pouring over pages and pages of documentation. It's also good to see a mixture of examples, ideally at least one simple example and a more complicated one.

Open source projects are often full of good examples in the documentation. There's no better way to encourage developers to use (and potentially help develop in future) your latest open source project than to provide them with a plethora of examples. Here are a few I used recently: Authlogic, Whenever and Cache Money

Discussion

Examples are great, but it's not feasible to come up with every possible use for the API. Why not let the developers do this, provide them with a great discussion environment; forums, mailing lists, IRC etc. This will also reflect well for prospective developers who will see an active community as a positive thing.

The PHP API docs are a great example of this, see the page for the strpos() function for example. Facebook has been making great progress on its API docs lately, but before that the best place to get information was in their developer forums.

Sandbox & Playground

For very large APIs its helpful to have a developer area where you can try out API calls with different inputs and see their outputs. Flickr have really nailed this with their API Explorer where you can try out any of their API with full debug output.

Going one step further, it would be really helpful to have access to a sandboxed version of the API. This allows creation of data that is completely separate and isolated from the live data but still operates in the same manner. For some application this is essential, particularly when developing with financial transactions. Luckily PayPal have a full sandbox environment where you can create accounts, balances, card and bank account all that respond to the public API.

Anything else?

I'd love to hear what else you think good API documentation needs and your opinions on any great or terrible API docs out there, drop me a comment below or contact me on twitter.

Building a Custom HTML5 Audio Player with jQuery

Thu, 02 Sep 2010 00:00:00 +0100

We recently built an HTML5 audio player for Tim Van Damme's The Box, a new podcast where he interviews people who make cool stuff. Tim wanted an HTML5 audio player on the site, and we put together some jQuery to hook up the player interface he designed. In this article we'll run through the code to explain how it works, covering a few caveats along the way.

Here's the player interface, and the markup for it.

<p class="player">
  <span id="playtoggle" />
  <span id="gutter">
    <span id="loading" />
    <span id="handle" class="ui-slider-handle" />
  </span>
  <span id="timeleft" />
</p>

As you can see, we have a few span elements for each component of the interface:

playtoggle is the play/pause toggle button.
gutter is the timeline track
loading is the bar indicating the loading or buffering progress
handle is the circular element that serves as the playhead, indicating current position in the audio file, and also acting as a drag handle to move to a different point of the audio
timeleft is the play time remaining, in minutes and seconds

We won't cover the CSS for the player here, but if you want to see how it's styled you can inspect the styles on the live site.

We're going to use jQuery to detect support for HTML5 audio, and if it's supported, we'll insert the audio player markup and the audio tag itself. This means that browsers that don't have HTML5 audio support won't see an audio player interface they can't use. You could optionally fall back to a Flash based player, but as Tim is already providing a direct link to the MP3 file elsewhere on the page, the audio player is seen as progressive experience enrichment here.

if(!!document.createElement('audio').canPlayType) {

  var player = '<p class="player"> ... </p>\
    <audio>\
      <source src="/path/to/episode1.ogg" type="audio/ogg"></source>\
      <source src="/path/to/episode1.mp3" type="audio/mpeg"></source>\
      <source src="/path/to/episode1.wav" type="audio/x-wav"></source>\
    </audio>';

  $(player).insertAfter("#listen .photo");

}

The detection code on line 1 is taken from a guide to feature detection on the superb Dive Into HTML5 site. If support is detected, we go right ahead and insert the player code (abridged at the "..." in this code), and the audio tag.

Your audio tag should contain three source formats. The first is OGG Audio which is an open standard, supported by Firefox and Chrome. The second is our old friend MPEG-1 layer 3 (MP3) which is a commercial proprietary format supported by Safari. The third is a plain old WAVE file, which is what Opera wants to hear. Browsers will attempt to use sources in markup order, so for example Safari would fail to read the OGG format source, and use the MP3 source instead, while Opera will fail on the first two sources and use the third WAVE source.

Now we need to start adding functionality to our player:

audio = $('.player audio').get(0);
loadingIndicator = $('.player #loading');
positionIndicator = $('.player #handle');
timeleft = $('.player #timeleft');

if ((audio.buffered != undefined) && (audio.buffered.length != 0)) {
  $(audio).bind('progress', function() {
    var loaded = parseInt(((audio.buffered.end(0) / audio.duration) * 100), 10);
    loadingIndicator.css({width: loaded + '%'});
  });
}
else {
  loadingIndicator.remove();
}

We first save variable references to several key elements of our player so that we can refer to them quickly without querying the DOM each time.

We then look for support of the buffered property on our audio tag. This should contain information about how much of the audio file has been buffered. At the time of writing, Firefox doesn't provide the buffered property at all, while Opera has the property but doesn't put anything in it. For browsers that do (and for future versions of Firefox and Opera that have full support for the buffered property), we set up an event handler for the 'progress' event (fired as loading progress is made). As the audio file loads, we calculate the amount of the file that has been loaded as a percentage, and then use this value for the width of our loading indicator. If buffered progress support isn't available, we can simply remove the redundant loading indicator element from the DOM.

Now we'll write an event handler for the 'timeupdate' event, which is fired whenever the current play time is updated, either as we're playing the audio normally, or when we seek to a new position within the audio file.

$(audio).bind('timeupdate', function() {

  var rem = parseInt(audio.duration - audio.currentTime, 10),
  pos = (audio.currentTime / audio.duration) * 100,
  mins = Math.floor(rem/60,10),
  secs = rem - mins*60;

  timeleft.text('-' + mins + ':' + (secs > 9 ? secs : '0' + secs));
  if (!manualSeek) { positionIndicator.css({left: pos + '%'}); }
  if (!loaded) {
    loaded = true;

    $('.player #gutter').slider({
      value: 0,
      step: 0.01,
      orientation: "horizontal",
      range: "min",
      max: audio.duration,
      animate: true,
      slide: function() {
        manualSeek = true;
      },
      stop:function(e,ui) {
        manualSeek = false;
        audio.currentTime = ui.value;
      }
    });
  }

});

First we calculate the play time remaining (in seconds), the position of the playhead as a percentage of total duration, and the minutes and seconds remaining.

We then update the displayed time remaining in the player interface, taking care to insert a leading 0 if necessary to the seconds figure.

If we haven't triggered this event by manually seeking (determined a bit further on in the code), we move the playhead position indicator along the gutter track. This basically just slides the indicator along as we listen, unless we're using the slider mechanism to move the playing position, in which case we don't want to interfere with the position of the indicator because the slider code will handle that for us.

Until the audio has started to load, the duration of the audio file is not available. We therefore check to see if the audio file has started to load, and proceed to set up the slider for the draggable playhead control. We're using jQuery UI for our slider here, and the basic configuration options should be self explanatory. We add two event handlers; one on slide where we set a flag that we are manually seeking to a new position, and stop, to unset the flag and tell the browser's audio player that it needs to move to our new position in the audio file.

The only thing left is the play/pause toggle button.

$(audio).bind('play',function() {
  $("#playtoggle").addClass('playing');
}).bind('pause ended', function() {
  $("#playtoggle").removeClass('playing');
});

$("#playtoggle").click(function() {
  if (audio.paused) { audio.play(); }
  else { audio.pause(); }
});

First we set up a couple of event handlers. When we start playing the audio, we add a class of 'playing' to the button so that it switches to the pause state. We then remove that class if we pause the audio playback, or reach the end of the file which fires the 'ended' event.

Our click handler for the play/pause button is very simple; play the audio if we're currently paused, otherwise pause it.

So that's it for a super simple audio player using lovely web standards instead of Flash. By going down this route, Tim's visitors can use his audio player on mobile devices like the iPhone and iPad, and the loading performance of the player is hopefully better than would have been achievable with a plugin based solution. Go and listen to The Box, and if you have any questions or ideas for improving the player, please leave a comment.

UPDATE: Fixed the code for Opera, providing the WAVE source and a more detailed check for support of the buffered property.

UPDATE 2: Fixed a typo binding the play event handler, which should be $(audio).bind not audio.bind. Thanks to @patdryburgh for helping find that.

Useful Resources:

Photo credit: Miikka Skaffari

Single-Line vs Multi-Line CSS, plus TextMate tips

Tue, 23 Mar 2010 00:00:00 +0000

Now that the stormy debate around whether or not designers should be able to code what they produce has died down, the new debate du jour is here! Or rather, it's back again. This one has been around before, and it's about whether laying out your CSS code on single lines (one selector per line), or multiple lines is better.

In the red corner; single-line CSS code formatting:

header h1 { float: left; width: 155px; height: 50px; }
header h1 a { display: block; opacity: 0.8; border-bottom-width: 0; }

In the blue corner; multiple-line CSS code formatting:

header h1 {
	float: left;
	width: 155px;
	height: 50px;
}

header h1 a {
	display: block;
	opacity: 0.8;
	border-bottom-width: 0;
}

The principal argument for coding on single lines is that you end up with a file that has all your selectors stacked up on top of each other, allowing for easier eye scanning when hunting for a specific selector.

The most common argument against is that when you're working in a source controlled environment (which of course we all are because we'd be idiots not to, right?) and a change is made to the CSS, there's no way with the single-line approach to quickly see which property was changed and how. Here's a modification to some multi-line CSS code:

And here's how the same modification would look if this was a single line statement:

It's obvious from the multiple-line example that the background-color property has been changed from transparent to "#fff", but in the single-line example the actual change isn't highlighted because differentiators only work on a per-line basis, so it just reports that the whole line has changed. It will be up to you to work out which property was modified. Now talk to me about "easier to scan".

For the most part, and complexities during source control diffs aside, this is simply a case of personal preference. Opinion is divided right down the middle as to which is better, with no fence sitters because really you just have to pick your preferred option (as a team if there's more than one CSS coder involved on a project) and stick to it for consistency. Saying that it's best practise to use one technique or the other isn't really helpful — just figure out what works for your situation and environment and stick with that.

My preference is for multiple-line CSS, because that's how I've always done it, I use source control on every single project (even solo projects), and my process for designing and front-end coding involves inspecting elements in Firebug, testing modifications to markup and style, then using Find ⌘F to jump immediately to the relevant selector I want to change. This involves no visual scanning for selectors in my code editor, and once you have the required keystrokes wired into your brain I guarantee that this is far quicker than scro

But enough of this petty bickering debate! Because I'm lucky enough to be a TextMate user, this is all completely academic. Here are two quick TextMate tips for multiple-line CSS coding to help with finding selectors, if you want the best of both worlds.

When editing a CSS file, you'll have a "Symbol Navigator" in right side of the status bar, which displays the currently focused selector based on cursor position. If you click the Symbol Navigator, a menu appears which displays all your selectors in a very easily scannable list. Simply click on one to jump straight to it.

You can also use the shortcut ⌘⇧T to pull up a "Go To Symbol" dialog, where you can type part of your selector and it will suggest matches. Use the arrow keys to move up and down the matches, and the Enter key to jump to that selector in your file. Hat tip to Doug Neiner for that one.

There's also a quick way to jump between what is effectively a selector only view and your standard multiple-line format, by making use of Code Folding. While editing normally and laying out code using the multiple-line technique, your code will look something like this:

But if you use Code Folding to collapse your code on all levels (and with vanilla CSS there will only be one level anyway) using the shortcut ⌘⌥0 then you'll reduce your display of your code to this:

This view will help you scan selectors quickly without the attributes getting in the way. When you've found the selector you want, pushing ⌘⌥0 again will return you to the unfolded view.

Notice by the line numbers that I'm not afraid of large CSS files, and why should anyone care about that? It's 2010 and we have smarter text editors these days than the days where a 2000 line file was a big deal because of all the scrolling we had to do. CSS should be minified as part of your deployment process to your production environment anyway, so this is a non-issue for your end users who are served the file.

So, go forth and code in whatever style you prefer. Hopefully this shows you that if you have the appropriate tools for the job, smaller details like code formatting start to matter a lot less which allows us to spend our energies on coding more and better things, instead of arguing about how we code.

Update: Sean Gaffney commented to tell us about another TextMate shortcut: ^⌥Q which properly converts multiple-line CSS to single-line. Thanks, Sean!

Update 2: Another killer TextMate tip, this one from Doug Neiner - Hitting ⌘⇧T while editing a CSS file will bring up a "Go To Symbol" dialog with a list of all your selectors, which is much cleaner than my method for searching using Find. This "easier to scan selectors" argument for single-line CSS is really shot to pieces if you're a TM user now. Thanks Doug!

A simple technique for web app maintenance modes using Apache

Thu, 18 Mar 2010 00:00:00 +0000

Sometimes we need to take our web apps offline temporarily (usually for scheduled major updates or emergency repair work), whereby regular users are shown a simple static page notifying them that the site is unavailable but still giving developers/admins full access to the site to test changes. Of course you could implement this in your application, but wouldn't it be nice if you could get your web server to do it all for you?

Its not as hard as it sounds thanks to some great Apache modules, most notably mod_rewrite.

Here are the basic rules we regularly use at Neutron Creations (I'll go in to more detail in a moment), these go inside our .htaccess file, but with some modification could go in the main httpd.conf file if you'd prefer.

# Part I
RewriteCond APP_ROOT/maintenance.txt -f
RewriteRule SUPER_SECRET_URL / [R,L,CO=AllowMaintenance:1:YOUR-DOMAIN.COM]

# Part II
RewriteCond APP_ROOT/maintenance.txt -f
RewriteCond %{REQUEST_URI} !^/maintenance/
RewriteCond %{HTTP_COOKIE} !AllowMaintenance
RewriteRule .* /maintenance/ [L]

# Part III
RewriteCond APP_ROOT/maintenance.txt -f
RewriteRule ^(.*)$ - [env=MAINTENANCE:1]

Header set cache-control "max-age=0,must-revalidate,post-check=0,pre-check=0" env=MAINTENANCE
Header set Expires -1 env=MAINTENANCE

This technique uses the presence of a file (APP_ROOT/maintenance.txt) to enforce a set of rules. To enter maintenance mode simply create this file with touch APP_ROOT/maintenance.txt, to leave maintenance mode remove this file with rm APP_ROOT/maintenance.txt. You'll need to replace APP_ROOT with whatever makes sense in your application environment.

I tried to set this relative to Apache's DOCUMENT_ROOT but could not get this to work, if anyone knows how to do this please do let me know in the comments. Thanks to ianjs for pointing out the correct syntax for this, you can use %{DOCUMENT_ROOT} in place of an absolute value for APP_ROOT to reference your document root

Part I

This provides developers (or anyone who knows the secret URL) with continued access to the app by setting a cookie that will be remembered for the session, and used in later rules. You will need to change YOUR-DOMAIN.COM to be the correct domain for your app, you could also add an expire time on this cookie if you wish (more info in the mod_rewrite flags docs).

You will also need to ensure the SUPER_SECRET_URL is set to something suitable, complex enough to be unguessable by users but easily read out if case you need to give it out over the phone.

Part II

This is the key set of rules for what we're trying to achieve, and reads like this: IF we are in maintenance mode AND the request is not for the maintenance page AND the AllowMaintenance cookie is not set THEN redirect the request to the maintenance page.

The maintenance page must be completely self contained within this maintenance directory, as any sub-requests outside of this path (for any CSS/JS assets used on the maintenance page for example) will be redirected to the maintenance page themselves.

With this directive, Apache will serve an internal redirect so the user will still see their original URL in their address bar, allowing them to refresh to try again later. If you want an external redirect so that the user sees the '/maintenance' URL instead, then just add the R flag to this RewriteRule.

Part III

This part could be quite important depending on how aggressively you cache HTTP requests in your application. Using an environment variable that's only set if you're in maintenance mode, we override any cache related headers sent from your app, making all requests expire immediately and telling proxies/caches to avoid caching anything going forward. You'll need mod_headers enabled for this part to work.

So there you have it, a very simple method to gracefully shift your web app into maintenance mode, completely independent of your application code itself. I'm sure this could be expanded upon and I'd love to see any ideas you might have for building on this technique, so drop a comment here and let me know.

Photo credit: tweek

Update: Its important to make sure that if you are NOT using an external redirect in Part II that you need to be careful linking to any assets as any relative paths will be relative to the url that user was trying to reach. Try to use absolute links where possible. I wasn't clear about this before, thanks to Dasha for pointing this out.

Update 2: I've been asked if it would be possible to serve a 503 (or similar) response to search engines to prevent them indexing the maintenance page. I have an alternate solution for this, if you serve an external redirect with a status code '307 - Temporary Redirect', by adding the R=307 flag in Part II, this will tell search engines to ignore the response and continue to index the original page. In addition you can add a 'Retry-After' header to inform them when to try again, just add this after the other Header directives Header set Retry-After 1800 env=MAINTENANCE which says to try again after 30 minutes.

Easier print CSS coding using Firebug and Web Developer Toolbar

Mon, 15 Mar 2010 00:00:00 +0000

Print stylesheets are an aspect of front-end engineering that rarely get any love, and they'll often be ignored until the last possible moment before launching a site. Sometimes they might even be forgotten altogether, but you should always make time to write at least a basic stylesheet that formats things neatly, and reveals any elements that only make sense on paper while hiding things like interactive controls that will be useless when printed.

But how to make the process of creating a print stylesheet as painless as possible? The obvious (and painful) way of working goes something like this:

Open/refresh your site in a browser
Pull up a Print Preview
Make changes to your print CSS in your editor
Close the Print Preview
Go to 1

It's only going to take a few laborious cycles through that loop before you're ready to give up and work on something more rewarding.

But wait, there's a better way! By using Firefox with its two close friends; Firebug and Web Developer Toolbar (you're all using these daily for front-end engineering anyway, right?), we can make working on a print stylesheet as simple as one for the screen. Observe.

First, open your website in Firefox, then we'll ask Web Developer Toolbar to render the page using your print rather than screen CSS by choosing CSS → Display CSS By Media Type → Print from the Web Developer Toolbar:

Now what you'll see is your website rendered using your print media type CSS. We could just launch Firebug and start attacking the DOM and our styles at this point, but if you need to reload the page to pull in changes made to your code via your text editor, you'll have to choose to view the print CSS again.

Thankfully there's an option for that; Options → Persist Features, which will retain our CSS media type preference across page reloads:

Now we can start using Firebug to inspect our page and play with styling and markup as normal, and if you need to reload the page, you'll still be viewing the print CSS afterwards.

So now that you have no excuses for not producing sensible print CSS, go and give this a try and patch up your site. We still need to do ours — I'll get around to that some day.

The site you see in the background here is Eric Meyer's 2002 article on print stylesheets, which despite being 8 years old is still relevant today, and even made use of a CSS3 attribute selector for some super progressive enhancement before we even knew what that term was.

Redesigning interfaces: learn from the old before you throw it away

Tue, 09 Mar 2010 00:00:00 +0000

Central heating, as a domestic utility, has not changed much over the decades. We burn fuel, our homes stay warm when its cold outside, and we generally regulate the service in one of two ways — either it's thermostatically controlled to maintain a constant room temperature, or scheduled according to rudimentary on/off time programming.

The interfaces to the latter control system however, have changed considerably over the years. Here's an old control interface, still in use in many homes (but I would guess no longer manufactured for sale in any new systems today), based on a mechanical rotary time switch.

To set the on/off periods, you move the small plastic tappet switches to either an inner (on) or an outer (off) position. The example here is set to run constantly from 10am to 10pm (hour 10 to hour 22). You can set the current hour simply by turning the inner dial, and you can switch between timed, and constantly on or off with the large switch in the lower right.

Compare this to a more modern interface, with an LCD display, and multi-function buttons.

These are just some of the buttons on this system. If you slide down the face plate, a further five are revealed which are used for actually modifying the schedule.

If you compare the manuals for these two systems, the manual for the rotary timeswitch is about as extensive as my description of it above. Just set the time by turning the dial, then put the switches into position corresponding to your desired on and off times, set the thing to timer mode, then go and get on with your life.

The manual for setting up a schedule on the digital system involves ten pages of A4, including (I kid you not) no less than four flow charts. This system supports per-day scheduling, so to set it you start with the current day, then you're forced to set up four discreet active periods of heating operation (if you want fewer than four, you have to set some of them back to back). Times are set (and the clock) using Plus, Minus and Enter keys, and when you're done setting one day you can copy and paste the times (no really) to other days, or set them individually. It's not hard once you've got the hang of it, but even the most basic schedule set up will involve about 20 key presses.

Now let's say that winter is passing into spring, but it's still pretty cold at the start and the end of the day so you want the heating on at those times but off while the sun is radiating you nicely during the day. If you have the timeswitch control, you can reach straight for the day time tappets and nudge them all to the off position with a swipe or two of one finger and then go back to training your cat to make a Reuben sandwich, or whatever it is you enjoy doing when you're not programming central heating systems. If you have the newer digital system, first switch the system to Edit mode, push enter repeatedly until the blinking digits represent the day, and then the time you want to change, then use the Plus and Minus keys to ... well you get the idea.

The timeswitch interface is perfectly simple, both aesthetically and in terms of functionality. When you come back to it months after setting a programme and needing to make an alteration, you can see at a glance the current schedule on the familiar clock face layout, and even complete reconfiguration will take a few touches and mere moments. It does have its downsides of course. You don't get per-day scheduling, and with moving clock parts inside it's neither as silent nor as reliable as its digital replacement, but its certainly a lot easier to actually use.

As far interface design goes, we've taken a big leap backwards with the digital heating control. When you're redesigning an interface for anything, please make sure you look at (and use!) what you're replacing to make sure you're not throwing away something that did the job far better than your proposed solution. You may have newer technology to hand which makes the old design look antiquated, but there will always be lessons to learn from any design you're inheriting and hopefully improving upon. Ignore them, and our lives will be full of endless Edit/Run mode, Plus, Minus and Enter button experiences, supplemented by multi-page manuals stuffed with flow charts.

Why TextMate?

Wed, 17 Feb 2010 00:00:00 +0000

I'm often asked "what text editor do you use?", and I respond TextMate. I'm usually then asked "so, why do you use TextMate?". There's many reasons I use it, and I've heard of developers switching from a PC to a Mac in order to use it. Can a text editor be so good that it can make a developer switch operating systems? Hopefully this article will introduce you to how powerful TextMate is and you can decide for yourself.

File Management

Lets start at the beginning with file management. As with most IDEs you have a list of files and/or folders, and TextMate is no different. This is displayed in the 'Project Drawer' (see right). This in itself is nothing special, anyone who has used or seen OSX knows that many apps have these 'drawers'. What is special about TextMate, at least for me, is that there is a much faster way of finding the files you want called 'Go to file', found under the 'Navigation' menu or on the keyboard shortcut ⌘T. This brings up a small dialog window listing all the files in the current project which is filtered as you type. As you can see from the screenshot below, this not only searches for exact matches but also any filename that contains the letters you enter in that order.

You can scroll through this list with the arrow keys (the first one is selected by default) and select a file to open with the enter key. TextMate will remember which files you have chosen for each entry and weight its list accordingly which allows you to create shortcuts to certain files, for example in nearly all of my Ruby on Rails projects entering 'rou' will select the 'config/routes.rb' file which has gotten me in to the habit of bringing up the routes file with the following keystrokes ⌘T, R, O, U, ↵

Editing

Once you have open the file you want to edit there are numerous ways TextMate can help you get things done quickly. One of my favourite is the ability to edit multiple lines at the same time. For example if you had the following code in your project somewhere and wanted to shorten the name of the variables.

var Button_Next = $('button.next'),
    Button_Prev = $('button.prev'),
    Button_Cancel = $('button.cancel'),
    Button_Finish = $('button.finish');

You could use find/replace to replace 'Button_' with 'btn', or you could use TextMate's column selection, hold ⌥ while selecting with the mouse, to select all those columns and overwrite them with 'btn' by typing normally.

How many times have you found yourself omitting line-termination semicolons in a language that requires them (JavaScript, Java, PHP, C etc) after a long period of coding in a language that doesn't (Ruby, Python etc)? TextMate has a great feature which you'll find very useful called 'Edit Each Line in Selection' found under the 'Text' menu or the shortcut ⌥⌘A. Select the text, press the shortcut and then add a semicolon, TextMate will add that character the end of all lines in your selection. Voila!

Here's a few more quick time savers

Generate a 'lorem ipsum' paragraph by typing 'lorem' and pressing tab
From the command line open a folder with the 'mate' command
Inline/incremental search, activate with ⌃S and type, press the shortcut again to go to the next match
Close the currently open HTML tag with ⌥⌘.
Toggle commenting on the current line/selection with ⌘/
Send a selection of text to pastie or any other application that will accept text with ⌃⌥⇧V

Bundles

Most of the functionality of TextMate is provided by Bundles. These are extensions that add coding snippets, language support or interfaces to command line scripts. TextMate itself comes with over thirty of these bundles supporting a wide variety of programming languages and other features. Most are specific to a particular programming language, but some of them are more general.

The TODO bundle has just one command 'Show TODO List' ⌃⇧T which scans through all your project files and displays all TODO, FIXME and CHANGED notes in your code with links to the lines within those files. These tags are customisable in the bundle configuration through regular expressions

The Math bundle allows you to do quick and simple calculations or base conversions (Dec, Hex & Oct). There's just one shortcut to remember - ⌃⇧C. Select some text and press the shortcut keys, this will then present you with the following menu allowing you to select what you want to do with the selected maths expression. Option 2 (Evaluate and Replace Expression) is extremely useful for replacing inline expressions (ie, 60*60*24) with their evaluated value (86400).

The Blogging bundle allows you to edit blog posts in most blog engines that support the MetaWeblog API, including Wordpress, Movable Type, Drupal and Typo among others. There's a great screencast over on the TextMate blog that covers using this bundle.

Add-ons

As well as these built-in bundles there are a number of third party bundles which can increase your productivity. Just drop any of these .tmbundle packages in to your ~/Library/Application Support/TextMate/Bundles directory.

The Ack in project bundle provides just one feature, with the schortcut ⇧⌘A, a GUI to the ack command. Ack is much like grep but is aware of programming languages and the files associated with them, so it won't waste time searching through log files, core dumps or any other files not related to the project.

The Javascript Tools bundle provides a quick way to run Douglas Crockford's excellent JSLint on your javascript code. By default this runs validation when you save a javascript file with the keyboard shortcut ⌘S and provides error/warning counts in a small tooltip. It also has a more thorough validation summary available on this shortcut, ⌃⇧V.

This Git bundle adds a great simple menu on the ⌃⇧G shortcut for working with git repos. You're all using git right ?

Finally the Zen Coding bundles (Zen CSS & Zen HTML) add loads and loads of snippets to make working with CSS/HTML much quicker. Paul Boag gives a nice review of this over on the boagworld website.

If you have any must-have bundles, please drop a link to them in the comments and I'll check them out.

Is that it?

These are only the features that I use regularly without looking up and just touch the surface of what is possible with TextMate. If you want to find out more about what you can do with TextMate I'd highly recommend "TextMate: Power Editing for the Mac" by James Edward Gray II (buy it from amazon). It has almost 200 pages of great information, including expert features such as creating your own bundles.

As always if you have any thoughts or criticism, either positive or negative, please leave a message in the comments.

Photo credit: batega

The value of remarkable customer experiences

Fri, 12 Feb 2010 00:00:00 +0000

A couple of weeks ago I was out for a friend's birthday, in a bar called The Lexington. It was a Saturday night, so we arrived early so we could order some food before the place got too crowded. After placing our order, 45 minutes passed, and I let it get to a whole hour before I queried the order at the bar. They apologised and said that the kitchen was really busy, and indeed the bar was getting busier by the minute with a few tables placing orders for lots of food.

After about an hour and a half in total since placing the order, I went to the bar, by now intending to cancel the order and ask for a refund. I was promptly told that "due to the unusual circumstances, the kitchen has been closed." That's right, they closed the kitchen, and didn't notify people who had unserved orders. I was refunded, but with barely an apology or any offer of compensation like a free or discounted meal on another occasion. So that's the last time I'll be going back to The Lexington, and I'll be sure to mention this story to anyone who asks about that bar, or for a recommendation for a bar in that area, and tell them to avoid it like the plague.

After leaving The Lexington, by now beyond mere hunger, we stepped into a burrito shop a short walk away called Chilango. The guy who served us looked exhausted (it was only ten minutes before they closed at midnight after what must be one of their peak times for the week), but he greeted us like long lost friends, asked if we'd been before and offered to explain the ordering process when we said it was our first time there. As we ordered he made welcome suggestions about which options would work best with what we'd picked so far, introducing ingredients like they were members of his band, and their origins.

While we were eating, another member of staff asked if we'd mind filling in a feedback card as they were new and wanted to know how they were being received. I gladly obliged, unable to think of anything to criticise, but I suggested that they offer a some varieties of salsa instead of just one (hey it was late, give me a break). When we left, we were thanked by all the staff at once, and the guy who was wiping tables near the door told us to enjoy the rest of our weekend.

Fast forward a week, and I get an email back from someone at Chilango, personally thanking me for my feedback and saying how much it means to them to hear that I enjoyed the atmosphere and the friendliness of their staff. They even said they'd look into finding some other kinds of salsa. As a result of that email and my remarkable experience, I'm writing this blog post which you're now reading, and I'd urge you to go and check them out if you're near one of their locations in the UK (London's Upper Street and Fleet Street, and Sheffield's Meadowhall, at the time of writing).

Do you see the difference there?

If you're in charge of selling a product or service, whether you own it or if you're just the person on the front lines talking to customers, please take a good look at the experience your customers are getting. You might be able to get by offering a mediocre experience if you're hosting live music and serving niche products like The Lexington (they specialise in bourbons and American whiskey), but wouldn't you rather provide an experience that's going to inspire your customers to write blog posts like this one, and tell all their friends about you? Which approach is going to grow you a customer base of fans instead of regular patrons?

If you want to know more about remarkable experiences and why they're so important, you need to read the works of Seth Godin - especially Purple Cow, but really everything he's written, including his blog.

Photo credit: Striatic

Send contact form submissions to Highrise

Tue, 09 Feb 2010 00:00:00 +0000

While adding a few features the back-end of our contact form logic I had an interesting thought; Wouldn't it be nice if these messages not only arrived in our email mailbox, but were also automatically entered into our Highrise account. Highrise is a web-based CRM from 37signals which we use for tracking contacts and project leads.

When the contact form is submitted, we do a little processing (with PHP) and then fire off an email to our contact address as normal. Highrise supports forwarding emails to a 'dropbox' address (more on dropboxes) which will then be imported as a message, creating the original sender as a new contact, or attaching it to an existing one if the email address matches with one of your contacts. All we have to do is add an additional step to reformat the email as a forwarded message, and send it to our dropbox address. Here's the code we use:

$body = "New Contact Form Submission\n\n";

// build message body
/***** <snip> ****/

// sanitise email/name
$name_safe  = mail_header_safe($_POST[&#039;contactName&#039;]);
$email_safe = mail_header_safe($_POST[&#039;email&#039;]);

// send to our contact list
mail($contact_addr, &#039;Contact Form&#039;, $body, "From: $contact_addr\r\nReply-to: \"$name_safe\" < $email_safe>\r\n");

// build highrise body
$new_body  = "\n---------- Forwarded message ----------\n" .
$new_body .= "From: \"$name_safe\" < $email_safe>\n\n";
$new_body .= $body;

// send to highrise
mail($dropbox_addr, &#039;Fwd: Contact Form&#039;, $new_body, "From: $contact_addr");

You will also need to make sure that the from address in these emails ($contact_addr in our code) is registered to a user on your Highrise account, otherwise the message will be discarded by the Highrise importing process. See the Highrise help page for more information on this.

As usual, if you've found this useful or have any improvement please leave a comment!

How to track traffic exiting your site

Wed, 20 Jan 2010 00:00:00 +0000

Thanks to the great work Google have been putting in, we can now gather an enormous amount of information about our site visitors for free using Google Analytics. One of the metrics that I have always been curious about is the 'Exit Page'. Sadly, all visitors do eventually leave our sites and this metric shows what was the last page they viewed before leaving. Even more sadly, that's as far as we can get with a basic Google Analytics deployment. Wouldn't it be great if we could see where our visitors went after leaving our site?

Well, with a little bit of javascript magic we can.

What you'll need on your site

Google Analytics (I'm using the new Asynchronous Tracking code but with a little modification it'll work with the standard tracking code)
jQuery (again with some modification this can be done without, but you're all using jQuery anyway right?)

How do we do it

We're going to be using the Event Tracking functionality of Google Analytics to track whenever someone follows an external link on our site. Firstly we need a way of determining which links are actually external. We could add additional classes to any links you want to track or we can use a little jQuery tickery to determine that automatically.

Haven't used jQuery before? Don't know what on earth the $() function is? Check out the getting started guide, you'll thank me later

$.expr[':'].external = function(obj){
  return !obj.href.match(/^mailto:/)
    && (obj.hostname != document.location.hostname);
};

jQuery Howto

This adds the :external filter allowing us to use $('a:external') to select external links.

All we need to do now is to use this selector to track events when these links are clicked. The documentation shows that the _trackEvent() method takes four parameters; Category, Action, Label and Value. Of these four, only the Category and Action are mandatory. We'll be using the optional Label parameter to pass the URL of the external site back to Google Analytics.

$('a:external').click(function(){
  _gaq.push(['_trackEvent','Outbound','Link',$(this).attr('href')]);
});

Taking it further

We're using a Category of 'Outbound', as these events are all related to off-site events, and an Action of 'Link' as we're looking at links. We can expand on this using a little more jQuery to change this Action based on some conditions, for example if the link is within another element, if it has a particular class or if the url contains a string.

$('a:external').click(function(){
  var link = $(this), type = 'Link';

  if ( link.closest('ul.comments').length == 1 ) type = 'Comment';
  if ( link.hasClass('amazon') ) type = 'Affiliate';
  if ( (/pastrami/i).test(link.attr('href') ) type = 'Tasty';

  _gaq.push(['_trackEvent','Outbound',type,link.attr('href')]);
});

You could also this technique for other similar events, that might not otherwise be tracked by Google Analytics, such as for downloads

// track downloads of Acrobat and ZIP files
$('a:not(:external)').filter(function() {
  return (/\.(pdf|zip)$/i).test($(this).attr('href'));
}).click(function() {
  var href = $(this).attr('href'),
      pos = href.lastIndexOf('.'),
      ext = href.substr(pos+1).toUpperCase();
  _gaq.push(['_trackEvent','Download',ext,href]);
});

All done

Once that is all in place and you've had some events tracked by Google Analytics you'll be able to see the events showing up in the reports. Here's a shot of the events shortly after the launch of the Neutron Creations site on the 19th January 2010.

So that's it. Now you can track where your visitors are going when they eventually leave your site. If you have any improvements or other uses for this sound off about them in the comments.

Photo: lansakit

New year, new site

Thu, 14 Jan 2010 00:00:00 +0000

As the dust settles on 2009, everyone is busy putting their plans for 2010 into action. For us, we're kicking our plans off with the unveiling of something we've been planning and building for a little while now - a new website for Neutron Creations.

Our first site, designed by Tim Van Damme, was built to communicate a small amount of information about who we were and what we did. The resulting one page design turned a lot of heads for its striking appearance, and it fulfilled its duty brilliantly by bringing us some of our first clients. As we found our feet with the business and completed our first projects, we knew it was time to expand the site into something more detailed. Looking back, it's clear that we owe a lot to Tim's design, and it's with mixed feelings that we move it to our little museum where it can be admired for years to come.

For the new site, we approached Elliot Jay Stocks who we'd already worked with, and told him we wanted something unconventional that didn't abandon the feel of the original site completely. It took us some time to get used to the radical ideas he came out with, but we believe he's really outdone himself and we're over the moon with our new home on the web. We'd love to hear what you think too, so please leave us your feedback in the comments, especially if you've spotted any teething problems. The new site is also home to our portfolio (which we'll be adding to soon) and a bit more blurb about what we're all about, as well as the blog you're reading now which we've dubbed The Particle Lab.

Our lab techs will bring you news on what we're up to, and will also share plenty of ideas, techniques and tricks we discover as we continue building web applications for ourselves and for our clients. We'll be talking about anything that pops into our heads that we think someone might find useful, from snippets of code from our weapons of choice such as Ruby (on Rails), HTML, CSS and jQuery, but also less technical topics such as our views on the state of our industry, or ideas for marketing your products and services. One thing we can guarantee is that there will be absolutely no top ten lists of any kind. On that you have our solemn word.

So please subscribe to this channel via our feed or email, follow us on Twitter, or just stop by once in a while for the latest post. We look forward to welcoming you back here soon.

The Particle Lab

Track traffic exiting your site, without jQuery

Building the new Neutron Creations site

Building WordStream

Safari SVG CSS animation bug

My favourite Ruby Gems

Basic testing for mobile devices in a desktop browser

Lab Report #6

Lab Report #5

Lab Report #4

Win a ticket to "The Web Is"

Lab Report #3

Lab Report #2

Back to Front show #20

Lab Report #1

Reserve your username

More about connect.me

Good API Documentation

Accurate & Up to date

Clear, concise and well structured

Examples

Discussion

Sandbox & Playground

Anything else?

More Reading

Building a Custom HTML5 Audio Player with jQuery

Single-Line vs Multi-Line CSS, plus TextMate tips

A simple technique for web app maintenance modes using Apache

Part I

Part II

Part III

Easier print CSS coding using Firebug and Web Developer Toolbar

Redesigning interfaces: learn from the old before you throw it away

Why TextMate?

File Management

Editing

Bundles

Add-ons

Is that it?

The value of remarkable customer experiences

Send contact form submissions to Highrise

How to track traffic exiting your site

What you'll need on your site

How do we do it

Taking it further

All done

New year, new site