Jalada

Tweetdeck, Echofon to get User Streams betas

2010-07-28T00:00:00+00:00

Twitter have announced that Tweetdeck and Echofon are going to be the first mainstream desktop apps to release beta versions of their software that use the new Twitter User Streams.

Twitter User Streams are streams unique to each user on Twitter which provide all the data needed to update a desktop application display in realtime. As it opens up, we will see a lot of applications becoming considerably more responsive, as they will no longer need to poll Twitter for updates at regular intervals; instead users will see new tweets, DMs, etc. in realtime.

I’m excited to see this transition to realtime, and I’m hoping that in the future Twitter will open up User Streams to web apps so that we can implement them on Twitterfall.

Update: Tweetdeck have posted about their new beta and how to get into it. Only 5000 accounts will get access at first, clearly Twitter are taking this very carefully.

RSSfall 2.0

2010-07-01T00:00:00+00:00

A couple of days ago I received an email from Julien Genestoux from Superfeedr saying they were about to release something cool, and would I be interested in cooking something fun up to demo it. I said I would, and got to work on a new version of RSSfall; ’RSSfall 2.0’.

RSSfall 2.0 uses Superfeedr’s new ‘track’ API that allows people to subscribe to keywords. These keywords are then applied in realtime to the thousands of new RSS entries Superfeedr is processing every second, and the results are returned using the PubSubHubbub protocol.

I then combined this API with my Comet server, Hydrometeor to provide an interface for web browsers to receive these realtime updates.

So far, this is little more than a demo (though there’s no reason why you can’t use it to monitor important keywords e.g. your brand). But I’m really excited to develop this further and see what Superfeedr have in store!

Superfeedr’s new API is like Twitter’s ‘Track’ but for RSS, I can see a lot of websites taking advantage of this great new source of data.

Superfeedrfall

2009-12-04T00:00:00+00:00

You may have seen RSSfall, which I blogged about a month or so ago. It uses PubSubHubBub (PSHB or PuSH or Hubbub for short) and Comet to stream RSS feed updates to your browser in realtime.

Someone else working in the realtime RSS space is Superfeedr. Superfeedr is an API for subscribing to RSS feed updates, a bit like Hubbub.

So what’s the point?

What’s great about Superfeedr is that they combine polling, Hubbub, rssCloud, and several other ‘Helpers’ to stay on top of RSS (for more, read their technology page). They guarantee that you will receive an update within 15 minutes. In fact, if you don’t, it’s free. That’s how much they believe in their system.

Julien Genestoux CEO and founder of Superfeedr contacted me about their competition to make something cool with their new ‘river’ API, which - to be brief - is a way of getting at the stream of data going through Superfeedr. He said he’d be interested in seeing a Superfeedrfall; RSSfall using their river API.

Within a couple of hours, I presented a very rough around the edges Superfeedrfall. It looks very similar to RSSfall, but you’ll notice that you receive a new update every 3 seconds or so (the speed of the stream they made available for the competition).

How does it work?

I put together a quick piece of Erlang that builds upon some of the code I’ve written for Twitterfall but instead connects to Superfeedr rather than Twitter. I then got that to parse and send the results to a Hydrometeor instance for it to send to any subscribers. When you visit Superfeedrfall, you are one of those subscribers, so you receive the update in realtime.

If you're forgetful, Things iPhone sync is useless

2009-11-23T00:00:00+00:00

Edit: I got a response from Cultured Code :) great to get such a quick response with a detailed answer.

Hi,

Thank you for your inquiry.

As you can imagine, wifi sync was only a first step. We are already working on a multiple Mac and iPhone over-the-air syncing solution. That’s the most requested feature currently. At present I can’t give you a time frame, though, when this is going to be ready.

When Push Notifications come to Things touch in version 1.5 Things will be able to update the application badge icon without launching the application itself.

Best, Roman

I’m forgetful. That’s why I bought the wonderful Things for Mac and iPhone by Cultured Code. One nice feature is that if your iPhone and Mac are on the same network, the two will sync whenever you run the iPhone application, no third-party slow web-based system required. The problem is it’s completely useless if you’re forgetful, because you’ll forget to sync when you’ve added new items. And let’s face it, if you’ve bought Things, that’s probably you.

There are a couple of use cases where the Things iPhone sync doesn’t work, and there are really simple ways to fix them.

Zzzz...Oh! I just remembered...

When I’m in bed, my mind has a tendency to remember things I need to do tomorrow, or come up with new ideas for Twitterfall. You’d think I could just fire up Things on my iPhone, quickly jot it down, and thanks to the wonders of iPhone sync it would transfer to my iMac humming away the other side of the room ready to remind me when I get up in the morning?

But no.

Even though when I load Things it immediately syncs with my iMac, it takes quitting and restarting Things once I’ve added the item before it’ll be on my iMac as well. This is because Things only syncs on launch. 20 seconds ago the application spoke to my Mac, chances are it is still there now, so why not sync the new item too?

Solution: If Things has just synced, when I add a todo item, sync it straight away. Then I can get back to sleep.

Out and about

I’m a university student. When I go onto campus I often remember things I need to do when I get back to the house, so I note them down on my iPhone. However like I said, I’m forgetful. By the time I get back I’ve forgotten I even wrote those items down on. I understand it can’t sync over the air with my Mac while I’m out, but what if my iPhone could remind me by setting the Badge icon for Things as just what is in my inbox - which is where I keep items I’ve written during the day? That way I’ll see it next time I use my phone.

Solution: Give me the option of just showing what is in my ‘Inbox’ on the iPhone badge icon to remind me I need to sync.

Both these solutions sound pretty trivial to me, and would really help my workflow. Now I just need to forward this blog post to the guys at Cultured Code :-)

Core i7 27" iMac Impressions

2009-11-21T00:00:00+00:00

I’ve been waiting for Apple to refresh their iMac product line for ages. On the 20th October it finally happened and by the 21st October a Core i7 27” iMac with 4GB RAM and a 1TB hard drive had my name on it.

After nearly a month of waiting, my iMac finally arrived.

No photo of the 27” iMac can quite convey just how large this thing is. It’s massive. I keep sitting back and going ‘Wow, it really is huge’. I’ve used dual/triple/quadruple/quintuple before, but there’s something different about having it all in one large workspace. I can keep Twitterfall, IRC, this blog post, a web browser, and a chat window all open and visible at once. Sure, I can do that with multiple monitors. But having everything within one environment definitely feels ‘cleaner’. It’s a lot more natural to throw windows around one big space than several big spaces.

As you can see from the second photo I opted for an Apple Wired Keyboard, so I can’t comment on what the wireless one is like. From what I can tell it’s very similar to the old one. Both have the same action as a MacBook (the white clamshell, not the unibody MacBook Pros, which are slightly different) and both are nice to type on.

The Magic Mouse is interesting. I like it, though I’ll see how my wrist likes it after extended use before committing myself to it. No problems with the bluetooth, ~~though it doesn't work in Window unless there's a sneaky trick I don't know about to get it to work~~ [Edit: Sparkesinator has pointed out that released an update for Bluetooth in Windows which adds support for the Magic Mouse, thanks!] I just plug in a spare USB mouse anyway. If you’re unsure about the Magic Mouse I suggest trying it out first to see how it feels, it’s certainly different to ordinary mice.

The Core i7 is understandably extremely fast. In my unscientific test HandBrake reported 20 minutes to encode a 1.5 hour long film on default settings. I haven’t done any other benchmarks, I’ll leave that to people better versed in the art. The Core i7 has the same Hyper-threading technology as the old P4s, meaning OS X (and Windows) see an 8-core machine. Rest assured it’s very fast, and based on benchmarks I’ve seen it’s worth paying the (relatively) small amount to upgrade from a Core i5 to a Core i7.

I tried to install Windows XP in Boot Camp so I could try out some games. I installed it successfully but discovered that the audio drivers (Cirrus Logic HD) have problems with the inbuilt microphone, causing it to distort and slow down. After a bit of a struggle, I managed to get Windows 7 installed in Boot Camp (I might post my own instructions sometime, but in the meantime Google around if you’re trying to do this yourself) and discovered that the audio works perfectly there. I imagine it would be the same for Windows Vista.

On the gaming side, I’ve tried out Aion, GRID, Mass Effect, and Mirror’s Edge in Windows so far, and the performance was stunning. In Aion the HD 4850 performs admirably, pushing out between 30 and 100 FPS (mostly around 50) at the native resolution of 2560x1440. Aion is based on the 5 year old CryEngine but it still impressed me, and it looked amazing on the 27” monitor. Grid, Mass Effect, and Mirror’s Edge also performed well at the native resolution with everything turned up fairly high (except antialiasing, which was a performance killer). I preferred playing them all at a lower resolution however because performance was that little bit better, and it negated a lot of need for antialiasing (running at lower resolutions naturally blurs the image slightly). The iMac is definitely no slouch in the graphics department, though you’re always going to be able to build a better gaming PC for cheaper (maybe not with a 27” display though).

Overall, I’m extremely impressed with all aspects of the iMac and can’t really find fault with it. I could moan about missing functionality like no Blu-Ray, no adaptors (yet) for using the Mini-DisplayPort input, no SSD option, lack of upgradeable parts, and so on. But for what it is, the 27” iMac really is a brilliant piece of computing equipment.

Weird font corruption on Aion at high resolutions?

2009-11-20T00:00:00+00:00

While trying to get Aion working on my 27” iMac, I noticed that the fonts looked a bit odd, almost as if they had little chunks taken out of them. I hunted around online for a solution but to no avail.

Turns out, at really high resolutions (though the same may apply to low resolutions depending on your font DPI settings I guess) the font just really isn’t meant to be scaled up that much (Aion increases the font size the larger the resolution). Simple go into the Game Options and turn down the ‘UI Size’. I put mine all the way down to 0.8 or so and it’s still all pretty large, and with the added bonus of the fonts not looking weird :)

There are still a couple of odd exceptions where the font isn’t quite right, but on the whole it looks a lot nicer.

As an aside, Aion runs really well on the new Core i7 iMacs.

Screenshot to come soon so you know what I mean…

Using Jekyll and Git to Blog

2009-11-16T00:00:00+00:00

I’ve been using Posterous for a few months now as a blog, and found some of the features really handy. It was nice composing my blog posts by email, and the simplicity of the service as a whole was really nice. Then I tried to write a blog post about netbooting the Ubuntu Live CD and it all went wrong. Posterous isn’t all that friendly about code snippets, and it inserted newlines and incorrect symbols all over the place. I’m a Computer Science student (see the lecture notes?) and I like computers, so I’m going to be writing blogs posts that include code. This made me see the light - I spent more time fixing that blog post than I did writing it. I want the barrier to entry to posting on my blog to be as small as possible. I needed to try something different.

I turned back to WordPress which I had used for quite some time, and very quickly I was reminded of why I stopped using it; I saw the memory usage on my server skyrocket, and a security fix was released the very next day. That wasn’t an option either.

I’d heard a bit about just using static files to create a blog which sounded interesting (certainly fixed my ‘memory usage’ issue) and investigated further. I came across Jekyll which is written by the guy who co-founded github. Github themselves use it to generate github pages. It seemed pretty interesting, so I forked a blog that uses it (happens to be the guy who wrote it) and adapted it for my needs. Note that at the moment the designs are quite similar, I’m hoping to alter this over time so my blog is a little more unique.

It’s been a lot of fun to use, and due to the way it’s designed it really complements a blog structure, with anything else you need. For example I’m using it to store my lecture notes (which I write in Markdown already where I can) and thanks to Jekyll using the Liquid templating engine it’s easy to generate the ‘dynamic content’ necessary to give access to your content.

Jekyll works really well with a version control system too. I’m using git with github, and all the source is available on my github. This means that I can get to my blog from anywhere, keep a copy of it (and run a copy of it) locally, and use hooks to automatically update the live version of my blog if I wish. Feel free to take a look at the code and use any of it yourself.

Any comments on the new system much appreciated. It’s a work in progress and I may alter things as time goes on.

Installing Ubuntu with a graphical installer using tftp, dhcpd, and nfs

2009-10-23T00:00:00+00:00

If you have the facilities, it’s really handy to install Ubuntu using Ubuntu’s nice install environment without having to burn any CDs. Just download the ISO and use it in conjunction with some server stuff and boot any computer off the network to get a fully interactive graphical installation, just as if you’d put in the CD. I also have a gigabit network, so this is a lot faster than using a CD drive.

Without further ado, here is how to do it, in brief. I’m using another Ubuntu machine for the server stuff, other distros may vary:

Install a TFTP server

I use tftpd-hpa (which is the name of the package in the Ubuntu repositories). Make sure it’s all set up OK and is running. I think the configuration defaults to using /var/lib/tftpboot, so that’s the path I’m going to refer to in these instructions.

Download the appropriate netboot.tar.gz

You can find these netboot.tar.gz files on the Ubuntu website. Make sure you get the right one for your architecture and Ubuntu release. Extract it to /var/lib/tftpboot.

Download the Ubuntu Desktop ISO for your architecture.

Make it available via NFS

First, you’re going to need to install NFS if you haven’t already. I think the package is nfs-kernel-server. Then pick somewhere to mount the ISO; I use /srv/ubuntu. If you want to use that too the command is:

sudo mount -o loop /path/to/ubuntu-desktop.iso /srv/ubuntu

Once you’ve mounted it, you need to make sure the computer you’re going to install Ubuntu onto can reach it. You do this by configuring NFS using the /etc/exports file. My /etc/exports file looks like this:

/srv/ubuntu (ro,insecure,all_squash)

I know very little about NFS, but that sounds good enough to me. Restart NFS with sudo /etc/init.d/nfs-kernel-server restart. If you have any way of checking it’s available, do so. I personally check by trying to mount nfs://my_server/srv/ubuntu on my Mac.

Grab some files off the ISO

In order to boot into the LiveCD environment, the computer you are netbooting is going to need the kernel off the ISO. If you’ve been following along exactly and are installing Karmic Koala, these files are /srv/ubuntu/casper/vmlinuz and /srv/ubuntu/casper/initrd.lz. Older versions of Ubuntu may use initrd.gz instead, I’m not sure. It doesn’t matter either way. Copy these two files to somewhere in /var/lib/tftpboot. I make a directory called /var/lib/tftpboot/ubuntu-desktop and put them in there.

Add the option to boot this kernel when your computer netboots

In /var/lib/tftpboot/pxelinux.cfg/default, add the following lines (change any paths/server names if yours don’t match). Note I used my server IP address in the ‘nfsroot’ parameter because I’m not sure if this thing can do DNS resolution or not. Also note the line-breaks here for readability:

LABEL live-karmic
kernel ubuntu-desktop/vmlinuz  
append root=/dev/nfs boot=casper netboot=nfs \
  nfsroot=my.server.ip.address:/srv/ubuntu \
  initrd=ubuntu-desktop/initrd.lz quiet splash --

My pxelinux.cfg/default file looks like this. Again, beware of line breaks:

include ubuntu-installer/amd64/boot-screens/menu.cfg  
default ubuntu-installer/amd64/boot-screens/vesamenu.c32  

LABEL live  
kernel ubuntu-desktop/casper/vmlinuz  
append initrd=ubuntu-desktop/casper/initrd.gz boot=casper \
	netboot=nfs nfsroot=192.168.0.3:/srv/ubuntu --  
	
prompt 30  
timeout 30

Set up dhcpd to tell the computer to netboot

In your dhcpd configuration file, make a client section for the computer you want to netboot. Add filename=”/pxelinux.0” to this section. The section for the computer I wanted to netboot looked like this:

host ccube {  
  option host-name "ccube.haven.network";  
  hardware ethernet DE:AD:BE:EF:DE:AD;  
  fixed-address 192.168.0.5;  
  filename "/pxelinux.0";  
}

Don’t forget to restart dhcpd.

Profit!

Set your computer to boot off the network, and watch the Ubuntu LiveCD environment boot before your very eyes!

These are very rough guidelines, not intended to walk you through every baby step involved in netbooting into this environment. I assumed that if you want to do this, you probably already know a fair bit about Linux. There is more information about netbooting available all over the web. Just Google ‘Ubuntu netboot’.

Google Wave issues

2009-10-07T00:00:00+00:00

Looks like Google Wave is having lots of issues today. Their servers are probably having issues due to the number of new people who have got Google Wave invites today (including me). The help forum confirms that many others are experiencing the same issues.

Once things quieten down, I’ll be able to properly test it. For now I’ve only had one successful experience with Google Wave (I had a chat with @jongretar about using Erlang Web Development Frameworks and authenticated REST APIs), the rest has been littered with trying to archive waves, or get the site to load at all.

This is pre-beta obviously, and the technology is definitely impressive. But it’ll be more impressive once Google can handle it.

So if you feel like you’re missing out without a Google Wave invite, don’t worry. Right now you’re not missing out at all.

Google Attempts to Explain what a Web Browser is

2009-10-06T00:00:00+00:00

This article from Mashable is about Google’s video explaining what a browser is, but I hadn’t seen the interviews Google did asking people what a browser is. From their responses, I think a reasonable percentage of these people got confused because the questions probably went something like this: ‘Hi I’m Scott from Google, what’s a browser?’ Because people went into ‘I haven’t got a clue about computers’-mode, so latched on to the name Google. Particularly when then asked ‘What browser do you use?’.

Funny anyway.

RSSFall - what Realtime RSS looks like

2009-09-10T00:00:00+00:00

As some of you may know, I’m the lead developer at Twitterfall - a site for viewing tweets in a realtime manner. Twitterfall is something of a demonstration and experiment; although it is a fully featured product that people use every day it helped me learn about the emerging realtime web and how to develop for it.

Some of you may also know about the recent flurry of activity surrounding two new technologies: Pubsubhubbub, and rssCloud. Both technologies do more or less the same thing from a consumer standpoint and their implementations aren’t much different either. Pubsubhubbub and rssCloud make RSS realtime.

Over the past few weeks I’ve been researching the most popular of these two technologies at the time: Pubsubhubbub. And today I’m ready to show off some of what I have been working on.

It’s called RSSFall, and it shows you what realtime RSS looks like. Updates fall from the top of the page (much like Twitterfall) as soon as we receive them. It may take a few minutes for you to receive a new entry from the feeds we’re monitoring, so keep the page up for a few minutes and watch the news come in.

Obviously right now RSSFall isn’t much more than a visual demonstration of how realtime RSS can be. But I’ve got plenty of plans for this technology and can’t wait to show you what I’ve got in store. Plus, if you’re a company interested in the new realtime RSS technologies and want to investigate into integrating them onto your website or any other area, get in touch using the contact link on the page.

Lifehacker performs Browser Speed Tests

2009-09-03T00:00:00+00:00

Lifehacker have done various speed tests against the latest versions of Google Chrome, Mozilla Firefox, and Opera and also compared the results to their last round of testing. Results match up with my previous results between Chrome and Opera, but more interestingly the development version of Google Chrome (seen in the graph above as 4.0.203.2) performs significantly worse in some areas than Chrome 2.0 which is the current stable version.

Is Google Chrome bloating out by adding new features? Or is it just missing a few optimisations because it’s the development version? It’s interesting to see that Firefox performs better in memory management than any other browser considering its reputation for being a memory hog.

Twitter Tracking Links again?

2009-09-02T00:00:00+00:00

On August 25th Techcrunch noticed that Twitter appeared to be tracking outbound clicks on their web interface, but that the redirects seemed to disappear after the downtime afterwards.

It looks like the links are back, I was just using Twitter.com and noticed the outbound links, which are of the format:

link_click_count?url=http%3A%2F%2Fis.gd%2F2OoZi
  &linkType=web&tweetId=3720272969&userId=15620813

plus an authenticity token. Looks like they’re tracking the link, where it’s come from, what tweet it’s come from, and the userId of the user that clicked it (in this case me).

It’ll be interesting to see if this sticks around, and I bet it’s part of the analytics service they might be providing to paying customers. I’m not sure how it will stand up to bit.ly analytics (which of course, they blessed) especially as bit.ly analytics will work on any service, not just the web-based interface. They could also use this to muscle in on Tweetmeme, and they could track links by clicks rather than retweets and extend their trending topics system to include trending links, though how accurate this would be at determining popularity when it would largely depend on the number of followers (clicking takes a lot less effort than retweeting) is questionable.

Is my Server Back Up yet?

2009-08-19T00:00:00+00:00

While waiting for one of the Twitterfall servers to come back online after an air conditioning failure in the datacentre, x5315 and I realised that instead of keeping an eye on a ping command, we could be told when the server was back online with this really simple (Mac-compatible) command line:

ping -o server && say "the server is back up"

Also, if you use Prowl on your iPhone, you could use something like this instead:

ping -o server && curl -q --insecure \
"https://prowl.weks.net/publicapi/add?apikey=YOURAPIKEY&application=Server Monitor&event=The server is back up&description=Server up"

And when the server is back up, you’ll get a push notification on your iPhone.

Simple, but handy.

A Twitter Bookmarklet that Doesn't Suck

2009-08-16T00:00:00+00:00

I’ve tried a few Twitter bookmarklets in the past. Some don’t shorten the URL, some require you to either hand over your credentials or log in via oAuth (boring!), some don’t take the title from the page you are on.

So I’ve decided to make my own based on what I like doing. It was really easy (literally a few lines of PHP and Javascript), and it’s really simple to use. Here’s why it doesn’t suck:

It tries to shorten the URL using is.gd. If it fails for some reason it’ll just resort to using the unshortened URL. is.gd has a really simple API; I know bit.ly would be more useful, but I’ve never used the bit.ly API. Maybe one day, I made this in 10 minutes after all.
It uses a nice tweet format. It puts the title, and the shortened URL, into a nice format of “Reading ‘title’ URL”.
You don’t have to give me your password. It doesn’t use any fancy automatic posting that requires you to give over your credentials, it just takes you to the Twitter home page (provided you are logged in, if not then it will log you in through Twitter as normal) with the status filled in, so you can add your own comment or whatever.
It opens in a new window, so you can close it afterwards.

There might be some bugs, let me know if you spot anything or have any ideas.

Go here for installation instructions

For reference, the syntax is:

http://twitterfall.com/bookmarklet.php?url=URL&title=TITLE

Simple.

Javascript Aho-Corasick String Search Algorithm

2009-08-01T00:00:00+00:00

For the new Twitterfall Exclusions filter, we needed something that would efficiently search tweets for multiple substrings. The simplest solution would be to search through the tweet multiple times for each string, but this is dumb and inefficient, especially when you may have several tweets per second falling down. The next solution would be to craft a regular expression and use that, but that seemed inefficient to us and we’re computer scientists and we were sure there would be a better way.

After doing a bit of research, I stumbled across the Aho-Corasick algorithm, which is a string search algorithm that matches all patterns in a dictionary ‘at once’ by constructing a trie with sets of links from each node representing a string to the node corresponding to the longest proper suffix. It sounded like just what we needed.

I couldn’t find a Javascript implementation of the entire string search algorithm, so I combined Dennis Bryne’s Trie implementation and my own code to produce a Javascript implementation of this useful string search algorithm, great for filters. This is the string search function, which you can then combine with the Trie implementation on the afore-mentioned link.

// Aho-Corasick String Search Algorithm
// @author Jalada http://jalada.co.uk
// Arguments:
//   trie - a Trie as per http://is.gd/1Y9FT
//   s - string to be searched
function ahoCorasick(trie, s) {
	// Start at the root.
	var current = trie;
	// Nothing in state at the moment
	var state = "";
	// Split the string
	var split = s.split("");
	var j = 0;
	// We return everywhere in this loop.
	while (1) {
		for (i=j; i<split.length; i++) {
			var r = current.hasChild(split[i]);
			// Does this character exist in the children of where we
			//are in the trie?
			if (r) {
				// If so, append to the state, and traverse to
				// that child
				state += split[i];
				current = r;
				// Have we found a word now?
				if (trie.getWordCount(state) != 0) {
					return true;
				}
			} else {
				// If not, go back to where we started to match,
				//reduce i, and empty the state
				current = trie;
				i = i - state.length;
				state = "";
			}
		}
		// Reached the end of the string
		if (state == "") {
			// Just found nothing
			return false;
		} else {
			// Was in the middle of finding something, so possibly
			//missed something, so go back to check.
			current = trie;
			j = i - state.length + 1;
			state = "";
		}
	}
}

I haven’t had much of a chance to make sure this is the most efficient way of coding this algorithm. But it seems to work fine, and in some rough testing I did it seemed 3-5x faster than a regular expression of the syntax /This|That|The other|Whatever/ in the circumstances I tested. If you use this, and find a way of improving on it, be sure to let me know so we can include it back in Twitterfall :-)

Feel free to use this Javascript code. I’m releasing it under the MIT License.