Technology Treason

2010: The year the Netbook turns into the Web-book

2010-01-04T10:37:00.005+11:00

2010 is set to be a bumper year for Consumer Electronics. With people spending less outside the home they are focusing more inside and as just about everyone now has some monstrous TV it's the little things that count.

2009 was arguably the year of the Netbook. After Asus launched with their eeePC in 2008 a land-rush occurred last year with virtually every Notebook manufacturer providing an offering. HP's Mini and Acer's Aspire One ranges both did very well as did Asus with their eeePC.

At the end of 2009 however it became virtually impossible to get a Netbook that is truly still a Netbook. Acer, Dell, Asus and HP all fell back into their same tired old routine - bigger, faster, more capacity! And the Netbook experience suffered.

I've made this point before but the power of the Netbook is in the Network - not in how big a hard drive it has. Why do I need a 160GB hard drive when I have Terabytes of NAS and Gigabytes of cloud storage? I don't need 5 USB ports and I certainly don't need Windows.

Having said all of this the Netbook category has gone ballistic - having doubled from 16 to 33 Million units sold in 2009 - sales are worth about $USD11Bn globally (Display Search research for 2009)

My money is on the next generation - so called Web Books, Slates or Tablets. These devices are being actively invested in by a number of investors and represent a merging of several types of computing behaviour.

Architecturally, most are small form factor (10 inch or less), are either a tablet or have a folding range far outside the normal Notebook range (can be flipped over on itself entirely - so is just a screen), they are generally touch screen capable with many being multi-touch and the big one - most are not running windows (generally running flavours of Linux).

Behaviourally, the Web Book is designed to be a piece of Consumer Electronics. It's not a desktop replacement, it's not an office machine. It's a device that is a general purpose computer but built to use in the home as such it plays on the following:

It's relatively small and definitely light.
The processor is powerful but not an energy guzzler (Intel Atom's do brilliantly here as do ARMs)
The display is gorgeous and has high viewing angles so multiple people can see it
It uses wifi and may not even have Ethernet connectivity
Solid State disks are a must but are low capacity (you don't need more than 16GB in a machine that is connected to a network) thus saving on energy
battery life is a must - the longer the better thus every component is energy efficient
Ideally the screen is touch capable and ideally multi-touch (thus eliminating the need for a keyboard)

The device is permanently connected to the network and thus the Internet. It's there to connect with people, view photos, play your tunes, watch movies and read web pages. It's not there to write documents, do full scale design or programming (though people will use it to do this in a limited, fast fashion).

I've been excited about Internet tables, Slates, Web Books - call them as you will since Nokia released the N710 and with Apple, HTC (Google), Litl and others all about to play in this space in a big way over the next few months, there will be a lot of people asking for a Web Book or Internet Slate in their Christmas Stocking next year.

Expect to see masses of innovation in this space as companies that have not been too caught up in the Netbook scene enter the fray for the first time and start showing off some new ideas. Litl does this with their awesome Easel Frame style web book and both HTC and Apple will doo some great stuff on the user experience end of things too.

2010 will definitely be the year that the Internet goes increasingly mobile both inside and outside the house but the experience of it literally becomes more tactile and less bound to the keyboard.

Why I'm interested in AWS Spot Prices for EC2

2009-12-22T17:46:00.002+11:00

There's been a lot of chatter going on around the intertubes over the last couple of weeks since Amazon Web Services released their Spot Instances pricing model for EC2.

In a nutshell - AWS have created a compute market. Instead of charging the same price to every person for the same product they have basically created a market where people can buy compute time at less than a price they are willing to spend based on the current demand.

There's been some conversation about the fact everyone should just put the current demand price in as their maximum and this would game the system (the comments here for example) however this misses the point slightly. The Clouderati often talk about Utility Computing or Commodification as one aspect of Cloud Computing and what AWS have done is the logical conclusion of that - they have created a true market for the provision of computing time based on supply and demand.

Now what's interesting with the ideas some commentators have come up with regarding gaming is it assumes everyone's working the interest of everyone else. That isn't the case. Yes I know I could get some resource cheaply if I keep my bid low and am willing to wait for a period of time however I have clients and they have deadlines. That big compute job crunching all the marketing data needs sorting out this afternoon - so I'm going to put a high bid in for 50 nodes NOW! The market will accommodate that and those with low bids will be knocked off. Thus the market constantly corrects to the requirements of demand.

But it's the flipside of this which makes me really interested in EC2 Spot Instances. I can have a battery of servers doing work at little to no cost if I build my system correctly.

The critical element to this is I need to address availability correctly - that is I need to ensure that my entire system doesn't go down because I've been priced out of the market.

This is a really rough idea at this point but I'd love feedback around it - it's obviously based around some kind of online application that requires multiple nodes.

I have an instance which is the master. All parts of the stack could be retreated back to this server if it's needed.
I have Cloudwatch or some other monitoring system assessing the performance of my nodes so I can see when I have spare capacity or when I'm under utilised.
The master server has a series of heuristics looking at the current work loads and the current costs that each server is incurring versus the work it is carrying out. Thus low utilisation and low cost is okay but low utilisation and high cost would cause alarms to go off.
The heuristic set up makes reference to the demand pricing level and strives to always keep each instance below that price.
As the spot prices go up and over the demand price I immediately terminate expensive spot instances and start replacing them with lower price demand ones. As the price comes back under then I can replace demand prices with spot prices
The master server then creates instances as required to fulfill the work units that are required and link them into the system.
Each node is able to be switched off mid unit so the entire network is self-healing

So the only thing that would be required to get this up and running now is having a reliable system for creating nodes and getting them working into the network as quickly as possible and producing the heuristic system to monitor and create and destroy instances based on some rules that would create some intelligence around pricing.

Not least the system would need to determine whether a mix of different types of instances would be appropriate if there are large distinctions between their current spot price for given work units. For example:

If we were serving a bunch of web pages using some heavy duty memcached system then RAM is the most important commodity. Say I have an instance of 1.7GB RAM at 3C/hr and another instance of 7.5GB RAM at 15C/hour then my intelligence system needs to understand the component (Memcached) just needs buckets of RAM and that getting 5 instances at 3C/hour is better value than 1 at 15.

Importantly it can then ramp up towards that number based on what is actually required rather than doing the whole lot and then under-utilising.

So I think we're quite a way away from this type of system but my opinion is that this isn't out of the realms of possibility and importantly the market Amazon has created has allowed (I could almost say "is going to force") these types of architectural considerations to start being made.

Interestingly all of a sudden decisions I am going to make around infrastructure is going to be much more value based. It's not about ROI - it's about value and am I getting the best value from my infrastructure. IT teams that get this are going to make an absolute killing with the type of services they can offer and the prices they'll be able to do it for.

Am I off my rocker? I'd love to explore this idea further.

Prediction: 2010 will be the year Apple and Google have a cage fight

2009-12-21T10:46:00.005+11:00

The pre- match slanging is pretty much over and the location of the fight has been chosen. 2010 is going to be the year Apple and Google finally stop dancing around and actually get in the ring. Unlike a nice clean refereed boxing match (Apple V Microsoft) this is going to be a dirty underground cage fight complete with barbed-wire wrapped gloves - expect to see a lot of blood on the floor - and fanbois rucking in the concourses.

The ground is, of course, Mobile and the massive dominance both organisations have taken in this space over the last 12 months. Mobile is still a fast growing area of communications but smartphones is where it's at. There's no question Apple ignited the world's imagination of what is possible in the mobile space and capitalising on the fact that the fashionability of a phone is important in a way that RIM and Microsoft just didn't get.

Google have taken that to a whole different level with Android which just "gets" what it is to be a data capable and Internet connected phone. Couple this with some fashionability and the stage is set for an almighty fight.

Looking through the AdMob report for November, it's astonishing to see how fast Android has grown in the last 2 months (doubled on traffic requests through their network) but more importantly was the launch of the Motorola Droid and the whole Droid Does campaign. The Droid is one of the fastest selling phones of all time almost hitting iPhone 3Gs sales levels (which was working from an installed base upgraded) and is now accounting for about a quarter of Android device share - only behind the G1 which has been out for 18 months - expect to see that change over December.

Now Motorola have entered the fray and with Samsung and Sony Ericsson both scheduling major launches into Q1 2010 the mobile landscape is going to get increasingly messy as the iPhone isn't the only great phone out there. Indeed I think Sony is going to do a Motorola with the Xperia X10 as it is simply stunning and is a big name in the mobile space - especially in Europe. HTC have had a great lead but 2010 will see Motorola and Sony return to some dominance here - and they can fight Apple in the Fashionability stakes.

The biggest challenge for Apple is how to combat Google on the phone itself. Outside of iTunes, Apple has little in the way of first party apps for the iPhone and whilst it has a huge developer network it is definitely alienating them through it's App Store management nightmares. Many developers are developing for both iPhone and Android devices - especially those using Web technologies for building and apps like Phone Gap to cross-package.

A lot of what makes the iPhone really useful are Google applications (native Gmail, Maps and most importantly Search!) - Apple has no way to combat this. Are they going to deny Gmail or Search like they did with Google Voice?

Apps that are available on both platforms and services that are available "in the cloud" eg Maps, Comparison Shopping etc dilutes Apple's position as it's only point of differentiation becomes fashionability and both Sony Ericsson and Motorola have competed for over a decade against Nokia by building highly fashionable phones.

I'm not sure this fight will be a death match but all the signs are there for a battle of epic proportions. Both are likely to be extremely battered by the time they come out the other side and would be wise to hold a little bit in reserve in case Nokia's Maemo platform takes off the way they are expecting it to - at that point things could get really messy.

SPDY could gain acceptence very quickly - with some product innovation

2009-11-13T12:12:00.002+11:00

Google have announced some early findings about their research into a faster protocol to reduce latency times due to good old fashioned HTTP. HTTP was designed as a really simple protocol to delivery (primarily) text content over the Internet and thus was born the Web.

One of the problems with HTTP is that it only really allows a single request to be serviced at any one time. The reason this doesn't APPEAR to be the case is because modern browsers create multiple connection threads that connect independently to the server and it gives the appearance of things downloading in parallel. It's a neat hack and works because we have good network speeds and mast processors to manage all this multi-tasking. Go back to a Pentium II with Netscape 2 and you'll watch the glacial procession of elements loading in from the top and goes down the page.

The Google project page talks a lot about why HTTP pipelining doesn't work and some of the technical architecture behind SPDY which I won't cover here other than to say that it's great we are seeing this type of innovation at the protocol level. What's most interesting for me however is how we get it in production.

There is a lot of nay-saying going on around this suggesting that because of the size of the Web you'll never get people to shift to a new protocol HTTP:// won, let's all leave it at that because there are too many web servers and web browsers to convert. This is what I want to address in this post.

Yes - there are fair too many legacy browsers to deal with to make this transition happen. Look how many IE 6 browsers are still in use, but we'd also have to shift all the Mozilla users, Chrome users (easy because of forced update) and Safari users as well. Not to mention all those pesky mobile devices that are springing up.

Dealing with the web servers is a much more straightforward issue. There really aren't that many in the scheme of things. Indeed much of our existing infrastructure runs multiple servers, Apache alongside a lightweight server like nginx and this is increasingly common.

As such there's nothing stopping me dropping in a SPDY server alongside my existing infrastructure for those users that can directly access it (Chrome 4, Firefox 5, Safari 6 and IE 10 for example).

But let's not stop there. A network admin could create a software appliance at the Firewall or Internet Gateway level for the corporate network that took HTTP requests, turns them into SPDY requests and then proxies these back. Now I have doubly fast Internet connectivity without upgrading my connection. For the price of a box that is well worth it.

For home users we could do the same thing. This protocol is software - it runs on TOP of TCP so because of that a Firmware upgrade of your average Netgear or Linksys home router could get you the same benefits as those above. ISPs could force this remotely on certain systems (Cable for example) or provide info on how to do it such as through a web, phone or personal service.

So for all the nay-sayers out there - this is a MASSIVE opportunity to speed up the web and people need to think outside the browser sometimes. QoS was delivered at the router level based on intelligent packet analysis - that speeds up network traffic massively but it's a software change not a hardware one.

I don't think it will be long until we see Netgear and Linsys start promoting this like they did with the WiFi standards and force adoption because it makes a great marketing case to do so.

I'll be trying this out at the rawest state to see if we can make it work and if I can, watch how fast our servers and network gateway get upgraded before I embark on upgrading client machines.

AdMob purchase by google paves way for interesting developer funding

2009-11-10T10:14:00.005+11:00

It's just been announced that Google is set to buy AdMob for $750M in an all-stock deal. This is the third biggest purchase Google has ever made (the only two bigger are YouTube and DoubleClick).

AdMob started in 2006 so they have capitalised very well for a 3 year old business. Indeed they've been cash positive for a while now so this is a great acquisition by Google. The full gory details of the deal can be found here and a press site by google here

We know this is all aligned to Google's interest and in particular their big appetite presently for anything Mobile. However this also opens up some enormous opportunities for developers.

This acquisition brings with it some great opportunities for in-application display advertising that is delivered contextually but also based on Google AdWords auctioning technology. Along side this I can then use the same advertising account to drive ads on my mobile website that compliments my application and then use standard ads on my main website that provides additional information / community support etc.

All of a sudden a possible revenue opportunity opens up that was kind of there previously but wasn't very smart. Over the last 18 months in particular we've been watching the rise of free-ad-supported applications as well as paid-no-ad versions of the same application. I would expect to see a lot more of the ad-supported apps once this deal goes through.

The reason for this is twofold:

1. As a developer I can manage all of my advertising spaces with one vendor. I don't really want to have to deal with all these businesses I just want to get some beer money for my app that I'm spending my non-work hours producing.

2. With contextual ad serving, I can make certain elements of data within the application available and use that to generate calls to the Ad Server - much the same way AdWords works with a web page or in Gmail. This means the ads that are served will be more relevant to the content which should lead to higher Click Through which then leads to potentially more revenue for me (see note above about beer money)

This makes a lot of sense for an advertiser as well. Certain applications have huge amounts of uptake - twitterific on iPhone or Twidroid on Android for example. Imagine having contextual ads served based on the content of your twitter stream. Twitter might resist it but it could make some serious cash for the app developers.

Overall I think this will really blow the top of mobile advertising. Advertisers who have been a little shy in the mobile space will be comforted by the fact it's Google doing it. App and mobile site developers stand to gain some good funding from it and it be relevant for their audiences and as the world goes increasingly smartphone mobile mad over the next 18 months this will be worth serious $Billions in the next 5 years or so.

Cross posted to Citrus Agency Blog

Crown Oaks Day Racing Challenge

2009-11-05T07:09:00.005+11:00

Last night I was writing code to play around with an idea I had rather than studying the Form Guide. See today I am off to the races (Crown Oaks day at Flemington) with some clients - hence why I should have been studying the form guide and not playing around with Erlang.

So I've decided to try an experiment:

Can the wisdom of the Twitter crowd outperform both blind luck and the bookies favourites with regards to return on bets during the racing.

Now we all know blind luck should lose. Betting on a winner at random is probably not going to get a single hit but against the favourite should prove interesting.

What interests me most is that betting on a race is actually contributing to an Information Market. Now theoretically information is held by all the various agents (people betting) and each one doesn't have a full picture but together the market becomes efficient and pushes down the odds of certain horses winning and then having long shots.

In smaller races where you have not so many people betting this works and either the favourite or a horse will relatively low odds will win. At large race meetings this doesn't work because a lot of people bet randomly (based on name, birthday number etc) and because of this it creates a lot of noise in the market so it breaks down.

So here's the challenge.

I'll start this from Race 3 at Crown Oaks Day today.

Each of the races is displayed below with a link to the field list. I'll be making a bet based on complete randomness (random number of the horse) and following the bookies' favourite. I'll then take the majority from messages to my twitter account (@ajfisher) for that race and place a bet following that. Simply send me a tweet "@ajfisher Race: NUMBER Horse: Name or Number"

Race 3
Race 4
race 5
Race 6
Race 7

Also you can use the tag #crowd-oaks if you want.

So can a smaller crowd provide more wisdom and outperform the bookies and complete blind luck on a big race day. Let's find out. It'll be fun either way.

The only reason why Linux isn't ready for prime time desktop

2009-10-11T12:58:00.004+11:00

Okay, so this title's probably a bit misleading as there are probably a few reasons but as far as I'm concerned there's only one thing stopping my final transition to desktop Linux for complete every day usage.

Presenting

In my job I do a lot of presenting. I give major milestone presentations on projects, I present to the business on things that are going on, I present in pitches where we are attempting to win new business and recently I've started presenting at conferences.

I would not use my Linux desktop (and I have combinations of Ubuntu and KUbuntu 9.04, CentOS) to present with at all - even if someone paid me.

Before I say why I'll also lay out my Linux credentials. I use RHEL, Ubuntu and Centos EVERY day. All of my home computers are Linux based, I have a Linux PDA, I prefer my Ubuntu desktop for work and I administer numerous Linux (CentOS and RHEL) servers - via command line - all of the time. I've used it for over a decade and am more than happy with it and more than happy to hack on it to get stuff working.

However, there comes a point where I am not going to entrust a complete presentation that our business or my reputation relies upon to Linux's extremely flaky graphics system.

Yes, I know laptop Linux is problematic (but if the rest of the desktop is stable why not my second video out?)

Yes, I know that graphics card support (particularly from ATI) is very closed so there's lots of reverse engineering going on (but again if I can have one video out working why not two?).

I'm not sure why this is the case - I think it's a combination of X.org config and poor tools for configuring multiple screens with different resolutions but it definitely needs a lot of work to go ready for prime time.

I was at a conference this week and I had built my entire presentation in my Gnome desktop using FLOSS tools like Open Office Impress, had a great looking presentation and was legitimately keen on presenting using either my Ubuntu or Centos desktop. After hours of mucking around however I didn't feel supremely confident in just walking up to the podium, plugging in my laptop and "It Just Works"^TM. It's just too hit and miss.

I don't generally experience this with Linux in general and Ubuntu specifically although I am aware of other people saying it. For me 99% of the time it does actually just work.

So I defaulted back to my dual-boot Windows partition and presented from that instead. This was the partition that I had considered nuking because I hadn't used it in about 6 months. In this instance though I didn't have any other choice - and sure enough it did just plug in and go.

I still presented from Open Office Impress though (which is a fantastic bit of software I might add!) and I think I was the only one at WDS09 that presented with it (and I'm sure no one could tell I wasn't using PowerPoint or Keynote).

Desktop experience is exactly that - an experience and our experience, particularly when we are doing something social with a computer can affect our mental state quite substantially.

If I'd have taken the decision to present using Ubuntu I would have felt worried about whether my laptop would work and I would have been nervous and probably would have delivered a terrible presentation. In contrast because I knew I wasn't going to have any support issues I felt confident, in control and delivered what I hope was a good presentation to the audience.

Ubuntu are trying to address many of these issues with the Paper Cuts project but that's really aimed at business. Apple have addressed similar issues (hardware compatibility) by having a presenter's kit (which you buy) which provides all kinds of adapters to go from Mac to just about every video input type. Microsoft addressed this years ago from Windows 2000 with a great set of dual head tools that made it simple and a standardised way for vendors to incorporate them and it is extremely rare for it to fail.

Business use is one of the areas that Linux (and especially Ubuntu) has got a real opportunity to shif users across as there are so many other business benefits but users want a single consistent desktop so they aren't going to build on one desktop and present on another - it's too inconsistent.

For me this issue on presenting and graphics support isn't so much a paper cut as it is a gaping flesh wound and it really needs to be addressed.

Web Directions South 2009 - Cloud Sourcing the Business

2009-10-09T13:26:00.008+11:00

I recently gave a presentation at Web Directions South which was fantastic (the conference I'm referring to here!). The session was on cloud computing and I hope everyone got something out of it. I've put the presentation itself over at slideshare

Cloud Sourcing the Business

View more presentations from Andrew Fisher.

My speaking notes are below as the presentation won't make much sense without it as it's mostly images or single statements.

Slide: Introduction

Cloud computing is a fast moving part of the IT landscape right now and, media hype aside, it does actually represent a fundamental shift in the way we produce and consume IT services.

[CLICK]

Slide: Cloud computing simplified

Partly because of where Cloud Computing resides on the hype curve but also because of it's pace of change, understanding the landscape can sometimes be confusing and it's probably best explained by this slide.

Today I'm going to talk about what cloud computing actually is in practice, how cloud sourcing can be used in your business as well as it's future direction and some of the traps you need to be aware of.

[CLICK]

Slide: What is cloud computing?

Cloud computing encompasses a huge array of services, technologies and techniques. Marketers would have you believe everything plugged into the Internet is a cloud. To understand the distinct areas of cloud computing let's try a little experiment.

Hands up if you are right now using some kind of Cloud Computing Service in your business?

Okay so if you are using some kind of infrastructure service such as S3, EC2, vCloud or RackCloud put your hand up.

If you're using something like Google App Engine or Force.com actively right now, stick your hands up.

Okay, so what about those people using Gmail, Google Docs, Office Live, Hotmail, Twitter or Facebook or something similar to that?

So nearly all of you are using cloud services right now without perhaps even realising it. So my work today is done as you're all experts...

As you can see, cloud computing covers a lot of different services and disciplines. Partly because of press coverage the things you expect to be Cloud Services such as infrastructure are there but so too are things you don't think of such as google maps or Gmail.

The networking company F5 commissioned some research not long ago with many CTO and CIOs of large US businesses on what they considered to be THE definition of cloud computing. They trialled 6 front runners for what cloud computing is.

None stood out from the rest as a sufficient or complete definition.

Googling Cloud computing will give you a heap of results and nothing really convincing either.

[CLICK]

Slide: Infrastructrure

Hardware guys generally believe if there's no infrastructure then it can't be a cloud.

Slide: Web 2.0

Software guys like me generally take the line that it's all about Web 2.0 style applications.

[CLICK]

Slide: NIST Def

The National Institute of Standards and Technology in the US have come up with this working definition. The full thing runs to a couple of pages, but here's the guts of it,.

Cloud computing is a pay-per-use model for enabling available, convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and so it goes on...

Obviously this is a definition by committee in order to be able to get as wide a consensus as possible

NISTs definition is great for someone who is firmly entrenched in the cloud community whether as a vendor, seasoned consumer or consultant but is a bit impenetrable for everyone else coming in.

So for the purposes of today let's go with something a little more straightforward.

[CLICK]

Slide: Cloud Computing Definition

Cloud computing is: A Network based Service, that is available on demand such that the end consumer considers it not their problem

I think it sums up the spirit at least of what NIST is trying to say.

Let's dive into this a little.

[PAUSE]

It's a NETWORK BASED SERVICE. Cloud computing provides a service over a network of some kind.

[PAUSE]

Clouds are provided ON DEMAND. It is imperative that you can turn the service on and off as well as scale it up or down as needed. Cloud Computing is all about using a commodity so I should be able to stop using it if I need to or only use it part of the time.

[PAUSE]

Finally Cloud Services are Not My Problem. Or yours. Offering the service is going to be making someone lose sleep somewhere but it shifts the provisioning, management and maintenance issues away from the consumer towards the provider.

It's like flicking on a light switch. To make my lights go on I don't need to understand how the power gets across the cables or indeed connect the wires up myself each time I want to make it bright. It's not my problem – someone else looks after this for me. It's like Gas, Water or Electricity.

[PAUSE]

This definition is simple but allows us to explore what cloud look like and some of the ways they can work for you right now.

[CLICK]

Slide: Flowers?

Before we get into the actual types of clouds and what they can do specifically, it's worth touching on where these things live.

Clouds can be Public or Private. It doesn't matter whether you are available to everyone on the internet or just one little part of it. Service clouds just made available to your business are just as important as the massive public ones that are made available to everyone.

[CLICK]

Slide: Public clouds

Public clouds are those located out on the Internet and provided for a fee to parties that wish to consume them.

[CLICK]

Public Clouds are the ones that are being talked about most out in market such as those provided by Amazon, or Google.

I would use a public cloud because I either don't want or don't have the expertise to run one privately. Likewise as a business I can look at a public service and think that will do the job perfectly and I just pay to use it - why bother going and building my own?

[CLICK]

Slide: Private Clouds

Private clouds are typically owned or leased by a single organisation for their purposes only.

[CLICK]

Private clouds are experiencing very large amounts of growth as many very large organisations such as banks and governments start flexing up their IT to provide more services that can be combined and consumed on demand by internal divisions.

[CLICK]

Slide: Hybrid Clouds

Much like a hybrid car where you combine something like eco-friendliness with performance, it's also possible to have Hybrid Clouds

[CLICK]

As you'd expect Hybrid clouds are where businesses are using a combination of public and private components. This provides a huge amount of flexibility to the business as decisions can be taken around which services you are happy to have public and those you prefer to keep private.

[PAUSE]

So now we know where these clouds are located, it's time to look at what they can do.

[CLICK]

Slide: what can cloud computing do?

So now we know where these clouds are located it's time to look at what they can do.

[CLICK]

Slide: Service

Remember in our definition I talked about how Cloud Computing was some kind of service delivered over the network? Well unimaginatively every time you're looking at something cloud related you'll always see it called Something as a Service. The word Service in Cloud land is almost interchangeable with commodity and Cloud Computing is really all about taking some piece of technology and providing it in the most commoditised manner possible.

[CLICK]

Slide: Cloud Computing Landscape

The current Cloud Computing Landscape is made up of four major service areas- Infrastructure, Platforms, Software and Data.

Infrastructure and Software as Services are currently the most well known. These have the highest uptake from consumers at the moment – Amazon's Infrastructure Services alone generates over $250M in revenue a year. However there are more services you can use as seen by platforms and Data so let's take a tour around the Cloud Computing Stack.

[CLICK]

Slide: Infrastructure as a Service

This is most commonly used form of Cloud Computing currently. It's commodity hardware or infrastructure that is available for consumers of storage or processing.

[CLICK]

Slide: Server room

With infrastructure you can go and put your own servers in your own room and manage them yourself or else you can go and use some else's who have built it on a drastically different scale.

Usually these are paid for based on some kind of metering such as GB of storage or hours of processing time.

The consumer usually doesn't need to manage the the underlying infrastructure, they just put the pieces together and configure it as they like. Infrastructure Clouds have really taken off over the last 18 months or so because of the investment businesses such as Amazon, Go Grid and RackSpace are making in their User Interfaces making it quick and easy to provision storage or full servers or appliances.

Slide: Aws

As you can see, being able to manage your infrastructure services

[CLICK]

Slide: Go Grid

Is as simple as clicking a mouse.

[CLICK]

SLIDE: Using Infrastructure Services.

So how can I use this? With Infrastructure Clouds I can go and create everything from a single virtual server to host some site or project all the way through to creating a virtual data centre – in minutes.

Because I am a massive cloud geek I just decided one evening to create and run a Neural Network using 300 machines in Amazon's EC2 cloud – just because I could. It cost me about $50 and worked perfectly.

Many Infrastructure vendors also provide storage clouds where you can upload files and pay for the storage and traffic.

In practice, these services are great for trial projects or for when you need massive scalability very quickly.

SLIDE: Melbourne cup

Last year for the Melbourne Cup we deployed a Hybrid Cloud to help with web site demand during the Melbourne Cup carnival. Most of the year, the Victoria Racing Club's website is busy but manageable with pretty average infrastructure.

The Melbourne Cup is Australia's biggest sporting, cultural and social event with hundreds of millions of viewers world wide.

Over the Melbourne Cup Carnival, the site visits ramps up considerably with over a thousandfold increase in traffic, most of which is over only a couple of days and 90% is during about 6 hours on Melbourne Cup Day.

Using Amazon's S3 service as a Content Delivery Network, we hosted all our heavy assets such as images, video and flash files. Amazon's EC2 service was used in order to deploy servers on demand as they were needed for extra capacity.

Alongside this we then had a private cloud at Web Central that could be configured how we needed it based on the types of traffic we experienced.

The solution proved successful with nearly a million visitors on Cup Day and at peak over 200,000 files being transferred a minute.

On top of this though, it was also extremely cost efficient because VRC don't have servers sitting around doing nothing for 360 days a year as all of that extra capacity was switched on just prior to Melbourne Cup and switched off 2 days after.

Using Infrastructure Services in these circumstances helps you maintain a great user experience and also makes your costs much more manageable and reduces your provisioning time.

A lot businesses are using this right now to help with managing their heavy files. Look where Twitter hosts it's images...

[CLICK]

SLIDE: S3 Amazon Screen shot

That's right – Amazon S3

[PAUSE]

Given the sheer numbers of people visiting twitter now this makes complete sense for them to be able to manage the costs of image hosting and delivery.

At last count they had over 60 Billion items in there which is growing at a rate of about 10 Billion every few months so there's plenty of people using this and similar services out there. The infrastructure services are getting better all the time as competition is driving innovation.

[CLICK]

SLIDE: Platform as a Service

Platform clouds will be immensely popular over the next couple of years. They are 1 level up from Infrastructure services and these clouds take away the “machine” part of the cloud giving you a platform fully configured and ready to deploy your application.

[CLICK]

Slide: Diving platform

This means developers get the on-demand parts of Infrastructure clouds without having to concern themselves with managing a virtual data centre or configuring machines.

Certain assumptions are made at the platform level – for example Google App Engine restricts you to certain languages you can use. However in making these assumptions, it means developer has a defined jumping off point. Google App Engine and Force.com are seeing an explosion of developers who are embracing the platform and coming up with new ways to use it.

[CLICK]

SLIDE: Logo Slide.

At the moment platform clouds are in their infancy however they are growing rapidly. The two most well known and heavily used right now are Google App Engine and Force.com which is the platform underpinning Salesforce.com

The single biggest benefit of Platform Clouds is simply Speed to Market. The development process is drastically shortened because many behind the scenes decisions have been taken for you. As such the testing time is shortened because if you're using components of the platform they are already tested so you only need to test your code not the Platform's. Additionally you also don't need to go away and build an entire system stack – databases, servers, APIs etc – it's all there waiting for you to come along and do something with it. Ongoing it means a lot less maintenance as well.

Developers are looking at Platform Clouds in order to spend more time working on innovations and less time on replication. As a developer, why would I want to spend time writing yet another application that puts some customer information into a database off the back of a web form and then write some little app that allows the customer service team to make sense of it. It's this behaviour that Platform Clouds are addressing and the side effect is that not only do I get the benefits of having a complete system to work with that's developing new features all the time, I get to produce it quickly.

Nucleus Research looked at 17 projects done using Force.com earlier in the year and found that speed of delivery had up to a 10 times increase. The average across all projects was about a 5 times increase in speed.

Imagine what you could achieve right now if the projects your business is doing could be in market in one fifth of the time it's currently taking you? Just consider that for a second...

Developers are citing similar benefits of App Engine as well as some of the other platforms out there though little official research exists just yet.

As these platform clouds really mature over the next year or so the pressure will be on for businesses to start using them a lot more heavily as it will be uncompetitive to build equivalent systems from scratch.

[CLICK]

SLIDE: Software as a Service

Salesforce.com has been the Pin Up of SaaS for a very long time – in many ways they defined what it was and probably started a lot of the Cloud Computing discussion as they showed very early on it was possible to commoditise a service and deliver it over the Internet.

The reason Software clouds exist is because in the traditional Software industry there is a dirty little secret – maintenance costs more than innovation. Most Software Vendors have a small team coming up with new ideas and developing software then a massive one looking after the support and maintenance of these new ideas. Software Vendors have product lifecycles where they support certain versions of a product until they can justify not doing it any more and then hopefully force the customer to upgrade. During this lifecycle various customers could be on many different release versions and all of this adds to the cost of support.

The heart of Software as a Service is that there is only ever one version of a product, delivered over the Internet and all customers use the same version until the next upgrade. Immediately much of that time used for maintenance and support is available for sales, marketing and innovation.

Is it any wonder that Salesforce is now on it's way to dominating the global CRM space.

[CLICK]

Slide: SaaS Collage

Most software is now moving in this direction and is a big part of what enables Web 2.0 and is transforming the way developers deliver and maintain software for their user base. Look at software like Google Docs – who'd have thought even a few years back it was even remotely possible to use a spreadheet like Excel in a web browser.

Consumers of Software Clouds are generally pretty ignorant of all the underlying platform and infrastructure and are only interested in configuring the application itself.

[CLICK]

SLIDE: Gmail screenshot.

There are many applications of Software Clouds, far too many to go through today and my advice for any business procuring any new piece of software for the organisation is to make sure you seriously consider a Software Service in your selection process.

As an example let's look at Gmail. I'm guessing a lot a people here are using GMAIL for your personal email however Google has a fully supported Enterprise edition that works for your domain. You can still use outlook if you want or use the web client or mobile client if you prefer.

Email for a business is as important as cashflow however for many businesses the management and in particular cost of management of email is significant. On Premises email is something you have to maintain – backups, mailbox cleanouts, archiving, system upgrades etc are all the standard tasks done by an IT manager whether employed or contracted. Even with all this time spent, email still occasionally goes down.

Taylor Woodrow, a large construction business in the UK had much this scenario. They are a large business with about 1,800 employees and they were managing all their email in house using their IT team.

Last year Taylor Woodrow switched over to Enterprise Gmail, moving away from their on Premises corporate email they'd had for years. Whilst there are still costs with Gmail such as licenses to pay – Gmail costs $50 per user per year - and you still need someone to administer the accounts the cost savings for the business were massive. Over the year they saved about $2 Million compared to their previous mail software.

Given that large organisations have a scale of efficiency and they can get great cost savings per user, the benefits to smaller businesses are proportionately greater.

For an Australian business that has between 20 and 50 employees it's estimated that many businesses will save on average about $100,000 a year in total costs due to reduced licenses, man power, backups, hardware requirements etc.

That cost saving could be ploughed back into marketing, sales or other IT projects such as building your own Software Cloud.

Again, this is just one Software Service example but it shows how it's possible to start realising cost savings and get a better solution. Whilst I'm sure everyone here is broadly happy with their email system, are you now or will you ever be the best email providers? Probably not. Gmail can do it better and cheaper than you can. This is the crux of most Software Services – they can be provided, maintained and constantly upgraded to a much higher standard and for less cost than you can do it for and that's why we seeing them grow so rapidly.

[CLICK]

SLIDE: Data as a Service

Data as a Service is the newest part of the cloud computing landscape and I see this as having the most opportunity to shift how we create and consume data and is the one you can get involved in right now..

[CLICK]

SLIDE: NY Times Visualisation

Data is being provided everywhere and we are just now starting to see the interesting ways that people can use it. Data as a Service provides the consumer with the ability to retrieve and publish data – whether their own or that which is publicly provided - via the service.

The consumer gives no consideration to the underlying software, platform or infrastructure, only whether the data is available or now.

Data clouds are being used to expose data out to end consumers such as via Retail or Search APIs so those consumers can take them, remix them and provide them outwards again.

Data as a Service is built for mash ups – think about all of those sites with mashups of people's tweets plotted on google maps

[CLICK]

Slide: Zappos

or this great Zappo's site that shows what products people are looking at on a map showing where they are located.

[PAUSE]

SLIDE: DaaS Applications.

Data Clouds are so new that there aren't that many examples to show you yet though over the next 18 months we'll see an explosion in the data that's being made available.

Organisations that are starting to do this are people like the US Government with their Open Government or Government 2.0 initiative. Lower level governments are starting to make their data available such as the San Francisco city government who are making available data sets for things like crime data, highway information, and lists of public trees of all things.

Whilst these data clouds are very new we are seeing some interesting business models develop out of them. A US company called Jigsaw has recently become self funding. Jigsaw specialises in managing personal data by crowd sourcing contact information from participants and then they use that data to sell to marketers who want to cleanse their own subscription or marketing lists or who want to acquire new customers with very specific requirements.

Other businesses are looking at creating industry based data clouds to manage source information such as where livestock have been reared, transported and eventually slaughtered before making their way to market thus providing a traceable and authentic source for produce certification.

[CLICK]

Slide: Layar

The other area where Data Services are starting to appear is in the burgeoning field of Augmented Reality. Consumer ready AR is really being driven by and will continue to be driven by Data Clouds as huge data sets are made available to consumers via their mobile phones or PDAs.

Layar for Android and soon for iPhone is one such app that allows data sets to be viewed on your mobile device. Perhaps obviously at the moment this is skewed towards things like tweets, photos from Flickr or Wikipeadia entries that are near to you but this will change as organisations start to make their data available.

The examples on screen now show you where your local 7-elevens are and some points of interest in Sydney.

This space is very much in its infancy right now but it's definitely an area you can get involved in massively – if you're sitting on data make services available to get at it and watch the things people start doing that you never considered.

[CLICK]

SLIDE: Cloud stack display again

We've taken a tour around the cloud computing landscape and you can see there are a lot of areas that you can use. There are also some extremely niche areas I haven't talked about such as Desktop as a Service where your computing desktop is all cloud sourced however we'll leave them for today.

One really interesting thing that I've noticed about Cloud Computing is that as you go up the Cloud Stack consumers tend to USE more of those services at the bottom to PROVIDE more of the those services at the top. There's this knock on effect of one type of cloud computing providing the means for another type to occur and it's this effect that is driving much of the growth of the newer cloud types.

Now we've seen what's available let's take a look at the future to see where cloud computing is headed.

[CLICK]

SLIDE: Cloud Computing Future.

Cloud Computing is rapidly changing shape right now.

In 5 or 10 years the landscape will be as different again as it was 5 years ago.

Industries and consumers will be changed by how they interact with Cloud Services.

The fundamental paradigm shift that is occurring will allow IT to go back to being an enabler of business. Taken altogether, cloud computing becomes IT as a Service and businesses that embrace this philosophy will be able to do some great things.

Here's some examples.

[CLICK]

Slide: Manufacturing

Manufacturing will be changed by almost infinite processing power that can be provided on demand coupled with advanced computational techniques such as Genetic Programming to explore new product designs and get competitive advantage.

[CLICK]

Slide: Health Services

In health, for a long time governments in particular have been trying to make health related IT services better, providing a seamless service from one end of the health system to the other. Hybrid Cloud Services will be at the heart of our new health systems where any health professional will have proper access to medical records so they can treat patients much better.

[CLICK]

Slide: Start Ups

Start up businesses with great ideas will be able to be in market quickly, driving innovation and competing immediately with their bigger competitors, amplifying their disruptive potential within their chosen industries.

[CLICK]

Slide: Science & Research

All scientific centres will have access to supercomputer grade processing and storage which will drive new discoveries rather than just those with extreme budgets. Presently a lot of cutting edge science and research is done by centres that have almost unlimited funds. Now those with much more modest budgets will be able to use On Demand and Cost Effective cloud services to help their research.

[CLICK]

Slide: New media

All online sites and campaigns will be able to deal with large scale traffic and maintain high levels of availability, not just those with large budgets. Developers will be able to take advantage of this scalability and push the boundaries of the content they can deliver and the applications they can create.

[CLICK]

Slide: New Markets

New markets will open by providing guaranteed access to data for consumers such as to their music and videos no matter where they are in the world – this is at the heart of services such as Hulu.

Opportunities for businesses guaranteeing management and security of users personal data or industry data in so called “Data Banks” will start to present themselves. How much would you pay to have on demand access to all of your personal information all in one place that is searchable and secure?

[CLICK]

Slide: Geography

As a side effect of all these Cloud Services becoming available, businesses will no longer suffer from geographic location. Businesses in Australasia, South America and Africa will compete directly against Europe and North America. Competition will be based on knowledge and innovation not on available capital.

As you can see, all types of organisation stand to benefit from Cloud Sourcing.

[CLICK]

Slide: Warning sign

Whilst I've painted a picture of almost infinite computing resources being available, great bits of software being produced and a veritable mountain of data that's about to come online it's not all rosy.

With the pace of change of the technology outstripping everyone's ability to react quickly both within Government and Business, Cloud Computing is currently in a similar position to the Wild Wild Web of the early 90s.

Clarity is the most important requirement for any organisation producing or consuming cloud computing services.

Here's a short guide to some of the things your business needs to address before you start Cloud Sourcing.

[CLICK]

Slide: Disaster Recover and Risk Management

Disaster Recovery & Risk Management are no less important than they are in your business right now! Just because clouds are someone else's problem doesn't mean the risk management and recovery issues AREN'T your responsibility.

If you want 100% availability you have to work for it whether you are using Cloud Services or not. 100% availability is only possible by combining services whether at your DC or via the cloud.

As an example, GMAIL had a recent outage and caused a Chicken Little style panic that “The Cloud was falling”. Taken in context however, even with the number of people affected Gmail is still more reliable than corporate non-Cloud email which fails quite often.

Disaster and Risk management are no less important just because you are using Gmail or some other cloud email service. You need to have a policy to deal with outages just as you would do normally.

[CLICK]

Slide: SLAs

SLAs at the moment are the worst part of the Cloud Computing industry. Some vendors don't have any commitment, some do and there's a lot of variation between. Before committing, you need to assess the Service Level Agreement and decide if it's right for your business and right for the type service you're buying and what you're going to use it for. Some applications don't need highly available Service Levels but others do.

Slide: Data policies

Understanding the operational risk with regards to data is hugely important in a Cloud Computing environment as your data is probably going to be more public than you are used to.

Existing data policies may need to be updated due to the additional risk of being in a Cloud Environments. Many Vendors are actually better at managing this than their clients as they are doing it day in and day out.

Slide: Question mark

Before selecting a vendor it's important to ask the following questions and understand the responses and what they mean for your business

[CLICK]

1. How safe is your data?

Who has access to the physical machines?

Can the DC be cross contaminated? In large data centres it is possible for your data to be available even after you've switched off your service. Can this become available to someone else down the line?

Can you access your data to move it if you want to leave? Lock in and standards around this are a massive issue that is still not completely addressed

[CLICK]

2. How safe is your uptime?

What happens if a major hub is unavailable because of some Internet issue?

Who do you call when it all hits the fan at the Vendor end? How is their support provided? How do you support your consumers?

Who is monitoring your services?

[CLICK]

3. Are there any border concerns?

Laws are different between Australia and places such as the US and Europe.

Crossing international boundaries may bring with it additional legal requirements or may mean your services or data are subject to different laws than you are used to. Legal counsel here is imperative.

[CLICK]

Slide: Traffic light

Don't let me put you off. Forewarned is fore armed and none of the issues are insurmountable with the right policies put in place by your business.

If governments are using or considering Cloud Services then you are almost guaranteed that the issues with SLAs, Uptime and Data concerns are being addressed right now. Few businesses are as stringent in their requirements as multinationals and government agencies.

Also, bear in mind that many of these issues should be taken into account for any of your current IT services anyway so knowledge and policies you have in place now are still relevant for cloud service provision.

The important thing to remember is that you need to assess risk against the possible opportunity and make an informed business decision before embarking on any IT project.

[CLICK]

Slide: Seismic shift.

Cloud Computing is THE single most disruptive technology available to businesses right now.

We are in the middle of a fundamental change in the way IT works for the business.

This is being driven by the convergence of extremely cheap telecoms alongside sophisticated software and commodity hardware has led to an explosion of available computing power and storage. For the first time in our computing history we are approaching the point where actual deployable computing resources are going to outstrip our ability to use them. This creates a vast pool of cheap resource that will be used in innovative and disruptive ways.

If you are not using cloud computing actively within your organisation soon you WILL be left behind whilst your competition streaks away.

[CLICK]

Slide: Ticking Clock

Competitive market pressures are making this transformation occur so rapidly you have only a handful of years left to make this shift.

[CLICK]

Slide: Revolution

Cloud computing is as much of a revolution as the Internet was to business in the 90s but will have occurred in less than half the time.

Why? Because Cloud Computing enables not only the commoditisation of technology services but the democratisation of it as well.

In the same way that cheap PCs enabled an information explosion and reformed the way we communicate, Cloud Computing changes the way technology is used by everyone. Large or SMALL.

When the cloud computing revolution is complete there will be virtually NO distinction between the large and small business with regards to the scale and quality of the technology they can deploy to market.

[CLICK]

Slide: Zero

The cost to deploy these technologies will be almost ZERO

[CLICK]

Slide: Time to market

The time to be in market will be days and weeks not months or years.

[CLICK]

Slide: Infinite availability

The flexibility and scalability of the services available will be virtually INFINITE.

[CLICK]

Slide: What can I do now?

How can you start with Cloud Computing now?

If you're a business get your CRM onto something like Salesforce, get your email to Office Live or Gmail.

If you're a developer, get to grips with something like Google App Engine and deploy your next app there so you can build it faster.

If you're about to launch a campaign, use some infrastructure services to support it from the outset using something like S3, EC2

Are you sitting on a pile of data? Make it available through a data API. Chances are if it's about products you'll start selling more of them.

Here's some tips on how to sell cloud computing into your business.

Slide: Cost focussed business

If your business is cost focussed – and let's face it, most are at the moment – Cloud Services are Operating Expenditure rather than CAPEX. There's no depreciation, financing or generally up front costs. You pay for what you use and nothing more.

Cloud services are generally cheaper than maintaining Infrastructure, Platforms and Software of your own.

Providing Data as a Service might lead you to discover additional revenue streams or result in more sales of your core product.

[CLICK]

Slide: Process focussed business

If you are a process oriented business then you need to look at starting with with small, low risk trials and projects.

Look at primarily Software services for a starting point.

Make sure you do your homework around SLAs & legals. Most Cloud vendors realise there are issues and are attempting to fix them.

You can also consider setting up your own private cloud. VMWare, Melbourne IT, GoGrid and Rackspace all do this and I'm sure would be very happy to help out.

[CLICK]

Slide: Innovation focused businesses

If you are an innovation business the key thing to stress is that you can be in market faster by using Software Services or developing on Platform Clouds. Your focus needs to be on how you can innovate using these tools and how they stop you from having to replicate and go over ground that has been covered many times before.

For innovation businesses you can trial things out with much less commitment. If the results are poor, switch off the service and the costs stop. If you have a hit you can scale up quickly to accommodate your new market.

Innovation businesses will typically combine many kinds of services together in novel ways. Experience across all parts of the Cloud landscape will provide significant competitive advantage in market.

[CLICK]

Slide: Last Slide

Hopefully this presentation has helped provide some insight into what Cloud Computing is, what it can do and what you need to be aware of before jumping in. Cloud computing IS a fundamental shift in the way business works.

Cloud computing is not about a single technology or a single service. All of the elements come together differently for each and every business. Cloud Computing is primarily a shift in the way of thinking about IT and how IT can enable the business to radically transform the way it operates.

Those businesses who embrace Cloud Computing will have a significant business advantage, limited only by their ideas and their willingness to execute on them. How are you going to Cloud Source your business.

This article was cross posted to the Citrus Agency Blog

An unofficial endorsement of the Android platform?

2009-07-29T12:08:00.006+10:00

As TechCrunch reported: Pigs Fly as Facebook and Google work together on an Android App - there's been a few indicators that this might be happening, particularly with some random mentions here and there on Twitter but no one was really expecting anything to occur given the competitiveness between the two businesses.

What's most interesting about this (particularly from my standpoint as an Android user) is that it will be the only other official mobile client besides iPhone; which really endorses the Android platform as the second runner to iPhone. And in acknowledging that, it also indicates that Facebook are considering that Android will have substantial traction in the coming year - not least when you consider there are two dozen Android based phones slated to hit the market in the rest of the year which could make a serious dent in iPhone's penetration.

iPhone launching with the Facebook client has been largely cited as one of the big levers in it's sales. Smart phones have been around for a decade and there have been sleek devices previously (Nokia 7710 for example was just one big touch screen 3 years before the iPhone launched) but the mind of the consumer wasn't fired by the opportunities it could provide to them. You only had to look at the marketing by the telco's around the iPhone launch to see that the Facebook client was the Killer App for the smart phone in terms of hooking people in. It gave them a very tangible benefit to owning what would have been the most expensive handset they'd have bought to date - "I can keep in touch with my friends besides calling them..."

With an official Facebook client for Android, the same endorsement has been conferred and one of the key marketing differentiators has been removed. I'm tipping late 2010 to be an interesting time as Apple and Google really go toe to toe and start slugging it out - which will be fantastic for innovation in this space.

Case Study: Django + Agile = Sportsgirl redevelopment

2009-07-26T21:46:00.007+10:00

I've decided to write this one up because there isn't much on large scale and high speed Django development as yet and this is all still fresh in my head so it's worth getting down on paper (or screen as it were).

The agency I work for, Citrus, works with Sportsgirl, an iconic Australian Fashion Retailer and we were commissioned to help them build a community component to their site to help create a social shopping experience. The store was already there and was built as a bespoke Flash / .NET application and we had the opportunity to sit this on a different box in the data centre anyway. We thought this would be a fantastic opportunity to use Django and is exactly what it's designed for.

Architecturally we are using a LAMP stack using RHEL 5, Apache 2, mySQL (yes I know but it's to do with hosting) and obviously Django. Process wise we actually use an agency version of Agile that allows a collaborative effort between Designers, Application Developers and User Interface Developers.

Overall we were on a fixed deadline that meant the production phase was less than 8 weeks from sign off to go live including production of the site, interface and design then lock downs for content population and testing.

To make this work, everything was based around the platform - we chose as a base Django 1.0 and then layered into it a stripped down version of Pinax (we currently use v0.5.1 - the current official release, with updated apps) that has user profiles, avatar and gravatar functionality, photologue photo / image management, blog, pyBB forums, user voting and commenting.

With an established platform all three teams could start working concurrently much more effectively. This is one of the biggest benefits of Django and working to a framework and platforms like it because code can be prototyped so fast to a development build that everyone can see what they have to play with - Thanks to @jtauber and team at Pinax for that as well.

From there it was a case of lots of designing, interface creation, development and review to get it into it's final state ready for testing.

During this time we also worked on the flash home page produced by our flash master complete with nice collision detection, and full modularity so maintenance on this is all about creative not about development every time there's a refresh (very often on this brand). We'll cover this in more detail at some point.

The final phase saw deployment to the live environment which we did in Amazon EC2 for launch. We did this primarily for scalability reasons as the launch was going to be pretty large and promoted both on and off line.

As part of our final testing we also performed a lot of optimisation, this was based around optimising queries Django was making to the DB on both ends and we also then rolled out our delivery optimisations.

The first part of this was to implement memcached which is simply one of the best pieces of software presently available for data driven applications. On launch day we had a cache hit rate of over 80% which meant only 20% of all possible queries were going through to the database. With a couple of hundred thousand people visting the site during the launch phase this was instrumental in keeping particularly RAM usage low on the DB server as well as removing any bottlenecks to the Database due to latency.

We used nginx alongside apache to deliver all the static files on the site (not least because the imagery is so hi-res it was killing Apache to serve it!!). I'd often wondered how well this would work with a reasonably trafficked site but I wasn't disappointed. nginx dropped the load off the apache server which struggles for both CPU and memory (even with static files served outside of Django) from peaks on pre-live at 90% CPU and 70% available RAM + SWAP to 25% peaks on CPU and 30% RAM which is what Django was using to deliver pages with Apache's overhead.

The site went live on July 8, 2009 coinciding with a very large in store, off line and online campaign that drove quite a bit of traffic to the site. The server functioned exactly as required and with the optimisations peaked at only about 60% utilisation.

Overall this was a great project to work on not least because of the Agile process coupled with a technical foundation that allowed us to work even more collaboratively. 8 weeks for a major site launch is hard work for everyone at all levels no matter what their involvement. A great team helps with this but having the benefit of fantastic Open Source platforms to get our clients into market makes this even more achievable. Even less than 2 years ago I'm not sure I'd have attempted what the team achieved.

The Golden Age of mobile? Soon maybe...

2009-07-22T09:24:00.001+10:00

Some would say that it's already been in the heady days of GSM Data and WAP, some would say it stalled when European clients pulled all funding from mobile internet apps in the post-dot-com-crash GPRS days, some would say that with the advent of the iPhone we're there in all it's shiny-coverflow-enabled-finger-waggling-goodness.

It seems like every second person is now weilding some kind of internet enabled device and in Europe and the US the penetration is even higher than Oz although we are racing for a frontline position showing that reasonable access is more important than either coverage or cost.

Half way through 2009 it's interesting to look at some of the predictions for this year - particularly where mobile is concerned and take a quick stock.

The big 3 (Apple, MS, RIM) of last year are now well and truly the big 6 with highly competitive offerings from Android who we all knew had aspirations, the re-emergence of Palm with a life-recharging elixr known as Pre and of course Nokia firmly touting its Maemo platform that's been in development for many years and is arguably the most stable and feature rich of all.

Costs for data access across the globe are plummeting with Vodafone in the UK offering the first truly unlimited data packages on phones, showing we live in a commodity market that is almost free. EU laws limiting the charges for call and data roaming will see uptake rise as people start using their phones across countries as well.

Applications obviously make up a huge part of what our mobile experiences are like and I think if anything 2009 will go down in history as the year of the widget or micro app. Whilst iPhone still only supports Objective-C and Cocoa and their iron control is starting to hinder their progress on this front there are enough people keen to try and make a buck that the ecosystem around applications is phenomenal with over 50,000 available at last count.

Nokia, MS, RIM and Palm all have app stores however these are fledgling compared to Apple's and of all the other players Android is the only one that can be considered a contender with approaching 20,000 apps available, the vast majority of which are free. Android has a very hands off approach to this so its interesting to see what makes it through compared to Apple's more militant approach. Being Java based is also helping Android be the largest growing development community too as it's super quick to get up and running.

So where will we be in another 6 months? Will we look back and think 2009 is where it all started?

I think it's a little premature. We are really at the start right now. Much of what we are doing on phones right now isn't much more than we were doing 5-6 years ago just with a bigger screen and prettier graphics.

My money's on 2010 when we see a real rise of Augmented reality applications hit the phones. This is the area that will truly show what carrying the entire Internet around in your pocket can do and has been the spur for this part of computer science this year where it had languished for over a decade.

When my phone can alert me when my friends are nearby, interact with environmental sensors, buzz me when a store within 500m is having a sale on an item I'd previously shown interest in, automatically adjust its settings dependent on where I am and the privacy level I want to adopt and filter all of the information on the Internet into a 3 inch screen in a way that is contextual and meaningful then I think we'll consider the Golden Age has started.

It's been a while...

2009-07-01T09:20:00.000+10:00

So what with a relocation from London to Melbourne to take on the Technical Directorship of one of Australia's leading digital agencies and all that entails, keeping this blog up to date has been kind of problematic, especially with the lazy-man's alternative being Twitter and a corporate blog that I contribute to as well.

I've reassessed this blog though and it's going to be taking a slight departure but hopefully should be all the more useful because of that.

EEEPC's power is in the network not the machine

2008-05-10T05:08:00.000+10:00

Over here at Technology Treason we lurve ASUS' EEEPC. We liked the idea when they first came out and specifically trawled around Hong Kong computer markets to find one not long after they were launched. It's not the Apple Air or iPhone kind of aesthetic lust, we're talking about true "in sickness and in health" type love when it comes to the EEEPC.

Indeed for someone to now take this device off me it really would have to be from my cold, dead, rigamortis set fingers - and then only with a saw.

Go online and look at reviews. They fall into two camps - those who think it's great as a second machine that just happens to do a lot of funky things (see latest Linux Format June edition for a classic example) or those that just don't "get it" any wonder why the hell anyone would want a tiny-weenie machine when you can get a low spec dell for a few hundred quid now.

There is third camp however who are starting to realise that a linux based UMPC is truly a brilliant bit of kit and it's really because of the network it sits on not the thing plugged into it.

I've had mine for about 5 months and realistically I've installed about half a dozen bits of software - 10 at a push. I can do docs, review spreadsheets, skype, web browse hell even play games if I want and when hooked to a network I can do all of these things with all of the files I could possibly want.

My machine comes home and it auto connects to my home network, synchs to my media server and can play all my media files out of the box. I can check my mail and actually read it without squinting without firing up the laptop. I can connect from home to work via a VPN and mod some files for a client without leaving the sofa or the garden and be doing wht I want before the laptop has finished booting to a desktop.

At work I can use it for presentations and taking notes on projects without printing stupid amounts of documentation and hefting my laptop along with me.

It's not a replacement computer - it's a tool. A finely shaped, infinitely configurable tool. All the things I want in my phone but will never get because of the lack of keyboard,mouse and processing power and without it being much bigger.

The thing is I'm a techie, if I'm talking a walk down the street phone and wallet are it. If I'm going somewhere then it's satchel with camera, book, PSP and now EEEPC in place of a laptop.

ASUS have released details recently of a new version designed to hit off the people who think the other is too small. I don't know myself. Small is beautiful and in this case perfectly formed.

Ubuntu 8.04 - truly desktop Linux

2008-05-02T19:51:00.002+10:00

I'm quite an Ubuntu fan, having followed the project since more or less it's original inception. Given the general lack of problems with it thus far you'll notice very few entries on this blog about it.

Indeed various clients of mine are running Ubuntu servers that are easily maintained, easily managed and just generally easy and have been for several years. It's not necessarily an industrial strength OS - see Fedora for example - but for quick deployment, great security and stability and a modifiable tool (thanks to its Debian base) that just gets the job done you don't really need to look much further.

But I think that's about to change. You see Ubuntu 8.04 (the latest version that also happens to be a Long Term Support version) has markedly shifted the goalposts of what I expect from a Linux distribution.

I've been wanting to be convinced to move to a complete Linux desktop for the better part of 10 years but there's always something holding it back - lack of support for a media type, lack of drivers for particular bit of hardware, issues to do with wireless, no power management for my laptop, I can't run some third party apps like Skype... but that has all now changed - and changed in a massive way.

As I'm want to do, every time a new version of Ubuntu comes out it is duly installed on my Acer Travelmate notebook - a very good test of whether an upstart OS "just works" or not. The machine is about 2 years old but it has some quirks such as it has an ATI 3D card embedded on it's motherboard that was difficult to get working properly even on XP, it also has an inbuilt webcam - again with proprietary Acer drivers and it has gigabit Ethernet. Oh and it's widescreen.

About 9 times in 10 I don't even get to a working desktop without some hackery of graphics drivers, x.org files and I've even had a couple of "bomb-proof" distros just not even boot up to a command line.

I've been around Linux for a long time and I know what to expect, I'm an enthusiast and advocate so none of this surprises me at all and I'm prepared to work through the issuses. Most of the time I get to a working desktop with some sort of graphical interface that is mostly not widescreen, with no 3d support, sometimes wireless and without the use of the webcam. Linux isn't aimed at desktop use - it's just a side effect of people using it for development who wanted some creature comforts whilst working - notably the Gnome and KDE bods.

Imagine my surprise when I booted Ubuntu 8.04 and I logged into a graphical desktop that detected wireless and gigabit ethernet, properly displayed my screen in widescreen mode, gave me the option to run my ATI drivers easily and then configured the 3D in a few seconds and on top of that gave me a working webcam that I'd never had running under Linux and you could tell from the whoops of joy that here was something worth formatting my hard drive for.

Every device I threw at it was auto detected and installed in moments, flash drives, USB devices, a weather station, even an old MP3 player than needed proprietary Sony software to synch on XP. All handled with aplomb and with scarcely a pause by the processor.

What the hell was going on? How did we go from solid and okay 7.10 to this awe inspiring 8.04 in just six months? Had Mark Shuttleworth finally given his soul to Beelzebub in exchange for the most promising distro to date?

Then the answer came to me in a word: Vista.

Vista - that problematic and misbegotten child of Redmond that has been causing havoc in the IT world for nearly a year now. I haven't installed it on a work machine, neither has any other techie I know that wants to "Get Things Done". My dad had it and tried it daily for 6 months - he's now back on XP. I know corporate users who've had it on new machines and reverted to XP in order to decrease the amount of support required for users.

Shuttleworth and his cabal of Elite Ubuntu coders have recognised a change is in the air - particularly in Europe that is Linux's stronghold - we have an opportunity to put Linux on the desktop of millions of users who might upgrade to Vista but are worried about its impact. Couple this with a slight economic downturn and people are worried their existing hardware just won't work with Vista thus leading to a higher upgrade cost.

This latest LTS version gives novice and power users alike the ability to do anything they want with their desktop and it just works. It gives corporate users the knowledge that they have the security of support for 5 years without the rug being pulled out from under them.

We bit the bullet this week and put all our support team onto this version exclusively - WinXP was nuked off their machines. The development team are all dual booting but the number of XP desktops seem to be fading from view at a very fast rate as the requirement to just "drop in" on Windows becomes less necessary.

We've been saying it for nearly a decade but "this year is the year for desktop Linux" and with 8.04 Ubuntu the excuses for moving OS can now be left at the door thanks.

Easy product or class rating system

2008-04-28T04:02:00.000+10:00

So you've got a lovely little ratings system going on your site. All of a sudden though you get slashdotted, dugg or just your marketing starts working and you have thousands of users all rating your products / services / systems / posts / videos etc and your pages start to creak.

"It's the shared web space you're on," say your techies, "it can't handle the users" and duly bounce you to a better hosting environment at triple the cost along with the migration charges.

From time to time I come across this problem when I've either picked up code from someone else or else a techie asks me how to optimise a page that's running really slowly. In this particular instance it was caused by a ratings system in the style of Amazon or YouTube - basically a user is displayed a product and then people rate it as to whether it's any good. The real problem came when they had a list of products, each of which had it's individual ratings displayed.

The cause of this very slow page however had nothing to do with shared hosting or otherwise or direct server load - it was all down to some naive coding executing what my old CS lecturer would call an O(n)² process.

What the coder had done was get a list of products, then for each product gone back to the database and got a list of all the rankings ever made and then averaged them out. Nice and simple but frightfully inefficient and that which caused the problem I've highlighted.

This isn't the first time I've seen this and I've been asked how to build them numerous times as well so here's a well optimised method of doing it in general terms.

Consider first that calculating the average when you insert into the database is going to be computationally less expensive than calculating it every time you perform a select when a user hits the page. This sounds obvious but it's stunning how often it's overlooked.

Make two extra fields for your product table, one called average and the other called user_count or something. On your insert of the rating into the ratings table, run a trigger or else add some code that will update the product table with the updated count and a new average calculated from the ratings info.

Now when you select the product data you pull down the average and user count as part of that select and they are just simple static fields, thus adding no more computational load than the original select or view does already.

This gives you a nice little rating system that's not heavy in terms of processor load. However we can improve things once step further if you aren't interested in the data.

The option I'm providing below is good if you are just after a running average and don't care about the individual ratings being kept. I did a project recently where we weren't worried about keeping individual ratings data because the site wasn't going to be up for very long and it didn't add anything to our system to have it.

This option uses a running weighted average in order to just update the data in the product table without requiring a ratings table at all.

Some useful background maths though:

If I have a set {3, 4, 4} and take it's average I need to add the numbers and divide by the number of entries. Thus this set's average is (3+4+4)/3 = 3.67

Now suppose I've precalculated this average as I've suggested above and stored it without the individual ratings, I now want to add another rating, 2 to the set.

Intuition says to do something like this: (2 + 3.67)/2 = 2.83 which is actually wrong. Looking at the set {3, 4, 4, 2} we can guestimate that the average is going to be somewhere more between 3 and 4 than it is 2 and 3 as we've calculated above.

Thankfully a technique from statistics gives us an option here which is to use a weighted average instead. This is useful for adding sets together that have different numbers of elements within them but maintain the averages by skewing the data using proportional averages (or a weighted average).

The general formula for this is:

Avg_w = (Avg₁ * (n₁ / (n₁+n₂))) + (Avg₂ * (n₂/ (n₁+n₂)))

Where:

Avg₁ is the average of the first set
Avg₂ is the average of the second set
n₁ is the number of elements in the first set
n₂ is the number of elements in the second set

In our example this simplifies even further because our second set is actuall only one item. So let's work this through:

Avg_w = (3.67 * (3/(3+1))) + (2 * (1/(3+1)))

= (3.67 * 3/4) + (2 * 1/4)

= 2.75 + 0.5

= 3.25

Which is the answer we're after for our average.

As we know all the base line average data in the product table and we know the value of the rating we're tracking, it's a very simple function to update this instead of doing another insert into a ratings table and we just keep on doing it for every rating that has been added.

Computationally this is a very inexpensive process and whilst I'm more than happy to be shown otherwise I think this is about as good as it gets in terms of optimisation.

The key thing is we've now reduced an O(n)² operation to O(n) which is a drastic improvement as n tends towards infinity.

Phorm over function?

2008-04-22T17:43:00.002+10:00

Phorm is, and will continue to be for some time I think a hugely divisive issue online. BBC have another story today about it, this time having spoken to the various security companies like F-Secure, McAffee etc about whether they will flag a message to the user about whether Phorm has been enabled or not.

Phorm management have come out saying "it's only a cookie", the same as many other sites use to provide tracking (such as Google Analytics), interactivity (such as shopping carts or ID maintenance on numerous retail sites), or a small amount of memory (configuration information for the BBC home page for example).

The difference, though, is that the information is being used differently because data is being shared.

This is what got the Information Commissioners Office's back up because sharing data between companies without users opting in is a breach of the Data Protection Act - "But not if it's anonymous data" say the legal eagles from Phorm - and technically they are correct. This is a case of adopting the letter of the law rather than the spirit of it.

Tim Berners-Lee came out saying he would move ISP if he found out they were using Phorm and whilst I admire his line I fear the vast majority of consumers won't care or rather just won't be bothered to switch - just see how many people actually switch bank or utilitiy companies.

For me this is a case of the slow erosion of privacy at the hands of our ISPs. In a massively competitive market where margins are being squeezed ever tighter, the sale of their user data to Phorm must have seemed like the proverbial golden goose.

It won't take long for someone to cotton onto the flip side of this and market aggressively on the privacy front. Talk Talk made huge inroads as an ISP on the back of their "The Internet should be free" campaign with regard to price (being bundled as it was with other services). Who will be the first to play the "Internet should be private" card and sign up to a deal not using Phorm or other tracking software?

In my cynical world view, I think the security firms have realised this and it is 99% of the reason for why they are looking at it all as the anti-spy, -mal and -virus software is worth billions.

In real terms Phorm isn't actually that clever a piece of technology - most of what has been achieved is in the brokering of deals between ISPs and content owners and then a bit of clever gluing in the middle.

In the end Phorm will either be a great white elephant and just slip off the radar the way many technologies and companies have done or else it may actually be a spur to drive privacy legislation forward in line with our digital behaviour - how long it will take to do this however is the question as government is typically a long way behind technology in terms of law-making.

Can Yahoo really get things so wrong?

2008-04-21T20:43:00.004+10:00

Update - The guys at Yahoo came to our rescue after tracing through the "network" somewhat to find someone that knows someone at Yahoo to help us out. Unfortunately their techies couldn't explain why we'd been bloack listed either but we are now officially on their whitelist so big thanks to the guys for helping us out.

Yahoo are one of the original dotcoms. They've been around for a long time so they should know their business. Imagine my surprise when one of my clients starts complaining that their confirmation emails to yahoo email accounts are permanently being binned as is everything else they send - including personal communications.

Like most mail providers, free or otherwise, Yahoo have a spam policy that will look at an inbound email and then drop it in your inbox or spam folder depending on how it is classified.

As with most techies I have about a dozen email addresses at various providers in order to test exactly these sorts of issues. Especially given that the goalposts are changing all the time.

Sure enough even a personally addressed confirmation email was killed as it came into my yahoo account. "Ah ha," said I, "they've been blacklisted". So off one goes and checks the various blacklisting sites and there's nothing there. Hmmm.

It transpires that yahoo have just taken it on themselves to block that domain. Weirdly though, a personally addressed mail to me from the client with only the word "test" in the subject line is still considered Spam yet an email from some random address that doesn't reply, containing several instances each of the words "penis", "cock", "viagra" and "cialis" made it through to my inbox completely unscathed. At this point the phrase about arses and elbows definitely comes to mind.

Trying to get Yahoo to do anything about this issue is similarly problematic as there are no feedback channels to deal with this problem at all.

So overall we've just had to advise people to not use Yahoo or to check their junk mail periodically and read the mail there.

Security 101 : The user should be able to authenticate

2008-03-16T04:21:00.002+11:00

Are you listening Barclays?

I like security - particularly data security and in very particular data security that protects my personal information (unlike a certain Uk government department a few months back).

However, I've been around this game long enough, worked for a bank long enough and built more web applications capturing user data for long enough that I know there is one fundamental truth when it comes to data security and that is: pragmatism.

When I was at Uni I was told, "The only secure system is one that has no network connection, no keyboard or mouse and most of all no users" (and I apologise Dr Fekete for bastardising your phrase but you can't have done a bad job for me to remember it 15 years later!).

However the flip side of all of this was that depending on the data being protected, the security protocol should be appropriate without undue burden placed upon the user. Which is why logging into flickr is trivial but logging into your bank should and is a more arduous affair.

Banks are very secure enviroments which is good because the last thing I want is some 13 year old script kiddie making off with the tens of pounds in my bank account. Having said that, the bank should never make it difficult for me to get to the tens of pounds in my account due to security reasons.

At the moment though banks are running very scared and they are nailing the customers because of it. On my recent trip to Australia I had my card stopped no less than three times because Barclays decided that the activity looked fradulent.

Initially I thought something serious had happened but a call to Barclays got them to right the problem which was part of their new security measures. The next time it happened was because Barclays decided that it was time for me to come home and that I shouldn't be using my card in a Fraud Capital of the world like Sydney. The third time it happened though it locked my account out entirely and I was told I would have to come into a branch with identification documents to sort it all out - except there aren't any in Australia and I was leaving the next day for Hong Kong. Luckily a very understanding parent lent some cash.

I applaud Barclays' sentiments - they really were trying to protect my account, however it would appear as though client / bank trust has disappeared and I can no longer say "I want access to my money globally" without alarm systems going off all over the place. If I was backpacking I'd have been in serious trouble as without a bailout I literally had about 10c in my pocket.

Upon return to the UK Barclays' statement was along the lines of "Sorry but we're dealing with a lot of fraud and it's better to be safe than sorry". Tell this to one of my employees who just had £3K wiped out of their account due to identity theft (spent on local UK products and didn't fire off a single warning) and they are being told they have to prove it wasn't them...

In a way I feel sorry for Barclays because they are damned one way or the other - on this issue though it should just be a case of phoning and doing a vocal authentication then saying "I'm abroad for 4 weeks allow any transactions from xyz country until I say otherwise". In this manner everything other than DDs occuring in my home country should be treated as fraudulent and everything authorised abroad should be fine...

Bring on the chip in my hand is what I say...

DVD Jon strikes again

2008-02-20T12:38:00.002+11:00

At Technology Treason we love DVD Jon or Jon Lech Johansen as he is more commonly known. This great Norwegian famously broke the DVD encryption put in place by the big firms with the release of some software primarily aimed at allowing DVDs to be played on computers and unlocking the regionality of DVDs and DVD players.

When he released DeCSS he ran afoul of the US DMCA and was almost charged, he was then indicted by Norwegian authorities acting on behalf of the US who actually did go to court twice to try and convict him of hacking. Both times they failed and decided not to go to the Supreme Court.

Imagine our complete amusement in the office when we find out he's now trying it on with Apple via iTunes.

iTunes is a love it or hate it product - if you are part of the Apple / Steve Jobs faithful it is obviously the greatest thing on earth, if you know nothing about technology it's a simple product that allows you to use one of those "fangled new digital music type thingies".

If you are a techie you see it as a proprietary lock in and try and avoid it like the plague. The main issue for most techies is you can't play your music on anything other than your PC / Mac that has iTunes installed and your iPod / iPhone / iTouch.

I've railed against lock in for time immemorial - just a quick count of my personal items puts the following music players at my disposal - mobile phone (x2 because my wife has one that can play music too), MP3 capable stereo, PC (x3 - my office, my home and laptop), PSP, Xbox, a real MP3 player and my Nokia Internet Tablet - 10 devices at my personal disposal that I want to play music from and indeed do play music from.

The thing is, I know how to do all of this so I just push the files around on memory cards or over my network (streaming from my media server for example) onto the various devices. For many people this isn't possible and Apple's enforcement of the iTunes lock ins firmly violate the right I have to play my music (or video) on whatever device I choose at whatever time I choose. I also vote with my wallet and don't buy tunes from Apple.

What DVD Jon has done with his software (available from DoubleTwist for free) is allow you to take files that are locked into iTunes and essentially it plays the file, re-encoding it into a format you can play on other devices (I haven't looked properly but presumably OGG or MP3).

Just to rub salt into the wound he's going to cause Apple and the US music industry he's decided to let you share your files with friends as well. One wonders how long it will be before a writ arrive from the RIAA and Apple... I'm sure they'll be racing to get in first.

So well done Jon - keep up the good work and keep fighting the good fight - media we have legitimately purchased is ours to use on any device we own for our personal use.

Eventually the media industry will wake up and realise where they've been going wrong. Perhaps if EMI had taken notice of the way the world was going they wouldn't have had to cull a couple of thousand staff.

Why industries can still be revolutionised on the web

2008-02-12T15:56:00.000+11:00

I'm a bit of a cynic really. Anyone that's trawled through the depths of this blog will know that I have a fairly acid tongue when it comes to technology. I am a walking example of the phrase "familiarity breeds contempt".

One of the projects I've been involved in rececntly has started generating press just by virtue of it being better than anything that has preceded it in this particular industry - I personally would have preferred them to be commenting about the content but any press is good press as they say.

By rights I should have a nice warm fuzzy feeling about having a site people talk about and it's always great to receive recognition for a job well done - especially for my more junior staff who have worked damned hard on the site - however it is disappointing that we still exist in an age online where just applying some good design, good information architecture and some well balanced technology is enough to turn an entire sector on it's head.

Apologists will hold up their hands and say "we're a young form of media - it's going to take time". I however am not in this camp - how much time do we need?

Personally I find it untenable that there are still sites being built using non-standards based HTML and CSS, that sites beyond a couple of holding pages are built using things like Dreamweaver and not content managed, that good structural web design is something that still amazes people rather than being the norm and that information architecture still hasn't found its way to the hearts of 95% of the digital agencies that service the web.

I am constantly lamenting the state of most industries' websites generally. Take a tour around the leisure industry and find a website for a hotel anywhere in the world. Look at most ecommerce sites for even big retailers and certainly go anywhere online in the government, volunteering or political sectors and you are sure to be assaulted by bad design, bad technology and most importantly bad information architecture.

Even five years ago there were excuses that bore merit - changing web standards and platforms, variation of internet connection speeds and different levels of web penetration in different markets. These excuses don't exist any more. And to be honest why was it when I was learning my craft as a developer all those years ago that I was told about things like usability, information design and later information architecture but the junior developers and designers now are not...

This is why there are still industries to revolutionise if you have the contacts, the desire or the contracts to do it. Here is my short list of the biggest problem industries:

1. Tourism and leisure - get some good design and photos, don't use bog stanard templates and for goodness sake stop sending my credit card details in unencrypted email.

2. Holiday / travel booking - get some fuzzy logic in your scripting. If I can't fly tomorrow but I can fly the next day tell me without making me guess. Also make it easy for me to bounce back and forth between different trips without having to start again. Remember all those lectures about how to maintain the state of a system in Computer Science... this is what they were for.

3. Retail - Keep your site updated with accurate stock levels. I also shouldn't have to go to the end of the check out process to find out what the shipping charges are. Do a detection on my regional settings or IP address and take a best guess and say it's a guess. 95% of the time you'll be right and I'll stop having to go back and forth.

4. Service Media - When will you learn that a flash site turns off most people as does a splash page. At least have an alternative HTML site so I can find your phone number / contact email or address. Also remember that table based design was around in 1997 - time to get with the times guys.

5. Volunteering / politics - Yes I know you are on a budget but just because someone you know or your favourite intern just happens to have a copy of dreamweaver doesn't make them a professional web designer or developer. More harm than good is done by casual development - find some budget, find someone aligned to your cause and they'll do it cheaper or for kudos value and develop a site worth looking at.

6. Government - Just because a turd is shiny doen't make it worth anything. Above all make sure someone in the procuring department knows the difference between HTML and CSS and you won't get shafted. Government expenditure online is extortionate for the value achieved. Given the amount of paperwork done for any bit of government work it is amazing that Information Architecture isn't put right to the centre of the brief... how many people using direct.gov.uk would that help?

So get stuck in and lets see some other industries and sectors turned on their head. It's about time the biggest information resource in history got a bit of a spit polish and had all the kinks straightened.

The state of Oz technology

2008-01-31T16:53:00.000+11:00

Well rarely does an entire country entice me to start ranting (and at this point I'll point out I am in fact Australian) but by crikey Australian technology hasn't really moved in the last 5 years.

Now I appreciate this is a sweeping statement and I'll point out that the technology I'm talking about primarily is media based - mobile / web / internet. I have also had the benefit of living in London for the better part of 10 years so I've been at the hub of what is going on.

What I don't understand is why is it that for a nation that was at the forefront of new media ten years ago are we now in a position where nothing has shifted for the last 5. SMS is still massively underutilised and the idea of an SMS shortcode in Australia is a joke - 8 digits is only 2 shorter than a mobile number so is hardly short! Indeed everything to do with mobile is still more expensive, slower and less polished than we are used to in Europe. I went to Vodafone when I got here and asked for a pay as you go sim card for my phone that had pay as you go data on it... I was met with blank stares - Telstra and Optus were both the same.

General Internet access is similarly expensive and slow compared to what we are used to in Europe. Given a relatively modern telecommunications infrastructure, why telcos are flogging the ADSL route instead of fibre / cable begs the question of why so many roads were dug up in the capital cities to facilitate this in the late 80s and early 90s.

What is also interesting is the lack of FOSS out here. Linux is relatively popular but no where like it is in Europe. Indeed corporate America has it's laser telescopic sight firmly trained on the Australian market and even getting Linux hosting is no where as simple as getting a site hosted on a windows server. Linux certification and knowledge is still seen as a specialist skill.

Overall I'm disappointed that Australia hasn't maintained it's lead in internet technologies. In part people like me are to blame for starting our careers here and then being drawn to the brighter lights of the UK and the US where visas are easily come by, pay levels are higher and the ability to work on cutting edge technologies are plentiful.

Perhaps we are on the verge of a change in Australia and I hope that some of the ground lost can be regained over the next five years.

The warm glow of site launch

2008-01-09T09:08:00.000+11:00

I've been in this game a long time but there is still nothing sweeter than launching a site after spending a months building it with your team and the client. As a TD, site launch brings a mix of emotion - fatigue from the lack of sleep for the 10 days prior to launch, relief that the site is launching on time and on budget and the client seems happy with it all and finally worry about whether the thing will work as expected, what will everyone else think about it and by god I hope the server doesn't fall over on Day 1 under load...

My grandfather was an engineer for Philips and he described to me the same feelings when they were launching a new product so I have a sense that irrespective of discipline, team based endeavours in engineering always foster the same heady mix of emotion fuelled by relief, adrenaline and fatigue.

Whilst I am an old hand at this within this industry these days, having been here since the dawning, it is great to watch members of the team for whom this is the first of many site launches in their career and their happiness that it is done and their complete pride in their work.

Having seen photos of workers completing railways and other major constructions in the 19th and early 20th century one can't help notice the parallels of young engineers completing a job regardless of whether they are working with steel, glass or lines of code.

Potent messages of impotent industries

2007-12-22T00:56:00.000+11:00

I should probably know better than to open my mouth but the obvious has to be stated on this one. For anyone that is netSavvy enough to know what BitTorrent is, the news that TorrentSpy has just lost its court case against the MPAA isn't exactly surprising.

Hearking back all the way to Naptser we seem to have an annual tag teaming of court cases brought about by the RIAA and the MPAA in order to bring these "nasty pirate companies" such as TorrentSpy to heel.

Sites documenting the ins and outs of the case are plentiful so I won't go into detail. (For more info see the BBC report as it's quite neutral)

After every one of these cases new technologies spring up to either to protect people's privacy better or make the technology better (Naptser giving way to Kazaa and others which gave way to the BitTorrent protocol).

The recording and movie industries are worried because they are no longer the gate keepers to content and can charge what they like for it. As such the "dirty pirates" must be prosecuted even if they are, as in TorrentSpy's case, nothing more than a pointer to where the content is being held.

The great amusement in this particular case is that the only reason the MPAA "won" in this instance is because of TorrentSpy's refusal to provide the tracker and user data because this was a breach of Dutch Data Protection laws. As such the MPAA won by default.

Had this truly been a court case, it would have come to light that TorrentSpy provide a framework for people to post tracker data about any files they have on their machines and indeed they don't have copies of any of the physical files. The MPAA probably would have still had them closed down but their legal case was always going to be shaky.

So TorrentSpy will be closed, they will be bankrupted but there will be a dozen smaller companies waiting in the wings to see if they can bleed the MPAA that little bit drier.

You see the big problem here is that the MPAA can't let up now. It doesn't have the mechanics in place to distribute online properly (unlike same music where iTunes and others provide the service) not least because of the antiquated territorial boundaries films get sold by.

As such we'll be seeing another legal case next year - maybe ISOHunt will be next - and another company collapsed but then dozens more set up for a brief stab at providing content to the people.

The quote from the MPAA spokesman is great:

"The court's decision... sends a potent message to future defendants that this egregious behaviour will not be tolerated by the judicial system," John Malcolm, the MPAA's executive vice president and director of worldwide anti-piracy operations, said in a statement.

"The sole purpose of TorrentSpy and sites like it is to facilitate and promote the unlawful dissemination of copyrighted content. TorrentSpy is a one-stop shop for copyright infringement."

What's most amusing is that according to many sources, music being downloaded from "official" sources is almost as much as that being downloaded illegally. Surprising how given the tools, a cessation of hostility towards the users and a price point that accurately reflects the product being sold and the consumer comes to the party once again.

The MPAA still has a lot to learn about the Internet - one wonders how much it will cost them in legal fees in the mean time.

My top 5 jQuery seasonal wishes

2007-12-21T02:41:00.000+11:00

I've waxed lyrical about jQuery before, I've been using it a lot to do worker code which I just can't be bothered to hand write any more. Not least because jQuery handles all the little browser inconsistencies for me so the code I actually call into a page is infinitely more maintainable, especially if someone follows behind who maybe isn't so up to speed with JavaScript as I am.

However, use a tool for long enough and closeness breeds contempt as they say. In this vein (and regular readers will know I don't do complimentary very often) and in the spirit of seasonal "Listing programmes" of every style, these would be the top 5 things I'd like to see incorporated into jQuery in the next year.

5. Documentation - Starting off slowly and easily I'd definitely like to see some better documentation. Ideally I'd like to say that new sublibraries aren't included until their documentation is properly up to scratch. Some areas are very well documented other areas are sketchy at best.

4. Wait(msecs, callback) - part of the effects sublibrary, we have all kinds of effects to enable objects to slide, fade and animate but we don't have a wait command. What I would give to have a command that you can just append to a sequence of animations and then wait for a period of time before calling another function or stepping to the next instruction.

As you can see from my jQuery Slideshow the common way to do this is to call animate() with the same instruction as your last step with a callback. It's not big or clever but it does the job.

3. fadeToggle(speed) - again part of the effects sublibrary; we have slideToggle which is a great bit of code, call it and the object either slides open or shut depending on it's state. It would be great to have the same thing with fade rather than writing detection code and then calling fadeIn or fadeOut.

2. State detection - Another worker function would be really useful here to actually determine the state of an object as to whether it is on or off in display terms. I am fully aware I can use document.getElementById(objname).style.display or equally $().css.display() however this will return "none" if it's off, but it could also return "block inline table table-cell list" etc depending on what it is.

Ideally I'd like $().displayState() and it would return "on or off" or indeed true or false as a boolean so it would make display code even easier logic wise.

And finally,

1. Cast to DOM object. One of the best things about jQuery is it's query language. Using elements from the CSS and Xpath specifications pulling objects out of the document is so much easier than using DOM traversal methods.

However sometimes the jQuery functions just aren't enough and we need to cast an object to real JavaScript to play with it - a simple method of doing this would mean the power of a great interrogation language along with the ability to cast to a real DOM object.

I fully expect someone to come kick me now telling me I can do some or all of these things and indeed the functions I'm asking for exist aleady however the documentation as mentioned in number 5 is lacking in some areas so it isn't obvious if it is doable.

Obviously this is a little toungue-in-cheek as if I was that worried about these issues I'd write the code myself and submit it to the team for inclusion in the next version. Indeed perhaps that could form the basis of one of my New Year's technology resolutions.

Happy Holidays all.

SMS Bamboozlement...

2007-12-19T21:33:00.000+11:00

I'm doing some work for a client at the moment who's industry is particularly technophobic. The absolute cutting edge is a bit of YouTube video thrown willy nilly into a page. I'd also point out that design is something that rarely makes an appearance in this particular industry.

So it was pretty refreshing when we went to them with a series of ideas from the more commercial sectors of New Media and one of the things they latched onto was SMS. Queue annoyance though when we had already got everything ready to go other than to push the big green "launch" button and another company got involved and started talking about location aware services and high end data capture etc.

At this point the client dissolved into a mess of indecision - "Why weren't we doing all of this?" was the question, to which the answer was "Because you don't need to - primarily because your text messaging service is built around raising revenue through donations!"

I've had this happen in the past, notably with SEO companies. I do pity the poor clients who get stuck in these situations where they've finally decided to push their technology base along but then get waylaid by all the glittery, flashing and hypnotic LEDs.

At the end of the day it is important to remember why you are doing something and not get sidetracked (and not get ripped off). Once a strong foundation of technology is laid there is always something new you can build - you don't have to have every shiny present under the tree to have a great christmas.