This is a transcript of my speaker notes from a presentation given at the Churchill Club in Melbourne on Thursday, 23 February 2012. The overall topic was a technology briefing on IoT technologies and business opportunities and had other speakers talk on the specific technologies involved and the networking needs of these technologies. My view (presented below) was to take a less technical stance and present examples and opportunities to provoke thought on the topic. Due to the format that was used this wasn’t the actual talk though it was used as the basis of my discussion points and my opening overview.

Introduction

Good evening – my name’s Andrew Fisher and for the next 15 minutes I’m going to talk about the humanisation of the Internet of Things.

Let me ask a question. How many of you here tonight have got primary school aged kids? Put your hands up.

So right now, with just a box of Lego Mindstorms and a couple of hacks – all readily findable on Google and YouTube – your average ten year old now has the ability to conceive, design, build and deploy an Internet Connected Thing in their bedroom. When I was ten Lego was mostly fire trucks and spaceships – with some Technic if you were really lucky.

As a young teen getting into electronics I was almost always disappointed. The pay off given the amount of effort required was too low. So the lure of programming – with it’s fast iteration and almost infinite malleability was too strong and thus it became my career.

So if kids are capable of playing with this stuff in their living rooms then you can be sure there’s applications we haven’t thought of yet.

Tonight I want to talk about the how the Internet of Things technologies are being used by humans not just by machines, like how the Sensor Commons is starting to drive social change, how our physical health and well being are being enhanced by technology and how web-facilitated products are redefining our interaction with physical objects and our sense of where the physical-digital boundary lies.

Some Context

First of all some context for what is happening and why the changes we’re seeing are happening so quickly now.

Over the last five years or so it would appear that the other half of Moore’s Law – the bit showing that any given processor will get exponentially cheaper over time has finally caught up with mine and many other people’s teenage aspirations.

The Internet of Things is being driven by both sides of the Moore’s Law coin – on the one hand we have access to vast amounts of high performance computation available to us whether though PC or Cloud based architecture and on the other we finally have access to enough computing power to put inside physical devices at a price that is so cheap it is almost disposable.

However high performance and inexpensive computation are not sufficient to drive the technological changes we’re currently seeing. Along side this we need cheap, ubiquitous communications, and in many places around the world we now have this type of access in both WiFi and digital cellular networks.

So what happens when you have cheap computational power in a physical object coupled with cheap and almost infinite processing power available via cloud services which are facilitated by ubiquitous networks both wired and wireless? What happens to the the world when the price of physical computation and networking drop to the point where it can be given away for free in a McDonalds Happy Meal toy?

Well, we’re almost there now and I have some examples to share.

The sensor commons

Let’s start with the most obvious Internet of Things use case – sensor networks and telemetry. But instead of how networks of sensors are creating smart homes, smart cities and everything in between I’m going to focus on how sensors coupled with the web are driving social changes through a mechanism I call the Sensor Commons.

Deriving from Open Source, Creative Commons and Open Data movements, the Sensor Commons are being built by grass roots organisations in order to solve the problems they are seeing in their communities. By sharing designs, code and the data they collect, they are providing a means for creating understanding at a local level that are not being achieved by centralised direction.

At the heart of these projects are data aggregators and brokerages such as Open Sense, ThingSpeak and Pachube – who, via web based APIs are providing the ability to push and pull bits into the platform via whatever arrangement of atoms you would like to use. Whenever you hear about a tweeting pot plant, chances are it’s using one of these systems to do it via their web APIs.

Much like the rest of the Internet however, for every hundred tweeting pot plants, there’s a project much more worthy and they don’t always start in an architected fashion.

Last year after the Japanese earthquake for example, some hackers decided to put together an open sourced radiation sensor in order to monitor the amount of radiation in the wake of the Fukushima incident. Notably they were doing this due to the unreliability and non-public nature of official sources. Very quickly, many sensors sprang up around Japan and the Pacific to monitor the radiation levels. Commercial and scientific sensors also had their data pushed to Pachube to aggregate it too. So within only a couple of weeks, pachube became one of the most definitive sources of public radiation data in the world that was good enough to see trends.

Other developers used the data and mashed it up to create mobile apps and more importantly used Google Maps to visualise what the levels of radiation were in real time in general understandable terms. This made it easy for people to start understanding whether 50 micro seiverts was high, low or otherwise – whether it was normal or whether it could make you sick.

This availability of cheap physical devices coupled with ubiquitous networks facilitated by web technologies allowed this to happen in both a time period and geographic scale that few governments would have been capable of achieving. Even five or six years ago, trying to do this would have been almost impossible to achieve.

And now there are many such projects doing similar things – from Don’t Flush Me in New York attempting to change people’s awareness and understanding of storm water flooding in the sewage network to the Air Quality Egg project being undertaken in Europe and US which seeks to provide inexpensive air monitoring devices in numerous homes in order to assess air quality that is typically only being measured in one location in most cities with sensors that cost tens of thousands of dollars apiece.

This humanisation of sensor data through mechanisms like Sensor Commons shows how we will begin to interact with and understand our environments better. From systems that show how much energy a given building is using in real time to visualisations projected into civic spaces that show the levels of satisfaction of residents with local government services. The mashing up of data like this will prove very interesting.

Meta-Products

If networked sensors are the beginning point of many Internet of Things discussions, Meta-Products are it’s natural evolution. These are products and services that are built from scratch to take advantage of this new computational and communications architecture and without a significant web component to their design would be be able to exist. They generally create so much value that they totally change the way we behave as a result.

Health and well being

The classic example is Nike+. What’s interesting is that the mechanics of the sensor in the shoe existed previously yet the wrapping up of the Nike+ web services that sat on top of it was what drove it into the mainstream. That ability to track your own performance as well as compare and compete with others was the killer feature.

And with that, they created a new product category that is changing the way we view exercise, health and well being. From the FitBit which monitors your activity levels and can monitor your sleep through the WiThings scales that track your weight by simply stepping onto it. We’re even starting to see heart rate and blood pressure monitors for the very serious bio data geeks out there.

What I’m most surprised about is that there aren’t similar devices yet for pets even though we’re quite used by now to the whole idea of the Internet of Cows – livestock tagged with RFID chips being moved around networked farms and other parts of the supply chain for traceability reasons.

Health and well being is a huge market that is still in it’s exploratory phase but there is plenty of space for well thought out products and services. Just this week approval has been given for a team to move to production of medical implants that receive signals wirelessly from the GP at the appropriate time for a drug to be administered – in order to help patients on serious medication regimens so they don’t need to do so many injections and to help with compliance. Welcome to the future of how we’ll be administered drugs in our old age.

Web facilitated physical objects.

Meta-products can also use the web to orchestrate better physical efficiency than existed before. Take flexicar or Go Get as an example. Has anyone here subscribed to their services?

The beauty of Flexicar and the services like it is it’s simplicity – which is in turn driven by web thinking about a physical world problem. This is what happens when physical objects become discrete, track-able data points that can be interacted with via the web. The clue to the thinking here is that each vehicle has a unique name – proving it’s individuality. This shifts the conversation from being about a car in a fleet to the car that’s around the corner from me now.

Not only has the model changed the way we hire cars; it’s also changing the way we view car ownership. Many urbanites are choosing to not own vehicles as they are expensive and see a low frequency of use. Given a means of efficiently managing all the different people who want the car at different points in a day, this model becomes extremely viable. The number of Flexicars springing up around suburbs such as Prahran and St Kilda are testament to how desirable this method of ownership is becoming.

This type of meta product is shifting our behaviour from ownership to access – driven by smart networked objects intermediated by clever web services. There are huge opportunities in this space for urban environments where people are just drowning in redundant clutter – tools such as lawn mowers and other large, expensive tools would be an obvious place to start on this front but we’re starting to see similar behaviour in fashion as well.

Digital Shadows and the physical / virtual boundary

One final example I wanted to share is in the realm of kids toys – which is always an interesting space to see technology opportunities in the future. The idea I think was sound, but it was probably a little ahead of it’s time and was a commercial failure but it’s worth mentioning regardless.

Disney built a product called Clickables, which was physical jewellery and other objects that you bought. The product was marketed at Tween girls and was derived from the insight that these girls generally behave in a gift culture – that is, respect and social standing are raised by gifting each other – typically through helping others out and making and giving things to their significant friends.

The physical jewellery could be paired with a friends by clicking them together and that would create a friend relationship in a virtual world played online. Here girls could exchange virtual goods and go on adventures together. Obviously the product outcome was that additional physical artefacts could be bought and gifted to another person in the real world and that would unlock achievements, special goods and activities within the virtual one.

As I say, the product itself was a flop and was probably indicative of Disney’s lack of network thinking as a Web veteran would put it.This was because the value of the virtual environment was dependent on the number of physical devices in the real world. For any given circle of friends, in order for the virtual world to be useful they all had to have a clickables product linked together – a classic network effect issue. Someone taking another crack at this now would probably have a better run, given the drop in development costs associated with doing this type of work now that would see greater ubiquity of devices.

Conclusion

And so this that brings me full circle to the question I asked at the start; what happens when web connected, physical devices are so cheap that the hardware is disposable. What happens when a smart device can be handed out for free with a Happy Meal or as part of conference schwag?

As yet we don’t quite know where this will go, but those who have web thinking in their core DNA and who can also become skilled in designing and building physical things will architect new products and services. This will redefine whole product segments across toys, wellness, and health care all the way down to changing our relationships with ownership and the way we understand our physical environment.

The web of things for me is about the humanisation of the web, bringing it into our physical spaces and things; in order to enhance them both. This trend is just in it’s beginning, but the effects it will have on our physical world and the way we interact with it will be profound.

The Internet of Things, a term being bandied to the point of almost meaninglessness now it’s hit the mainstream of the NYT and the BBC. Yet, while the mainstay of the media struggles to describe how and why smart sensor arrays are going to mean you spend less time in traffic, ultimately pay more for your electricity but make sure your fruit is always fresh, there is a quiet revolution taking place.

The action taking place is the creation of what I call the Sensor Commons. Why is this a revolution? Because as a population we are deciding that governments and civic planners no longer have the ability to provide meaningful information at a local level.

Two posts summarise this activity and its implications beautifully for me.

The first, by Ed Bordern from Pachube, is on the creation of a community driven Air Quality Sensor Network. His passionate call to arms highlights that we have no realtime system for measuring air quality. Further, what data does exist and has been released by governments is transient due to the sampling method (ie that the sensor is moved from location to location over time). Summarising a workshop on the topic, he discusses how a community oriented sensor network can be created, funded and deployed.

The implications of this quiet revolution are discussed by Jauvan Moradi in his post on how open sensor networks will affect journalism. Jauvan discusses how citizen data will re-establish the localised roots of journalism by reporting on issues that matter locally and with accurate, real time data to help drive the story. Obviously Jauvan has an interest in media so he’s taking that slant yet this is but one of the many implications of the Sensor Commons.

We don’t know what we’re going to get when we arrive at a point where there is hyperlocalised data available on any conceivable measure – sound levels, temperature, rain levels, water quality, air quality, the number of cars passing a location in real time. The needs are going to be driven purely by local communities – by bottom-up interest groups that have access to cheap technologies to enable the sensor creation as well as a local need or concern that drives action.

I gave a talk at Web Directions in October this year on the Web of Things. The last third touched on the notion of community created and led data – citing the nascent Don’t Flush Me project in New York and the spontaneous self-organisation of radiation data in the wake of the Fukushima Disaster.

Through observation of many of these projects, as they mature one of the issues I have is that many of these endeavours require deeply technical knowledge in order to be effective. For the true Sensor Commons, as I see it, we need to have deep engagement with the population as a whole, regardless of technical ability or knowledge.

What is the Sensor Commons?

Before I get into the fundamental requirements of a Sensor Commons project it’s worth defining what I mean by the term. For me the Sensor Commons is a future state whereby we have data available to us, in real time, from a multitude of sensors that are relatively similar in design and method of data acquisition and that data is freely available whether as a data set or by API to use in whatever fashion they like.

My definition is not just about “lots of data from lots of sensors” – there is a subtlety to it implied by the “relatively similar in design and method of data acquisition” statement.

In order to be useful, we need to ensure we can compare data relatively faithfully across multiple sensors. This doesn’t need to be perfect, nor do they all need to be calibrated together, we simply need to ensure that they are “more or less” recording the same thing with similar levels of precision and consistency. Ultimately in a lot of instances we care about trended data rather than individual points so this isn’t a big problem so long as an individual sensor is relatively consistent and there isn’t ridiculous variation between sensors if they were put in the same conditions.

In my definition of the Sensor Commons geography doesn’t matter. You can be as hyper localised as measuring the sewage level of a borough as in the case of Don’t Flush Me or measuring radiation on a global scale. The scale upon which you operate is dictated by the type of thing you’re measuring. For example measuring water quality in two unlinked water courses makes almost no sense, in two different oceans it makes even less with regards to comparability.

The 5 requirements of the Sensor Commons.

We’re at a very early stage of the Sensor Commons and attempting to define it may be foolish, however by observing many different types of projects around the world I believe there are five critical requirements for getting a Sensor Commons project off the ground and making it a viable endeavour. A Sensor Commons project must:

• Gain trust
• Become dispersible
• Be highly visible
• Be entirely open
• Be upgradeable

Each of these will be dealt with in the sections below. A project that has these characteristics will generate a momentum that will exist beyond a core group of technical evangelists and will find a home within the mainstream community.

Gaining trust

Many sensor commons projects shine the light on our human behaviour. Ostensibly the goals are noble – to try and understand our environs such that we can make them better and change our behaviour – yet we must stay on the side of data and fact and not move towards blame; others can carry that torch. For example the project that seeks to close down the local smoke stack due to its impact on air quality will have a hard time fostering trust due to their agenda. We all want to have clean air but my kids go to school with the kids whose parents work in said smoke stack – how will I look at them when they lose their jobs?

In the section on dispersal I’ll talk about using existing community assets and infrastructure and trust plays a part in this. If you are piggy-backing the local library’s WiFi so you can get a network connection down in your stream bed it is imperative you don’t abuse their network by sending or requesting too much data – or harvest anything you shouldn’t do.

Trust is provided by having stated project objectives, clear policies around what data you’re going to capture, where it will go and how it will be used and available. Someone responsible for dealing with these issues and being the “go to person” for any issues or questions that arise will provide credibility as well as probably opening up some opportunities for partnership as well.

Note how trust requires no technology, merely an understanding of it. This is a perfect role to engage non-technical team members in, especially those who can articulate why the project is important to the community.

As an example the Don’t Flush Me team have done an excellent job of this, they have built trust with the authorities who are granting them access to the sewerage system – there’s no blame being cast, they are simply trying another way to help a community known problem. Similarly they are building trust with the community by creating a valuable resource for people who care about their local environment.

Become dispersible

One of the biggest issues facing Sensor Commons projects is that of dispersion. Projects that seem like such a good idea fall at the hurdle of widespread adoption. Understanding how you can disperse your sensors properly means that like a dandelion seed on the wind you’ll find plenty of places to put down and ensure success.

There are many factors that contribute to this which are discussed below:

Price

This is pretty obvious but can go overlooked. What is the total cost of the sensor? Don’t forget that as well as the material cost of components you need to factor in someones time to build the sensor (especially if it’s short run and will be hand built by the project team). You also need to factor in ongoing cost – for example if you have a remote sensor that uses the 3G network you’ll need a data plan that is paid for month by month. Similarly if it breaks down can it be fixed (if so for how much) or is it a straight replacement?

Price is a big factor in dispersal. Taking Dont Flush Me as an example, the cost of the sensor and the data plan make the project unwieldy without donations. While I’m sure this will work in the end, this isn’t the path towards quick dispersal. Contrast this with say the radiation data gathered by individuals globally – while the sensors themselves were relatively expensive, the network cost was negligible and thus led to higher uptake.

Tapping into local assets

If you can gain trust with the community then you get the opportunity to try and use community assets to help with dispersal. If you can use things like WiFi on your project why not talk people locally and see if they would be willing to “host” one of your sensors on their network so long as you don’t do anything silly. I used to belong to a kitesurfing group in the UK and we wanted to get a networked wind meter at our local beach so we could see if it was windy enough to go kitesurfing. As we all drank in the local pub on the beach anyway, the owner allowed us to mount a weather station on his roof and use his Internet connection so we could publish the data so we could all see it.

Do you have a local library that is on a main road? Might make a good location for an air quality sensor that uses their WiFi to stream the data back up. Libraries, schools, local council buildings are all community infrastructure – it’s worth a conversation to see if you can use it for your project.

In Melbourne you can walk along some of the suburban creeks that run into our bay and never be out of range of a WiFi connection for its entire length. Surely some of the people who live along that river would have an interest in the quality of the water and would share their WiFi with you? If you’re a local organisation you probably know some of them or know someone that knows some of them already.

Utilising local assets can dramatically drop the cost of a project, meaning more units, greater dispersal and better community engagement too.

Units should be self contained

The sensors themselves should be as simple and self contained as feasibly possible. Utilising batteries or solar power makes sense and if you can use WiFi then it’s even better. WiFi modules for things like Arduino use are becoming pretty cheap now so won’t blow your budget too much. It’s still cheaper to run cable if you can however this is another barrier to dispersal. It’s one thing to ask someone if you can put the sensor in the creek next to their house and send the data through their WiFi, it’s another entirely to ask them to route a cable across their yard, in a window and across a room to plug it into their router.

Don’t underestimate the cable factor – I’ve had my wonderful and generally relaxed wife draw a line at visible cables taped to the decking so I could get data from the back yard into the house.

What level of technical knowledge is required to deploy?

To gain higher levels of dispersal, drop the technical knowledge required to deploy. There’s a reason why Linksys and Netgear own the home router market – because anyone with some very basic instructions could deploy a box and get their home Internet up and running reliably. If you have a difficult package to deploy it means your technical members of your project will need to do it. This may not be a problem if you’re doing small scale projects but if you have say hundreds of nodes this becomes an issue.

An API makes your data dispersible too

Once you have your data wrap it in an API so it becomes dispersible too. This doesn’t need to be a grand piece of software engineering, either let me have all of it and document what you’ve got or else provide me a method of querying your data over a period (from X to Y) for the entire node array or individual nodes in the array. Make it lightweight and expressive, such as JSON and you’ll provide a data set that can be readily used, integrated into other systems or mashed up with other data sources easily.

Adopt permissive licensing

Permissive licensing for your hardware, software and data allows it to be used and improved upon by others. You probably haven’t considered all of the uses people will come up with for your project so let others help you.

Be highly visible

There are two aspects of visibility that should be considered; first the visibility of the device itself and second, the visibility of the data created.

With respect to the sensor itself if it is in a public place then you should endeavour to make it visible and also provide information about what it is there for. Occasionally you’re going to get vandals trash your stuff – there’s not much you can do about it. However if you take the opportunity to explain what it is and what the project is about then it becomes harder for someone to vandalise a community project than something put there “by the man”.

Once you have the data then look for ways not just to make it public but also ways to make it visible. The Neighbourhood Scoreboards project by Martin Tomisch and team from the Sydney University Design Lab showed how visibility of data at a community level could affect behaviour.

Imagine engaging with a local council that has a display on the side of their building showing what the overall air quality score was in real time for the borough? These sorts of Civic Displays could become quite common place as different projects feed data into them. There’s probably an opportunity for civic art to incorporate data from these types of projects and display it in interesting ways to the local population.

By creating visibility of the data we can raise awareness or affect behaviour which is often the goal for many of these projects.

Data should be visible online as well – not simply by making the data sets available but also highlighting some meaning as well. What I found most interesting about the self-assembly of the radiation data on pachube in the wake of the Fukushima incident was that it wasn’t “real” until it was on a google map. Prior to that point there were dozens of data streams but it was too hard to interpret the data. Making your data visible in this instance means making it approachable for people to gain understanding from it.

Be entirely open

Openness in this day and age is almost expected but it’s worth pointing out that the projects that open source all of their code, schematics and data will do better than those that don’t.

The other part of openness however is about the wider project context. This type of openness is about the transparency of the project objectives and the findings, documenting any assumptions about your data such as it’s likely error rate and whether you’re doing any manipulation of the raw data to derive a metric.

Government data sets and sensor networks are steadfastly closed but there is a lot of weight paid to them because they have an implied lack of error and high precision. Ostensibly this is because they are supposed to be “well engineered”, rigorously tested and highly calibrated devices – why else would one sensor cost $50,000?

With radiation data on pachube as an example, there was much made in April about how reliable it was given that it wasn’t calibrated, the sensors were probably sitting on peoples’ windows and that they were only consumer grade. Precision was never the intent for those deploying the sensors however so the argument was moot – ultimately the point was to assess trend. If my sensor has an accuracy level of ∓ 20%  then it’s always going to be out – probably by a similar amount. However if it goes up consistently over time, even though it’s out by 20% the trend is still going up – and I wouldn’t have known about that unless I was using a more deployable sensor because the government one is probably 200km away.

Having a culture of openness and transparency makes up for the error and lack of precision in the data. By “showing your workings” it opens up your data and method for critique and from there allows room for improvement. It also provides a method by which you can agree or disagree with the assumptions if you want to use the data and make an informed decision underpinning the data set.

Be upgradeable

The final requirement is to be upgradeable. One of the benefits of Moore’s Law is that not only do we get more computing power for the same price over time but that we get the same computing power for less dollars over time. Consider a humble arduino – something that is more powerful for about $40 than a multi-thousand dollar 286 PC back in the late 80s.

Being able to upgrade your sensor network allows you to take advantage of all the developments happening in this space. Adequately modularising your components so they can be switched out (eg switching to WiFi from cabled Ethernet) as well as abstracting your code (not doing heavy processing on your sensor, offloading it to the acquirer then processing it) make upgrading easy over time.

This means your project gets better over time rather than stagnating.

The Sensor Commons Future

Smart Cities are all well and good and IBM, Cisco and others are more than welcome to their ideas and products to make our urban infrastructure more clever – we need it more than ever now. For me this vision is narrow in that the top-down view made from a very tall tower provides an architecture that doesn’t seem to solve problems at a local level. Humans, by our nature are highly localised beings – whilst we may have to travel long distances to work we only travel a few kilometres from where we live and work once we’re there. As such we develop profound connections to our local environments – this is why we see “friends of” groups spring up for local parks, creeks or other reserves and why communities lobby so heavily for protection of these spaces. This type of technology enables us to interact with our environments differently.

If you think this is all naive data-driven techno-utopia think again.

Governments are starting to look at ways they can push their data into platforms like Pachube to make it accessible. Germany is in the process of doing this with its radiation data.

Individuals and project groups are already using tools like Pachube, Thingspeak and Open Sense to aggregate data from their local environment (eg: C02 levels).

It’s becoming almost trivially easy to create the sensors and the web tools are there to hold the data and start the process of understanding it. The chart below shows the temperature in my back yard in real time for the last week.

The access we are getting to cheap, reliable, malleable technologies such as Arduino and Xbee coupled with ubiquitous networks whether WiFi or Cellular is creating an opportunity for us to be able to understand our local environments better. Going are the days where we needed to petition councillors to do some water testing in our creeks and waterways or measure the quality of the air that we are breathing.

The deployment of these community oriented technologies will create the Sensor Commons; providing us with data that becomes available and accessible to anyone with an interest. Policy creation and stewardship will pass back to the local communities – as it should be – who will have the data to back up their decisions and create strong actions as a result.

If you have a project that is creating a Sensor Commons I’d love to hear about it. I’ll list them down here as I get them.

I saw this tweet from Tim O’Reilly the other evening, and duly read the post attached. The premise is that search in the Android Market and in iTunes is fundamentally broken – so badly busted in fact that AltaVista could have produced a better search engine. Enter into the mix, Chomp who profess to solve the app search problems by doing things like indexing comments about the app, links to it from blog posts etc in order to give it a more “google-style” ranking.

Whilst I technically applaud what Chomp have managed to do, they have almost no revenue to speak of and a dismally small audience for such a service. My sense is that the reality of app store search is that almost no one actually uses it to search for apps in a general sense – they use it for recognised brand terms or keywords which lead them, without fail, to exactly the application the user is after.

No one I know (and I talk to a LOT of people about mobile device usage) have ever said to me that they couldn’t find the app they were looking for. When was the last time someone you know lamented the fact they couldn’t discover an app based on broad criteria? I’m tipping never.

If you’re a mobile maven or a serious app hipster then yes search is broken for this context (yes, I fall into this camp too). For the other 200 million (and rising) users around the world they actually don’t care about the latest, most niche app in the market all they care about is that they can find the specific app they are after. And how do they find what app they are after? Marketing.

Pure, simple, traditional, marketing.

The app store is not the web and because it’s not the web it doesn’t get the web’s benefits (relatively easy to search, standards based, hyperlinked content). Apps are actually a lot like, wait for it, applications – and if you’ve ever had anything to do with real application development then you’ll know there are huge marketing teams (the same size or bigger than the dev teams) who exist purely to market the bejusus out of the product that’s been created.

When you play this game it’s all about awareness and conversion – typical  marketing activities and those that are doing really well in the app stores understand this model very well. The application is promoted as a product, it’s highlighted on the website, there are emails sent out raising awareness with a download link and it’s promoted on TV and all the other channels one advertises products.

Let’s face it, no one goes to iTunes and searches for “app to see what my friends and family are up to” – they type in Facebook. Similarly no one searches for “restaurant review guides” – they type in Yelp, or Urbanspoon or whatever other brand they related to out on the web.

And I can almost guarantee you that no one has ever searched for “kill pigs with irate avian projectiles using a catapult game”.

Apps, like any other product are also subject to referral. People write about them – how many blog posts are entitled “The 10 best [CATEGORY] apps for [PLATFORM] ever“? If you like something you tell someone about it and if you really like something you install it for them on their device. I’m not sure either of my parents have ever even seen an app store but they have plenty of applications installed on their phones and tablets courtesy of us kids.

One of the most telling aspects of the recent Android 4.0 launch was that they demoed in passing how using NFC you can share any arbitrary piecs of content by touching phones together. Applications was one of the demos of this and clearly the Android team has seen some data to indicate that personal referral is a major way that apps are shared.

Which leads me back to the start – that apparently search is broken in the app store.

Mobile devices are one of the most highly monetised and increasingly optimised sales channels currently in operation. Every search, application view, download and uninstall can be measured by person, by location and probably a heap of demographic inference by virtue of the rest of your account with Apple and Google.

In my view it is inconceivable that if there were even a reasonable number of searches occurring within the app stores that didn’t convert into a download that search wouldn’t be fixed. This is Google’s core business – they have the means to fix it – if something was broken. And you can be sure Apple would either buy or build something that worked as well. If even 1% of searches were resulting in a lack of discovery, that would translate to potentially hundreds of thousands of dollars, if not millions over time. That’s revenue that would be just left on the floor. Revenue like that tends to get picked up by data-centric businesses pretty quickly.

The other factor in all of this is that over the last 15 years we have been trained to believe that the web has the answers. Google is always there in a web browser, only a few seconds away, regardless of mobile device or desktop. A couple of little app stores that we only use once every couple of weeks isn’t going to become our first point of reference compared to the entity that answers our questions multiple times a day, every day.

Ultimately there’s nothing new about this at all – how do I choose what bit of video editing software to install on my Ubuntu laptop? I go to the web, I read reviews then I download and install the ones I want to check out; eventually selecting one that meets my needs. Synaptic doesn’t do this for me and it’s one of the best (and longest lived) app stores in existence.

So I don’t believe that the app store search is broken and I think there is definitely a place for more editorially driven referral. Looking at what nvidia is doing with the Tegra Zone on android is a good indicator of how this could work (magazine content to promote android games designed for the tegra graphics chip).

Ultimately I’m not convinced that apps are our long term future on mobile any more than they were on the desktop and we will start to see a rationalisation whereby the best survive and the others just fade away into nothing or at least transition into web apps, just like a lot of desktop software did over the last 10 years. At the end of this process though we’ll end up with smaller app stores, more editorially controlled, with higher quality, more relevant applications and at that point this whole search debate is meaningless. When was the last time you required search to find some “office productivity software” in Dick Smith or Best Buy or PC World? While you ponder that I’m off to use an app that “displays short, character limited messages from people all around the world” on my phone.

Recently, at Web Directions this year John Allsopp revisted a classic piece of web writing, “A Dao of web design“, 11 years since writing it. The presentation stood out as a passionate defence of the things that made the web great including it’s universality and openess.

One point he made was a rebuttal to a comment that the “problem” with the web was one of control. This was becoming the defence for why the mobile app model was the future but it’s been raised before – that the failings of the web were its lack of control, its slow speed of evolution and its difficulty.

Full disclosure: I’m a web guy. I come down on this argument the same way as John. The web has been 95% of my career thus far, has allowed me to travel the world, speak, write and have a stack of “firsts” that I am very proud of working on. 

Having seen the news that Internet Explorer has dropped below 50% browser share for the first time in over a decade I am insanely happy because it proves the point. The web doesn’t need curators or controllers, it needs champions. Internet explorer – and specifically IE6 has been the dominant controller of the web for nearly a decade.

We’ve had control – and we reaped it’s benefits – stable (though quirky) APIs and Web Standards implementations. For all it’s ills IE6 gave us okay CSS, good Javascript and a DOM that was largely workable and we really had only one platform to consider for a long time.

But we don’t need control, we need champions. Mozilla and Web Kit and Opera were our champions; but so to was the entire web community. By galvanising around web standards we made the case that control wasn’t sufficient for innovation or even good implementation.

As the desktop browser champions matured those of us in the community endorsed the browsers as being “mainstream friendly enough” so we installed firefox or chrome on every machine we could possibly “infect” with a decent, modern, secure browser. That’s assuming we didn’t just tell the person in question to go buy a Mac (so we could minimise the family and friends tech support calls). So momentum built…

When Nokia originally chose Opera and Mozilla as the core of their mobile web browser and Apple and Android chose webkit they endorsed the open mobile web – with all the consistency issues and difficulty that meant – yet the web browser is still the most used “app” on a mobile phone (outside of the phone and sms functions) and is the fuel of most of the web’s current growth. So momentum built…

When the big web properties said “no more IE6″ and meant it, license was given to every web developer and designer on the planet to say to their clients they weren’t going to support it any more – because just about every client uses at least one of Facebook, Twitter or Google at some point. So momentum built…

Now Microsoft – who famously “embraced and extended” the web and produced IE5 & 6, can’t even keep up with the innovation occuring due to the multitude of champions simply pushing the web further, day by day and month by month. With IE10 they will have just about caught up to the bar that has been set by Mozilla and Web Kit – after a decade of almost no innovation – controlling the web by fiat.

And so the champions have prevailed – in an unruly, largely disorganised, bickering, chattering, tweeting mass – but still they have prevailed. In the same way that science, when exposed to the transparency of the scientific method consistently brings new discoveries and innovation.

The position we have attained over the last 18 months and have hit now with this milestone is really a restarting of the process begun in the 90s. We are seeing large scale innovation across the web that languished for such a long period of time. This is why developers and designers are excited about their craft again after years of just about going nowhere.

But the new front is the Mobile App – with its fallacy of control that should hopefully be merely a speedbump in our history, like the ubiquitous onmouseover=”rollover()” of 2000. The web doesn’t need curators nor control; all it needs are champions who are prepared to make a statement, choose a path, have their views supported or revised and continue to evolve this amazing platform.

Post-WWII, after the dust had settled and old maps had been redrawn we entered a period of utopian future-gazing. This period, predominately lasting the 1950s and mostly found in the American and Soviet superpowers was characterised by our view that as humans we would triumph over any adversity. We held the view that all of nature was our to command – for we had just mastered the atom, the jet age had begun and we were on the cusp of sending humans into space. The dream of protein pills and personal jetpacks didn’t eventuate but the thinking of the time ushered in a new period that would last for another couple of decades – where considering big, idealistic futures was considered the norm and that one should get out and try and make them happen.

For me, Web Directions South 11 held much of the same passion and fervour. Finally, at least in Australia it seems, the shadow of the GFC is lifting and we are at the dawn of a period where a third of all humans are connected and we are starting to see the future that mastering the connected bit can achieve. Whereas in 2010 it was about the potential technologies like HTML5 and mobile technologies would bring us, in 2011 it is about our application of these technologies into new ways of working, new styles of design and the creation of a future for everyone.

Much of the conference theme was about just getting out and doing stuff, providing echoes of the 1950s sentiment that we live in a time where it is the makers’ world, that we are architects of the future and our place within it.

Indeed, WDS11 showcased a future where the web transcends the boundaries of mere browsers, where it is now the platform for our mobile worlds but increasingly our physical ones as well. As designers the call was put out that we will be designing physical objects, as developers that we will be creating means for greater utility of the web in these future environs.

Web Directions this year was a showcase in diversity in thinking and technology, like an atomic age World Fair. We had sessions on Augmented Reality merged with the Web of Animals merging with classical architectural design applied to the web merged with Physical and Ubiquitous (PDF) computing. We had visions of our future mixed with an appraisal of our current state of being. Alongside the whimsical we had the practical too, conveying the knowledge of our craft that will raise the bar yet again next year to greater levels.

And we had calls to arms in the style of presidential challenges past.

A few weeks ago I read a piece by Neal Stephenson on how Science Fiction writers had lost their talent for dreaming big ideas and articulating futures that would inspire scientists and others to attempt to fulfil them – a sentiment I totally agreed with. What I’ve since realised is that scratch the surface of the web community and there are big ideas being dreamed. Whether it’s a future where all data is interlinked seamlessly so we can create better stories about our past or one where we begin to interact with render-ghosts, those half real avatars of the robot world through the medium of the web there are futures being envisioned and articulated – and they are being made real.

Many people I talked to – both presenters and delegates kept saying over and over that this Web Directions was making them think bigger, that it wasn’t all just about skill transferral, that they were inspired to do more.

I think we exist now at a point of change, where the web has hit some unintended critical mass. The web is now permeating the fabric of humanity on a nearly global scale and Web Directions managed to tap into the web’s subconscious – highlighting the patterns that will develop over the next couple of years and take shape around the mainstream.

Regardless of whether I get my personal jetpack or not, this neo-futurism wave that seems to have swept the web community (and the maker community more generally) will at least help shake off the shadows of a post-GFC world, and sometimes a bit of unbridled futurist optimism can’t be a bad thing.

[Note I will update links from presentation synopses to actual presentations once they are uploaded]

If you were an attendee of Web Directions 2011 I’d love your view on whether you had this sense of the vibe?

A while back I criticised Adobe’s lack of web oriented strategy and in particular it’s bloody minded resistance to dropping flash. This was written during Adobe’s public spat with Apple as a result of iOS not supporting Flash and with no intent to do so.

As we’ve seen over the last 12 months, mobile devices are beginning to shape the web. At some point in the not very distant future mobile devices will be the dominant platform for the web – not high end desktop machines packing a lot of memory and quad core processors. As an android user who has flash available on both phone and tablet I concur with Mr Jobs, Flash on mobile devices is a terrible experience. Flash kills battery life and it causes a significant number of crashes. Even on the desktop Flash is respobsible for just about every browser crash I experience on a daily basis. Poorly executed flash ads built by junior designer / developers with no review seem to be the worst culprit.

It has been widely speculated that once Flash penetration drops below about 90-95% it will be untenable to continue supporting it within the development community and we are fast approaching that number. Many flash developers I know are looking at cross skilling. Quietly I’ve noticed the trend of brainstorms and solution sessions being less about “can we do this in the browser using HTML and Javascript?” and more about how far can the HTML 5 and related technologies be pushed. Websockets & JSON are the tools of the real-time; canvas is increasingly the tool of animation and give WebGL another 12-18 months to get some penetration and we’ll have 3D in the browser as well. No need for that crashy plugin any more.

Today Adobe have announced to the world not just that they’ve acquired Nitobi and Typekit but that they are also considering a future devoid of flash or at least one where it is no longer relevant to the mainstay of their product line. These acquisitions highlight the future of the web that Adobe is betting on, in much the same way they bet on the Web a decade ago by purchasing Macromedia.

Nitobi’s experience with phonegap brings some serious mobile credentials into the business, especially where the native blurs with the web in a cross-platform manner. Phonegap didn’t win the Mobile Web vs Native App war but it won some significant battles and showed manufacturers and developers alike that the web platform was the most viable for cross platform development in most cases – even on the native side. Adobe’s purchase of Nitobi should translate into tooling that will help designers and developers build higher quality mobile web and native apps and provide a good, solid revenue stream – it might finally reinvigorate the aging Dreamweaver product line too.

Typekit’s purchase is slightly different and marks Adobe’s understanding that the web, and in particular HTML and CSS, is the future of digital design. Typekit have done an amazing job in a small space of time, again almost single-handedly proving to the web community that we can have amazing typography online – at a time when web typography had been in the doldrums for nearly a decade. Given Adobe’s heritage in this space, this is a wise decision. On the surface this may not look like a huge revenue opportunity however I wouldn’t put it past Adobe to wrap typekit up in some tooling down the line as well as making a play at the agency end of town for one stop font management across domains. There’s some opportunity here I think Typekit have touched on but not had the resources to bring to bear. Adobe already has a lot of the design agency relationships and that’s a lot of potential revenue opportunities. Also consider that even a $15 a year subscription from 10 million bloggers is a tidy revenue stream.

These acquisitions are a great thing for Adobe. The purchases of Macromedia and Ominture have gone less well than anticipated as behomouth pieces of software were digested by the business. These smaller, more nimble, web oriented businesses will inject some creativity back into Adobe and hopefully some of their culture will rub off on the software giant.

A lot of people will lament the swallowing up of two extremely savvy web organisations that have shaped a big chunk of the digital landscape over the last 2 years. Maybe, but I’m quite excited by the potential this brings. With Adobe’s resources focussed in these areas, there’s a lot more to come from both Nitobi and Typekit and that will only bring benefits to the web – like flash did over a decade ago to bring interaction to the web but whose time is now past.

I’ve always loved the fact that in every smartphone there are a range of sensors attached to it. At the moment this is limited to GPS (position on the earth), orientation (how the device is tilting) and direction (how is it moving / accelerating) but we’ll get others such as temperature, light etc at some point too. While we’ve had access to these for some time at the native app level, exposing access to these sensors has been slow to filter through to the web browser. We’ve had GPS for a while using the GeoLocation API which is great but it’s a little too macro for playing with.

With the iPhone 4 and Gingerbread Android devices we’re starting to see this data become available. Mobile Chrome is lagging but Mobile Safari, Firefox for Android and Opera Mobile in developer mode are all providing access to the gyroscope and accelerometer data even though the spec is still in draft mode. (Note: Having just got a Honeycomb tablet today, it appears mobile chrome for Android now supports these features so we should see it available in handsets from Ice Cream Sandwich)

If you want to see what sort of data you can get access to take a look at the demo here if you have one of the browsers above.

Why would you even want this – a web browser is for browsing the web isn’t it?

It is, but increasingly it’s a tool for interaction as well. I started exploring this space not so much for the applications that you could create within the browser (eg games) but more out of interest in how you could use the web browser to facillitate interaction with the physical world.

I work with a lot of clients who use apps to create engagement and I know there’s a lot of download / install drop off. Even as a hardened mobile user, accustomed to installing many apps a day I get frustrated if I have to install something just to interact with a billboard or in store display (especially if it isn’t multi-platform).

If I’m standing in your store or in front of a piece of your media why are you asking me to download something?

With that in mind, I feel mobile web can come to the rescue. Why not use the web browser as a means of controlling a web app remotely. Could the web browser become a game controller in its own right, for you to be able to interact with media and with other people?

Have a look at the presentation I gave on this at Web Directions South, What do you know Melbourne in August this year. There’s slideshare and youtube available.

The YouTube screencast in particular shows how quickly people could jump on and start playing and interacting with a system that was literally only up for 90 seconds. Watching the smiles as people played this from where I was standing on the stage was gold.

Click here to view the video on YouTube.

How does it work?

Tank Tag is actually pretty simple. Using the device API events, DeviceMotion and DeviceOrientation, I’m pulling the sensor data from the phone and streaming it to a server over web sockets. The web server manages all the data and then provides it back to another browser (again using web sockets) that simply provides the view into the game using canvas (and actually handles all the game play). I’m using Django Socket IO for the server but any Socket IO based server would work and you could do this just as readily using node.js for example.

At the moment this is a proof of concept and a bit rough around the edges and the big caveat is that because the sensors generate so much data (literally as fast as the phone can produce it) that you have to limit the web socket stream. For my demo I ran this using a portable wifi hotspot and at an event / media location you could do the same thing. Trying it out over 3G was very laggy though some optimisation may help there but 3G isn’t really designed for realtime data streaming hundreds of times per second.

Applications

As described above and in my presentation there’s plenty of applications within the browser itself. The Is this an Earthquake site is a  cute application of the technology but is limited to  one user. I’ve seen a few labyrinth and bouncing ball demos as well but they are all limited to the individual’s context.

The big areas I see as opportunities for this technology are media interaction (individual or multiplayer) and physical interaction.

Media Interaction

As can be seen by the tank tag demo, using just a web stack it was possible to gain a large amount of multi-user interaction very quickly through a simple interface with an intuitive control method using the device as a controller. The opportunity for transforming media spaces (eg museums / installations) is huge – all that’s needed is a small server (A mac Mini would be sufficient) a wifi access point and a display or projector a combination that would cost less than a thousand dollars + display.

Advertising media spaces could utilise this technology too, say on train platforms where you have a lot of people standing around for up to 10-15 minutes not doing anything (and largely having their phones out already). What better way to drive brand engagement that providing a bit of fun or distraction?

Physical Interaction

This was actually my first thought in this space as for WDYK Melbourne I was planning on having a mobile phone controlled puppet on stage people could interact with. The idea was a bit cumbersome to work with in the 5 minute format but the technology works. An early stage example of this can be seen in the photos below where I’m using the web browser orientation values across each of the three axes to control the RGB channels on an LED thus creating an interactive light sculpture that merely requires a web browser and some motion to interact with.

Besides being a Django / Python guy, this was the main reason I chose Django Socket IO as the server – being in Python gave me access to USB devices so I could control the light (or servos) using the incoming data. I couldn’t do this easily (if at all) with Node.

Where to from here?

Well I’d love to see what people can do with this. I’ve released a library that is in early stages of cleaning up some of the differences between implementations of the DeviceOrientation and DeviceMotion events and the data they produce. Feel free to use, play with, test and fork that.

If you’re interested in Tank Tag and the arduino interaction stuff then that’s available as a repo on github. It’s a little rough at the moment but will be cleaned up over the next week as I stabilise it a bit more.

If you have an implementation or use case then bang it in the comments below and I’ll link them from here as well.

Web Intents, Web Activities, Web Actions – call them what you will, I’m going to call them the future of the web app. The idea isn’t new – for example the humble mailto: “protocol intent” has been around since about web 0.1 as a means of telling the browser to do something other than render a web page (in this case, fire up an email client and send a “mail to” whoever’s email address it was). Various application makers have hooked into this over time to give you more functionality direct from the browser eg Skype transforming all the tel: links to clickable phone numbers so you can Skype to directly (as well as doing some regex parsing to trying and find phone numbers in text too). Likewise there have been a few thoughts over the last 6 months or so attempting to develop out this idea with varying levels of success.

For the purposes of this post, I’m going to use the term Web Intent. I know it has android specific loading as well as infers an underpinning API binding but I’m going to assume here they are all interchangeable.

What makes web intents different than a protocol link is that they are fundamentally bound to the services we play with on the web. I look at the applications I have installed on my computer these days and other than the stock things I use to develop code, just about everything I use day to do day is a web app – twitter, gmail, gtalk, flickr, github etc. Installing applications on my computer is becoming a thing of the past, but I haven’t replaced this behaviour by installing loads of plugins / extensions for my browser.

Instead; in the tabbed browser world, I simply have tabs up one end of my browser window that are permanently there. I also have a couple of extensions that operate at a meta-level on the whole page I’m viewing – notably bit.ly, evernote and chrome2phone.

But all of this is back to front – and this is the beauty of the web intent – I want to bring my existing services to the attention of the page so it can interact with what I’m looking at right now without that web page needing to know anything specific about the Intent directly (eg what protocol or services it also has available).

It’s at this point that a demonstration is probably useful. This is the Mozilla labs view of Web Activities which is a similar concept. It’s a little contrived because nothing exists yet except the flickr plugin but it explains the idea. Go ahead, watch the video, this will wait.

Click here to view the video on YouTube.

So now you get the idea – as a user I can “install” personally useful web apps that do certain things and when I want to use them I can push the content to the app which then interacts with it in some fashion.

That’s all well and good but I can imagine my “app tray” beginning to look something like my twitter favourites or my browser bookmarks so will get out of control pretty quickly. Likewise, as a developer I struggle with this issue all the time. Just tonight I was updating the share plugin for word press and there are over 50 places I can share a link to – if I was to list them all the share system would be bigger than this blog post. That’s insane! I get into trouble about this all the time with clients – yes I know you still think everyone’s using mySpace but seriously no one’s sharing there – least of all from your professional services oriented blog post.

As a user I probably use two (either facebook or twitter as well as email), possibly three (add a bookmarking service of some sort) and rarely use four (say chrome to phone). I’m a power user – my mum just shares it to facebook – my wife just copies the link into an email.

So now I’m going to talk about real intents as I see this developing. I see this broadly aligned to the way Android Intents work, which as a developer was one of the strongest features of the early platform. It allowed developers to build upon others’ work without needing to worry about the specifics of the programming library (which is how you usually build on programming code) or the API (which is how you usually interact with other web services).

Real intents are exactly that, they capture the “idea” of what you’re trying to do and take action upon it. The best one I always think about is sharing. Say I have a page, a snippet of content or a photo and I want to share it. What do I do? If it was a URL I could bit.ly it, I could send it to my phone, I could email it, tweet it, facebook send it, facebook like it, digg it, ever note it, reddit it or all sorts of things. The reality is I’d probably just tweet it, I may email it and if I want it later it’d evernote it.

This is where intents come in. As the page author (or web application author) all I need to do is declare the intent “Share URL” and the browser will then query my services that match that intent and provide them to me. Only the web apps that can service that intent will appear and there’ll be extra points for a browser that tracks how often I use each one and surface them to the top of the list.

I could also have “Share image” which will render a different set of intents – including local and remote printing services for example. “Save for later” could sync it to my reader, send it to my kindle or phone, create a diary entry for me to read it later or just book mark it.

And so the realm of intents gets created and we come full circle with the web – the idea that there is an interconnection of information and services that are all interrelated and can interact with each other. To me this starts looking like the real semantic web – no top down taxonomies, no RDF, no microformats and no standards bodies arguing for 5 years on the best way to construct a “share protocol”. This is classic survival of the fittest. The same way links between sites and web apps themselves work – a democratic, user oriented, grass roots approach to self-organisation of services.

Within the Android community, certain “intent patterns” quickly emerged – sharing, video watching and photo manipulation being the most common. No one architected how these worked, no governing body decreed that to share a URL “one must call this method”. The beauty of this is that as an app developer I can choose which intents I’m going to “service” as well as which ones I’m going to “call”.

“But how are you going to keep track of all these intents?” I hear the cry going. It doesn’t matter, you’ll learn pretty quickly what’s important and we’ll coalesce around the main ones pretty quickly (by virtue of them being smart, logical and useful – like web sites and apps themselves). What will make this beautiful for the end user is if the application calls for an intent that isn’t installed and it goes off and searches the web for intents that match – giving the user the opportunity to install one.

We’re a little way off this nirvana in my view, but not as far as you’d think. With the clearing of some of the old browsers we’re pretty much on an ever more rapid upgrade path that will see more users with this technology in their hands more quickly. It will only take some early adopters on the web publishing side to start using it to gain traction very quickly. Imagine a day in 2013 when you can view a blog post or a news article and when you get to the bottom, instead of being assaulted by 20 share icons that look like a designer vomited on the page, you get one elegant button just saying “Share this”…

Our retail behaviour is a function of context and specifically whether I am in the right contextual mode to shop. Too often context is neglected when talking about mobile and social strategy in the retail sector – which assumes that just because I’m using my phone or engaging on facebook it means I want to purchase.

This short presentation explores how context shapes behaviour and specifically talks about how address each of these contexts in a retail scenario.

Presentation given at ad:tech Melbourne, March 31, 2011 (notes are available on Slideshare page)

This is a post I’ve been mulling over for a while and it now seems the right time to put my thoughts down around the issue of Android Fragmentation. There has been a lot of talk amongst the community about this and whilst some was fuelled by Steve Jobs (though they are remaining remarkably quiet currently) it seems like the fragmentation “issue” now has a life of it’s own amongst devs.

The start of the whole fragmentation debate goes back approximately a year to the end of 2009 / start of 2010. At this time Android version shares looked something like this:

• Cupcake (v1.5): ~ 30%
• Donut (v1.6): ~ 50%
• Eclair (v2.1): ~ 20%

By the end of 2009, we had seen 3 major upgrades to the Android OS in 6 months (1.5 in April, 1.6 in September and 2.1 in October). What has been forgotten is these shares coincided with releases of new handsets (which at the time were very few). Indeed the incredible rise of Eclair in less than 3 months to 50% of global market share of Android users was largely due to the success of the Motorola Droid which had significant uptake. It also illustrates that the total population of Android devices was still relatively small at this point so could be easily swamped by a new device and new OS this is less the case now as can be seen by low levels of Gingerbread (v2.3) penetration.

The reality is that prior to even Froyo (v 2.2 released in May 2010), Android was an operating system in pretty small total numbers (especially compared to iOS, Symbian and Blackberry) and was still rather niche with respect to it’s end user (firmly aimed at knowledgeable techies who wanted a hackable smartphone) – it was also very BETA software. At this point the Android team could afford to push a large number of OS upgrades, allowing for experimentation and feedback very quickly – they also gave themselves the opportunity to prove momentum of the operating system and to go to market with new stories, showing big improvements each time. Recall the almost slow hand clap that Apple received once it finally released Copy and Paste actions – something they could have pushed earlier were it not for their annual release cycle.

In 2010, after the release of Froyo, we see a significant slow down in version releases now the software has matured but the Android team is also bowing to pressure to do fewer releases over time because of slow carrier uptake on upgrades. This is often caused by carriers making customisations with the manufacturers and they get charged for doing updates as has become recently apparent.

The biggest problem the Android team had was at the point where Cupcake, Donut and Eclair were all more or less evenly split in share and there was some significant differences in the underlying APIs which could cause breakages. There were features that just weren’t available in older versions of the API and it’s from this root that the fragmentation issue stemmed.

Fragmentation isn’t just an issue for Android developers though. As a Web Developer I’ve had significant fragmentation issues with browsers over the years (if you’re old enough – remember going from 800×600 displays to 1024×768?). Adobe (previously Macromedia) Flash also caused significant issues when everyone wanted to use a feature of a new version (eg Video) which wasn’t available in portions of the installed base and let’s not even talk about the legacy of IE6.

It’s not often talked about, but even iOS has fragmentation issues – most of our clients want support back to the iPhone 3G (a widely used phone due to the 2 year contracts we have in Australia) – and there were such fundamental hardware changes between versions that it still causes us issues if we need to use something like the camera (in image processing for example there’s no auto focus). Almost every version of iOS requires different sizes of things like icons and this is particularly noticeable when you have to produce assets for the iPhone 4. Likewise as you move to the iPad there’s a different set of capabilities (no camera for example, a lot bigger display) and I’m sure as we move to the iPad 2 there’ll be a considerable number of inequalities across versions.

These numbers from the guys at Bump indicate that 10% of the population are still on iOS 3 which is not insignificant – and a lot more than 10% are using iPhone 3G and 3Gs’ so there’s hardware concerns as well as software ones to consider.

Just about every developer with some degree of experience has to deal with a fragmentation issue at some point. If you want to see real fragmentation in action, become a PC game developer for 18 months – just about every component can be changed in a PC and cause the computer to have very different capabilities and developer headaches.

We’re just starting to see Android tablets (and probably phones too) with Dual Core chips in them and it’s widely rumoured the A5 (the chip powering the iPad2) will be Dual Core too. What’s that going to do for your fragmentation once all your code needs to be multi-threaded?

Here are some methods to deal with fragmentation:

Stop Whinging

It’s not going to change any time soon and nor do I want it to. Having a rich ecosystem of device manufacturers creates innovation that benefits consumers and creates new opportunities for developers. Do you remember what phones were like in 2006 when Nokia was the only company “innovating” due to their massive global market share? Perhaps you would prefer to go back to your Blackberry from 2007? Stop whining about how hard your life is to support all these devices – not everyone can afford your high end Galaxy S from Samsung or upgrade to the new iPhone every time Apple release one. Accept diversity and accept that not every device will be pixel perfect and move on.

Understand your market and where it’s heading

The Android team have this page on their site for a reason – it’s to help you understand right now and historically what sort of devices are available in the wild right now – I’d love to see Apple do the same thing to help make it easier for developers to understand the devices in the wild. Once you know your numbers you can extrapolate trends and work from there.

For example we’re working on a project that might have some issues with Cupcake (~5% market share now) – we aren’t supporting it for the same reasons we aren’t supporting IE6. In order for us to try and achieve compatibility it would cost too much and we know Cupcake is on the way out. It’s largely on phones nearly 2 years old so as people upgrade over the next 6 months it will disappear nearly altogether.

Do you really need that API feature?

I’m a developer, I know what it’s like when you get new features – it’s like being an 8 year old in a candy shop with $10 in your pocket. If you know your numbers you can assess whether you absolutely need something or whether you just want it because it’s shiny.

Structure your code properly

We have an architecture called Model View Controller (MVC) – you’re using it right? If you don’t know what it is may I suggest some further reading before embarking on phone development. MVC gives us the tools to decouple the different concers of our code (data and logic from UI and interaction). If you design your application the right way, you might need to do some UI tweaks to deal with a device with a larger screen than others – but guess what? That’s all you’ll change. Also your users will love you for it because the app works perfectly on their device.

Progressive enhancement

This has been a big part of web development for some time now and in turn came from game developers. If a basic interface will do for older devices then use it then enhance the elements you want to when you know that the device has more capabilities. An example of this is GPS. On the original iPhone, GPS wasn’t available – but that doesn’t mean you can’t do location based on cell tower triangulation or using WiFi locations to give you an estimate of where someone is. It won’t tell you what street number they are standing at but it will give you suburb resolution. Build for this and then if you can get the GPS coordinates then great – you get better resolution as devices get better (especially over time).

Do you really need an app?

This is still a big area of debate but if you answer “no” to the above question, fragmentation issues within and across Operating Systems largely disappear. If you can implement what you want as a mobile website then not only will it be easier to build, it will be easier to maintain and you can support more devices in one go rather than building lots of different apps over at least two operating systems.

Similarly, a further question is “Do you need a totally native app?” Using something like Appcelerator or the Open Source PhoneGap gives you a lot of cross-platform consistency and allows you to be a lot more efficient in app development – especially if it’s something that could be created as a web app.

Listen to your customers

Release early and release a lot of updates – with all the best testing available you’re never going to simulate the mix of capabilities, OS and installed apps on someone’s device. Be clear about the status of your app and allow them to feedback – thank people for their support and taking the time to file a bug report – every bug seen is a step along the way to having a fantastic, well honed product.

Be specific about your support

Pick up any computer game and it has minimum requirements. Even Nintendo do it with different versions of the GameBoy or with requirements for Wii Motion Plus controllers. If you’re doing it for a reason have some cajones and stand up for your rationale. “Sorry, this won’t work on these devices because the chip isn’t fast enough to play the game” works well.

If you’re not going to support HTC Magics (MyTouch) – tell people. There’s no point beating around the bush and then getting a bunch of bug reports you won’t fix and annoy your customers which will turn into hate mail. Tell them upfront and they’ll thank you for it – if they see it more often it might even spur them to upgrade to a new phone in all it’s Gingerbread or Honeycomb glory. Likewise use your platform targeting in the build process so some of those Cupcake and Donut devices just don’t even see your app in the Market.

Fragmentation can be an issue if you let it become one. Embrace the diversity and the challenge – remember that the apps you’re building could be creating a whole new experience for someone that will change how they communicate or go about their daily life and that’s an exciting place to be working – better than maintaining a 30 year old banking system huh?